Skip to content

Commit

Permalink
fix
Browse files Browse the repository at this point in the history
  • Loading branch information
jxnl committed Aug 24, 2023
1 parent db6ebec commit adab8b2
Show file tree
Hide file tree
Showing 8 changed files with 108 additions and 37 deletions.
60 changes: 30 additions & 30 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,37 +123,37 @@ In this updated schema, we use the `Field` class from `pydantic` to add descript
!!! note "Code, schema, and prompt"
We can run `openai_schema` to see exactly what the API will see, notice how the docstrings, attributes, types, and field descriptions are now part of the schema. This describes on this library's core philosophies.

```python hl_lines="2 3"
class UserDetails(OpenAISchema):
"Correctly extracted user information"
name: str = Field(..., description="User's full name")
age: int

UserDetails.openai_schema
```

```json hl_lines="3 8"
{
"name": "UserDetails",
"description": "Correctly extracted user information",
"parameters": {
"type": "object",
"properties": {
"name": {
"description": "User's full name",
"type": "string"
},
"age": {
"type": "integer"
}
},
"required": [
"age",
"name"
]
}
```python hl_lines="2 3"
class UserDetails(OpenAISchema):
"Correctly extracted user information"
name: str = Field(..., description="User's full name")
age: int

UserDetails.openai_schema
```

```json hl_lines="3 8"
{
"name": "UserDetails",
"description": "Correctly extracted user information",
"parameters": {
"type": "object",
"properties": {
"name": {
"description": "User's full name",
"type": "string"
},
"age": {
"type": "integer"
}
```
},
"required": [
"age",
"name"
]
}
}
```

### Section 3: Calling the ChatCompletion

Expand Down
69 changes: 69 additions & 0 deletions docs/finetune.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
Instructor-FT CLI Documentation
Jobs Module
1. list (Jobs List)
Monitor the status of the most recent fine-tuning jobs.

Usage:

bash
Copy code
python instructor.py jobs watch
Options:

--limit: Limit the number of jobs to monitor (default 5).
--poll: Polling interval in seconds (default 5).
--screen: Enable or disable screen output (default False).
2. create_from_id (Create from ID)
Create a fine-tuning job from an existing ID.

Usage:

bash
Copy code
python instructor.py jobs create_from_id [ID]
Options:
# Finetuning

While function calling finetunes are not out yet I've build some additional features to make
finetuning a bit easier. After you intall the instructor library you'll be able to use the instructor commands like so:

## Using the command line

```sh
instructor --help
Usage: instructor [OPTIONS] COMMAND [ARGS]...

A CLI for fine-tuning OpenAI's models
Options:
--install-completion [bash|zsh|fish|powershell|pwsh]
Install completion for the specified shell.
--show-completion [bash|zsh|fish|powershell|pwsh]
Show completion for the specified shell, to
copy it or customize the installation.
--help Show this message and exit.
Commands:
files Manage files on OpenAI's servers
jobs Monitor and create fine tuning jobs
```

## Creating a finetuning job

Here we're going to assume you already have a file of jsonlines that you want to upload. `finetune.jsonl`

You can upload, kick off, and monitor your finetuning jobs easily

```sh
$ instructor jobs create-from-file transformed_data.jsonl
```

You'll first see some logging status since it takes time for data to be uplaoded

```sh
Monitoring file upload for <file-id>
```

One that completes you'll see a similar progress for kicking off the finetunes and ulimately the monitoring stages,

![finetune](img/finetune.png)
Binary file added docs/img/finetune.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Empty file added instructor/cli/__init__.py
Empty file.
7 changes: 2 additions & 5 deletions instructor/cli/instructor.py → instructor/cli/cli.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import typer
import jobs
import files
import instructor.cli.jobs as jobs
import instructor.cli.files as files

app = typer.Typer(
name="instructor-ft",
Expand All @@ -9,6 +9,3 @@

app.add_typer(jobs.app, name="jobs", help="Monitor and create fine tuning jobs")
app.add_typer(files.app, name="files", help="Manage files on OpenAI's servers")

if __name__ == "__main__":
app()
2 changes: 1 addition & 1 deletion instructor/cli/files.py
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ def download(
@app.command(
help="Delete a file from OpenAI's servers",
)
def delete(file_id: str = typer.Argument(help="The ID of the file to delete")):
def delete(file_id: str = typer.Argument(..., help="ID of the file to delete")):
with console.status(
f"[bold red]Deleting file {file_id}...", spinner="dots"
) as status:
Expand Down
2 changes: 2 additions & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,8 @@ nav:
- 'MultiTask': 'api_multitask.md'
- "Introduction: Writing Prompts": "writing-prompts.md"
- "Prompting Templates": "chat-completion.md"
- CLI Reference:
- "Finetuning": finetune.md
extra:
analytics:
provider: google
Expand Down
5 changes: 4 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[tool.poetry]
name = "instructor"
version = "0.2.4"
version = "0.2.5"
description = "Helper functions that allow us to improve openai's function_call ergonomics"
authors = ["Jason <[email protected]>"]
license = "MIT"
Expand All @@ -12,7 +12,10 @@ repository = "https://github.com/jxnl/instructor"
python = "^3.9"
openai = "^0.27.8"
pydantic = "^2.0.2"
typer = "^0.4.0" # Add this line

[tool.poetry.scripts]
instructor = "instructor.cli.cli:app"

[tool.poetry.group.dev.dependencies]
pytest = "^7.4.0"
Expand Down

0 comments on commit adab8b2

Please sign in to comment.