Skip to content

Commit

Permalink
Merge pull request #35 from Cainier/feature-1.2.0
Browse files Browse the repository at this point in the history 
Feature 1.2.0
  • Loading branch information
@Cainier
Cainier authored Dec 2, 2023
2 parents 5fdf47b + b9afe72 commit 8297df1
Show file tree
Hide file tree
Showing 7 changed files with 798 additions and 183 deletions.
230 changes: 123 additions & 107 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,66 +13,135 @@ npm install gpt-tokens
yarn add gpt-tokens
```

### Usage
## Support

### Basic Models

* gpt-3.5-turbo
* gpt-3.5-turbo-0301
* gpt-3.5-turbo-0613
* gpt-3.5-turbo-1106
* gpt-3.5-turbo-16k
* gpt-3.5-turbo-16k-0613
* gpt-4
* gpt-4-0314
* gpt-4-0613
* gpt-4-32k (Not tested)
* gpt-4-32k-0314 (Not tested)
* gpt-4-32k-0613 (Not tested)
* gpt-4-1106-preview

### Fine Tune Models

* ft:gpt-3.5-turbo:xxx
* ft:gpt-3.5-turbo-1106:xxx
* ft:gpt-3.5-turbo-0613:xxx
* ft:gpt-4:xxx (Not tested)

### Others

* Fine tune training (Not rigorously tested)
* Function calling (Not rigorously tested)

## Usage

### Basic chat messages

```typescript
import { GPTTokens } from 'gpt-tokens'

const usageInfo = new GPTTokens({
model : 'gpt-3.5-turbo-0613',
model : 'gpt-3.5-turbo-1106',
messages: [
{
'role' : 'system',
'content': 'You are a helpful, pattern-following assistant that translates corporate jargon into plain English.',
},
{
'role' : 'system',
'name' : 'example_user',
'content': 'New synergies will help drive top-line growth.',
},
{
'role' : 'system',
'name' : 'example_assistant',
'content': 'Things working well together will increase revenue.',
},
{
'role' : 'system',
'name' : 'example_user',
'content': 'Let\'s circle back when we have more bandwidth to touch base on opportunities for increased leverage.',
},
{
'role' : 'system',
'name' : 'example_assistant',
'content': 'Let\'s talk later when we\'re less busy about how to do better.',
},
{
'role' : 'user',
'content': 'This late pivot means we don\'t have time to boil the ocean for the client deliverable.',
},
{
'role' : 'assistant',
'content': 'This last-minute change means we don\'t have enough time to complete the entire project for the client.',
},
{ 'role' :'system', 'content': 'You are a helpful, pattern-following assistant that translates corporate jargon into plain English.' },
{ 'role' :'user', 'content': 'This late pivot means we don\'t have time to boil the ocean for the client deliverable.' },
]
})

// ┌───────────────────┬────────┐
// │ (index) │ Values │
// ├───────────────────┼────────┤
// │ Tokens prompt │ 129 │
// │ Tokens completion │ 20 │
// │ Tokens total │ 149 │
// └───────────────────┴────────┘
console.table({
'Tokens prompt' : usageInfo.promptUsedTokens,
'Tokens completion': usageInfo.completionUsedTokens,
'Tokens total' : usageInfo.usedTokens,
console.info('Used tokens: ', usageInfo.usedTokens)
console.info('Used USD: ', usageInfo.usedUSD)
```

### Fine tune training

```typescript
import { GPTTokens } from 'gpt-tokens'

const usageInfo = new GPTTokens({
model : 'gpt-3.5-turbo-1106',
training: {
data : fs
.readFileSync(filepath, 'utf-8')
.split('\n')
.filter(Boolean)
.map(row => JSON.parse(row)),
epochs: 7,
},
})

console.info('Used tokens: ', usageInfo.usedTokens)
console.info('Used USD: ', usageInfo.usedUSD)
```

### Fine tune chat messages

```typescript
import { GPTTokens } from 'gpt-tokens'

const usageInfo = new GPTTokens({
fineTuneModel: 'ft:gpt-3.5-turbo-1106:opensftp::8IWeqPit',
messages : [
{ role: 'system', content: 'You are a helpful assistant.' },
],
})

// Price USD: 0.000298
console.log('Price USD: ', usageInfo.usedUSD)
console.info('Used tokens: ', usageInfo.usedTokens)
console.info('Used USD: ', usageInfo.usedUSD)
```

### Function calling

```typescript
import { GPTTokens } from 'gpt-tokens'

const usageInfo = new GPTTokens({
model : 'gpt-3.5-turbo-1106',
messages: [
{ role: 'user', content: 'What\'s the weather like in San Francisco and Paris?' },
],
tools : [
{
type : 'function',
function: {
name : 'get_current_weather',
description: 'Get the current weather in a given location',
parameters : {
type : 'object',
properties: {
location: {
type : 'string',
description: 'The city and state, e.g. San Francisco, CA',
},
unit : {
type: 'string',
enum: ['celsius', 'fahrenheit'],
},
},
required : ['location'],
},
},
},
]
})

console.info('Used tokens: ', usageInfo.usedTokens)
console.info('Used USD: ', usageInfo.usedUSD)
```

## Calculation method

### Basic chat messages

> Tokens calculation rules for prompt and completion:
>
> If the role of the last element of messages is not assistant, the entire messages will be regarded as a prompt, and **all content** will participate in the calculation of tokens
Expand All @@ -83,72 +152,19 @@ Verify the function above in [openai-cookbook](https://github.com/openai/openai-

![openai-cookbook.png](openai-cookbook.png)

## Support Models
### Function calling

* gpt-3.5-turbo
* gpt-3.5-turbo-0301
* gpt-3.5-turbo-0613
* gpt-3.5-turbo-1106
* gpt-3.5-turbo-16k
* gpt-3.5-turbo-16k-0613
* gpt-4
* gpt-4-0314
* gpt-4-0613
* gpt-4-32k
* gpt-4-32k-0314
* gpt-4-32k-0613
* gpt-4-1106-preview
Thanks for hmarr

https://hmarr.com/blog/counting-openai-tokens/

Test in your project
## Test in your project

```bash
node test.js yourAPIKey

# Testing GPT...
# [1/13]: Testing gpt-3.5-turbo-0301...
# Pass!
# [2/13]: Testing gpt-3.5-turbo...
# Warning: gpt-3.5-turbo may update over time. Returning num tokens assuming gpt-3.5-turbo-0613
# Pass!
# [3/13]: Testing gpt-3.5-turbo-0613...
# Pass!
# [4/13]: Testing gpt-3.5-turbo-16k...
# Warning: gpt-3.5-turbo-16k may update over time. Returning num tokens assuming gpt-3.5-turbo-16k-0613
# Pass!
# [5/13]: Testing gpt-3.5-turbo-16k-0613...
# Pass!
# [6/13]: Testing gpt-3.5-turbo-1106...
# Pass!
# [7/13]: Testing gpt-4...
# Warning: gpt-4 may update over time. Returning num tokens assuming gpt-4-0613
# Pass!
# [8/13]: Testing gpt-4-0314...
# Pass!
# [9/13]: Testing gpt-4-0613...
# Pass!
# [10/13]: Testing gpt-4-32k...
# Ignore model gpt-4-32k: 404 The model `gpt-4-32k` does not exist or you do not have access to it. Learn more: https://help.openai.com/en/articles/7102672-how-can-i-access-gpt-4.
# Warning: gpt-4-32k may update over time. Returning num tokens assuming gpt-4-32k-0613
# [11/13]: Testing gpt-4-32k-0314...
# Ignore model gpt-4-32k-0314: 404 The model `gpt-4-32k-0314` does not exist or you do not have access to it. Learn more: https://help.openai.com/en/articles/7102672-how-can-i-access-gpt-4.
# [12/13]: Testing gpt-4-32k-0613...
# Ignore model gpt-4-32k-0613: 404 The model `gpt-4-32k-0613` does not exist or you do not have access to it. Learn more: https://help.openai.com/en/articles/7102672-how-can-i-access-gpt-4.
# [13/13]: Testing gpt-4-1106-preview...
# Pass!
# Test success!
# Testing performance...
# GPTTokens: 0.473ms
# GPTTokens: 0.097ms
# GPTTokens: 0.072ms
# GPTTokens: 0.079ms
# GPTTokens: 0.095ms
# GPTTokens: 0.066ms
# GPTTokens: 0.064ms
# GPTTokens: 0.068ms
# GPTTokens: 0.077ms
# GPTTokens: 0.08ms
node test.js yourOpenAIAPIKey
```

## Dependencies

- [js-tiktoken](https://github.com/dqbd/tiktoken)
- [openai-chat-tokens](https://github.com/hmarr/openai-chat-tokens#readme)
38 changes: 33 additions & 5 deletions index.d.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,20 +6,41 @@ export declare function getEncodingForModelCached(model: supportModelType): Tikt
*
* https://notebooks.githubusercontent.com/view/ipynb?browser=edge&bypass_fastly=true&color_mode=dark&commit=d67c4181abe9dfd871d382930bb778b7014edc66&device=unknown_device&docs_host=https%3A%2F%2Fdocs.github.com&enc_url=68747470733a2f2f7261772e67697468756275736572636f6e74656e742e636f6d2f6f70656e61692f6f70656e61692d636f6f6b626f6f6b2f643637633431383161626539646664383731643338323933306262373738623730313465646336362f6578616d706c65732f486f775f746f5f636f756e745f746f6b656e735f776974685f74696b746f6b656e2e6970796e62&logged_in=true&nwo=openai%2Fopenai-cookbook&path=examples%2FHow_to_count_tokens_with_tiktoken.ipynb&platform=mac&repository_id=468576060&repository_type=Repository&version=114#6d8d98eb-e018-4e1f-8c9e-19b152a97aaf
*/
export type supportModelType = 'gpt-3.5-turbo' | 'gpt-3.5-turbo-0301' | 'gpt-3.5-turbo-0613' | 'gpt-3.5-turbo-1106' | 'gpt-3.5-turbo-16k' | 'gpt-3.5-turbo-16k-0613' | 'gpt-4' | 'gpt-4-0314' | 'gpt-4-0613' | 'gpt-4-32k' | 'gpt-4-32k-0314' | 'gpt-4-32k-0613' | 'gpt-4-1106-preview';
export declare type supportModelType = 'gpt-3.5-turbo' | 'gpt-3.5-turbo-0301' | 'gpt-3.5-turbo-0613' | 'gpt-3.5-turbo-1106' | 'gpt-3.5-turbo-16k' | 'gpt-3.5-turbo-16k-0613' | 'gpt-4' | 'gpt-4-0314' | 'gpt-4-0613' | 'gpt-4-32k' | 'gpt-4-32k-0314' | 'gpt-4-32k-0613' | 'gpt-4-1106-preview';
interface MessageItem {
name?: string;
role: 'system' | 'user' | 'assistant';
content: string;
}
export declare class GPTTokens {
constructor(options: {
model: supportModelType;
messages: MessageItem[];
model?: supportModelType;
fineTuneModel?: string;
messages?: GPTTokens['messages'];
training?: GPTTokens['training'];
tools?: GPTTokens['tools'];
debug?: boolean;
});
private checkOptions;
static readonly supportModels: supportModelType[];
readonly debug: boolean;
readonly model: supportModelType;
readonly messages: MessageItem[];
readonly fineTuneModel: string | undefined;
readonly messages?: MessageItem[];
readonly training?: {
data: {
messages: MessageItem[];
}[];
epochs: number;
};
readonly tools?: {
type: 'function';
function: {
name: string;
description?: string;
parameters: Record<string, unknown>;
};
}[];
readonly gpt3_5_turboPromptTokenUnit: number;
readonly gpt3_5_turboCompletionTokenUnit: number;
readonly gpt3_5_turbo_16kPromptTokenUnit: number;
Expand All @@ -32,7 +53,14 @@ export declare class GPTTokens {
readonly gpt4_32kCompletionTokenUnit: number;
readonly gpt4_turbo_previewPromptTokenUnit: number;
readonly gpt4_turbo_previewCompletionTokenUnit: number;
readonly gpt3_5_turbo_fine_tuneTrainingTokenUnit: number;
readonly gpt3_5_turbo_fine_tunePromptTokenUnit: number;
readonly gpt3_5_turbo_fine_tuneCompletionTokenUnit: number;
get usedUSD(): number;
private trainingUsedUSD;
private functionUsedUSD;
private fineTuneUsedUSD;
private basicUsedTokens;
get usedTokens(): number;
get promptUsedTokens(): number;
get completionUsedTokens(): number;
Expand All @@ -55,5 +83,5 @@ export declare class GPTTokens {
*/
private static num_tokens_from_messages;
}
export declare function testGPTTokens(openai: OpenAI): Promise<void>;
export declare function testGPTTokens(openai: OpenAI, prompt: string): Promise<void>;
export {};
Loading

0 comments on commit 8297df1

Please sign in to comment.