Exceeded token rate limit of your current OpenAI S0 pricing tier

[gochenumich]

Hi all, when following the assignment of chapter 4 (prompt engineering fundamentals), I cannot make api calls to my azure openai deployment due to status code 429, as mentioned in title. Has anyone encountered the same issue and how did you solve it. I set up my azure openai deployment, following the setup guide in chapter 0. Can we complete this course using $0 pricing tier? Thanks in advance.

[DreamSpider]

It seems you’re experiencing a 429 status code issue when making API calls to your Azure OpenAI deployment. Let’s troubleshoot this! First, ensure that you’ve followed the prompt engineering techniques for Azure OpenAI GPT models. Specifically, there are two APIs where prompt engineering comes into play: 1. Chat Completion API: This supports GPT-35-Turbo and GPT-4 models. It expects input formatted as a chat-like transcript stored inside an array of dictionaries. 2. Completion API: This supports older GPT-3 models and has more flexible input requirements (a string of text with no specific format rules). Make sure you’re using the correct API for your model. Additionally, consider the following: * System Message<https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/advanced-prompt-engineering>: Include a system message at the beginning of your prompt to prime the model with context, instructions, or relevant information<https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/advanced-prompt-engineering>1<https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/advanced-prompt-engineering>. * Rate Limits: Check if you’re exceeding rate limits (hence the 429 status code). Adjust your rate of API calls accordingly. * Prompt Validation: Validate the responses generated by the model even when using prompt engineering effectively. Not all scenarios generalize well, so testing is crucial. If you’ve covered these points and still face issues, let me know, and we’ll explore further solutions! 😊

…

________________________________ From: gochenumich ***@***.***> Sent: June 11, 2024 7:08 PM To: microsoft/generative-ai-for-beginners ***@***.***> Cc: Subscribed ***@***.***> Subject: [microsoft/generative-ai-for-beginners] Exceeded token rate limit of your current OpenAI S0 pricing tier (Discussion #399) Hi all, when following the assignment of chapter 4 (prompt engineering fundamentals), I cannot make api calls to my azure openai deployment due to status code 429, as mentioned in title. Has anyone encountered the same issue and how did you solve it. I set up my azure openai deployment, following the setup guide in chapter 0. Can we complete this course using $0 pricing tier? Thanks in advance. — Reply to this email directly, view it on GitHub<#399>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABMA2FYNWSCLZLLKXVRMVVLZG6NRNAVCNFSM6AAAAABJFLIGWKVHI2DSMVQWIX3LMV43ERDJONRXK43TNFXW4OZWHAYTAMBQG4>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[gochenumich]

Thanks, I changed the value of parameter max_token from 1024 down to 512, and that resolves it. Initially I just thought the code should be able to run without modifications