Fix: Anthropic vertex provider

[nalaso]

Edit
Updated here - https://sdk.vercel.ai/providers/community-providers/anthropic-vertex-ai

community package - https://github.com/nalaso/anthropic-vertex-ai

---

This pr provides support for claude models on vertex ai by introducing a new package, ai/anthropic-vertex.

Solves the issue #2441

It can't be added to ai/google-vertex because the vertex ai sdk doesn't support third-party models (claude models).
It can't be added to ai/anthropic because vertex models require the google-auth-library for authentication (possible though). Let me know if i should implement this way

example
`import { anthropicVertex } from '@ai-sdk/anthropic-vertex';
import { generateText } from 'ai';

const { text } = await generateText({
model: anthropicVertex('claude-3-haiku@20240307'),
prompt: 'Write a vegetarian lasagna recipe for 4 people.',
});`

[fry69]

For some reason I had to remove the starting / from this string to get this provider to work. Otherwise I get an error:

13 |     statusCode === 429 || // too many requests
14 |     statusCode >= 500),
15 |     // server error
16 |     data
17 |   }) {
18 |     super(message);
         ^
AI_APICallError: Not Found
 url: "https://europe-west1-aiplatform.googleapis.com/v1//projects/projectId/locations/europe-west1/publishers/anthropic/models/claude-3-5-sonnet@20240620:rawPredict"

      at new APICallError (.../ai/node_modules/.pnpm/@[email protected]/node_modules/@ai-sdk/provider/dist/index.mjs:18:5)
      at ../ai/node_modules/.pnpm/@[email protected][email protected]/node_modules/@ai-sdk/provider-utils/dist/index.mjs:458:15

Note the double slash after the API base URL. I have no clue where this additional slash comes from and this was the only place I knew where I could get rid of it. It is most certainly the wrong place.

[fry69]

FYI: I just published this PR as a community provider via NPM registry -> https://www.npmjs.com/package/anthropic-vertex-ai-provider

Repository: https://github.com/fry69/anthropic-vertex-ai-provider

This is just a first cut to see if everything is working so far (publishing etc). I am willing to develop this a bit further, implement the LanguageModelV1 spec and write some tests for this provider.

[socket-security]

New dependencies detected. Learn more about Socket for GitHub ↗︎

Package	New capabilities	Transitives	Size	Publisher
npm/@ai-sdk/[email protected]	environment, network	0	1.27 MB	vercel-release-bot
npm/@ai-sdk/[email protected]	None	0	167 kB	vercel-release-bot
npm/@types/[email protected]	None	0	1.98 MB	types
npm/[email protected]	Transitive: environment, network	+1	719 kB	google-wombot
npm/[email protected]	None	0	40.1 MB	typescript-bot

View full report↗︎

[lgrammel]

fwiw i don't think this is the ideal approach bc of the code duplication. what i have in mind is more like the openai & azure provider work (azure builds on top of openai) but I haven't had the time to take a look. I'd probably also integrate it into the vertex provider in some way.

[nalaso]

I was wondering if there's a way to get this into the official package. Because claude has one of the top ai models, and vertex is an ai platform from google. Obviously, many people will be waiting to see this work.

In the case of Azure OpenAI and OpenAI, it's possible to get them to work under the same package since they don't require additional dependencies. https://stackoverflow.com/questions/76711533/how-to-use-the-python-openai-client-with-both-azure-and-openai-at-the-same-time))

For anthropic using vertex, it requires the google auth library, which is irrelevant to include in a package used by users who only access the Anthropic endpoint.

[lgrammel]

azure openai and openai are in separate packages:

• https://github.com/vercel/ai/tree/main/packages/azure

[lgrammel]

Re getting this in: exploring a solution is high up on my todo list, I might get a chance in the coming weeks to see how it can work. it'll require changing the anthropic provider

[lgrammel]

In the meantime would love to see this as a community provider published to npm

[nalaso]

Till then may I make changes such that it will be displayed here - https://sdk.vercel.ai/providers/community-providers/custom-providers?

[nalaso]

I already have the package(need to refractor). I am using it specifically for one of my project.

[fry69]

I just want to mention that the provider as is has problems and does not work (at least not for me). That is why I fixed it and published it as a community provider (after reporting problems with solutions seemed to have no effect for a few days).

But if you all have other plans I happily remove my repository (can't unpublish on NPM though, as far as I know).

[nalaso]

Let me check. I have an updated code as a separate package where everything works. Last time I checked "anthropicVertex" also works if you specify all the options in the env. I kept this as draft because I thought only I would need this, so I didn't update the PR. Happy to see someone else is actually using anthropic through vertex. And yeah, you don't have to delete the repo. I'll just push it. Am happy if u contribute there.