Artificial intelligence is becoming increasingly personalized and specialized. At Salesforce we recognize the growing need for businesses and developers to integrate their chosen large language models (LLMs) into the Einstein AI Platform. To address this need, we're sharing API specifications, sample code, and other developer tools with the open-source community.
As a companion to this repo, the Einstein Platform Cookbook features sample code, recipes, and formatted versions of the API specifications.
Sample content:
- Specification: LLM Open Connector API
- Recipe: LLM Open Connector + Amazon Web Services
- Recipe: LLM Open Connector + Groq
- Recipe: LLM Open Connector + SambaNova
The Bring Your Own Large Language Model (BYOLLM) Open Connector is designed to provide powerful AI solutions to customers, independent software vendors (ISVs), and internal Salesforce teams. With this connector, you can connect the Einstein AI Platform to any language model, including custom-built models.
The BYOLLM Open Connector is a commitment to community-driven growth and innovation. By allowing users to integrate any LLM—from those models hosted on major cloud platforms to those models developed in-house—we're opening up a world of possibilities for enhanced, bespoke AI applications. This capability not only caters to the needs of large enterprises looking to leverage specific models like IBM Granite or Databricks DBRX, but also supports smaller teams eager to experiment with open-source models. With features designed to ensure ease of use, such as a streamlined UX in Einstein Studio and API specifications closely based on the OpenAI API, this connector empowers our users to enhance their AI-driven applications while maintaining high standards of security and compatibility.
- Clone this repository.
- Implement an HTTP REST service using the LLM Open Connector OpenAPI specification. This service can contain three endpoints:
/completions,/chat/completions, and/embeddings.- The
/completionsendpoint is used for simple text generation use cases. It is required for Einstein Studio. - The
/chat/completionsendpoint is used for chat-based use cases. It is required for Prompt Builder and Agentforce. - The
/embeddingsendpoint is used for embeddings use cases. It is optional.
- The
- Test your service connection using Bring Your Own Large Language Model (BYOLLM) in Einstein 1 Studio.
- Blog post: Bring Your Own Large Language Model in Einstein 1 Studio
- Help content: Bring Your Own Large Language Model
You can now use your LLM from anywhere that can access generative models from Einstein Studio.
The Models API connects your application to large language models (LLMs) through the Einstein Trust Layer. It supports the BYOLLM feature of Einstein Studio, including models created with the LLM Open Connector.
To get started, see the Models API Developer Guide.
Your contributions to this repository are welcome! Refer to the Contributing Guide and Code of Conduct to get started.
If you like the resources that you see here, consider adding a ⭐ on GitHub. It helps other people discover them!
Please report any security issue to [email protected] as soon as it is discovered. This library limits its runtime dependencies in order to reduce the total cost of ownership as much as can be, but all consumers should remain vigilant and have their security stakeholders review all third-party products (3PP) like this one and their dependencies.
All code in this repository is licensed under an Apache License v2 license.
The shorter version of license text must be added as a comment to all Salesforce-authored source code and configuration files that support comments. This rule applies to file formats like HTML, CSS, and JavaScript, which aren't exactly source code, but are still critical to the project.
/*
* Copyright (c) 2023, Salesforce, Inc.
* SPDX-License-Identifier: Apache-2
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
Rights of ALBERT EINSTEIN are used with permission of The Hebrew University of Jerusalem / CMGWorldwide.com.