May I ask if DJL can use ChatGLM-6B（ https://github.com/THUDM/ChatGLM-6B ）Are you unfamiliar with DJL and would like to use it in company projects? Can you give me some examples?

[kkangert]

May I ask if DJL can use ChatGLM-6B（ https://github.com/THUDM/ChatGLM-6B ）Are you unfamiliar with DJL and would like to use it in company projects? Can you give me some examples?

[frankfliu]

@kangerts
We didn't try this model. But you should be able to try it out with our DJLServing: https://github.com/deepjavalibrary/djl-serving

Here is a few deploy python model examples: https://github.com/deepjavalibrary/djl-demo/tree/master/djl-serving/python-mode

And Large language model deployment blogpost:

• Deploy Falcon-40B with large model inference DLCs on Amazon SageMaker
• Build custom chatbot applications using OpenChatkit models on Amazon SageMaker
• Deploying LLMs On Amazon SageMaker With DJL Serving
• Achieve high performance with lowest cost for generative AI inference using AWS Inferentia2 and AWS Trainium on Amazon SageMaker
• Deploy large models at high performance using FasterTransformer on Amazon SageMaker

[kkangert]

@kangerts We didn't try this model. But you should be able to try it out with our DJLServing: https://github.com/deepjavalibrary/djl-serving

Here is a few deploy python model examples: https://github.com/deepjavalibrary/djl-demo/tree/master/djl-serving/python-mode

And Large language model deployment blogpost:

• Deploy Falcon-40B with large model inference DLCs on Amazon SageMaker
• Build custom chatbot applications using OpenChatkit models on Amazon SageMaker
• Deploying LLMs On Amazon SageMaker With DJL Serving
• Achieve high performance with lowest cost for generative AI inference using AWS Inferentia2 and AWS Trainium on Amazon SageMaker
• Deploy large models at high performance using FasterTransformer on Amazon SageMaker
  Okay, thank you.