README.md

Phi-3 Mini with ONNX Runtime

Phi-3 Mini-4K-Instruct ONNX models

Clone the model

git lfs install
git clone https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-onnx

This project uses a the CPU version save in the folder Phi-3-mini-4k-instruct-onnx\cpu_and_mobile\cpu-int4-rtn-block-32.

Add a reference to the nuget package:

Microsoft.ML.OnnxRuntimeGenAI and Microsoft.ML.OnnxRuntime.