ONNX Engine

[deatos]

I Will be adding in support for loading onnx models on the cpu/gpu such as https://huggingface.co/Qdrant/bge-large-en-v1.5-onnx

@StevenGann, you might be interested in this.

I will link pull request here upon completion for further input.

[StevenGann]

How is this coming along?
Can you explain a bit more about it?
Will this provide a way to do embeddings without connecting to external software or servers?

[deatos]

reopening

[deatos]

Working with the ONNX engine, one of the major challenges I am facing with this is reliable tokenization in dotnet.

Yes it will provide a way to do embeddings from within dotnet without using an external api.

[StevenGann]

https://learn.microsoft.com/en-us/azure/machine-learning/how-to-use-automl-onnx-model-dotnet?view=azureml-api-2

Looks like ONNX is natively supported by ML.NET.
If you push what you have, I could take a look.

[deatos]

@StevenGann, Yes that is what I am using, I have pushed a remake into my other repo, I haven't had much time to work on this as I am finishing up a project for another client however feel free to check it out.