Swift bindings for llama.cpp thanks to which you'll be able to run compatible LLM models directly on your device.

• Lightweight and easy to use
• Works on macOS and Linux
• Supports streaming via structured concurrency
• Swift 6 ready!

• Unit tests
• Model downloads from URL and HuggingFace

Use swift package manager:

.package(url: "https://github.com/srgtuszy/llama-cpp-swift", branch: "main") 

Here's a quick example on how to use it. For more, please refer to an example app in example/ folder.

// Initialize model letmodel=tryModel(modelPath:"<model path>")letllama=tryLLama(model: model) // Results are delivered through an `AsyncStream` letprompt="what is the meaning of life?"fortryawaittokeninawait llama.infer(prompt: prompt, maxTokens:1024){print(token, terminator:"")}