Start a runpod serverless with the docker container svenbrnn/runpod-ollama:latest. Set OLLAMA_MODEL_NAME environment to a model from ollama.com to automatically download a model.
A mounted volume will be automatically used.
| Variable Name | Description | Default Value |
|---|---|---|
OLLAMA_MODEL_NAME |
The name of the model to download | NULL |
See the test_inputs directory for example test requests.
Streaming for openai requests are fully working.
See the embed_model directory for instructions.
This project is licensed under the Creative Commons Attribution 4.0 International License. You are free to use, share, and adapt the material for any purpose, even commercially, under the following terms:
- Attribution: You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
- Reference: You must reference the original repository at https://github.com/svenbrnn/runpod-worker-ollama.
For more details, see the license.