Instructions to use Tesslate/OmniCoder-9B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Tesslate/OmniCoder-9B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Tesslate/OmniCoder-9B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForImageTextToText

processor = AutoProcessor.from_pretrained("Tesslate/OmniCoder-9B")
model = AutoModelForImageTextToText.from_pretrained("Tesslate/OmniCoder-9B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Tesslate/OmniCoder-9B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Tesslate/OmniCoder-9B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Tesslate/OmniCoder-9B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Tesslate/OmniCoder-9B

SGLang

How to use Tesslate/OmniCoder-9B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Tesslate/OmniCoder-9B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Tesslate/OmniCoder-9B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Tesslate/OmniCoder-9B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Tesslate/OmniCoder-9B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Tesslate/OmniCoder-9B with Docker Model Runner:
```
docker model run hf.co/Tesslate/OmniCoder-9B
```

About the omni model

by BikoRiko - opened Mar 13

Discussion

BikoRiko

Mar 13

To tell about this model from me

BikoRiko

Mar 13

Hi your model usse the name Omni I already created a withsite with Omni models I won't sue I'm just saying make a max coder version as I already released Omni max coder 3.1 witch is so advanced et codes 3d games and other apps at eas

Enderchef

Mar 13

Anyone can name their model what they want. It shouldn't matter. The word "omni" isn't owned, and there are hundreds of models with the name. Even Qwen and/or Deepseek have/has an omni model.

baton4ik

Mar 14

Only one way to find out which is the real Omni model.

BikoRiko

Mar 14

I didn't say those stuf about owning the word omni I want yall to make a max coder version for coding

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment