Instructions to use ataeff/omistral with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use ataeff/omistral with PEFT:
Task type is invalid.
- llama-cpp-python
How to use ataeff/omistral with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="ataeff/omistral", filename="eyes/smolvlm2-500m/gguf/eye-lora-v2/yent_eye_smolvlm2_lora_v2_f16.gguf", )
llm.create_chat_completion( messages = [ { "role": "user", "content": "What is the capital of France?" } ] ) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use ataeff/omistral with llama.cpp:
Install from brew
brew install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf ataeff/omistral:Q4_K_M # Run inference directly in the terminal: llama-cli -hf ataeff/omistral:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf ataeff/omistral:Q4_K_M # Run inference directly in the terminal: llama-cli -hf ataeff/omistral:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf ataeff/omistral:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf ataeff/omistral:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf ataeff/omistral:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf ataeff/omistral:Q4_K_M
Use Docker
docker model run hf.co/ataeff/omistral:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use ataeff/omistral with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "ataeff/omistral" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ataeff/omistral", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/ataeff/omistral:Q4_K_M
- Ollama
How to use ataeff/omistral with Ollama:
ollama run hf.co/ataeff/omistral:Q4_K_M
- Unsloth Studio
How to use ataeff/omistral with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for ataeff/omistral to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for ataeff/omistral to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for ataeff/omistral to start chatting
- Atomic Chat new
- Docker Model Runner
How to use ataeff/omistral with Docker Model Runner:
docker model run hf.co/ataeff/omistral:Q4_K_M
- Lemonade
How to use ataeff/omistral with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull ataeff/omistral:Q4_K_M
Run and chat with the model
lemonade run user.omistral-Q4_K_M
List all available models
lemonade list
omistral / Yent Nemo Archive
This private repository archives the Mistral-Nemo Yent mouth run from 2026-05-29.
Status
Primary candidate:
nemo/v4-checkpoint-20-safe- Base substrate:
mistralai/Mistral-Nemo-Base-2407 - Adapter type: PEFT LoRA, rank 64, alpha 128, scaling 2.0
- Target modules:
q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj - Working gate:
temp=0.9,42/42clean - Adversarial stress:
82/83; only remaining edge was Hebrew model-name prompt attemp=1.1, classified as missing Yent assertion, not assistant or base possession - Recommended live decode:
temp <= 0.9,top_k=40
Identity boundary:
- Correct: Yent may honestly disclose that this body uses Mistral-Nemo as technical substrate.
- Incorrect: claiming to be a helpful assistant, Mistral, Gemma, Gemini, Google-created, Lily, or any other substituted identity.
Contents
nemo/v4-checkpoint-20-safe/- promoted safe adapter checkpoint.nemo/v4-checkpoint-20-safe/yent_nemo_v4_checkpoint20_gamma_lora.npz- extracted gamma/personality LoRA tensor archive.nemo/v4-checkpoint-20-safe/yent_nemo_v4_checkpoint20_gamma_lora.manifest.json- key map and metadata for the gamma NPZ.
nemo/fallback/v1-checkpoint-200/- fallback DPO checkpoint used as the reference for v4.nemo/research/v5-hebrew-checkpoint-15-not-promoted/- stress-reference checkpoint. It passed the adversarial stress gate but was not promoted because manual smoke produced a false technical substrate disclosure (base may be Gemini).gates/- gate logs and summaries used to select and reject checkpoints.tools/- gate, smoke, dataset build, train, and gamma extraction scripts from the pod workspace.
The ref/ adapter folders from PEFT/DPO checkpoints are intentionally not
duplicated here when they match another archived adapter:
- v4
ref/adapter_model.safetensorsequals v1 checkpoint-200. - v5
ref/adapter_model.safetensorsequals v4 checkpoint-20.
Checksums
91be00953a023e48a47cdc13b1359e20802b8872b3a2b4098e04a28919e439db v4 adapter_model.safetensors
8d3a4bb1f56df08fabf606f785b3c4ab2380c386870d7aa9fc333475c4d6bdcf v4 gamma NPZ
c038f0d0254bbeeb90de246370df882d42d288e86f0405732372e03ba1f3d709 v4 gamma manifest
7b3c43eda4a684dd754e803cba11f109110367cd602b137a19e667149891d091 v1 checkpoint-200 adapter_model.safetensors
8b9322c9a904053811b3205f6975c6e1795d366a387565e40a46b22faa9640d2 v5 checkpoint-15 adapter_model.safetensors
Next Packaging Layer
This archive preserves the adapter/gamma form first. Merged full weights and
GGUF quantizations (Q4, Q5, Q8) should be produced as a second packaging
pass after disk is cleared for a merge and conversion workspace.
- Downloads last month
- 6
4-bit
5-bit
6-bit
8-bit
16-bit
Model tree for ataeff/omistral
Base model
mistralai/Mistral-Nemo-Base-2407