Instructions to use jncarlo/tinychats-generator with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use jncarlo/tinychats-generator with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="jncarlo/tinychats-generator", trust_remote_code=True)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("jncarlo/tinychats-generator", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("jncarlo/tinychats-generator", trust_remote_code=True)

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use jncarlo/tinychats-generator with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "jncarlo/tinychats-generator"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jncarlo/tinychats-generator",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/jncarlo/tinychats-generator

SGLang

How to use jncarlo/tinychats-generator with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "jncarlo/tinychats-generator" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jncarlo/tinychats-generator",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "jncarlo/tinychats-generator" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jncarlo/tinychats-generator",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use jncarlo/tinychats-generator with Docker Model Runner:
```
docker model run hf.co/jncarlo/tinychats-generator
```

Description

This is phi_1-5 fine tuned to generate synthetic tiny chats

Example:

input: A joyful conversation between a human and a robot, word - car, verbosity - 4 output: <|system|>

The model generates something like this:

The Joyful Adventure of a Human and a Robot in a Car

<|user|> Hi, Robot! I'm going on an adventure in a car. Do you want to come with me? <|assistant|> Hello, Human! That sounds like so much fun. I've never been in a car before. <|user|> It's going to be amazing. We'll see new places and meet new friends. <|assistant|> I can't wait to see the world from a different perspective. What's the first stop on your adventure? <|user|> The first stop is a beautiful park. We'll have a picnic and play games. <|assistant|> That sounds like a lot of fun. I'll bring my favorite robot toy to play with. <|user|> Great idea! We can have a robot and a human picnic together. <|assistant|> I'm sure it will be a memorable adventure. What's the next stop on your journey? <|user|> The next stop is a big city. We'll see tall buildings and busy streets. <|assistant|> I've never been to a big city before. I'm excited to see all the lights and sounds. <|user|> It's going to be a bit overwhelming, but we'll stick together and have fun. <|assistant|> I'm glad we're going on this adventure together. It's going to be the best. <|user|> Me too, Robot. I'm so glad we're friends. Let's make the most of this adventure. <|assistant|> Definitely, Human. I can't wait to see what else we'll discover on our journey. <|user|> The journey is just beginning. There's so much more to explore and learn. <|assistant|> I'm ready for anything. With you by my side, I know we can do anything. <|user|> That's the spirit, Robot. Together, we can conquer any challenge.

Downloads last month: 6

Safetensors

Model size

1B params

Tensor type

F16