Instructions to use Manuel030/alpaca-opt-6.7b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Manuel030/alpaca-opt-6.7b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Manuel030/alpaca-opt-6.7b")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Manuel030/alpaca-opt-6.7b")
model = AutoModelForCausalLM.from_pretrained("Manuel030/alpaca-opt-6.7b")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Manuel030/alpaca-opt-6.7b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Manuel030/alpaca-opt-6.7b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Manuel030/alpaca-opt-6.7b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Manuel030/alpaca-opt-6.7b

SGLang

How to use Manuel030/alpaca-opt-6.7b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Manuel030/alpaca-opt-6.7b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Manuel030/alpaca-opt-6.7b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Manuel030/alpaca-opt-6.7b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Manuel030/alpaca-opt-6.7b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Manuel030/alpaca-opt-6.7b with Docker Model Runner:
```
docker model run hf.co/Manuel030/alpaca-opt-6.7b
```

YAML Metadata Warning:empty or missing yaml metadata in repo card

Check out the documentation for more information.

alpaca-opt-6.7b

Source Code: www.github.com/Manuel030/alpaca-opt

Training and inference code for OPT models based on the Alpaca training protocol. You can get the trained model from the huggingface hub:

from transfomers import OPTForCausalLM

model = OPTForCausalLM.from_pretrained(
    "Manuel030/alpaca-opt-6.7b",
    )
tokenizer = AutoTokenizer.from_pretrained("Manuel030/alpaca-opt-6.7b")

Why?

Most base model are based on Llama and hence do not have a permissive license.

Outputs

Instruction: Tell me about alpacas.

Response: Alpacas are a species of large South American camelids. They are characterized by their long, coarse wool and their unique coat patterning. Alpacas are commonly used for their soft, warm wool, which is used in a variety of products, including clothing, blankets, and yarn. Alpacas are also used for their fiber, which is used in the production of rope and twine. Alpacas are also known for their friendly and docile nature. They are popular companions for people who enjoy the outdoors, and are often kept as pets or used as herd animals. Alpacas are also known for their unique personality traits, including their unique vocalizations and their unique way of walking. Alpacas are native to South America and are found in Argentina, Bolivia, Chile, Colombia, Ecuador, Peru, and Uruguay. They are also commonly found in other parts of the world, such as in Australia, New Zealand, and Europe. Alpacas are a popular livestock species, and are often used to help raise llamas and other types of livestock.

Instruction: Tell me about the president of Mexico in 2019.

Response: The president of Mexico in 2019 is Andres Manuel Lopez Obrador (AMLO), who took office on December 1, 2018. AMLO is a leftist politician and former mayor of Mexico City. He is known for his populist platform and his opposition to the country's political establishment. He has promised to bring about a more equal and just society and to address the problems of corruption and inequality. AMLO has also promised to address the country's problems of crime, violence, and poverty. He has also been critical of the United States' immigration policy and has pledged to protect the rights of migrants and asylum seekers. He has also promised to fight corruption and to reduce the country's dependence on foreign oil.

Credits

tloen/alpaca-lora

Downloads last month: 9