lavita/ChatDoctor-HealthCareMagic-100k
Viewer • Updated • 112k • 3.47k • 110
How to use vikash06/doctorLLM with Transformers:
# Use a pipeline as a high-level helper
from transformers import pipeline
pipe = pipeline("text-generation", model="vikash06/doctorLLM") # Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("vikash06/doctorLLM")
model = AutoModelForCausalLM.from_pretrained("vikash06/doctorLLM")How to use vikash06/doctorLLM with vLLM:
# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "vikash06/doctorLLM"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "vikash06/doctorLLM",
"prompt": "Once upon a time,",
"max_tokens": 512,
"temperature": 0.5
}'docker model run hf.co/vikash06/doctorLLM
How to use vikash06/doctorLLM with SGLang:
# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
--model-path "vikash06/doctorLLM" \
--host 0.0.0.0 \
--port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "vikash06/doctorLLM",
"prompt": "Once upon a time,",
"max_tokens": 512,
"temperature": 0.5
}'docker run --gpus all \
--shm-size 32g \
-p 30000:30000 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env "HF_TOKEN=<secret>" \
--ipc=host \
lmsysorg/sglang:latest \
python3 -m sglang.launch_server \
--model-path "vikash06/doctorLLM" \
--host 0.0.0.0 \
--port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "vikash06/doctorLLM",
"prompt": "Once upon a time,",
"max_tokens": 512,
"temperature": 0.5
}'How to use vikash06/doctorLLM with Docker Model Runner:
docker model run hf.co/vikash06/doctorLLM
Sample Input on Postman API:
Number of epochs: 10 Number of Data points: 2000
The instruction should be reasonable to ask of a person with general medical knowledge and should not require searching. In this task, your prompt should give very specific instructions to follow. Constraints, instructions, guidelines, or requirements all work, and the more of them the better.
Reference dataset: https://github.com/Kent0n-Li/ChatDoctor