Qwen3 8B

Qwen3 8B, AWQ 4-bit, on vLLM; fits 12 GB cards. Chat in your browser, plus an OpenAI-compatible API.

Engine: vLLM · AWQ
Size: 6.1 GB
GPU memory: 11 GiB or more
Runs on

Pick a machine

No machine can run this right now.

New machines appear as hosts connect them.

A smaller version runs now:

Qwen3 8B Q4_K_M (llama.cpp, any GPU) from $0.12/h

Have a GPU? Earn with it See all machines

Doesn't fit (3)

1× Apple M5 Max (40-core GPU) · New hostno inference runtime for this machine's backend1× Apple M3 Max (30-core GPU) · New hostno inference runtime for this machine's backend1× Apple M4 Pro (20-core GPU) · New hostno inference runtime for this machine's backend

Hardware that can run this

Nothing that can run this is online right now.

3 more devices could run it, none online

Hardware	Fits	Est. speed
RTX 3080 Ti · 12 GB	AWQ, Q8	89–127 tok/s
RTX 3090 · 24 GB	AWQ, Q8, FP16	91–131 tok/s
RTX 4090 · 24 GB	AWQ, Q8, FP16	98–141 tok/s

Speeds are estimates from memory bandwidth and weight size, not measured benchmarks. A price is the cheapest listed offer for that hardware.

Template details

Rating: No ratings yet
Runs: 0
Min VRAM: 12 GB
Backend: Set by the machine you pick
Version: vllm-qwen3-8b-awq-v1.0.0
Image digest: 4895d4c39da6…
Health check: SSH probe on the leased endpoint

We match GPUs on their measured runtime and dedicated VRAM. A hardware report doesn't qualify another vendor's backend.

#llm #chat #openai-api #vllm #awq

Reviews

No reviews yet.