Qwen3 8B AWQ (vLLM)
Qwen3 8B, AWQ 4-bit, on vLLM; fits 12 GB cards.
Needs 11 GiB VRAM · CUDA on NVIDIA
No GPU big enough is free right now.
New machines appear as hosts connect them.
No speed benchmarks yet, so we match on price.
Template details
- Rating
- No ratings yet
- Runs
- 0
- Min VRAM
- 12 GB
- Backend
- CUDA · NVIDIA GPUs
- Version
- vllm-qwen3-8b-awq-v1.0.0
- Image digest
- 4895d4c39da6…
- Health check
- SSH probe on the leased endpoint
OpenAI-compatible API on the leased HTTP endpoint (per-job API key from the rental's access panel). Weights (6.1 GB) are warmed into the host cache and mounted read-only. Thin AmpleRun wrapper (templates/images/workspace, variant serve-vllm) over vllm/vllm-openai:v0.30.0-cu129@sha256:a67f8f186d4567612ac37a55bd82295006002b18858eb12a8af2f05f86c2ae3f (digest verified 2026-09-26); NOT BUILT, so image_digest is a placeholder that can be neither published nor rented. Licence: Apache-2.0 (https://github.com/vllm-project/vllm/blob/main/LICENSE). Weights: Apache-2.0 (Qwen/Qwen3-8B-AWQ) (https://huggingface.co/Qwen/Qwen3-8B-AWQ). Needs an NVIDIA driver that supports CUDA 12.9 or newer. Status: not hardware-tested.
We match GPUs on their measured runtime and dedicated VRAM. A hardware report doesn't qualify another vendor's backend.
#llm #chat #openai-api #vllm #awq
Reported on real GPUs
People report these themselves. We haven't checked them.
No reports yet.
Sign in to report how this template runs on your GPU.
Reviews
No reviews yet.
Sign in to leave a review.