Run an open model as your own API, then check how fast it answers for your users. GridPup deploys the model on the cheapest cloud GPU that fits it, behind an https address with API keys, and bills by the hour. Its latency test times any LLM endpoint, yours or a provider's, from three regions.
Popular LLMs and the cheapest machine for each
GPU memory each model needs to serve with vLLM, and the cheapest machine that fits it right now. Open a model on GridPup to deploy it or see every GPU that fits.
| Model | GPU memory needed | From /h |
|---|---|---|
| Llama-3.1-8B-Instruct meta-llama · licence approval needed | 21 GB | $0.207 |
| Qwen3-8B Qwen | 22 GB | $0.207 |
| gpt-oss-20b openai | 18 GB | $0.175 |
| Qwen3-14B Qwen | 37 GB | $0.335 |
| Qwen3-32B Qwen | 77 GB | $0.975 |
| gpt-oss-120b openai | 75 GB | $0.975 |
| Mistral-7B-Instruct-v0.2 mistralai | 20 GB | $0.175 |
| DeepSeek-V3.2 deepseek-ai | 784 GB | $83.44 |
Speech to text
Whisper models answer OpenAI-style /audio/transcriptions requests once deployed.
| Model | GPU memory needed | From /h |
|---|---|---|
| whisper-large-v3-turbo openai | 3.9 GB | $0.071 |
| whisper-large-v3 openai | 13 GB | $0.130 |
| distil-large-v3 distil-whisper | 7.4 GB | $0.071 |
Machines and prices from GridPup, updated through the day. More models on open models and their GPU needs.
How deploying works
- Pick a model. Choose one from Hugging Face or bring your own. GridPup shows the GPU memory it needs and every GPU that fits.
- Pick the machine. The cheapest GPU that fits is chosen for you, or pick another. You see the price per hour first.
- Get your API. GridPup starts the machine, loads the model and gives you an https address and API keys.
- Stop when done. You pay by the hour while it runs; stop it any time.
Endpoint latency test
Already serving a model, or choosing between providers? The test sends real requests from AWS in US East, Frankfurt and Mumbai and measures what people there see.
Reaching it
DNS, connecting and TLS from each region: how far your endpoint is from your users.
First token
From the request to the first token of the answer, and the first answer token after a model's thinking.
Speed and load
Tokens per second for each user and in all, with 1, 4, 16 and 32 users at once: where it slows and where rate limits start.
Voice and speech
Time to the first audio for voice APIs; for speech to text, time to the first words and the word error rate.
Works with OpenAI, Anthropic, Gemini, Groq, Together, Ollama, vLLM, TGI, Triton and your own JSON, WebSocket or gRPC API. Start a test on GridPup.
Before you deploy
Benchmark the model first
Tokens per second and users per GPU on H100, A100, RTX 5090 and more, so you rent the right GPU.
Fine-tune it
Train or fine-tune your model on a cloud GPU from the browser, then deploy the result.
Try hosted models first
Ask GPT, Claude, Gemini and Grok the same question side by side in the AI Playground, pay per answer.
Compare GPU prices
Live prices per GPU-hour for H100, A100, L40S, RTX 4090 and 70+ other GPUs across clouds.
Frequently asked questions
How much does it cost to host an LLM as an API?
You pay for the GPU machine by the hour while the API runs. Small models such as Qwen3-8B or Llama 3.1 8B run on machines that cost well under a dollar an hour; the table on this page shows the cheapest machine for popular models right now. Large models such as DeepSeek V3 need eight data center GPUs.
Is the API compatible with OpenAI's?
Chat models are served with vLLM, which offers OpenAI-style /v1/chat/completions, so tools and SDKs written for OpenAI work by changing the base URL and key. Speech models such as Whisper answer OpenAI-style /audio/transcriptions requests.
How do I measure my LLM API's latency?
GridPup's endpoint latency test sends real requests to your endpoint from AWS regions in the US, Europe and India and reports DNS, connection and TLS time, time to first token, tokens per second, and how these change with 1, 4, 16 and 32 users at once.
Which endpoints can the latency test check?
Your own vLLM, TGI, Ollama or Triton server (a raw IP and port are fine) or an API such as OpenAI, Anthropic, Gemini, Groq, Together or Deepgram, over HTTP(S), WebSocket or gRPC. To test your own server you show it is yours with a token file or DNS record.
Which models can I deploy?
Thousands of open models from Hugging Face: chat models (Llama, Qwen, Gemma, Mistral, gpt-oss, DeepSeek), vision models, speech to text (Whisper), image generation (Stable Diffusion, FLUX) and embeddings. Licence-gated models such as Llama need approval on Hugging Face first.