The GPU server vs VPS for AI question usually starts as either-or: rent a GPU and run everything on it, or stay on a VPS and hope the model is fast enough. In practice, most AI applications are made of parts with very different needs, and only one or two of those parts need a GPU. Splitting them is cheaper, sturdier and easier to move between regions. This article shows which parts go where, how the two connect, and when you can skip the GPU altogether.
For how location fits into these choices, see the GPU cloud by location guide.
What each kind of server is good at
A Cloudj VPS is a stable, always-on Linux or Windows server in one of 14 countries, with hourly or 1, 3, 6 or 12-month billing. It keeps its address and its disk, and it is the natural home for anything that must be reachable all the time.
A Cloudj GPU instance is a container on an independent host machine with 1, 2, 4 or 8 GPUs, billed per minute from your balance. The host has no hardware SLA, each machine has an "available until" date after which the instance and its disk are removed, and if the host goes offline the instance does not move elsewhere. It is excellent for heavy computation and should hold nothing you cannot recreate.
That difference suggests the split.
GPU server vs VPS for AI: which parts need a GPU
| Component | Where it belongs | Why |
|---|---|---|
| Domain, TLS, reverse proxy | VPS | Must stay reachable at a fixed address |
| Web front end, API, authentication | VPS | Light CPU work, always on |
| Database, conversation history, user files | VPS (with backups) | Must survive GPU changes |
| Job queue and scheduler | VPS | Coordinates GPU work, survives restarts |
| Agents and tool calling that use hosted model APIs | VPS | No local model, so no GPU needed |
| Embeddings for a small corpus, rerankers at low volume | VPS often enough | Small models run acceptably on CPU |
| LLM inference beyond small models, image and video generation | GPU instance | Needs GPU memory and compute |
| Fine-tuning and training | GPU instance | Needs GPU, temporary by nature |
| Batch transcription, captioning, large embedding jobs | GPU instance, short-lived | Heavy, then finished |
The rule of thumb: state on the VPS, compute on the GPU. The VPS front end with a GPU backend article walks through a full setup.
How the two connect
The simplest link is an SSH tunnel from the VPS to the GPU instance, so the model server is never exposed to the internet:
ssh -N -L 8000:localhost:8000 -p <port> root@<gpu-host>The host and port come from the GPU instance's Connection tab. Your API on the VPS calls http://localhost:8000 and streams responses back to users. Run the tunnel under systemd or a similar supervisor so it reconnects on its own. When you replace the GPU machine, you change the host and port in one place.
For queued work, the VPS can instead push jobs to the GPU and collect results with rsync, which suits batch processing.
Choosing locations for each part
Put the VPS near your users. Cloudj VPS locations include Singapore for Southeast Asia, Frankfurt in Germany for Europe and eight cities across the United States, among 14 countries in total.
Then put the GPU as close to the VPS as the live inventory allows. GPU location is a country filter: open the GPU deploy page, enter the VPS's country code (for example SG, DE or US), and if nothing suitable is listed, add neighbouring codes. For batch jobs, the GPU can be anywhere, so take the best price. Stock changes during the day; check the console rather than assuming.
When CPU inference on a VPS is enough
You do not always need a GPU. A VPS can serve:
- Small embedding models for a few thousand documents or modest query volume.
- Classic ML such as gradient boosting, logistic regression or small scikit-learn pipelines.
- Tiny quantized language models through CPU runtimes like llama.cpp, for internal tools with a handful of users who can wait a few seconds.
- Agents that call hosted model APIs. The heavy model runs elsewhere; your VPS only orchestrates. The AI agent VPS in the United States article covers this case.
- Light OCR and speech on short files at low volume.
Signs you have outgrown CPU: response times users complain about, CPU pinned at full load, or a model that simply does not fit in RAM at a usable speed. That is the moment to add a GPU instance behind the same VPS, without rebuilding anything else.
If you are weighing a bigger fixed machine instead, VPS vs dedicated server explains that trade-off.
Costs of the split
- The VPS is a steady, predictable cost.
- The GPU runs only when needed, billed per minute; the price shown at deploy is locked for that instance. Stopped instances pay storage only but release the GPU, and destroyed instances cost nothing.
- Bandwidth on the GPU is charged per GB at the machine's listed price, so keep large static files on the VPS side.
Live "from" prices for each GPU model are on GPU cloud.
FAQ
Can I run my whole app on a GPU instance?
You can, but it is risky. GPU hosts have no hardware SLA and machines end on their "available until" date. Keep the app and data on a VPS.
Do I need a GPU for a chatbot?
If it uses a hosted model API, no. If you run an open model of meaningful size yourself, usually yes.
Should the GPU be in the same country as the VPS?
For interactive serving, as close as possible. For batch jobs, it does not matter.
Does Cloudj offer both VPS and GPU instances?
Yes. You can run the VPS front end and rent GPU instances from the same Cloudj account.
Start with the split
Run your API on a VPS near users, then open the GPU deploy page and add a GPU only for the model. Compare GPU options on GPU cloud.