GPU server Singapore for AI inference: a setup guide

This is a hands-on guide for teams who need a GPU server Singapore for AI inference: an LLM API that answers users across Southeast Asia without a long round trip to another continent. We will search for a GPU in Singapore on Cloudj, size the VRAM, install an inference server ourselves, and put a Singapore VPS with Nginx and TLS in front so the public endpoint stays stable even when the GPU machine changes.

For the broader picture of Singapore GPUs, see the GPU server Singapore article.

The architecture in one paragraph

Users call https://api.example.com. That domain points to a small Cloudj VPS in Singapore running Nginx with a TLS certificate. Nginx forwards requests through an SSH tunnel to the GPU instance, where vLLM serves the model on a local port. The GPU never exposes the model port to the internet. If the GPU host goes away, you deploy a new one, restart the tunnel, and users notice nothing. The VPS front end with a GPU backend article explains why this split is worth it.

Step 1: size the VRAM

Pick the GPU by the model you plan to serve. As general guidance for a single card:

Leave headroom. Concurrency and context length grow the KV cache, and running out of memory under load is worse than a slightly larger card. L40S and RTX A6000 are datacenter and workstation cards that need a higher account level; the console tells you if one is not available to you.

Step 2: search Singapore and neighbours

GPU location is a country filter over a live list of host machines. Open the RTX 4090 deploy page and set the Country field to SG. Each result shows the country (no city), reliability, max duration, download speed, bandwidth price and a Datacenter badge where relevant.

If a machine is listed, check two things before deploying: the "available until" date should be well beyond your planned service period, and reliability should be high, since this is a user-facing service. If nothing fits, stock changes during the day; widen the search to nearby countries:

SG, MY, TH

Then test latency from your users' networks before going live. For a service, also set a minimum reliability. Live "from" prices per model are on GPU cloud.

Step 3: install the inference server

Deploy with the Ubuntu + CUDA template and an SSH key. None of the console templates comes with vLLM preinstalled; you install it yourself. Connect using the host and port from the Connection tab:

ssh -p <port> root@<host>
nvidia-smi
python3 -m venv /opt/vllm && . /opt/vllm/bin/activate
pip install vllm

Start the OpenAI-compatible server bound to localhost only, with an API key:

vllm serve Qwen/Qwen2.5-7B-Instruct \
  --host 127.0.0.1 --port 8000 \
  --api-key "$VLLM_API_KEY" \
  --max-model-len 8192

Run it under tmux or a process manager so it survives your SSH session. The vLLM documentation covers quantized models, tuning and supported architectures. Ollama is a simpler alternative for small teams; the same front-end pattern applies. For a longer walkthrough of vLLM itself, see the vLLM inference server article.

Step 4: tunnel from the VPS to the GPU

On a Singapore VPS, create an SSH key, add its public key to the GPU instance's ~/.ssh/authorized_keys, then open a persistent tunnel:

ssh -N -o ServerAliveInterval=30 -o ExitOnForwardFailure=yes \
  -L 127.0.0.1:8000:127.0.0.1:8000 -p <port> root@<host>

Wrap it in a systemd service (or autossh) so it restarts on failure. Now http://127.0.0.1:8000 on the VPS reaches vLLM on the GPU, and nothing on the GPU is publicly exposed beyond SSH.

Step 5: Nginx and TLS on the VPS

A minimal server block:

server {
    server_name api.example.com;
    location / {
        proxy_pass http://127.0.0.1:8000;
        proxy_http_version 1.1;
        proxy_buffering off;
        proxy_read_timeout 300s;
    }
}

proxy_buffering off lets streamed tokens reach the client as they are generated. Issue a certificate with certbot, add rate limiting, and keep the API key check. The Nginx reverse proxy with SSL guide covers the full setup.

Step 6: health checks and failover by hand

vLLM exposes a /health endpoint. From the VPS, poll it every minute:

curl -fsS http://127.0.0.1:8000/health || echo "GPU backend down"

Alert on failure. GPU machines are independent hosts with no hardware SLA, and there is no automatic move if one goes offline. Keep a deploy script that installs vLLM and pulls the model, so that you can bring up a replacement in SG or a neighbour and repoint the tunnel quickly. Return a friendly error from Nginx while the backend is down.

FAQ

Does Cloudj provide vLLM preinstalled?

No. You install vLLM or Ollama yourself on the Ubuntu + CUDA template, or build a custom public image if your account level allows.

Is a Singapore GPU always available?

No. Stock changes during the day. Search SG; if nothing fits, add MY and TH and test latency.

Why not expose vLLM directly on the GPU?

The GPU machine can change. A VPS gives you a stable domain, TLS and access control, and keeps the model port private.

Which GPU should I start with?

For 7B to 14B models, an RTX 4090 is a common starting point. Move to a 48 GB class card for larger models or heavy concurrency.

Start building

Open the RTX 4090 deploy page, set the Country to SG and see what is listed now. Compare live prices on GPU cloud before you deploy.