SSH GPU server access: connect to a cloud GPU and check it

Every Cloudj GPU instance is reached the same way: over SSH, with a key. There is no password login, so getting SSH right is the first skill you need. This SSH GPU server guide covers creating and adding a key, finding the host and the port, which is not always 22, what to do in the first few minutes after deploy, how to confirm the GPU with nvidia-smi, and the two habits that save most beginners: tmux and port forwarding.

Step 1: create an SSH key

If you already use SSH keys for servers, you can reuse one. Otherwise, create one on your computer. On macOS, Linux and Windows 10 or 11 (PowerShell includes OpenSSH):

ssh-keygen -t ed25519 -C "my-laptop"

Press Enter to accept the default location and set a passphrase. This creates a private key, which never leaves your computer, and a public key ending in .pub. Setting up SSH keys for a VPS explains keys in more detail; the same key works for GPU instances.

Print the public key so you can copy it:

cat ~/.ssh/id_ed25519.pub

Step 2: add the key and select it at deploy

Add the public key to your Cloudj account in the console. When you deploy a GPU, you must select at least one SSH key; deploy will not proceed without one. If you work in a team, add each person's key and select all of them.

Choosing the machine itself, including its country, is covered in choosing a GPU server country in the console.

Step 3: find the host and port

Once the instance is running, open its Connection tab. It shows:

• the host,
• the port, which is often not 22,
• the full SSH command, ready to copy.

The command looks like this:

ssh -p PORT root@HOST

Always copy the port from the Connection tab rather than assuming 22.

The first few minutes

A newly added key can take up to 3 minutes to be accepted after the instance starts. If you see Permission denied (publickey) right after deploy, wait a moment and try again before you change anything.

Step 4: make connecting easier

Add an entry to ~/.ssh/config on your computer:

Host gpu1
    HostName HOST
    Port PORT
    User root
    IdentityFile ~/.ssh/id_ed25519
    ServerAliveInterval 30

Now ssh gpu1 connects. ServerAliveInterval sends a small keep-alive every 30 seconds, which helps on connections that drop idle sessions. Update the entry when you deploy a new instance.

Step 5: check the GPU

Once connected, confirm what you rented:

nvidia-smi

You should see the GPU model, its memory, the driver version and the highest CUDA version the driver supports. A few more useful checks:

nvidia-smi --query-gpu=name,memory.total,memory.used,utilization.gpu --format=csv
watch -n 2 nvidia-smi
df -h

The first prints a compact summary, the second refreshes every two seconds while a job runs, and the third shows free disk space. Disk size is fixed at deploy, so check it early.

If you use the PyTorch template, confirm PyTorch sees the GPU:

python -c "import torch; print(torch.cuda.is_available(), torch.cuda.get_device_name(0))"

Running PyTorch with the Jupyter template covers that template in detail.

Step 6: keep long jobs alive with tmux

If your SSH connection drops, anything running in that session stops. Run long jobs inside tmux:

tmux new -s train
python train.py

Detach with Ctrl+b then d. Reconnect later and reattach with tmux attach -t train. If tmux is missing, install it with apt-get install -y tmux.

Step 7: reach web tools with port forwarding

Jupyter, ComfyUI, model APIs and dashboards listen on a port on the GPU instance. Do not expose them publicly; forward the port over SSH instead:

ssh -p PORT -L 8188:localhost:8188 root@HOST

Then open http://localhost:8188 in your browser. Change the numbers for other tools, for example -L 8888:localhost:8888 for Jupyter or -L 11434:localhost:11434 for an Ollama API. Running Ollama on a GPU server in Asia uses this pattern.

Moving files

rsync -avP -e "ssh -p PORT" ./data/ root@HOST:/workspace/data/
scp -P PORT root@HOST:/workspace/results.zip .

Note that scp uses a capital -P for the port. rsync -P resumes interrupted transfers, which matters for large models.

Troubleshooting

The instance page also shows status, GPU utilisation and the last lines of the container logs, which help when a container does not start.

FAQ

Can I log in with a password?

No. GPU instances accept SSH keys only.

Why is my port not 22?

The SSH port is assigned per instance and can be any number. Always use the one in the Connection tab.

Can several people use the same instance?

Yes. Add each person's public key to the account and select all of them at deploy.

How do I open Jupyter or ComfyUI?

Forward the port with ssh -L and open it on localhost in your browser.

Next step

Open the GPU deploy page, deploy a small instance with your key selected, and run through the checks above. Live "from" prices for every model are on GPU cloud.