NVIDIA L4 cloud instances are built for a specific job: serving small and mid-size models and processing video efficiently, rather than training large models fast. That makes the L4 a natural candidate for inference that should run close to users, where you may want one modest GPU in several regions instead of one big GPU in one place. This article explains what the L4 is good at, how to search for it near your users on Cloudj, and how it compares with consumer RTX cards for the same job.
For the general method of choosing a GPU region, start with the GPU cloud by location guide.
What NVIDIA L4 cloud instances offer
The L4 is a datacenter GPU on NVIDIA's Ada Lovelace architecture with 24 GB class memory. In NVIDIA's lineup it is the low-power, single-slot inference card, with hardware video encode and decode engines (including AV1) that make it popular for video AI pipelines. Use the model page for its listed specifications rather than relying on memory.
Good fits for an L4:
- serving 7B-8B language models, quantized mid-size models, embedding models and rerankers;
- speech recognition and text-to-speech;
- image classification, detection and OCR on streams;
- video pipelines that decode, run a model per frame and re-encode.
Less good fits: large-model training, or image and video generation where raw compute decides how long users wait.
L4 vs RTX 4090 vs RTX 3090 for inference
All three have 24 GB class memory, so the question is not "will it fit" but "how does it serve".
| L4 | RTX 4090 | RTX 3090 | |
|---|---|---|---|
| Class | Datacenter inference card | Consumer, Ada Lovelace | Consumer, Ampere |
| Memory | 24 GB class | 24 GB | 24 GB |
| Power design | Low power, single slot | High power | High power |
| Strength | Efficient steady serving, video engines | Raw speed for generation and heavy models | Same memory at an older generation |
| Account level | Datacenter class; may need a higher level | Available to new accounts | Available to new accounts |
How to read this: if your users wait on long generations (image creation, long chat replies), a 4090 usually finishes each request sooner. If you serve many short requests steadily, or run a video pipeline, an L4 can be the tidier choice. The only reliable answer is a test with your model and your traffic. Per-minute billing means you can deploy each card for an hour and measure. The consumer vs datacenter GPU guide discusses the trade-off more broadly.
On account level: datacenter cards need a higher account level than new accounts have. If the console says "This GPU class is not available at your account level", contact Cloudj support from the console, or start with an RTX card while that is arranged.
Search for an L4 near your users
Location on Cloudj is a country filter over a live list of independent host machines. Open the deploy page with the L4 preselected, deploy an L4, and enter the country code nearest your users. Some examples:
| Users mostly in | Try | Then add |
|---|---|---|
| Southeast Asia, Vietnam | SG | HK, JP |
| Japan, Korea | JP | KR, TW |
| Central and Western Europe | DE | NL, FR |
| North America | US | CA |
If a machine is listed, check its reliability, max duration and bandwidth price, then deploy. If not, add the neighbouring codes, loosen filters, or try the 4090 or 3090 in the same country. Stock changes during the day, so check the console rather than assuming. The details of the filter are in how to filter GPUs by country.
One small GPU per region
Because an L4 is a modest card, some teams serve from several regions at once: one in SG for Asian users, one in DE for Europe, one in US for the Americas. Each region gets a short network path, and no single machine is a single point of failure.
That pattern needs a little plumbing:
- a front end per region, or a global router, that sends each user to the nearest GPU;
- the same model and container image in every region, so behaviour is identical;
- health checks, because every listing is an independent host with no hardware SLA and an instance is not moved if its host goes offline;
- a plan for each machine's "available until" date, when the instance and disk are removed.
For a Singapore-centred setup, the LLM inference in Singapore article covers the serving stack.
Measure what users feel
For interactive inference, the number that matters is time to first token or first result as seen from the user's network, not GPU speed alone. Test from where your users are:
curl -o /dev/null -s -w "connect=%{time_connect} first_byte=%{time_starttransfer} total=%{time_total}\n" https://api.example.com/healthRun it against your front end with the L4 behind it, then with an RTX card behind it, and compare. The real-time AI latency guide explains how to set a latency budget.
FAQ
How much memory does the L4 have?
24 GB class. The model page lists the exact usable figure.
Can a new account rent an L4?
Datacenter GPUs need a higher account level. The console tells you if your level does not allow it; contact Cloudj support from the console.
Is the L4 better than an RTX 4090 for inference?
It depends on the workload. The 4090 is usually quicker per request; the L4 is efficient for steady serving and video. Test both with your own model.
Is there always an L4 in Singapore?
No. Search with SG; if nothing is listed, add nearby codes or check back later.
Find an L4 near your users
Open the L4 deploy page, enter the country code nearest your users and see what is listed. Live "from" prices for the L4 and RTX cards are on GPU cloud.