Is the RTX 4090 the cheapest way to serve an LLM?

Per hour, yes — it rents from $0.35/hr. Per token on a model you would actually put in production, no: 24 GB of VRAM cannot hold a 70B-class model at usable quality, so you are comparing an 8B model to a 70B one. If an 8B model passes your quality bar, the 4090 is outstanding value. If it does not, the comparison is meaningless.

How much cheaper is an H100 than an A100 per token?

The H100 rents for more per hour but serves roughly 2.3× the tokens per second on the same model, so it works out to about $0.5174 per million output tokens versus $0.7063 on the A100 at current rental prices. Faster silicon that costs more per hour can still be cheaper per unit of work — that is the whole point of measuring in tokens.

Do these numbers include idle time?

No — every figure here assumes the GPU is busy 100% of the time, which nothing in production is. Multiply by your realistic utilization: at 25% utilization a GPU costs four times these numbers. That correction is usually larger than the difference between any two cards, and it is covered in our self-hosting break-even guide.

H100 vs A100 vs L40S vs RTX 4090: Cost per Token, Not per Hour

Every GPU rental page in existence sorts by dollars per hour. It is the wrong number. You are not buying hours, you are buying tokens — and the card that wins on price per hour routinely loses on price per token, because it is either slower, or too small to hold the model you actually wanted to serve. Here is the same comparison done properly, using live rental prices from our GPU pricing table.

The two rankings disagree

Sorted by rental price, the order runs RTX 4090 → L40S → A100 → H100. Sorted by what it costs to produce a million output tokens, it runs RTX 4090 → H100 → L40S → A100. The H100 is the most expensive card here by the hour and the cheapest 80 GB card by the token — it costs 1.7× the A100 per hour and still comes out ahead, because it does the work faster than it charges for it:

GPU	VRAM	Model it can serve	~Tokens/sec	From $/hr	≈ $/1M output tokens
RTX 4090 (24GB)	24 GB	8B-class open model	220	$0.35	$0.4419
H100 (80GB)	80 GB	70B-class open model	800	$1.49	$0.5174
L40S (48GB)	48 GB	30B-class open model	300	$0.67	$0.6204
A100 (80GB)	80 GB	70B-class open model	350	$0.89	$0.7063

Rental prices are the cheapest public on-demand rate we track per card, checked 2026-08-02. Throughput figures are conservative mid-points for a batched vLLM-class server on a single card — treat the ratios as the signal, not the decimals.

Why the cheap card is not cheap

The RTX 4090 (24GB) is the cheapest thing on the list per hour, and if you compare it to an H100 on a spreadsheet it looks like a steal. The catch is in the VRAM column: 24 GB cannot hold a 70B model at usable quality. It is an 8B–14B card, and an excellent one.

So the honest reading of that table is not "the 4090 costs less per token than the H100." It is "an 8B model on a 4090 costs less per token than a 70B model on an H100" — which is true, and tells you nothing, because those two setups do not do the same job. The only fair comparisons on this page are the ones in the same model class: H100 against A100, both serving a 70B.

H100 vs A100: the one real head-to-head

Same 80 GB, same models, different speed. The A100 rents for $0.89/hr against the H100's $1.49/hr — cheaper by the hour. But the H100 pushes roughly 2.3× the tokens per second, so per million tokens it comes out at $0.5174 against $0.7063.

Rule of thumb: pick the A100 when your GPU will sit idle a lot — you are paying for hours you waste, so cheap hours win. Pick the H100 when you are throughput-bound and the card stays busy — you are paying for work done, so fast work wins. The crossover is roughly the point where the hourly price ratio equals the throughput ratio.

Now compare it to just paying an API

Take the production-relevant number rather than the flattering one: a 70B-class model on an H100 at $0.5174/1M output tokens — and remember that assumes 100% utilization, which does not exist. Against live API pricing:

A frontier API model (OpenAI GPT-5.6 Sol) charges $10.00/1M output. Self-hosting wins on raw price by a wide margin — if you can match the quality with an open model, which is the entire question.
The cheapest API model we track (Mistral Mistral Nemo) charges $0.03/1M output — already cheaper than your best-case self-hosted number, before you have configured anything. It is a much smaller model, but so is anything you were going to fit on one card.
Drop that H100 to a realistic 25% utilization and it costs $2.07/1M. That single correction reorders the whole comparison, and it is why most teams who run this math end up staying on an API.

The full version of that argument — redundancy, engineering time, the utilization trap — is in self-hosting vs API: the real break-even math. Model your own traffic in the API cost calculator, then price the GPU side on the rental comparison.

How to choose, in four lines

Decide the model first, not the GPU. The model's VRAM requirement eliminates most of the list before price is even relevant.
Then pick on utilization. Bursty and idle-heavy → cheaper hours (A100, L40S). Saturated and throughput-bound → faster silicon (H100).
Price the marketplace against the datacenter. Peer marketplaces are the cheapest rates on our table; managed clouds cost more and interrupt less. For checkpointed training and batch jobs the cheap route is fine; for user-facing inference, pay for reliability.
Recompute quarterly. Rental prices and API prices both fall, at different speeds. A decision that was correct two quarters ago is not evidence about this one.