AI Startup Gross Margin Benchmarks and the Compute Ceiling

What is a good gross margin for an AI startup in 2026?

Published figures put a typical AI product between 45 and 60 percent gross margin, roughly 20 to 35 points below the 80 percent software median for SaaS, and what decides where your own company sits is the share of revenue its compute takes.

ICONIQ’s State of AI 2026, published in July 2026 from a second-quarter survey of about 300 executives at software companies building AI products, reports an average gross margin of 45 percent on AI products in 2025, and its respondents project 53 percent for 2026 and 59 percent for 2027. Bessemer Venture Partners’ State of AI 2025, published in August 2025, split its AI startups into two cohorts: Shooting Stars, the capital-efficient growers, at about 60 percent, and Supernovas, the ten fastest-scaling companies it surveyed, at about 25 percent and often negative.

The SaaS reference point is the 2026 SaaS & AI Performance Benchmarks published by Aleph and Benchmarkit on 1 June 2026: a median software gross margin of 80 percent across 342 B2B SaaS and AI-native companies on 2025 actuals. It is a benchmark for software, not for AI products, and it is the number an AI founder is most often measured against by mistake.

ICONIQ and Bessemer do not say what their respondents put in cost of revenue, and Aleph and Benchmarkit define it for software, with hosting, support and third-party APIs, but publish no AI cut. A 55 percent margin can be at the market, or it can be a 45 percent business with part of its compute in research and development. The number that tells the two apart is not on this list: it is the share of revenue that compute takes, measured on a definition you can reproduce.

Which figure to compare yours to is our judgment, not a published rule. For a company between $1M and $7M ARR, the ICONIQ average of 45 percent for 2025 is the nearest peer figure, because it covers the broadest population of companies building AI products. Bessemer’s Shooting Stars, at about 60 percent, are a selected group of the best performers: a target, not a peer group. And the 80 percent SaaS median applies only to the part of your revenue that runs no inference. Read your margin net of credits and at list price against the same figure.

Why are AI gross margins lower than SaaS?

Because an AI product pays for compute on every request a customer makes, while a SaaS product serves additional usage at almost no marginal cost. A SaaS company’s hosting bill grows slowly and mostly in steps; an AI company’s inference bill grows with every prompt, every generated output and every heavy user. The 20 to 35 point gap in the published figures is, in our reading, mostly that one line.

AI gross margin benchmarks by source and cohort

Published AI gross margin benchmarks range from about 25 to 60 percent because each one measures a different population with a different statistic.

One reading matters more than any single row: the ICONIQ figures for 2026 and 2027 are what builders expect, not what they achieved, and a projection should not be compared to a result.

How much of revenue does compute take at an AI startup?

In our 9 engagements with AI companies, inference and serving compute ran at 19 to 38 percent of revenue, net of cloud credits and with training excluded. The 9 are AI companies between $1M and $7M ARR whose books we took over between September 2025 and August 2026, from an in-house team or an outside provider.

Compute means production inference and serving: model API calls behind the product, GPU or cloud instances answering customer requests, inference endpoints. Training, fine-tuning, evaluation and research runs are excluded. Cloud credits are deducted, so the figure is what the company actually paid. Revenue is total revenue over the same period.

On that definition, one line of cost takes between a fifth and nearly two fifths of every dollar of revenue, before hosting the rest of the product, before support, before third-party data or implementation. For a company still drawing on cloud credits, the range is a floor: the same compute at list price takes a larger share. If you do not know your own ratio, measuring it is the first thing our finance work for AI companies does.

Is inference 23 percent of revenue at AI companies?

No: the 23 percent figure from ICONIQ’s January 2026 State of AI snapshot is model inference’s share of total AI product cost at scaling stage, up from 20 percent at pre-launch, and not a share of revenue.

Cost and revenue are different denominators. A company whose total AI product cost roughly equals its revenue will see the two shares land close together. A company still spending well ahead of revenue will see compute take a larger share of revenue than of cost, and a founder who reads 23 percent as a normal share of revenue will think a 30 percent ratio is a problem when it may be ordinary, or think 23 is safe when it is not.

The question that decides your margin is the revenue one. A cost share tells you where your spending goes; only a revenue share tells you how much gross margin is left.

What your compute share does to your gross margin ceiling

Gross margin can never exceed 100 percent minus compute’s share of revenue, so a company whose inference and serving compute runs at 19 percent of revenue is capped at 81 percent before any other cost of revenue, and one at 38 percent is capped at 62.

Everything else in cost of revenue comes off that ceiling: hosting for the rest of the product, third-party data, burdened support, implementation. With the 8 points of hosting and support used in the example below, a company at the high end of our range lands near 54 percent, close to the ICONIQ projection for 2026, and a company at the low end lands near 73 percent, well above every published AI figure. Which of the two you are is decided mostly by the compute line.

Ben Murray of The SaaS CFO, in a draft framework published in May 2026, flags an AI-native company whose AI revenue is less than 3 times its inference cost, which is compute above about 33 percent of revenue when all revenue comes from the AI product, sets 4 times as the floor to aim for, and treats 5 times or more as healthy. For the AI tier of a SaaS product his thresholds are higher: a warning below 5 times, a floor of 8 and healthy at 10. On his AI-native thresholds, the example below sits under the floor at list price, with revenue at 3.3 times compute, and the high end of our range, at 2.6 times, sits in his warning zone.

Why an AI startup’s reported gross margin can overstate the real one

In the AI companies whose books we took over, all 9 carried direct costs outside cost of revenue, and correcting the classification moved gross margin by 6 to 13 percentage points. The comparison with SaaS companies and the full population behind that figure are on our SaaS gross margin benchmarks page.

Two other mechanisms widen the gap between a reported AI margin and the real one, and both are judgment from client work rather than measured counts. Cloud credits make compute look cheaper than it will be: the reported margin is accurate this month and wrong as a forecast, because the subsidy ends on a date the company knows in advance. And training mixed with inference distorts in both directions: a fine-tuning run left in cost of revenue understates the margin for that month, while inference left in research and development overstates it every month.

Getting compute onto the right line, every month, is accounting work rather than strategy, which is why it belongs to controller work for AI and usage-based companies. A margin built on misplaced compute is not a conservative number or an aggressive one. It is a number about a different company.

How to calculate compute as a percentage of revenue

Four steps produce two ratios: the one comparable to our range, and the one an investor will rebuild.

1. Isolate production inference and serving compute for the last closed month. Model API invoices, GPU or cloud instances serving customers, inference endpoints, wherever they sit in the ledger today.
2. Remove training, fine-tuning, evaluation and research runs, even when they sit on the same cloud invoice. Tag them by project or account so the split repeats every month without a spreadsheet.
3. Record the cloud credits applied to that production compute in the month. The amount paid is the net figure; the amount before credits is the gross figure.
4. Divide each by total revenue for the month. Compare the net ratio to the 19 to 38 percent range above; a single month can swing by several points, so use a three-month average if your usage is uneven. The gross ratio is your margin after the credits end, and the number a careful investor will put in the model.

Example: a $300,000 month. A company recognises $300,000 of revenue in a month, all from its AI product. Production inference and serving cost $90,000 at list price, of which cloud credits covered $18,000, so it paid $72,000. A $25,000 fine-tuning run on the same cloud account is excluded.

The net ratio is $72,000 on $300,000, or 24 percent, inside our range. The gross ratio is $90,000 on $300,000, or 30 percent. The ceilings are 76 and 70 percent, and revenue covers compute 4.2 times net of credits and 3.3 times at list price.

The rest of cost of revenue is $9,000 of hosting for the rest of the product and $15,000 of burdened support. Gross margin is $204,000 on $300,000, or 68 percent, net of credits, and $186,000 on $300,000, or 62 percent, at list price. Both sit above every published AI average in the table, ICONIQ’s 45 and 53 and Bessemer’s 60. The 6 points between them are exactly what the credits are worth this month, $18,000 on $300,000, and the company should plan as if it were the 62 percent business it will be.

The two checks that tell you the numbers are comparable. First: is every cost of serving customers in cost of revenue, meaning production compute, hosting, third-party APIs and data, and customer support, or does some of it still sit in research and development or general and administrative expenses? Second: can you produce last month’s production inference and serving compute from the ledger alone, separately from training, with the credits applied to it? If either answer is no, your ratio is not comparable to anyone’s, and neither is the gross margin built on it.

What moves an AI startup’s gross margin

Pricing that charges for the usage that drives inference moves AI gross margin fastest, because it makes revenue rise with compute instead of leaving compute to grow under a flat fee.

After pricing, three levers move compute’s share by points: routing simple requests to smaller and cheaper models, caching repeated requests and embeddings, and moving predictable load to committed capacity. Each lowers the numerator of the ratio above without touching the product. How to model compute per unit of usage, and which lever to pull first, is the subject of our AI compute cost guide. The order of these levers is our judgment from client work.

What does not move gross margin, despite appearing on most lists: cutting research compute and freezing engineering hiring. Both are operating expenses. They extend runway, which matters, but they leave the margin exactly where it was.

When your AI gross margin changes a decision

A compute ratio measured on a clean definition changes three decisions that are usually made on an unmeasured one.

Pricing. A flat seat price on a product whose cost rises with every request loses margin on exactly the customers who use it most. If your compute ratio is at the top of our range, the pricing model is the first conversation to have, which is the work of fractional CFO support for AI companies.

Fundraising. An investor will rebuild your margin at list price and with training taken out. A company that presents 68 percent and is rebuilt at 62 has not lost 6 points of margin, it has lost the reader’s confidence in every other figure in the model.

Runway. Credits ending raise cash burn by the full amount they were covering: $18,000 a month in the example above, from a date you already know. Put that date into the plan now, and use the SaaS runway calculator to see what it does to the number of months.

How Offset Partners works on AI gross margin

We work only with SaaS and AI companies, so separating inference from training, putting compute on the right line and tracking credits are routine monthly work here rather than a clean-up discovered during diligence.

The diagnostic starts by measuring your compute share on the definition on this page, net and gross, and placing your margin against the published figures that actually apply to you. We publish the range we measure in our own AI engagements so you can check your number against ours before you pay anyone. And if your ledger already produces both ratios and they hold up when rebuilt, we will tell you there is nothing here for us to fix.

Sources

External figures on this page, with the date each was published.

• AI product gross margin averages of 45 percent in 2025, 53 percent projected for 2026 and 59 percent projected for 2027: ICONIQ, 2026 State of AI: The Builder’s Economy, published July 2026. Population: a Q2 2026 survey of about 300 executives at software companies building AI products, 87 percent in North America, revenue from $1M to above $1B.
• Model inference at 20 percent of total AI product cost at pre-launch and 23 percent at scaling stage: ICONIQ, 2026 State of AI Bi-Annual Snapshot, January 2026. The chart’s population is the 202 respondents to ICONIQ’s December 2025 survey. The chart, labelled as a share of total AI product cost, is reproduced in Vista Equity Partners, Understanding Inference, May 2026.
• Shooting Stars at about 60 percent and Supernovas at about 25 percent gross margin: Bessemer Venture Partners, The State of AI 2025, published 13 August 2025. Population: 20 high-growth AI startups from Bessemer’s portfolio and beyond, of which 10 Supernovas.
• Median software gross margin of 80 percent: 2026 SaaS & AI Performance Benchmarks, Aleph and Benchmarkit, published 1 June 2026. Population: 342 B2B SaaS and AI-native companies on full-year 2025 actuals, with gross margin figures drawn from 228 to 232 of them.
• Draft thresholds for the ratio of AI revenue to inference cost: Ben Murray, How to Calculate the Inference Efficiency Ratio, The SaaS CFO, published 11 May 2026. The author presents the thresholds as a draft.
• The worked example is illustrative, chosen to show the method, not a client.

The compute share, 19 to 38 percent of revenue in 9 engagements with AI companies, is Offset Partners’ own data. Definition: production inference and serving compute, training and research excluded, net of cloud credits, divided by total revenue. Population: AI companies between $1M and $7M ARR whose books we took over between September 2025 and August 2026, from an in-house team or an outside provider. The 6 to 13 point classification correction is detailed on our SaaS gross margin benchmarks page.

Related resources

• AI compute costs: a finance guide for founders
• SaaS gross margin benchmarks
• SaaS runway calculator
• Burn multiple explained
• SaaS finance readiness checklist

Related services

• AI company finance support
• SaaS controller services
• Fractional CFO services
• Book a SaaS finance diagnostic