How to Calculate & Forecast LLM Costs: Formulas, Token Estimators, and Unit Economics

Overview #

A financial engineering guide for AI software teams: mathematically model token consumption, estimate prompt-to-completion ratios, calculate blended gross margins, and project monthly API infrastructure budgets.

The Core AI Cost Formula #

Monthly Cost = Sum over all requests i [ (Prompt_Tokens_i * Input_Price_per_Token) + (Completion_Tokens_i * Output_Price_per_Token) + (Cached_Tokens_i * Cache_Price_per_Token) ].
Because output tokens are priced 3x to 5x higher than input tokens across every major provider, minimizing generated output length has an outsized effect on total expenditure.

Typical Token Ratios by Application Type #

• Customer Chatbot: 800 prompt tokens (history + system prompt) to 150 completion tokens (5.3:1 ratio).
• Code Generation / Autocomplete: 3,000 prompt tokens (surrounding files) to 80 completion tokens (37:1 ratio).
• Document Summarization: 15,000 prompt tokens (raw text) to 500 completion tokens (30:1 ratio).
• Autonomous Agent / Code Refactor: 8,000 prompt tokens to 2,000 completion tokens (4:1 ratio).

Modeling Unit Economics and User Gross Margins #

If a SaaS product charges $29/user/month and the average user conducts 40 queries per day (1,200 queries/month), average cost per query must remain under $0.015 to maintain an 80%+ gross margin. This mathematical constraint dictates model tier selection (e.g. routing 90% of traffic to lightweight models).

Code Example: Cost Estimator and Unit Economic Calculator #

def calculate_monthly_ai_cost(
    daily_active_users: int,
    queries_per_user_day: int,
    avg_input_tokens: int,
    avg_output_tokens: int,
    input_price_per_m: float, # e.g. $3.00 for Claude 3.5 Sonnet
    output_price_per_m: float # e.g. $15.00
):
    total_monthly_queries = daily_active_users * queries_per_user_day * 30
    total_input_tokens = total_monthly_queries * avg_input_tokens
    total_output_tokens = total_monthly_queries * avg_output_tokens

    input_cost = (total_input_tokens / 1_000_000) * input_price_per_m
    output_cost = (total_output_tokens / 1_000_000) * output_price_per_m
    total_cost = input_cost + output_cost

    return {
        "monthly_queries": total_monthly_queries,
        "input_cost": round(input_cost, 2),
        "output_cost": round(output_cost, 2),
        "total_cost": round(total_cost, 2),
        "cost_per_user": round(total_cost / daily_active_users, 2)
    }

print(calculate_monthly_ai_cost(1000, 20, 1200, 250, 3.0, 15.0))
# Prints detailed monthly expense projection

Frequently Asked Questions #

Q: How many words is 1,000 tokens?

In English, 1,000 tokens is approximately 750 words (about 3/4 of a word per token). For non-English languages or source code, token counts per word are higher.

Q: Why are output tokens so much more expensive than input tokens?

Input tokens are processed in a single parallel matrix multiplication (compute-efficient), while output tokens are generated sequentially one by one, requiring continuous memory bandwidth access.

Q: How does prompt caching affect cost forecasting?

Prompt caching reduces recurring input token costs by 50% to 90%, significantly altering monthly projections for multi-turn chat applications.