FinOps for AI: Managing Token Spend Like Cloud Infrastructure

AI can make a feature more useful, but it can also make its costs harder to predict. Unlike a fixed software license, AI spending can change with usage, the number of model calls, the amount of information processed, and the models involved. A feature that costs very little at launch can become significantly more expensive as more customers use it and the underlying workflow becomes more complex.

An AI feature may start with a single model call and a short prompt. Over time, the same feature might make several calls for a single user request, carry more information between steps, or use a second model for tasks the first one cannot handle well. If these changes are not tracked, the increase in spending can be difficult to explain when the bill finally arrives. That is why AI costs need more than a monthly review of the final invoice. Teams need to understand what is driving the spend, which products and workflows are responsible for it, and whether the cost is justified by the value the AI delivers.

Cloud teams have dealt with a similar problem for years. As cloud usage grew, businesses learned that infrastructure costs need regular tracking, clear ownership, and coordination between finance, engineering, and business teams. This led to FinOps, a practice designed to help organisations manage cloud spending while still getting the value they need from their infrastructure.

AI FinOps extends this approach to AI workloads. It brings the same discipline to model calls, tokens, inference, data processing, and other costs connected to running AI applications. The goal is not simply to reduce the AI bill. It is to understand where the money is going, why it is being spent, and whether the results justify the cost.

This has quickly become a priority for businesses. According to the FinOps Foundation’s State of FinOps 2026 research on AI budget ownership, 98% of surveyed organisations now actively manage AI spending, compared with 31% two years earlier. The shift is also changing the question businesses need to ask. Instead of looking only at the price of a model call or the cost of a token, teams need to understand the full cost of running AI across the business.

Watch this video to see why one AI request can trigger multiple model calls, how that drives up costs, and where FinOps brings visibility and control.

{% video_player “embed_player” overrideable=False, type=’hsvideo2′, hide_playlist=True, viral_sharing=False, embed_button=False, autoplay=False, hidden_controls=False, loop=False, muted=False, full_width=False, width=’1920′, height=’1080′, player_id=’398531236577′, style=” %}

Why AI Spending Is Different From Traditional Cloud Costs

Cloud costs are generally easier to follow. Teams can see what they are paying for and how changes in compute, storage, or data transfer affect the bill. AI makes the picture less straightforward. The cost of an AI feature depends on the model used, how often it is called, how much information is sent, how much it generates, and which other services are involved. The FinOps Foundation also points to factors such as token usage, changing model pricing, and the growing number of AI services involved in a typical application.

Take customer support as an example. Two requests may look almost identical to the customer, but their costs can be very different. One might need a quick response from a smaller model. Another could involve several model calls, customer information, a search through internal documents, and a more capable model before the final response is ready.

AI agents make this harder to see. What looks like one simple action to the customer may actually involve several steps in the background. If those steps aren’t tracked, teams may know that AI spending has increased without knowing what is actually causing it. So, instead of looking only at the overall AI bill, teams need to ask questions such as:

• Which applications or workflows are driving the most spend?
• Which models are being used for different tasks?
• How much does a customer request or business task actually cost?
• Are additional model calls improving the outcome enough to justify their cost?

The goal isn’t simply to spend less on AI. It’s to understand where the money is going and whether that spending is creating enough value. That makes AI cost management less about watching a monthly bill and more about understanding the economics of the workflows running behind it.

Five Metrics To Track To Gain Visibility Into Token Usage

Understanding where AI costs come from is only the first step. The harder part is getting enough visibility to understand which workloads are costing the most, why they cost that much, and whether the spend is justified by the result.

A monthly AI bill can tell you how much was spent. It usually cannot tell you what caused the increase. So here are a few practical metrics that become useful:

1. Cost per token

Cost per token gives the team a simple way to compare model usage across different applications and providers. It looks at how much was spent against the number of input and output tokens processed. It is useful for understanding model usage, but it should not become the main measure of AI efficiency. Two workflows can use a similar number of tokens and still deliver very different business results.

Deloitte makes a similar point in its discussion of AI spending, noting that traditional technology cost models need to evolve as tokens become an important unit of AI consumption. Its guidance also highlights the importance of real-time monitoring and forecasting as AI usage grows.

2. Cost per request

Cost per request gives the team a more practical view of what an application is actually spending.

For example, if a customer support application handles 100,000 requests in a month, knowing the average cost of those requests makes it easier to understand how changes to the model, prompt, or workflow affect the overall bill.

It can also help identify sudden changes. If the average cost of a request increases after a new feature is introduced, the team has a clear starting point for investigating what changed.

3. Cost per workflow or outcome

For applications that use agents, cost per request may still leave out an important part of the picture. One user request can trigger several model calls, searches, tool calls, and retries before the task is finished. In that situation, it makes more sense to measure the cost of completing the entire workflow.

Depending on the application, this could mean tracking:

• Cost per customer issue resolved
• Cost per document processed
• Cost per insurance claim reviewed
• Cost per qualified lead generated
• Cost per completed workflow

This changes the conversation from how much AI is being used to what the business is getting from that usage.

4. Spend by model, application, team, and use case

A single company-wide AI bill does not tell you enough. Suppose the monthly spend has increased by 35%. Without proper attribution, it is difficult to know whether the increase came from a new customer-facing application, higher usage of an existing system, a particular team, or a change in model usage.

Breaking spend down by model, application, team, and use case gives finance and engineering a much clearer picture of what is happening. This becomes especially important when an organisation uses several model providers or has AI applications running across different cloud environments.

5. Token usage by input versus output

Input and output tokens should also be monitored separately because these can point to different problems. If input token usage is increasing, the application may be sending longer prompts, more conversation history, or too much retrieved information to the model. If output token usage is increasing, the model may be producing longer responses than the application actually needs.

Looking at both numbers separately makes it easier to identify where the extra usage is coming from and what needs to change.

The goal is not to create another set of dashboards for finance and engineering teams to check every morning. The goal is to make AI spending traceable. When costs change, the business should be able to identify the application, model, workflow, or usage pattern behind that change and decide whether the additional spend is justified.

Five Practical Ways to Control AI Spend

Once a business can see where its AI money is going, the next step is to decide where that spending can be controlled without hurting the quality of the application.

These five practices tend to give teams the most control over AI spend without requiring a complete rebuild of what they already have.

1. Route workloads to the right model

Using the same model for every task is convenient, but it is rarely the most sensible approach from a cost perspective.

Consider a customer service application. Checking an order status or answering a common question may not require the same model used to handle a complicated complaint involving several policies and a long customer history. If both requests are automatically sent to the most expensive model, the business is paying for capabilities that the simpler request does not need.

A better approach is to set clear rules for which types of work go to which models. High-volume and straightforward tasks can use a lower-cost model, while more difficult cases can be moved to a stronger model when the situation calls for it.

McKinsey’s 2026 research on enterprise AI spending identifies model selection and routing as important areas for reducing unnecessary AI consumption. Its research also notes that organizations can route workloads based on factors such as cost, quality, latency, risk, and availability rather than treating one model as the default for everything.

2. Control token consumption at the application layer

A large part of AI spending is determined before the model even starts working. The way an application handles prompts, context, retrieval, and conversation history can have a direct impact on how many tokens each request uses. For a deeper look at these application-level decisions, see Daffodil’s guide to building token-efficient AI architectures.

An application may send the entire conversation history with every request, include documents that are not relevant to the current question, repeat the same instructions across multiple calls, or allow the model to generate much longer answers than the user needs.

Take an internal HR assistant as an example. If an employee asks about the company’s parental leave policy, there is little reason to send the model the entire HR handbook. Retrieving the relevant policy section first can reduce the amount of information the model has to process.

The same principle applies to long conversations. If an agent has already processed 30 previous messages, sending all 30 back to the model every time can quickly increase usage. Summarizing older parts of the conversation and keeping only the information needed for the current task can reduce that overhead.

3. Set budgets, limits, and usage policies

AI applications need boundaries, particularly when they can make repeated calls without a person being involved.

Imagine an AI agent that is supposed to search a supplier database and prepare a purchasing recommendation. If an API failure causes the agent to retry the same action repeatedly, a small technical problem could turn into hundreds of unnecessary model and tool calls.

A sensible setup could include:

• A monthly budget for each AI application
• Spending limits for individual teams or projects
• Approved models for different types of work
• Alerts when usage moves well above the normal range
• Limits on agent retries and repeated tool calls
• Maximum execution time or steps for longer-running workflows

These controls do not need to prevent experimentation. They simply give teams a way to catch problems before they become expensive. For example, if a new internal AI assistant suddenly uses three times more tokens than expected after a prompt change, an alert can bring the issue to the team’s attention before the next billing cycle.

4. Track anomalies, not just monthly totals

AI applications need boundaries, particularly when they can make repeated calls without a person being involved. Setting clear budgets, spending limits, and alerts can help teams catch problems before they become expensive. This follows the same basic thinking behind FinOps-based cloud cost optimization, where usage patterns and cost anomalies are monitored continuously rather than reviewed only after a large bill arrives.

Suppose an AI claims-processing system normally completes a case in eight model calls. If that number suddenly rises to 20, something may have changed. Perhaps a tool is failing, an agent is repeating a step, or a new prompt is causing unnecessary additional work.

These changes can be monitored across:

• Token usage
• Cost per request
• Model usage
• Number of model calls
• Number of retries
• Workflow completion time
• Cost per completed business task

This gives engineering teams a chance to investigate the cause while the problem is still small, instead of discovering it weeks later through an unexpectedly high invoice.

5. Connect AI spend to business value

Cost reduction should not become the only goal. Consider two customer support systems. The first costs $0.05 per interaction but resolves only 60% of customer issues without human help. The second costs $0.10 per interaction but resolves 90% of them.

Looking only at the AI invoice makes the first system appear cheaper. Looking at the entire support operation may tell a different story because the second system could reduce the amount of work passed to human agents and shorten the time needed to resolve customer problems. The same thinking applies to other AI applications. A document-processing system may cost more per document but save employees several hours of manual work. An AI sales assistant may use a more capable model but generate better-qualified leads. A claims system may cost more per case but reduce processing time and improve consistency.

This is why AI FinOps should connect cost with the outcome the business actually cares about.

What Actually Drives AI Costs?

Before you can control AI spending, you need to know what is creating the bill in the first place. The model invoice is usually the most visible part, but it is rarely the whole story. Costs can build up across the model, the way it is used, the infrastructure behind it, and the number of steps involved in completing a task.

Here are the areas worth watching most closely:

1. Token consumption

Tokens are one of the most direct drivers of AI costs. The model processes the information sent in the prompt and generates an answer, and both sides can contribute to the bill depending on the pricing model. The amount of text involved can vary considerably between requests. A short customer question may require very little context, while another request may include a long conversation history, retrieved documents, instructions, and tool results.

This difference becomes even more important in agentic applications. A single user request may lead to several model calls, with each call processing its own input and producing its own output.

The result is that token usage needs to be tracked at the application and workflow level, not just as a monthly number on a model provider’s invoice.

2. Model selection

The model used for a task can have a major effect on its cost. Not every request needs the most capable model available. A simple classification task, basic content transformation, or straightforward customer question may be handled well by a smaller model, while a complex reasoning task may justify using a more capable one. This is why many production AI systems use more than one model. The application can route different types of work to different models based on factors such as complexity, accuracy requirements, response time, and cost.

Google Cloud makes a similar point in its guidance on building a cost-effective generative AI strategy: managing AI cost is not simply about choosing the cheapest model. Teams need to find a balance between cost, performance, and the requirements of the workload.

The important part is to make model choice a deliberate engineering decision rather than allowing every workflow to default to the same model.

3. Inference and compute

Running an AI model also requires computing resources, and these costs become especially important when a business runs models on its own infrastructure.

For teams managing their own AI workloads, the bill can include GPUs or other accelerators, servers, storage, networking, and the infrastructure needed to handle changes in demand. Keeping enough capacity available for peak traffic can also mean paying for resources that sit underused during quieter periods.

Inference can therefore become a significant part of the ongoing cost of an AI application. Google Cloud’s recent work on inference optimization highlights the need to manage both performance and resource efficiency as AI workloads move into production.

This is also where traditional cloud practices still have a role. Better resource utilization, sensible scaling, and understanding when infrastructure is being used are just as important for AI workloads as they are for other cloud applications.

4. AI agents and workflow complexity

The number of model calls behind a feature can change the economics of an AI application considerably.A traditional chatbot may make one model call to answer a question. An agentic application can involve planning, tool use, additional reasoning, retries, and follow-up calls before the task is complete. That makes the workflow itself an important cost factor.

For example, an AI assistant that helps employees prepare a report may need to search internal documents, retrieve information from a database, check the results, and then produce the final report. If each stage involves another model call, the cost of completing the task is much higher than the price of a single response would suggest.

This is one reason AI FinOps needs visibility into the complete workflow, not just individual model usage.

5. Supporting infrastructure

The AI model is only one part of the technology stack. Applications may also depend on vector databases, embeddings, storage, networking, monitoring, orchestration tools, and third-party AI services. These costs may appear across different infrastructure and vendor bills, which makes them easy to overlook when teams focus only on model usage.

A useful way to think about it is: the model invoice tells you what you spent on the model, not what the entire AI application cost to run. Getting the full picture means connecting model usage with the infrastructure and workflows around it.

Build an AI FinOps Practice That Scales With Usage

Managing AI costs does not require a complete overhaul from day one. The important thing is to start building visibility before usage becomes difficult to control.

Begin by tracking token and model usage, then connect that spend to the applications and workflows generating it. From there, teams can set practical baselines such as cost per request or cost per completed task, add budgets and alerts, and use the data to identify where changes will have the biggest impact. Over time, those insights can guide decisions around model selection, prompts, retrieval, infrastructure, and agent workflows.

The bigger shift is in how AI spending is viewed. An API invoice can tell you how much was spent, but it cannot tell you whether that spending was justified. Businesses need to understand which applications are driving costs, what each workflow costs to run, and whether the result is valuable enough to support that expense.

This is where the principles of FinOps become useful for AI. The goal is not to make every AI task as cheap as possible. A better goal is to keep spending under control while making sure the system delivers the outcome the business actually needs.

Daffodil’s work across generative AI development and AI engineering brings these considerations together, helping businesses build AI applications with the right balance of performance, scalability, and cost.

As AI becomes part of more products and business processes, treating its costs as something that can simply be reviewed at the end of the month will become harder to sustain. The organizations that build visibility and cost discipline early will have a much clearer picture of where their AI investment is going and what it is delivering.