Category: FinOps

In this new reality where we have the privilege to not only take apart cloud infrastructure and re-engineer it for efficiency, but also bite down on AI implementations, it is becoming increasingly clear that we need strategies for optimisation.

One of the most important handles to finetune a workload in the cloud is rightsizing a VM. That can be through classic resizing, moving to a container set-up, moving to an autoscaling solution, going serverless, … to be fair, that last one is not really rightsizing, but the concept is there: making sure you run the appropriate compute resources for your workload.

Now, consider the following:

You are chatting with your favourite new LLM, asking it to fix a typo in an email, a script, … Under the hood, that little request just walked through the same reasoning stack that could solve a PhD level physics problem, or a complex math problem. Nothing was rightsized whatsoever. You just drove your Ferrari from one end of your driveway to another top open your mailbox, basically.

That is the state of AI spend right now.

Everyone is default routing every prompt to the biggest brain available, the same way we used to spin up an m5.4xlarge to run a task Lamba can perfectly handly. Slowly we are starting to fix that behavior in the cloud. For AI however, it almost feels like none of the lessons from Cloud are taken into account.

The instance family problem, but for intelligence

Rightsizing a VM starts with one simple question: “What does this workload actually need?” Not what it might need on its worst day, not what feels safe, what it needs right now, based on evidence. Yes, might and safe should play a role, but if you do the cloud right you let the cloud handle deviating scenarios. (Read: elasticity)

Model selection deserves the exact same diligence. Summarizing a support ticket does not need the same advanced model that is refactoring five interdependent services at 2am. But because switching models still feels like a decision, a manual toggle, most people just leave it on the expensive default and eat the bill. What probably also is a factor, is that there’s still a desire for models to improve and a hope that new models are more and more reliable. While that’s understandable, this is why you’re probably bleeding money.Anyone who has ever fought a team out of oversized reserved instances knows this pattern by heart.

Enter the router

Some products landed recently that made this analogy impossible to ignore, and they came from very different corners of tech.

Cursor shipped a Router that sits inside the IDE and classifies every single coding request before it ever touches a model. Simple, familiar edits get bounced to something cheap and fast. A gnarly multi-file refactor gets escalated to a frontier model. You pick a dial, Intelligence, Balance, or Cost, and the router steers along that frontier for you. Think of it as an autoscaling group, except instead of scaling instance count based on CPU load, it is scaling model tier based on task complexity. Same instinct, new axis.

Ramp took the harder, more forensic road. Their internal gateway does not just classify and forward, it learns. It watches latency and failure rates per model per hour, notices things like a discount tier quietly degrading during business hours, and adjusts routing in real time using statistical methods built for exactly this kind of decision under uncertainty.

That is a system running its own continuous audit of the fleet, the same forensic reflex FinOps people apply to spot pricing and reserved instance coverage, just pointed at tokens instead of vCPUs. Both approaches land in the same place: send the request to the cheapest model that still clears the bar. Everything else is implementation detail.

There’s however still an important gap:

Here is the part that should make every FinOps practitioner sit up. Rightsizing a VM has over a decade of tooling behind it. CPU and memory utilization graphs, sizing recommendations, guardrails, rollback plans, precedent and an established accepted culture. If you get it wrong, you know within a day and you fix it.

Model rightsizing has none of that maturity yet. ‘Clears the bar is doing an enormous amount of unaudited work in every one of these router pitches. Whose bar. Measured how. Nobody is publishing the equivalent of a CloudWatch dashboard for reasoning quality, at least not yet.

Which is exactly the gap a forensic FinOps practice should be walking straight into. We already know how to ask uncomfortable questions about whether a resource matches the evidence of need. We just have not built the token era version of that discipline. The utilization metric of tomorrow is not CPU percent, it is something closer to task complexity per dollar, and right now almost nobody outside these router vendors is even trying to measure it independently.

Rightsizing was never really about the VM. It was about refusing to pay for capability you are not using. So let’s apply that same logic to AI tasks, where we try and run only for what we actually need to achieve.

FinOps entered the world as a discipline focused on controlling costs and enabling efficient cloud investment to unlock business success and mission achievement.

While this is still how FinOps operates in most organisations, it has evolved into something much more interesting. A while ago, I made the case that FinOps had become an intelligence layer. I would like to extend that idea one step further: FinOps is a security discipline.

Let me explain.

Cost as a risk is heavily undervalued. We tend to look at cost management through dashboards, policies, reports, and budgets. Yet cost is fundamentally tied to an organisation’s ability to operate, invest, and grow. It determines what can be built, protected, improved, or sustained. Well-managed costs enable investment in backups, security improvements, research, skilled staff, resilience, and innovation. Poorly managed costs limit those investments and expose organisations to risk. Cost is not just a financial metric; it is a measure of organisational health and resilience.

Viewed through that lens, FinOps is not simply a component of good cloud operations, nor is it solely a governance function. FinOps is a defensive practice. Its purpose is to protect the organisation by reducing financial and operational risk. As a defensive discipline, FinOps secures the resources required to invest in stronger security controls and better operational outcomes. It provides visibility into infrastructure usage, network flows, and consumption patterns. It helps detect harmful cost anomalies before they become budgetary incidents.

As an example: in a world increasingly shaped by AI, where spending can scale at unprecedented rates, FinOps provides the controls needed to monitor, manage, and protect the organisation from financial exposure.

Speaking from the heart: We may have started as analysts. Some of us became engineers. Today, I believe our role has evolved further. We are operators working to secure the organisations we serve. It is with that mindset that FinOps practitioners should approach their work: not simply as cost managers, but as defenders of organisational resilience. By protecting the resources that make everything else possible, we help our companies thrive and achieve their mission.

FinOps data has always been sensitive, and FinOps practices have always been something worth protecting. As FinOps has matured into a key enabler of business decision-making, it has also evolved into something much broader than cost management. It now sits at the intersection of observability, anomaly detection, inventory management, and investigation.

FinOps has become an intelligence layer.

Many FinOps practitioners have probably experienced this already. At some point, someone from security, compliance, or operations has likely asked for a list of resources that could be affected by a vulnerability or operational issue. More often than not, FinOps teams are the only ones with a complete and accessible view of everything an organization is running.

Knowing what exists often matters more than controlling it.

Without deliberately aiming for it, FinOps has become a critical function that helps modern cloud-based organizations operate effectively while protecting their intellectual property, infrastructure, and architecture.

As this role continues to expand, I believe FinOps will rise to the challenge.

In the past, I have described FinOps as consisting of three primary responsibilities:

• Rate reduction
• Waste reduction
• Transparency

Today, I believe transparency is increasingly becoming the primary objective, with rate reduction remaining a close second. Waste reduction is gradually becoming a cultural responsibility shared by everyone who builds and operates in the cloud.

The rise of AI will only accelerate this shift. As organizations adopt AI at scale, the demand for transparency across infrastructure, applications, and consumption patterns will become even more important.

Into Uncharted Territory

I have spent nearly a decade helping organizations reduce waste, and I will probably never stop looking for inefficiencies. In many ways, I approach inefficiencies the same way security researchers approach vulnerabilities.

That said, it is becoming increasingly clear that FinOps needs more researchers and fewer advocates.

Rather than constantly pushing for frameworks , FinOps professionals should focus on boots on the ground impact. Show people what they can influence. Show them what is going wrong. Present the evidence and allow teams to act.

And when action is not taken, FinOps must sometimes step into the role of investigator, forensic analyst, and operator.

During a recent conversation with a peer in the industry, I described the current transition this way:

“With the advent of cloud, the procurement process was completely disrupted. Suddenly, anyone with a credit card and a browser could acquire infrastructure. Now AI is creating the same disruption for applications and software. Anyone with a license can build solutions that drive resource consumption.”

This is where the next major challenge for FinOps emerges.

Historically, FinOps practitioners have focused on understanding cloud architecture. Going forward, they will also need to understand software systems, application behavior, and how those systems consume infrastructure resources.

The evolution of FinOps as an intelligence layer makes this possible. The more visibility we gain, the more connections we can make. The more connections we can make, the better we can understand the true drivers of cost, performance, efficiency, and risk.

The Real Shift in FinOps

FinOps entered the technology landscape with frameworks, methodologies, and strong guiding principles.

Phrases such as:

• “It’s not about saving money, it’s about making money.”
• “Showback drives behavioral change.”

helped establish FinOps as a governance and management discipline.

Those principles remain valuable, but the environment has changed.

Today, engineers, architects, and product teams are far more aware of costs during the design and development process. This mirrors what happened with security over the last decade, where security considerations gradually became part of the normal way people build systems.

The Conclusion

As cost awareness becomes embedded in engineering culture, the role of FinOps naturally evolves.

We provide the visibility that allows teams to demonstrate leadership. We provide the context that supports informed decisions. We transform data into understanding.

And just as we built deep expertise around cloud infrastructure, we will build the same expertise around AI. By doing so, we will help our organizations adopt these technologies successfully, make better decisions, and ultimately achieve their mission.