Is token based pricing the right model for enterprises?

Strong financial planning is a core tenant for CIOs. Understanding and managing to a clear budget that addresses the value of their investments plays a key role in the success of the organization. CIOs often have their own ‘methodology’ as to how they manage their financial planning over the course of the month, quarter, year and across multiple years. As a CIO, I had my own way of managing between more structured activities and variables ones that were less predictable. One of the hallmarks of a CIOs success is to effectively manage the financial planning of their organization and as best as possible reach financial predictability.

In the past few years, the alure of AI and its potential has thrown a bit of a monkey wrench into the financial works. AI is highly consumption-based and brings a new and highly variable cost into the picture. Organizations have struggled to effectively understand and manage their AI spend. Unfortunately, many organizations have quickly burned through their budgets for AI spend and are looking at ways to better understand and manage the spend.

At the same time, providers have moved towards token-based pricing for AI workloads. The method of pricing will vary a bit between providers. But what is a token and how can enterprises relate to tokens?

Understanding value over cost

Like the conversations we had with cloud about consumption- based spend, AI is driving a similar sounding conversation. With cloud, the conversation was often “We are spending too much a specific provider.” This statement was often in the context of cost, not value.

With cloud, the problem was that enterprises were more focused on the sheer spending and a number without fully understanding the value they were getting from that spend. To be fair, fully understanding cloud costs and tying those costs to specific outcomes was challenging. It still is…even today. Major providers like AWS and Google are going to great lengths to help customers understand their cloud spending using their own tooling and frameworks from FinOps and TBM Council. More on FinOps and TBM Council in a separate post.

Unfortunately, AI is following in cloud’s footsteps and making some of the same mistakes by focusing on consumption cost versus value. To be fair, understanding AI’s true costs is incredibly more complicated than cloud was/ is.

Pricing models galore…

To complicate matters further, providers are using many different pricing models. Pricing models span seat-based pricing, subscription-based pricing, consumption-based pricing and most recently, token-based pricing.

Because AI is so new, the industry does not have a solid baseline to understand the true costs that go into specific actions. Providers are making their best guess at what they think their costs are and then passing that to the end-user. At the same time, there is still plenty of subsidizing happening along the AI value chain. That is to say that providers along the value chain are subsidizing costs to drive up demand. This is expected and typical behavior when creating a new market opportunity.

Broadly, providers are starting to gravitate toward token-based pricing as a common pricing model. Tokens are units of consumption that AI workloads consume while performing their work. The problem is: What really is a token?

What is a token and is it a viable unit of measurement?

What in the heck is a token and can I tie it (them) to a business outcome? In a word, no.

A token, is not a token, is not a token. The definition of a token varies between providers in terms of how it is used. There is a rule of thumb that a token equates to 4 characters in English. However, variables such as language changes, how a model breaks up words, capitalization and other factors can greatly impact how many tokens are needed for a specific task. This leads to greatly different outcomes in terms of token consumption and therefore costs.

Different models and providers treat and value tokens differently. This leads to further variability in the space. These unknowns and the inability to predict the consumption for an outcome are creating challenges for enterprise CIOs looking to embrace innovation while effectively managing their budget.

The bottom line is that token-based pricing is problematic on many levels. It is just too granular and inconsistent for enterprises to effectively manage.

CIO Perspective

As a CIO looking to effectively manage AI costs, can token-based pricing be the right metric today? No. At the same time, seat, subscription and outcome-based pricing is also problematic as each are not granular enough to represent the resources required to complete a task or action. So, what is the answer?

The reality is that we still must work this out. ‘We’ being each provider in the value chain along with end-users. It is likely we may gravitate to a more hybrid pricing model. One that combines factors from different models. Even if the definition of a token were to become more standardized, it is still too granular for enterprises to effectively manage. It would be like a pricing model based on managing compute cycles.

Using an analogy, today we are asking new car buyers up front to determine how many times they will start their car, stop at a stop sign/ stoplight, how many city miles they will drive and how many freeway miles they will drive. Up front. At the same time, we are driving all over the place and exploring where we can go with this new car.

It is impossible to answer the question up front. After the fact, sure. The problem is that enterprises need more clarity up front on what the costs look like so they can make educated value decisions.

As a sidenote, this is one of the reasons why enterprise CIOs are moving some AI workloads on premises to control their costs and take some of the variability out of the equation.

So, back to the base question…Is token-based pricing the right model for enterprises. On the surface, no. But a more sophisticated approach is needed.