Under Pressure: The Hidden Infrastructure Crisis No One Is Talking About

AI, power, and thermal demands are breaking old assumptions — efficiency and smart utilization are now mission-critical.

For decades, we operated with a mindset of unlimited resources. Unlimited data center capacity, unlimited cooling capabilities, unlimited bandwidth, unlimited power resources. The only constraint was money.

Today, in addition to physical constraints, the explosion of data is driving new pressures for IT organizations. In addition to the sheer amount of space being consumed by data, AI is driving even more demand for modern, sophisticated, high-performance infrastructure.

The combination of these two issues are likely to drive headwinds that will slow innovation.

There are also two related gaps not being discussed. First, the gap caused by over-provisioning of resources. And second, the gap between current demand and forthcoming demand.

Maximizing resource utilization

At a surface level, companies have a plethora of resources at their disposal yet are not maximizing their utilization. Today, that shows up in several different ways.

In firewalls, companies are largely underutilizing their firewalls with only 10-20% of the features in use. Aside from the physical resource underutilization, enterprises are paying for features they are essentially not using.

We also see this in terms of significant over-provisioning of resource allocation in cloud and on-prem data centers. This is largely due to a long-standing methodology where running out of resources was not an option. The answer was over-provisioning to ensure that never happened.

As we moved into the cloud era, this methodology carried forward which led to companies facing extreme costs from cloud-based alternative resources. Was this a shortcoming of the cloud provider? No. It was a combination of good marketing and poor planning before a move to cloud.

Here are a few pieces I’ve written about these issues in the past:

• Eight ways enterprises struggle with public cloud (2017)
• Why are enterprises moving away from public cloud? (2017)
• Eight reasons why enterprises move workloads from public cloud back on-premises (2020)

Cue the physical constraints

In the last several years, demand on core infrastructure has skyrocketed. New business demands to gain better insights faster is driving demand for more sophisticated applications. Which, in turn, drives demand for newer technology and innovation.

Today, there are several physical constraints that are quickly coming to the surface. Some of those examples include:

• Liquid Cooling: Newer, high-performance processors like Nvidia’s Blackwell GPUs, which are commonly used for AI workloads, can no longer be cooled with air alone and require liquid cooling. The idea of liquid cooling in a corporate data center or even some colocation facilities has long-since been seen as taboo.
• Copper > Optical: Copper is reaching its physical limitations in both speed and heat. As one example, copper interconnects are simply no longer fast enough to keep up and the shift to optical is needed. Copper-based solutions also give off more heat than their optical counterparts furthering the thermal challenges.
• Power and Cooling: More sophisticated workloads, such as AI, are requiring exponentially greater computing resources which require more power and cooling capabilities. Power grids are already straining under the demand, and providers are looking at new sources such as nuclear to supply these requirements.

Unfortunately, there is no quick, physical solution to any of these issues. 

We have crossed the threshold where we no longer have unlimited resources and need to look toward innovation, efficiency and intelligence.

Changing the culture: The drive for innovation, efficiency and intelligence

We can no longer operate in a tech culture assuming unlimited resources. Our culture and thinking must change.

The solution is getting smarter about how we use the resources we already have. As we seek innovation that drives greater efficiency through better intelligence, we can still accomplish growth while being smarter about how we consume resources.

CIOs need to start contemplating strategy changes to how they consume resources. AI is a double-edged sword. On one hand, it consumes greater resources. On the other hand, it will likely be the solution to its own problem.

AI provides a great equalizer to accelerate thinking and understand of how to progress maturity on so many levels. Resource consumption is just one of those.

Industry impact

Industry players from hyperscalers to OEM hardware providers will need to rethink how they address this change in demand and sophistication. Bigger, faster, better is quickly going to lose steam to those leveraging data insights to better utilize infrastructure resources.

From architecture, to design, to marketing, to partners, to pricing, there is a significant change quickly coming down the road.