By R K Anand
If you were in Silicon Valley in the late 1990s, today’s AI boom might feel familiar. In 1998 and 1999, internet traffic was roughly doubling every six to nine months. That was extraordinary growth in demand, and still, that pales in comparison to what we’re seeing unfold now.
Today, the four largest U.S. cloud companies are looking at more than $700 billion in capital spending this year, and the cumulative AI infrastructure spending is projected to reach $3 trillion to $4 trillion by 2030.
Nvidia arguably fired the starting gun in AI chips when Jensen Huang hand-delivered its DGX-1 to OpenAI in 2016. Today, it is a household name deeply embedded in the AI ecosystem, working with near-exclusive supply and investment deals.
With such a huge market under a seeming monopoly, where can competition — and the innovation that follows it — come from?
Part of the answer is that scale costs you speed, though when going up against an execution machine like Nvidia this is simply the baseline you have to solve for. The pace of change in AI outruns anything we saw in previous tech cycles: Model generations turn over in weeks, not years. The hardware beneath it has to keep shifting to serve them, and designing from first principles has real merit.
Lessons from Davids that came before
Hardware suppliers of the dot-com era followed the same peaks and troughs. Cisco, founded in 1984, dominated internet routing for most of the decade, and in March 2000 it was the highest market capitalization company in the world.
Then Juniper Networks, a startup where I was a founding engineer, shipped its first core router in 1998. Juniper went from a fairly anonymous upstart to 17% of the core router market at the end of 1999, then again to 38% in early 2001.
Challengers now can lean on history for some guidance.
First, don’t build a cheaper version of the incumbent’s product. Build a far better one. At Juniper, our bet was that the then-adolescent version of the internet was going to demand an exponentially higher throughput than before. Introducing an entirely new architectural approach to how routers are built, along with an innovative design philosophy focused on performance and reliability, we didn’t set out to sell a better router, so much as redefine what a core router had to be.
The boom in AI inference is today’s version of that strain on throughput. The initial hyperfixation on training AI models is fading, and the industry is collectively moving on to using them — known as inference. Inference is where AI’s promised productivity gains actually materialize, putting trained models to work across every task. As the enterprise applications for AI grow and become more complex, so too does demand for inference compute.
Google recently reported that it now processes more than 3.2 quadrillion tokens a month, up from 480 trillion a year earlier, as AI moves from chat to agents that plan and act on our behalf. Serving tokens at scale is a different beast from when it was just for model training, with different constraints on memory, power and cost.
The core principle we use applies to all sectors: Challengers have a better chance at optimizing for the market that will come, not the market that is already well-serviced by the incumbent.
Be known by your track record
Challengers start without the incumbent’s biggest asset: a reputation. Buying from the market leader is the safe choice, so a startup has to replace that trust with evidence. Every technological edge you can demonstrate and validate makes the next conversation easier, and over time that compounds into the credibility that an incumbent may take for granted.
To aid this, make migration as easy as possible. No one is wedded to a product out of loyalty. Inertia is one of the biggest strengths of an incumbent: the ease of continuing to do what has been done.
Unless there are clear economic and operational benefits, scale and performance advantages, more often than not, integrating new systems will not be worth the work involved in switching suppliers. Take any technical friction out of the equation, and you have a fighting chance with your improved product.
Uncertainty is your friend
The internet transformed the world in ways that we couldn’t have predicted, and I have little doubt that the relentless pace and momentum of AI is resulting in changes that are quickly making our lives and work unrecognizable from the recent past. This is only going to accelerate. Monopolies in frontier technology look insurmountable right up until the work changes. What AI does is rapidly morphing and evolving.
There is comfort in that for founders, and in something else too: A small team can now achieve so much. Challenging Cisco in the late 90s and early 2000s meant building a nimble and dynamic organization, albeit much smaller — one that is far more capable, reactive and product-focused. Juniper numbered a few hundred people by the time it had significant market traction, compared to the thousands of employees at Cisco.
Our own company is now around 120 people, attempting what once took an army. That is what should keep the incumbents watching.
R K Anand is co-founder and chief product officer of Tensordyne, an AI inference company. He was a founding engineer and executive vice president of the data center business unit at Juniper Networks and began his career designing microprocessors at Sun Microsystems.
Illustration: Dom Guzman
Stay up to date with recent funding rounds, acquisitions, and more with the
Crunchbase Daily.
