Are AI Neoclouds Shaping Data Center Strategies?

Something fundamental is changing in AI data centers — and it's not the servers or the cooling systems. It's the traffic itself.

New measurements from Backblaze reveal that neocloud-driven workloads are producing sustained data transfers between storage systems and compute clusters at rates ranging from 100 Gbps to 1 Tbps. Those aren't burst numbers; that's continuous throughput, moving enormous volumes of data in highly coordinated flows that look nothing like the network activity traditional data centers were designed to handle. For operators, architects, and anyone building or buying infrastructure right now, understanding what's driving this shift — and what it demands — is no longer optional.

What Is a Neocloud, and Why Does It Matter?

The term "hyperscaler" gets thrown around for any large cloud provider, but neoclouds occupy a distinct category. These are infrastructure providers purpose-built for AI compute — companies like CoreWeave, Lambda Labs, and IREN that aren't running general-purpose cloud workloads. They're running GPU clusters at scale, specifically to support model training and inference for AI developers who need raw compute without the overhead of a full enterprise cloud platform.

Neoclouds aren't just smaller hyperscalers — they're a structurally different kind of infrastructure business, optimized for workload types that major cloud providers weren't originally designed to serve efficiently.

That specialization is precisely why neocloud traffic behaves so differently. The workloads are more homogeneous, more synchronized, and more demanding of sustained bandwidth than the mixed enterprise traffic that shaped conventional data center design assumptions over the past two decades.

The Traffic Shift That's Breaking Old Assumptions

Traditional data center networks were engineered around a particular model of traffic: many short-lived flows, distributed across many endpoints, with peak demands that were high but brief. Load balancers, congestion control protocols, and switching architectures were all tuned to that reality.

AI clusters don't work that way. When a model trains across thousands of GPUs, those GPUs need to stay in constant communication with each other and with storage systems throughout the training run. The result is what Sameh Boujelbene, vice president at Dell'Oro Group, describes as "larger, more synchronized, lower entropy elephant traffic" — a precise characterization that's worth unpacking.

Elephant flows in networking parlance are long-duration, high-volume transfers. Low entropy means they're predictable and structured rather than random and bursty. Synchronized means many of these large flows are moving in coordinated patterns simultaneously. That combination — large, predictable, and synchronized — is categorically different from what conventional network hardware was designed to manage.

In practice, this translates to sustained pressure on east-west network capacity: the traffic moving laterally between servers and storage within a data center, rather than north-south traffic moving to and from users. East-west optimization has been a priority since the rise of distributed computing, but AI neoclouds are pushing that requirement to a new extreme. A 1 Tbps sustained transfer rate isn't a number you architect around as an edge case; it becomes the baseline expectation.

What Operators Are Being Forced to Rethink

The network implications cascade quickly. Switching fabrics that were adequate for enterprise mixed workloads become bottlenecks. Congestion management protocols designed for bursty traffic behave poorly under the constant pressure of synchronized elephant flows — they either overreact or underreact, neither of which is acceptable when GPU idle time costs money by the second.

Data center operators serving neocloud tenants are revisiting several core assumptions:

Oversubscription ratios, which have historically allowed operators to provision less physical bandwidth than theoretical peak demand, become dangerous in AI environments. If every flow is sustained and synchronized, there's no statistical smoothing to bail you out.

Congestion control architecture needs rethinking at the protocol level. Technologies like RDMA over Converged Ethernet (RoCE) and protocols designed for high-performance computing environments are gaining traction in AI data centers precisely because they were built for the kind of predictable, low-latency, high-throughput transfers that model training demands.

Backend fabric topology also comes under pressure. Spine-leaf architectures that work well for distributed workloads may need to give way to configurations that prioritize all-to-all bandwidth between GPU nodes — a substantially more expensive and complex design.

The insider reality here is that many colocation operators who rushed to court AI tenants are now discovering that their existing network infrastructure simply wasn't built for this traffic profile. Retrofitting is possible, but it's not cheap, and it's not fast.

Where This Goes From Here

The Backblaze data and Dell'Oro Group's analysis are early signals, not anomalies. As AI model training runs get larger — and there's no sign that trend is reversing — the demands on storage-to-compute bandwidth will only intensify. Models that currently require 100 Gbps sustained throughput will be succeeded by workloads requiring multiples of that.

The data centers that capture the next wave of AI infrastructure investment will be the ones that were designed from the start for elephant traffic, not the ones trying to retrofit for it.

This has concrete implications for site selection, network procurement, and even real estate decisions. Facilities with access to abundant power are already at a premium for AI workloads — but power alone isn't the differentiator anymore. Network architecture is becoming an equally critical filter. A 200 MW campus with insufficient east-west switching capacity is going to lose AI tenants to a 50 MW facility that's properly architected for sustained, high-bandwidth, synchronized flows.

For developers and investors evaluating data center assets, the practical takeaway is this: start asking harder questions about network topology, not just power capacity and cooling. The operators who understand that AI neoclouds have permanently changed what "network ready" means — and who have actually built for it — are the ones positioned to win the next several years of AI infrastructure buildout. Everyone else is playing catch-up on an increasingly expensive curve.

Explore the InfraSale Marketplace for cutting-edge data center solutions!

[INTERNAL LINK: AI neoclouds]

[INTERNAL LINK: data center strategies]

[INTERNAL LINK: network architecture]