Teaching Hyperscale Systems to Remember: An Interview with Anish Reddy Yennapusa

The AI buildout has a strange blind spot at its center. The industry is constructing computing capacity at a historic rate, with data centers projected to absorb [$6.7 trillion in global investment by 2030](https://www.mckinsey.com/industries/technology-media-and-telecommunications/our-insights/the-cost-of-compute-a-7-trillion-dollar-race-to-scale-data-centers). Running all of it badly keeps getting more expensive too: [Splunk's latest downtime research](https://www.splunk.com/en_us/form/the-hidden-costs-of-downtime.html) puts the annual cost of unplanned downtime for the Global 2000 at $600 billion, roughly $300 million per company. Yet for all the dashboards this money buys, hyperscale systems share a quiet flaw. When incidents repeat, when architectures evolve, when experienced engineers move on, the reasoning behind past decisions tends to vanish. The data survives. The understanding often does not.

[Anish Reddy Yennapusa](https://www.linkedin.com/in/anishreddy652), a software development engineer at a hyperscale cloud provider, has spent his career on that flaw. He is the lead developer of a global monitoring and telemetry platform spanning more than 400 data centers, and the author of MEMORY-AWARE INFRASTRUCTURE: Designing Hyperscale Systems That Observe, Remember, Understand, and Adapt, which argues for treating operational memory as a first-class architectural capability rather than an afterthought.

We spoke with Anish about why more telemetry has not produced more understanding, what a system that remembers actually looks like, the assumptions that broke along the way, and what has to happen before AI agents can be trusted to act on physical infrastructure.

Hyperscale systems generate more telemetry than any machines in history, yet you argue their operators still don't understand them. Isn't observability a solved problem by now?

Solved is not the word I would use. The industry has become excellent at collecting signals and surprisingly bad at keeping their meaning. [Grafana Labs' 2026 observability survey](https://grafana.com/press/2026/03/18/grafana-labs-4th-annual-observability-survey-reveals-a-field-at-a-crossroads-ai-economics-complexity-and-the-enduring-power-of-open-source/) of more than 1,300 practitioners found that complexity and overhead remain the top observability concern for 38% of organizations, and that alert fatigue is the single biggest obstacle to faster incident response, cited by 30%. More data did not fix those numbers. In several ways it made them worse, because every new signal adds one more thing a tired engineer has to interpret at 3 a.m.

The deeper problem is that observability answers only one question: what is happening right now. It cannot answer the question that actually resolves incidents, which is why the system behaves this way. I have watched teams re-diagnose essentially the same failure nine months apart because the person who understood it the first time had changed teams and the postmortem was a PDF nobody could find. The system saw everything and remembered nothing.

Your book argues for treating operational memory as a first-class architectural capability. What does that mean in plain terms?

It means the infrastructure retains its own operational experience and uses it. Today a monitoring platform can tell you a rack is running hot. A memory-aware system can also tell you that this rack ran hot last October under the same workload pattern, that the fix was a firmware quirk rather than a cooling failure, and that the engineer who diagnosed it left a note explaining why the obvious answer was wrong. Same signal, completely different outcome. One triggers a ticket. The other triggers understanding.

I did not arrive at this from theory. Building telemetry systems across a fleet of more than 400 data centers, where no two sites report their status quite the same way, teaches you that the hardest part of scale is not volume but accumulated difference. Every site carries its own history of hardware generations, vendor integrations, and improvised fixes. If your platform treats each reading as context-free, that history works against you. If the platform remembers it, the history starts working for you.

How do you actually build memory into a system? What does the architecture look like?

Three ingredients, in my experience. First, high-fidelity telemetry, because memory built on coarse data is just a blurry photograph. I spent a year building a polling engine that collects infrastructure telemetry at one-second granularity across hundreds of data centers, and that resolution is what makes retained experience worth anything. Second, retained context: decision logs, incident records, and baselines that stay attached to the signals they explain instead of scattering across wikis and inboxes. Third, feedback loops that let past experience shape future behavior, so the platform's alerting and optimization get sharper with every incident instead of starting from zero.

The financial argument practically makes itself. [Uptime Institute's 2026 outage analysis](https://uptimeinstitute.com/about-ui/press-releases/uptime-announces-annual-outage-analysis-report-2026) found that 57% of operators' most recent major outages cost more than $100,000, and one in five exceeded $1 million, with power failures still the leading cause. Repeated incidents are expensive teachers. A system with memory pays that tuition once. On our platform, unifying detection and learned baselines cut mean time to detect critical events from hours to real-time alerting, and new sites now come online in days rather than weeks because the platform carries what it learned from the previous 400.

What did you get wrong, or nearly wrong, building at this scale?

My first instinct was that more data was always better, and I pushed polling frequency as hard as the design allowed. What I learned the expensive way is that telemetry collection taxes the very hardware it measures. At one-second intervals across thousands of devices, the observer effect is real: you can degrade the thing you are trying to protect. I had to back off and rebuild around adaptive polling that adjusts its intensity to what each device can tolerate. It cost us a redesign I had not planned for, and it made the system far better.

The second wrong assumption was that I could normalize every vendor's data into one clean schema up front. Across hundreds of sites built by different vendors in different years, that rigidity shattered almost immediately. The tradeoff I chose was resilience over completeness: the platform drops a malformed reading rather than stalling the pipeline, which is part of how it holds 99.9% uptime. At this scale, a monitoring system that goes down while being perfect is worse than one that stays up while occasionally missing a sample. I would rather explain a gap in the data than explain an outage in the monitoring.

You also joined the judging panel for the Beta University x AMD AI Agent Hackathon. What separates a real agent project from a good demo?

The failure modes, every time. A demo shows you the impressive path: the agent works, the room applauds. When I judged, I looked for teams that could show me the boring path: the retries, the degraded behavior when a tool call fails, the audit trail of why the agent made a specific decision. The strongest projects treated their agent's reasoning as something to be instrumented and inspected, not something to be admired. Several of the best teams had effectively given their agents a working memory of prior runs, which meant the system improved over the weekend instead of repeating its mistakes. Judges notice that.

Judging also sharpened how I build. Hackathon teams optimize for a three-minute demo, and production systems optimize for the thousandth unglamorous Tuesday, but the discipline underneath is identical: an agent that cannot explain its reasoning is a demo, not a system. That is as true inside a data center as it is on a hackathon stage, and it is a large part of why I wrote the book.

Gartner expects a third of enterprise software to include agentic AI by 2028, yet also predicts over 40% of agentic AI projects will be canceled by the end of 2027. Who is right?

Both, and the tension between those numbers is the most honest thing in the industry right now. [Gartner's forecast](https://www.gartner.com/en/newsroom/press-releases/2025-06-25-gartner-predicts-over-40-percent-of-agentic-ai-projects-will-be-canceled-by-end-of-2027) is really one prediction, not two: agents will spread, and the ones deployed against systems they cannot truly understand will be canceled. Look at where agents actually perform. [Stanford's AI Index](https://hai.stanford.edu/ai-index/2025-ai-index-report/technical-performance) found that on the RE-Bench evaluation, top AI systems outscore human experts four to one on tasks with a two-hour budget, but humans pull ahead as the horizon stretches, doubling AI performance at 32 hours. Agents excel at bounded, short-horizon work, which is exactly the work where complete context fits inside the task.

That is where memory becomes the deciding factor. An agent acting on infrastructure without operational memory is reasoning from a single frame of a very long film. It will re-diagnose solved incidents, repeat retired decisions, and eventually get its project canceled for it. Practitioners already sense this: in Grafana's survey, 95% said it is important for AI to show its reasoning. Reasoning about what, though? My answer is what the book keeps circling. If agents are going to act on infrastructure, their decisions have to be grounded in what that infrastructure has already learned.

What has to change before we trust AI to run physical infrastructure?

Operational memory has to stop being a side effect and start being engineered deliberately, budgeted and reviewed the way we budget compute and network capacity. That means platforms that retain decision context as a matter of design, postmortems treated as assets that feed the system rather than documents that satisfy a process, and evaluation standards that ask not only whether an agent acted correctly but whether it knew what the system had already been through. None of this is exotic. It is unglamorous, which is exactly why it gets skipped, and why it keeps costing people outages.

As for me, this is where the next phase of infrastructure engineering gets interesting. I am excited by the challenges we break down as data center fleets grow at a record pace and AI takes on a greater role. I want to keep working at that intersection of software and physical infrastructure, while sharing and mentoring others on what we learn along the way. Hyperscale systems are only going to become more complex. The question is whether we operate them as we have for years, or finally design them to remember.