Why AI Shopping Assistants Need a Real-Time Presentation Layer: An Interview with Senior Software Engineer Aditi Patodiya

Generative AI is changing how consumers search, compare, and decide online, but the experience only works when the answer arrives fast, safely, and in a form the customer can actually use. McKinsey estimates that generative AI could unlock between $240 billion and $390 billion in economic value for retailers, which makes the engineering layer behind AI commerce increasingly important. 

Aditi Patodiya, a Senior Software Engineer with nearly 10 years of experience, builds large-scale backend systems behind a generative AI shopping assistant on a major consumer technology platform. Her HackerNoon article, How Enterprise AI Systems Simulate Memory Without Breaking the Token Budget, demonstrates her public technical writing on production AI infrastructure, especially how token budget constraints shape enterprise AI architecture. In her current role, her work on large-scale AIsystems focuse on a different layer: the real-time content presentation service customers see.

Aditi, what was the core problem you were solving with this project?

The problem was that a generative AI shopping assistant cannot feel truly real time if the customer-facing presentation layer is slow, fragmented, or stateless. When an answer is generated, the system has to assemble streaming text, check it for safety, enrich it with product details and imagery, and transform it into what the customer sees across the shopping app, mobile web, and website.

Before this project, the older system was less capable. It could not stream content in the way the assistant needed, and it could not retain context within the flow of a customer request. I conceived, designed, and led the delivery of a new real time, streaming, stateful content presentation service that now powers the answer experience for the assistant.

Why does the presentation layer matter so much in generative AI shopping?

People often focus on the model, but the model is only one part of the customer experience. A shopping assistant has to produce an answer that is safe, useful, visually complete, and fast enough to feel conversational. If the answer text arrives but the product context, safety checks, or final screen transformation lag behind, the experience breaks.

My work focused on the layer that turns raw generated output into the final customer experience. I consolidated several steps that had previously required coordination across three or more teams. Streaming answer assembly, content safety, enrichment, and transformation all moved into one service. That made the system faster to evolve and gave future assistant features a shared platform to build on.

What was your specific role in building the service?

From January 2024 to July 2024, I owned the service end-to-end. I created the technical vision, authored the foundational design documents, defined the high level and low-level architecture, and owned the interface contract between the service and the upstream orchestration system. I also led a team of seven to eight engineers, divided the work into streams, sequenced deliverables, and drove the project through production launch.

The core implementation work was also deeply hands-on. I designed the streaming workflow, the stateful presentation layer, and the way the service retained context within a customer request while assembling, enriching, checking, and transforming the answer into a single real-time stream.

What made this difficult at consumer-commerce scale?

At that scale, I had to design the service as critical path infrastructure, not as a standalone feature. The service processes 100% of the assistant’s production traffic, which means every customer request goes through it. Under normal conditions, it served about 52,000 requests per second on average, and during during a major peak shopping event it handled up to about 150,000 requests per second. That meant my design had to account for foundational infrastructure concerns, not only feature behavior.

I also led large-scale load simulation before launch, including an exercise that simulated roughly 6,500 requests per second. The goal was not just to prove the service worked in isolation. It had to survive real traffic patterns, safe deployment requirements, automatic scaling, throttling, monitoring, and operational safeguards.

How did the new architecture improve feature delivery?

The biggest change was removing the multi team bottleneck. In the prior approach, a customer experience change could require three or more teams to coordinate code changes across separate steps. That slowed down iteration and made the system harder to reason about.

By consolidating answer assembly, content safety, enrichment, and final transformation into one service, the architecture created a single place where the presentation experience could evolve. That matters because the assistant is not static. Every new customer facing improvement depends on the system being flexible enough to support it without forcing a long chain of coordinated changes.

Where did cost and infrastructure efficiency come into the design?

Generative AI at scale is expensive because every extra request, every duplicated workflow, and every unnecessary service hop becomes real cloud compute. IDC projects that accelerated servers will exceed 75% of server AI infrastructure spending by 2028, which shows why efficiency in the serving path matters so much.

A later enhancement to the service retained context across an entire customer conversation turn. That design consolidated multiple per-turn requests into a single stream and cut the service’s request volume by roughly 2.5×. That is important because the same customer experience could be served with less backend request pressure, which directly lowers the amount of compute needed to handle the same traffic.

How did you handle safety and launch readiness?

Content safety was part of the core service design, not an afterthought. Moving safety checks into the new service created an internal security and operational readiness requirement, and I personally owned the security risk assessment. That work determined the scope of the required internal security recertifications.

For launch, I authored the rollout strategy and formal change management approvals. We progressively enabled the service for live customers rather than treating launch as a single switch. I also drove the operational runbook, monitoring, alerting, storage and logging optimizations, and safe deployment safeguards that the service needed before it could sit on the critical path for the assistant.

What makes this service first of its kind within the assistant platform?

It was the first service in this assistant platform to combine a modern, high performance, two-way streaming protocol with a stateful presentation layer at this scale. The previous system was non-streaming and stateless. This service brought real-time streaming and context retention into the customer facing presentation layer.

For me, the novelty was not only using streaming. It was applying that architecture to the part of the assistant system that produces the answer experience customers actually see. Comparable large-scale internal services used older streaming approaches, and the modern protocol had only a single internal precedent on the AI inference side. Applying it to a stateful, customer facing presentation layer at at consumer-commerce scale was a major architectural step.

Why should consumers care about infrastructure they never see?

Consumers care because they experience the effects of the infrastructure, even if they never see it. A slow answer feels broken. A fragmented answer feels untrustworthy. A response that streams text but fails to present the right product details or imagery does not feel like a shopping assistant. It feels like a disconnected chatbot.

Deloitte found that 33% of surveyed consumers planned to use generative AI in their shopping journey, more than double the prior year. As more shoppers bring AI into product discovery, review analysis, and purchase decisions, the expectation becomes simple: the assistant has to respond naturally, safely, and instantly. That only happens when the backend presentation layer is engineered for real production traffic.

How does your work connect to your broader role in the AI engineering community?

My work is about taking generative AI out of the demo environment and making it reliable at consumer scale. That means designing systems that are fast, stateful, safe, measurable, and maintainable under heavy load. It also means thinking beyond a single launch and building a platform that other teams can extend.

That same engineering standard shapes how I contribute outside my day-to-day engineering work. As a peer reviewer for the 34th European Conference on Information System (ECIS 2026), I evaluate technical work through the lens of rigor, scalability, and real-world usefulness. In my own systems, those principles show up as architecture that can handle every assistant request, support peak shopping traffic, reduce request volume by roughly 2.5×, and give future AI shopping features a stronger foundation.

What is the main lesson from this project for companies building AI assistants?

The lesson is that a better model is not enough. If the system around the model cannot stream, enrich, validate, transform, retain state, and scale safely, the customer experience will fail no matter how capable the model is.

For AI shopping assistants, the presentation layer is not cosmetic. It is the infrastructure that turns generated text into the final answer experience customers see. My work was about building that layer for real consumer-commerce scale, where millions of customers expect the assistant to respond instantly and where every future feature depends on the foundation being strong enough to carry it.