I let an AI agent run my amazon ads for 2 months, sharing the real numbers (they're not great)
so quick context, I have a small self publishing side business, 10 books on amazon. Managing the ads was eating like 6h a week of my life and I was losing money anyway, so back in may I built an agent on top of claude and basically gave it the keys. It runs by itself every 3 days, pulls the reports from the amazon ads api, changes bids, kills keywords, creates campaigns, whatever it decides. I just get a report.
june numbers: ~$680 royalties, ~$757 ad spend. so net -$75. yeah I know. But before the agent I was losing more AND doing all the work myself, so I count it as progress lol
some stuff I learned the hard way:
the first version almost killed my best campaign. The agent found my winning keywords and "helpfully" duplicated them into a new campaign with higher bids... so it started outbidding itself in the auctions. My main campaign went completly silent for 2 days before I understood what happened. Now theres a hard rule, it has to check every live keyword before creating anything, and duplicates can only bid 80% of the original. Never happened again.
another thing, amazon attribution takes ~3 days to consolidate. If you let the agent judge the last 48h of data it will kill keywords that were actually converting, the sales just hadn't shown up yet. We exclude the last 2 days from every cut decision now.
also I stopped putting rules in the prompt. Everything lives in a yaml file (monthly cap, max bid change per cycle, max new campaigns per 72h...) and the agent reads it before every single action. When I want to change something I edit the yaml, not the prompt. And every action it takes goes to an append-only log, so when a run crashes mid cycle (it happens more than you'd think) the next one picks up where it left instead of doing everything twice.
the biggest lesson though... the ads were never the real problem. The agent optimized its way to breakeven and then just plateaued. Because no amount of bid optimization fixes a product page that doesn't convert, someone on reddit told me exactly this and it hurt because its true. So now I'm shifting it towards the actual product side, price experiments with auto revert rules, listing improvements (it drafts, I approve), that kind of thing.
stack if anyone cares: claude with scheduled runs + python client for the amazon ads api + an open source keyword tool called kdp-scout + yaml guardrails + jsonl changelog.
happy to share more details on any of it, the guardrails file, the dedupe logic, whatever. curious what others would do here, keep pushing the automation or admit that at this scale manual would be fine too?
To your closing question, the split that decides it is not automation versus manual, it is which parts of the 3-day cycle have a right answer. Pulling reports, the 80 percent duplicate-bid rule, excluding the last 2 attribution days, enforcing the caps, all of that has a right answer, so it should be plain code that runs identically every cycle with no model call in the loop. The model earns its keep only where you would also have to think, which keyword themes to try next, how to shape a price experiment, and those already flow through your drafts-I-approve gate. Your yaml-not-prompt move is the same principle applied to config, and pushing it through the rest of the pipeline is what makes the agent cheap and boring enough that manual stops being the alternative worth debating. It also kills the self-outbidding class of bug at the root, because a deterministic step cannot get creative with your winning keywords.
The full yaml I run for this shape of pipeline, budgets, timeouts, approval gates included: https://gist.github.com/renezander030/3725d3231a18e7121b7b6a2d5c...
The most useful part here is that you separated the agent's decision rules from the prompt.
For anything touching spend, I would want three guardrails before trusting it: a hard budget cap, delayed attribution windows so it does not kill good campaigns too early, and an append-only action log that makes every bid change explainable later.
When you move into price experiments, are you planning to optimize for royalties/ad spend only, or for a broader unit like profit per book after ads?
good question. Optimizing for net profit = royalties (ebook + paperback + KU page reads) minus ad spend, not just the ads line. KU pages was actually the eye opener, amazon's ads console doesn't see them at all, so a campaign can look like its losing money while the book earns fine through reads. The price experiments follow the same logic: the agent recalculates royalty-per-unit at the new price and judges after 14 days vs the old baseline, with an auto revert rule if units drop >30% without the margin making up for it. Max 2 experiments at a time so attribution stays readable. Curious what your third guardrail was btw, the preview cut it off — I have hard monthly cap, per-cycle change limits and an append only log, always looking for what im missing.
The third one I would add is a source-of-truth mismatch hold.
If the agent's optimization view is missing a revenue source or attribution is delayed, it should not be allowed to increase spend as if the ledger were complete. Hard cap + per-cycle limits protect the budget; this protects decision quality.
In your KU example, scale-up pauses until the revenue window catches up or a human approves it. The dangerous failure is not only a bad campaign. It is confidently optimizing against a partial picture.
The line about the ads never being the real problem hits close. Same pattern shows up in video ad creative: a bad hook kills traffic before your product page ever gets tested, but a great hook can't fix a page that doesn't convert either. Worth separating which one you're actually debugging before you touch either side.
ai agents are not a source of truth there just really advabced patternmatches when you ask it to complex tasks it struggles
This is honestly one of the more real use cases of “AI agents” I’ve seen — not hype, just replacing actual repetitive work.
The self-competing keywords bug is brutal but also super insightful. That’s exactly the kind of thing that doesn’t show up until you let something run autonomously. The YAML guardrails + append-only log setup is really solid too — feels like you’ve moved from “prompting” to actually operating a system.
Also agree with your biggest takeaway — ads optimization hits a ceiling fast if the listing doesn’t convert. That plateau you hit is pretty much proof the agent did its job.
At your scale, I wouldn’t roll back to manual. You’ve already offloaded the annoying part. Makes more sense to keep the automation and shift your effort to inputs that actually move the needle (listing, pricing, positioning).
Overall this feels like the right balance — automation for execution, you for strategy.
This matches what I keep running into. Agents get dangerous exactly where you hand them an open ended goal plus the full action space. Your best campaign got wrecked because nothing stopped it from competing with itself, not because the model was dumb.
The version that works for me is boring on purpose. One narrow job, a short list of actions it is allowed to take, and a rule it can never break, like never touch a campaign that is already profitable. Everything else stays a suggestion in the report that I approve.
Also worth checking, does the agent actually beat a dumb baseline. If a simple fixed rule (pause anything over target ACOS, nudge bids 10 percent) gets you close to the same result, the agent is adding risk without much edge yet. The net -75 is not the real signal. Decision quality versus that baseline is.
honestly the baseline question is the one that keeps me up at night lol. Fair hit. I don't have proof it beats a dumb rulebook yet, so I'm adding exactly that: once a month I'll replay the period with the simple fixed rules (pause at 15 clicks/0 sales, bid down at acos>100%, etc) vs what the agent actually did, and post the delta. My suspicion is the honest answer will be "the agent's real job was DISCOVERING the rulebook". Half my yaml exists because the agent screwed up once and we wrote the lesson down. Maybe the endgame is the rules run alone and the agent only handles the weird cases. What fixed rules would you put in the baseline?
The "rules live in a yaml, not the prompt" line is the most underrated thing here. I run voice agents in production and learned the exact same lesson the hard way — the moment behavior lives in the prompt, every tweak is a dice roll. Config file + append-only log is the difference between a toy and something you can leave running unattended. On your actual question: at 10 books, manual probably matches the ROI honestly. But you're not really building this for the ad margin — you're building the muscle for when it's 100 books, or a client's account, where manual truly doesn't scale. That's when the guardrails you're writing now pay for themselves. The plateau insight (optimization can't fix a page that doesn't convert) is the real gold, though — that applies to almost every "AI will fix it" pitch out there.
Really interesting build. I think the biggest takeaway wasn't the automation—it was realizing the bottleneck had shifted from ad optimization to conversion.
Also love the YAML guardrails approach. Feels much safer than burying all the logic in the prompt.
Definitely checking this app out ill leave feed back after review
The attribution lag lesson is huge. We had the exact same problem with Apple Search Ads — our automation was killing keywords that were actually converting because it was looking at the last 48 hours instead of waiting for the full attribution window. We ended up excluding the last 3 days from any negative decision. Small change, massive difference.
The plateau you describe is the most important takeaway here. Once your ads are reasonably optimized, every incremental dollar in optimization is worth less than a dollar spent improving the thing being sold. We only broke through our own plateau when we shifted focus from bid optimization to landing page conversion and pricing experiments.
The yaml guardrails approach is smart too. We went through the same evolution — prompts are too fragile for operational rules. Having a structured config file that the agent reads before every action makes it auditable and the rules don't drift when you update the prompt for something else.
One thing that helped us with the 'outbidding yourself' problem: we added a global view of all active bids per keyword across all campaigns as a pre-flight check. If the agent can see the full picture, it stops making locally optimal but globally stupid decisions.
the global bid view is a great idea, stealing it. Right now my dedupe check only fires before creating keywords, but you're right that the agent should see the full bid map before ANY decision, not just creation. Locally optimal but globally stupid describes exactly what mine did in june lol. Question, when your preflight catches a conflict does it hard-block the action or just flag it for review? And interesting that you landed on 3 days for ASA, we're at 2 for amazon and I still catch it occasionally judging something too early. The plateau point is where I am now, moved the whole focus to pricing + listing conversion this month, will share numbers when I have them
The part that stood out to me wasn't the automation—it was the point where the agent stopped improving results.
That's an interesting boundary. It suggests the bottleneck moved from campaign optimization to offer optimization. Once acquisition becomes efficient enough, the highest-leverage decisions often shift to the product page, pricing, and positioning rather than the ads themselves.