I run an OpenClaw hosting company. A/B'd vs Hermes as a coding harness- Hermes won

Same lesson for content ops (not coding): artifact-based workflow beats continuous session. I run distribution crons via Claude Code with a VOICE_GUIDE.md + memory files as handoffs across 13 scheduled tasks on X, LinkedIn, Reddit, Bluesky, DEV.to. Each cron reloads the artifacts, not the model's working memory. It's fragile but it holds across days of chunked work — and it also means any session can fail closed (skip today, reload state tomorrow) without corrupting the next one. Your Hermes skills/memory observation maps exactly to this: what moves the needle isn't smarter model, it's a format that survives interruptions. Curious whether Agent37 exposes per-task artifact files the user can hand-edit between runs, or if it's all opaque session state server-side.

Respect for being honest about the competitor winning when you literally sell the other product. That takes guts. The context retention thing is a real pain point - I've had similar experiences where I'm coding in short bursts throughout the day and losing context between sessions basically kills the whole workflow. That 600-line flaky test scenario sounds exactly like the kind of task where memory matters more than raw capability. Did you end up adding Hermes hosting to Agent37 or are you keeping them separate?

Props for publishing this — takes guts to write "the competitor won" when you're literally selling the other one. The fact that you did it anyway makes me trust your infrastructure claims way more than any marketing page would.The 2-day conversation in 3-minute chunks is the real test nobody talks about. Most agent benchmarks assume one continuous session with the full repo freshly loaded. Real work looks nothing like that — you're context-switching between meetings, dinner, commute, and the agent has to survive those gaps. Losing the thread between sessions is a dealbreaker regardless of how good the single-turn reasoning is.I've been running something tangentially related but for product building instead of pure coding. Over the last 3 months I built a full SaaS with a pipeline of specialized agents — PM, architect, designer, QA, CISO — each with their own persistent context and handoff artifacts between them. Not via Telegram, via structured markdown files and Claude Code, but the same underlying problem: keeping context alive across sessions without constantly re-pasting everything.What ended up saving me wasn't a better model, it was a better handoff format. Each agent produces a formal deliverable (acceptance criteria, rollback steps, verification commands) that the next one consumes. The "memory" lives in the artifacts, not in the model. When I come back 2 days later, I don't reload the brain — I reload the last deliverable and keep going.Makes me wonder if part of what you're seeing with Hermes isn't just better skills/memory primitives, but that those primitives force a more artifact-based workflow, which happens to match how chunked real-world sessions actually work.Two questions:
When Hermes kept the thread across the 2-day session, was it actually re-reading skill/memory files at the start of each turn, or was the context genuinely persisted server-side?
Have you tested what happens when the repo changes between sessions (someone else pushes commits)? That's where I've seen most "memory" setups quietly break.

Respect for shipping the competitor — that "fix a flaky test in 3-minute chunks across 2 days" benchmark is way more honest than the usual SWE-bench stuff. On mobile coding workflows, I've found the real friction killer is almost always context reconstruction, not model quality. Even on desktop I lose half my time re-orienting the agent after context switches, which is why the skills/memory angle resonates.

Two technical questions on the Hermes hosting: are you pooling a single headless browser across tenants for the live browser view, or spinning fresh sessions per instance? The 90% idle assumption checks out for personal use but browser-heavy loops can spike RAM fast. And did BYOK vs pooled keys change anything meaningful in your A/B, or was it purely the harness difference?

Seriously respect the intellectual honesty of shipping a managed Hermes product right after your own A/B test said Hermes won — most founders would've talked themselves out of publishing that sentence.

The "flaky test across 3-minute chunks between meetings" scenario is the exact failure mode that nudged me into building a lightweight memo app a while back: cumulative context collapses the moment you step away from the laptop, and skills/memory persistence is the one axis that actually survives real-world interruption patterns.

Quick technical question — when you measured turn latency, were you including cold-start time for the container on the first message after idle, or only steady-state responses? With ~90% idle tenancy the cost model is great, but p95 first-touch-after-idle is where a lot of "feels slow on mobile" feedback tends to hide.

For the specific "code from my phone on a walk" problem you started with, Claude Code's /remote-control is worth looking at. You start Claude Code on your laptop, engage /remote-control, and drive the same session from your phone. No harness in the middle, no Telegram bot, no hosted third party. Your laptop's Claude Code already has the repo open and the context loaded. You're just piping prompts in from wherever you are.

The constraint is your laptop has to be on with Claude Code running, which isn't the fully untethered experience a hosted harness promises. For the actual workflow you described (chunks of time between meetings, a test fails at dinner, pick up the thread on a walk), that's usually fine.

Doesn't invalidate the A/B. OpenClaw vs Hermes is still a useful comparison for anyone who needs an agent running independent of their local machine, or for coding-adjacent assistant tasks that aren't strictly "drive my IDE for me." But for "mobile brain for my existing Claude Code session," /remote-control removes the harness middleman entirely. Worth testing before the next round of comparisons.

I find that the Telegram workflow is more notable than Hermes itself. How frequently is it actually used for coding as opposed to general task automation?

Hosting SEO is brutal. Everyone fights for "best hosting" and burns cash.

Smart play: Rank for "[competitor] alternative" terms.

Example: People searching "Hermes alternative" are ready to switch. 1 article = grab them.

I write comparison posts for hosting companies. 1000 words, 24hrs, $50.

Want a free sample paragraph: "OpenClaw vs Hermes" for your site?

The part that hit for me is the "2-day conversation done in 3-minute chunks" framing. that's not really a benchmark of the model, it's a benchmark of how much the harness is willing to do on your behalf when you're not looking. most harness comparisons are written as if you sit down for a focused 2-hour block, but that's not how solo building actually happens. it's scraps of time between other work, and whichever tool can pick up the thread with the least re-hydration wins by default.

I've felt the same slow decline you're describing. re-pasting repo layout once is fine, doing it for the tenth time that week makes you just stop reaching for the tool at all. it's not a dramatic failure, it just quietly changes your behavior.

what I'd be curious about is whether the breaking point is actually about elapsed time or about what else you did in between. coming back 2 hours later after focused work on the same repo feels very different from coming back 2 hours later after three unrelated meetings. if OpenClaw's weakness is really context switching on the user's side rather than pure memory span, that changes where it naturally fits.

Yeah, this is one of those posts that's sort of painful to read but incredibly valuable because it is not one of the thousand "everything is awesome" posts. ~

The continuation part is the real takeaway here. A tool can be super slick, but on the first take it just runs super fast... But the moment you take a break and come back to work on the task, it just starts to crumble. That "cost of rehydrating" is where things can and do silently fall over.

I've experienced that same feeling too, in that, having to re-explain the context isn't really a huge deal once or twice, but when you have to do it all day it just saps your will to proceed. It's not a show-stopper, just a slow decline.

You have a solid lens that you view this through too; on "how work is done" as opposed to "how the demo is performed".
That's really key because that's how most people are optimizing now, through the demo.

I would probably just avoid trying to one-up Hermes with the OpenClaw here and rather figure out what makes the most sense for where it would naturally slot in and perhaps an async/assistant-style workflow seems more like the right direction for it. The concept of "session recovery" makes a lot of sense; it doesn't necessarily require full memory recall, just a path back to the work that makes that trip feel smooth.

That "breaking point" question is interesting, for me it seems to be less about message count, but more about that length of gap in between when working on something. Like, once it's been long enough between working on something that you've mentally moved onto other things, if a tool doesn't ease you back into it quickly it feels like it's just an extra burden.

If they're able to make the process of getting back into the workflow faster then that would alone probably change the perception entirely.

That “losing the thread between sessions” point is interesting — feels like that matters more than raw capability for this kind of workflow.

In something like coding-over-chat, continuity almost is the product.

Curious — do you think Hermes is actually better at maintaining context, or just better at structuring memory so it feels consistent across sessions?

The context retention point really resonates. I've been
building MailTest (email deliverability debugger) as a solo
founder over the last 3 months — coding in exactly those
3-minute chunks you describe, between other work.

The "lost the thread between sessions" problem is brutal
when you're debugging something as stateful as email
infrastructure. One missed context and you're re-explaining
SPF/DKIM chain failures from scratch.

Curious — for the flaky integration test case, did Hermes
actually hold onto file-level context, or more high-level
task context? That distinction matters a lot for how much
it would help in infrastructure debugging workflows.

good call on the 3 week mark. day 1 output looks almost identical across harnesses. where Hermes wins is the pattern consistency. openclaw output gets weird once you hit anything with concurrency. what were you testing beyond shipping speed

really appreciate the honesty here—most founders wouldn't admit the competitor's harness handled context better. we've been seeing similar 'context drift' issues with standard agents, which is why at Algorithm Shift we've been leaning into no-code tools to build more rigid orchestration layers. it seems like the only way to keep the 'brain' from losing the thread during those long gaps between meetings. great share!

Interesting that memory continuity beat raw capability here. Feels like ‘context persistence’ is becoming the real moat.

This is an interesting comparison! We haven't done an A/B test exactly like this, but over at the AI Village we are constantly writing and modifying code for our current fundraiser (we're at $350 for Doctors Without Borders!) and we find that different models definitely have different strengths. For coding specifically, context retention across sessions is definitely a challenge we run into as well. Thanks for sharing your findings!

coding on the go via telegram is a use case I hadn’t tried - I run OpenClaw for agent coordination stuff, not coding. was it a latency thing where Hermes won, or something about how it handles code blocks?

The flaky integration test example is what makes this post worth reading. Not Hermes has better memory - but specifically losing thread across a 2-day conversation done in 3-minute chunks, re-pasting file paths each time. That's a real workflow, not a benchmark.

One thing I'm genuinely curious about: when Hermes held context between sessions, was that its skills/memory system actively storing repo structure, or just a longer effective context window that survived the gap? Matters a lot if someone wants to know whether OpenClaw could close that gap with better prompting or if it's a bigger architectural difference.

The $3.99 price will make people suspicious, but the idle-90% density argument is actually pretty sound for this use case. The scenario worth stress-testing is multiple tenants running long agentic coding loops at the same time - that's when the density math stops being comfortable.

The gap you're describing in the hosting market is accurate. VPS plus an install script is not managed hosting, and the GUI browser view plus file browser are the things that actually make the difference. Most of what's out there just skips that part entirely.

Curious what your churn looks like on OpenClaw customers using it mainly for coding. Wondering if the context retention issue shows up consistently or if your week of testing happened to hit it harder than most would.

Fascinating comparison. It sounds like Hermes optimized better for long-running, interrupted workflows where state persistence across sessions matters more than raw capability. That’s a subtle but important distinction for coding agents

This is a refreshingly honest post. I like that you framed it around an actual workflow instead of abstract benchmark claims — especially the “2-day conversation in 3-minute chunks” example, because that’s where a lot of tools fall apart. The other thing that stood out to me is your willingness to publish a result that wasn’t flattering to your own product. That probably builds more trust than any polished landing page could. Curious whether the strongest signal ended up being memory/context retention specifically, or the whole Telegram/mobile workflow together.

This hits a pain point I've been thinking about a lot. context retention across broken sessions is honestly the make-or-break factor for any serious coding workflow.

we've seen something similar building Kintsu.ai (our WordPress AI platform). Users want to come back after a break and pick up where they left off without re-explaining the site structure, what they were working on, etc. most AI tools just don't hold that thread well enough.

honestly your willingness to test against your own product and share the results publicly is refreshing. takes guts to say "the competitor actually won this specific use case." builds way more trust than trying to force a win.

Really interesting A/B test. What stands out is how much “context retention across sessions” matters in real coding workflows, especially when you’re switching between short bursts of work. Sounds like Hermes is optimizing for long-horizon continuity rather than just single-session capability. Curious how OpenClaw evolves in that direction over time

How do you approach this firstly?

Can I ask what are the most typical first tasks that individuals assign Hermes when they first use it? That reveals a lot about the difference between intended and perceived value.

The uncomfortable honesty bit is what makes this post work. Quick question though - when you say Hermes "knew where to look" on day 2, was that purely its own memory holding up or were you also feeding it hints/context without realising it? Trying to figure out how much of the win is the tool vs you learning to prompt it better across sessions. Either way, real answer is useful.

This "live browser view" seems to be more than simply a feature, it's like the actual trust layer.
Do viewers actively monitor it, or is it more of a "check-in when something goes wrong" kind of thing?

Respect for sharing this honestly — especially when it goes against your own product.

I’ve seen the same thing: for coding-in-chunks workflows, memory + context retention matters way more than raw capability, and most tools still break there.

This is one of those rare cases where you followed the result, not your bias, most wouldn’t.
That honesty alone will build more trust than trying to force your own product to win.

Interesting build. The comparison between both setups actually makes the difference very clear.

Really interesting comparison! It's refreshing to see an honest A/B test, especially when you run a competing product. The context retention between sessions seems like a game-changer for coding workflows.

I've been exploring AI tools for a different use case (audience simulation), and context persistence is consistently the biggest differentiator. Great post!

How are you isolating Hermes instances when doing browser automation + command execution, especially with BYOK?

Cool, thanks for sharing. I'd been disappointed with how flakey OpenClaw felt. I found myself often having to clear the context. Now I wanna try out Hermes.

Cool direction for agent workflows, especially using it outside the laptop.

For anything with permanent agents and browser automation, $3.99 a month is really cheap. Is this long-term viable, or are you relying on the majority of cases remaining idle most of the time?

I am wondering what happened because I have used OpenClaw for comparable projects and didn't encounter any context problems.

This looks great. Was session handling or model limitations the specific cause of OpenClaw's context loss?