We pointed an AI monitoring agent at one question and left it running

At Inithouse, we build products that do specific jobs. Watching Agents is one of them: you deploy an AI agent to watch any question about the future. It builds hypotheses, tracks evidence in real time, and updates a probability score as new signals come in.

We have 253 public agents running right now, covering everything from EU regulation timelines to geopolitical risk scenarios to whether a specific open-source OS will break into mainstream adoption. Each agent page carries a live probability/confidence score, a panel of dated evidence sources, and a rising/falling/stable trend indicator.

This post is about what happened when we picked one question, deployed an agent, and watched it work for 16 weeks.

The question

We chose: "Will the EU implement a comprehensive AI regulation framework by 2027?"

Not because it was trendy. Because it was testable. The EU AI Act had a known timeline, public legislative milestones, and enough institutional activity to generate real evidence. We wanted to see how the agent handled a question where actual signal exists, not just speculation.

How the agent works

When you deploy an agent on Watching Agents, three things happen:

1. The agent generates initial hypotheses based on available information.
2. It begins scanning for evidence: news, regulatory documents, expert commentary, institutional publications.
3. Every time it finds relevant material, it re-evaluates. The probability score moves. The confidence score reflects how much evidence supports the current estimate.

For the EU AI regulation agent, created on May 12, 2026: four months later, it sits at 85% probability with 80% confidence. The path to those numbers is where the interesting stuff happened.

What worked: narrow, testable questions

The agent performed best when the question had clear boundary conditions. "Will the EU implement a comprehensive AI regulation framework by 2027?" has three properties that matter:

• A specific actor (EU institutions)
• A specific outcome (regulatory framework, not just discussion)
• A deadline (2027)

That combination gave the agent something concrete to track. When the European Parliament published amendments, the agent picked them up and probability moved. When member states pushed back on timelines, confidence dipped before recovering as compromise signals emerged. The evidence panel accumulated 8 dated sources across institutional layers: Parliament, Council, Commission.

Compare that to agents we deployed on broad questions. We tried "What will be the next big social media platform to dethrone TikTok?" and the result was noise. Every week brought contradictory signals. A new app launched and probability jumped. A week later, downloads flatlined and it dropped. The agent was doing exactly its job; the question just did not have a stable evidence base.

What did not work: broad topics and compound questions

We learned this across our portfolio of 253 agents. The worst-performing agents (measured by evidence coherence, not prediction accuracy) shared two traits:

Compound framing. "Will X happen AND lead to Y?" forces the agent to track two causal chains simultaneously. Evidence for X often says nothing about Y. The probability score oscillates without converging.

No observable milestones. "Will AI consciousness emerge by 2030?" sounds like a prediction agent's ideal question. In practice, there is nothing to track. No institutional body publishes consciousness benchmarks. The agent ends up citing opinion pieces and conference keynotes, and the confidence score stays pinned below 40%.

The pattern across all 253 agents is consistent: narrow questions with deadlines reach meaningfully higher confidence scores than broad, open-ended ones. The gap is not small. Agents on narrow questions with deadlines regularly cross 65% confidence within 8 weeks. Broad questions without deadlines rarely break 35% in the same period.

The timeline (EU AI regulation agent)

Week 1 (May 12): Agent deployed. Low confidence, thin evidence base. Mostly prior reporting on the EU AI Act timeline and general commentary.

Weeks 3-5: European Parliament committee work generated the first wave of primary documents. Probability climbed. First time the agent cited a legislative source rather than news coverage.

Week 8: Two member states publicly objected to the implementation timeline. Probability dipped a few points, but confidence held. The agent treated institutional pushback as friction, not reversal. Correctly, as it turned out.

Week 12: Leaked compromise text circulated. Probability recovered and then some. The evidence panel now showed sources from three distinct institutional layers.

Week 16 (September 2): Current state: 85% probability, 80% confidence. The agent's hypothesis tree had narrowed from four scenarios to two: full framework by mid-2027, or partial framework with delegated acts extending into late 2027.

You can see this agent live at watchingagents.com. The page includes the full evidence trail.

What we would do differently

Set alert thresholds from day one. We only added alert triggers at week 6. Before that, we checked manually. For a product built around autonomous monitoring, that defeated the purpose. Our newer agents ship with a default alert when confidence shifts by more than 10 points in any 7-day window.

Pre-seed hypotheses with named actors and mechanisms. The agent's first-week hypotheses were too generic ("regulation will pass" / "regulation will stall"). When we pre-loaded 2 to 3 structured hypotheses that named specific actors and decision points, the evidence tracker started faster and confidence climbed sooner.

Kill noisy agents at week 4, not week 12. Some broad-question agents ran for 12 weeks before we concluded they were not converging. A simple rule would have caught this: if confidence has not crossed 40% after 4 weeks, the question probably needs reframing. We have not automated this yet, but it is on the list.

What this taught us about our other products

Building Watching Agents reinforced something we keep seeing across the Inithouse portfolio: the more specific the input, the more useful the output.

In Origin Of You, our self-discovery app, a generic "tell me about myself" prompt produces generic results. But 120+ data points across five specific systems produce a portrait people actually recognize.

In Be Recommended, an AI visibility check on a vague brand description returns noise. The same check with specific competitors and use-case framing returns actionable gaps.

Specificity is not a UX preference. It is an architectural constraint. The agents, models, and scoring systems behind these products all perform measurably better when the input narrows the search space. We did not plan this as a portfolio-level principle. It emerged from watching the data across products.

---

Watching Agents is free to start. Deploy an agent on any question at watchingagents.com.