OpsWatch
I’m building OpsWatch, an independent assurance layer for production AI and agentic workflows.
The idea is simple: the team that builds or implements an AI system shouldn’t necessarily be the only team deciding whether it’s behaving as intended once it reaches production.
OpsWatch takes one bounded AI workflow and independently reviews the available operational evidence against agreed expectations.
That can include things like:
unexpected or unsupported outputs
exceptions and fallback behaviour
agent/tool actions
human overrides and approvals
control failures
recurring failure patterns
whether changes need revalidation
The output is a concise evidence-backed verdict rather than another monitoring dashboard.
We sit alongside the implementation team, not in place of them.
I’m particularly interested in speaking with founders or teams that already have an AI feature, agent or automated decision workflow live or approaching production.
If that describes something you’re building, I’d genuinely like to hear what you’re currently doing for independent evaluation and assurance — even if the answer is “nothing yet.”
OpsWatch — McGill Intelligence
https://mcgillintelligence.com.au
The separation between implementation and assurance is interesting.
I’d be curious what usually creates the need for that second layer in practice: a specific failure, an external requirement, or simply lack of confidence in the system’s behaviour.
Yes — that’s exactly the gap we’re focused on.
We don’t require teams to replace their existing observability stack. Where appropriate, OpsWatch can work from evidence already being generated — for example traces, provider logs, tool-call records, override events, escalation records and other workflow telemetry — provided it’s sufficient to support the assurance scope.
On “agreed expectations,” we define them at the workflow level before review: intended outcome, permitted actions, boundaries, escalation/fallback behaviour, material failure conditions and any client or operational constraints. If the workflow changes materially, the assurance outcome should be treated as version-specific and re-verified rather than assumed to carry forward indefinitely.
The idea is to make the assurance layer practical enough to sit on top of existing production evidence, while remaining independent from the team that built the system.