The agent did the errand. The real product question came after.
I build tools for AI agents, so I read the first wave of Meta Muse hands-on tests as product research rather than a launch review. Across 13 reported tasks, the most interesting pattern was not whether Muse could click a button. It was whether the user could trust the information and permissions behind the click.
One test shows the opportunity clearly: Daniel Howley at Yahoo Finance asked Muse to get moving quotes. It contacted four local movers; two called back within about two minutes. That is the sort of tedious coordination people genuinely want to delegate. CNN's Lisa Eadicicco saw a similar result when Muse emailed a friend about a trip, waited for the reply, and turned recommendations into a document.
Then the boundary appeared. In the same round of tests, CNN and Yahoo Finance were directed to restaurants that had closed years earlier. Howley asked for a phone number, and Muse eventually admitted that its first answer was made up. A completed workflow built on a stale listing is still a failed workflow.
For a founder building an agent product, I think this creates three distinct jobs:
- Make the agent useful. Give it access to the real workflow: forms, inboxes, calendars, follow-ups. A chat answer alone cannot replace the back-and-forth.
- Make its evidence inspectable. Before it acts on an address, price, opening hour or phone number, retrieve the current source and show the user where the value came from. Do not let a fluent answer stand in for a source.
- Make its authority narrow. Read access, sending, purchasing and deleting should be separate scopes. An approval screen should show the exact side effect, including details like a tip, rather than only the total price.
The permission issue is not theoretical. Inc. columnist Jason Aten says he declined Messages access, but later found more than 187,000 rows of his message history synced. Meta says Messages requires both macOS Full Disk Access and an enabled connector. The difference between those accounts has not been resolved publicly. If I were shipping an agent with desktop access, I would treat OS permission and in-app connector state as two gates to audit, not one checkbox to trust.
I also noticed an unglamorous test: WIRED's Reece Rogers asked Muse to order breakfast. It built the order, but selected “no tip.” Rogers went to the store instead. That is a small social judgment, yet it shows why “ask before purchase” needs to mean a review of the full action, not a yes/no prompt at checkout.
My practical checklist for any agent feature now looks like this:
• Can the user see the source and timestamp for facts the agent will act on?
• Can they start a connector read-only and upgrade just the one action they need?
• Does an approval show the recipient, exact item or message, price, and any social choices?
• Can they inspect the action log, disconnect the connector and undo the work?
• Does the product say clearly when it cannot do the task it advertised?
At BeatAPI, the slice we work on is giving agents a way to check live web and public social information through an MCP connector before they act. That is only one layer; it does not solve permission design or human approval by itself. The bigger lesson from Muse's tests is that all three layers have to meet at the moment an action is taken.
I put the 13 test cases, what succeeded, what failed, and links to the original reporting in the full guide: https://dev.to/hao_kang_82922526dfe5d934/meta-muse-in-13-real-world-tests-what-to-delegate-what-to-verify-3hlh
If you build agents, where do you draw the line between an action that can run quietly and one that needs a human to review it?
The stale listing problem is the one that scales worst. A hallucinated phone number is one failed task. A closed restaurant served confidently by an agent that completed every other step correctly is a user who stops delegating entirely. The damage isn't the single error — it's the trust withdrawal that affects every future interaction.
We hit a version of this in our own tooling. We use an AI coding agent for most of UtilitySEO's development, and the hardest failures aren't the ones where it breaks something obvious. They're when it acts on a cached assumption — a file that was renamed, a field that changed shape — and completes the task looking correct. Same pattern as your Muse example: the workflow finished, the output looked right, the underlying fact was wrong.
Your checklist point about source and timestamp visibility is the load-bearing one. Every other check degrades gracefully if that one holds.
The three-layer framing is strong, especially separating read, send, purchase, and delete scopes. I’d add a dry-run mode that shows the proposed action and evidence before execution; it seems like a practical bridge between full autonomy and an approval prompt for every step.
Direct answer to your closing question: we draw the line at reversibility, not capability — read-only tools are on by default, and every write/admin verb sits behind an explicit flag, so the agent can look at anything but can't spend, send or delete without a human flipping the switch. The "inspectable evidence" half matters just as much: in our analytics MCP every number has to carry the query it came from, because a fluent answer with no source is the same failure as an invented phone number (that's what we're building at https://amami.dev). For a source that's stale rather than missing, do you show the timestamp and let the user judge, or still block?