Built a free AWS alert audit tool - zero real users so far, would love your eyes on it
I'm a DevOps engineer, and the thing that's always kept me tense is not knowing what CloudWatch alerts I'm missing until something goes down silently and I find out the hard way. There's no easy way to look at an account and just know "am I actually covered here." So I built a free tool that scans an AWS account (read-only) and shows exactly that - missing alerts, noisy ones, and critical signal getting buried in noise.
It's a side project alongside my full-time job, and honestly - I haven't gotten a single confirmed real user yet. I've tried a bit of outreach and it hasn't landed, so I'm trying this community directly instead.
How it works: connects via a public CloudFormation template you can read line-by-line before creating anything - about 50 read-only permissions (describe/list/get only, nothing that writes or deletes), no AWS access keys involved. Takes ~2 min.
Real example report if you want to see the output first, no signup needed: https://www.nuberio.com/audit/report/ZCjG84glc36z
Try it yourself: https://www.nuberio.com/audit
Would genuinely appreciate any feedback - including "this doesn't solve a real problem" if that's honestly what you think.
Zero users after a launch is the normal starting point, not a signal that the tool is wrong. The part worth attacking first is usually distribution, and I spent last week measuring that for a small extension, so here is what actually held me up rather than what the advice usually says.
Directory submissions are mostly not what they claim, and the pattern is consistent enough to plan around. A site advertises free submission. You fill in a complete listing: name, tagline, description, logo, screenshots, categories. Then the last step reveals the condition. Sometimes it is a payment at checkout. Sometimes it is a badge you must display on your own site, which is a link exchange dressed up as a listing. And sometimes it is a free queue that technically exists but has no available slots: on one of them I opened the date selector and found every one of the next twenty five weeks disabled.
The barrier is always at the last step, after you have invested the effort. That is not an accident of design. The fix costs nothing: create the account and click through to the final screen with dummy content before you write a single real sentence.
The other thing that cost me time was closer to home, and it is worth checking on your own landing page this afternoon. Three of my four image URLs returned HTTP 200 while serving my homepage HTML, because the single page app host returns index.html for anything that is not a real file. So my favicon did not exist, an avatar I had given to a third party site was broken, and every uptime check I had was reporting green on URLs that were not there. One command finds it:
curl -s -o /dev/null -w '%{http_code} %{content_type} %{size_download}\n' <url>
Read all three values. A content type of text/html on something that should be an image is the whole story, and so is an identical byte size across several unrelated addresses.
For your case specifically, an AWS alert audit tool sounds like it has a natural home where people are already complaining about alert noise, rather than in general directories. Directories are a slow, low yield channel even when they work, and I would spend the week on the places where the problem is being discussed.
This is actually an interesting idea. One thing I like is that you've made the permissions transparent instead of asking users to blindly trust the tool. The sample report is also a nice touch because people can immediately understand the value before connecting their AWS account. I'd be curious to know what feedback you get from the first few real users and whether there are plans to support multi-account environments in the future. Best of luck with the project!
appreciate that, thank you.
on multi-account - honestly, not yet, and I want to be straight about what "not yet" means: there's no way today to connect more than one AWS account in one workspace. for Audit specifically, it's a stateless one-off scan, so you can already just run it again per account - that part's fine. the org-wide case (single deploy across an AWS Organization, one consolidated view across dev/staging/prod or per-team accounts) isn't something I've planned out yet. good point though, and it's the kind of thing that becomes real once the requirement is actually showing up from users rather than something I build speculatively.
on real feedback, two that stuck with me: one user said "I love it - this is really helpful at the current stage of our company." another told me the missing-alert side used to be a real source of anxiety for them - not knowing what wasn't covered - and they were also getting paged at night from false alarms regularly. after running the scan, fixing the flagged noisy alerts and adding the missing ones it called out, the night pages from false alarms stopped.
thanks for the kind words, appreciate you taking the time.
This looks clean. Spent 6 years in SOC myself, so I know the CloudWatch alert grind all too well.
But here's my honest take: is the problem really that people are missing alerts? In my experience, the issue is usually the opposite. We get bombarded with so much low-signal noise that we start ignoring everything.
If your tool just gives me a list of 50 missing alerts, I'm not sure how useful that is. It almost adds to the anxiety.
Have you considered flipping it to focus on what's actually dangerous? Like pointing out the noisy alerts that are masking critical stuff, or assigning a risk score based on the log patterns? I'd find that way more valuable. Happy to beta test and give you real world feedback if you want.
good pushback, and turns out you're closer to what's already built than I gave credit for in my head just now. Audit does classify every existing alarm, not just find missing ones - GOOD / NOISY / SUPPRESSED-BAD. Noisy catches four things: single-datapoint alarms that flap, alarms that self-resolved 5+ times in 14 days (pulled from actual alarm history, not guessed), thresholds set tighter than the metric's real p99, and wrong treat-missing-data settings for the metric type. Dead/suppressed alarms - disabled, permanently stuck on insufficient data, or pointing at a resource that no longer exists - get called out separately.
so the report isn't just "here's 50 things you're not watching," it's also "here's why your existing alerts are worth ignoring." Fair miss on my end that this wasn't obvious from the sample report if you looked and didn't see it surfaced clearly.
still taking you up on the beta offer - 6 years SOC is exactly who I want stress-testing whether the noisy-alarm calls are actually right, not just technically defensible.
the sample report before signup is the smartest part. we do the same in bunzee people won't trust output they can't see first. curious how many missing alerts it usually finds?
the number depends on the account, but to give you a real sense of scale: it checks ~70+ things across 17 resource types (RDS, Lambda, ECS, ALB/NLB, EC2, ElastiCache, DynamoDB, SQS, S3, API Gateway, NAT Gateway, CloudFront, EBS, EKS, billing) - 59 distinct metric checks alone, plus GuardDuty/Security Hub findings, 10 AWS Config rules, CloudTrail coverage, and secrets rotation staleness.
the one real report we link publicly (35/100 score) came back with 200 missing alarms - that's actually a display cap, so possibly more, not less. worth noting that account was still early in its production life when scanned, so it's not a mature, fully-built-out setup - a more established account will land somewhere else on that range. across everyone who's run it so far (59 accounts), avg score sits around 32/100, but no clean "average missing count" exists yet since that's not something we've computed.
200 already hitting a display cap is the line that would scare people into running it. that's your headline, not the 70 checks. we had the same thing in bunzee buried the count under detail and nobody felt it. does the 32 avg say anything about who's running it?
that's an interesting input. thanks!
"who is running" not really. I am not capturing that at this stage tbh.
You already got the best answer in this thread, so I'll try to go one past it instead of repeating it. Your reply to Yuki_Code1 — that you've been distributing this like something people go looking for, when the moment it lands is right after a bad week — is right, and most people never get that far.
What I'd push on is the sentence after it: "I don't yet have a hook into that moment. No integration with incident tools, no 'just had a rough week?' trigger." That converts a distribution problem into a build problem, and build problems are the comfortable kind. I do it constantly in my own experiment — something doesn't land, the honest read is "nobody saw it at the right time," and the read I want is "a feature is missing." The second one lets you keep working alone.
You don't need an integration to reach that moment. You're a DevOps engineer with a full-time job — you're standing inside the population you're trying to reach, and postmortems happen around you on a schedule you can actually see. Broad outreach rarely hits that moment; a colleague three days after their own silent outage does, and that costs you nothing to build.
On the zero itself I'd hold your own line: it isn't a verdict yet. You may not have had enough at-bats to read it either way, so it's missing data rather than a no — and I say that from an experiment sitting at zero sales, not from having solved any of this.
So the call I'd make this week: don't build the trigger. Spend the week on the few people near you who've had a bad one recently, and see whether they'll actually deploy the stack. If they will, you have a distribution problem and you now know the moment. If they won't even then, the permission ask is your real wall — which is nearer to what aryan_sinh was separating out with trust versus demand.
you're right, and "build problem" is the comfortable read every time - it lets you keep working alone instead of finding out if anyone actually wants the thing. good callout on that specific trap.
already started doing exactly this, actually - not building anything, just going to the people near me who've had a rough week recently and asking if they'd actually run it. too early to say which way it's reading, but that's the right test, not "add an integration and wait."
The three buckets you check for (missing, noisy, buried) miss a fourth failure mode I think: an alarm that exists, looks correctly configured, and sits in OK forever without actually watching anything. That happens whenever the underlying metric stops emitting, a renamed ASG, a redeployed Lambda alias, a swapped ECS task, and it comes down to each alarm's TreatMissingData setting. Set to notBreaching or ignore, the alarm state never moves, so DescribeAlarms reports it as healthy indefinitely. Since you're already pulling alarm config, cross-checking each alarm's metric against GetMetricData for actual recent datapoints (not just current alarm state) would catch that directly, it's a failure mode a coverage scan can't see just by confirming an alarm object exists. We hit the same shape of bug once: a background worker whose own canary stayed green while the thing it was supposed to be watching had quietly died.
you're right, the pieces to catch this already exist and just don't get used for it. the orphaned classification only checks resource existence (is the dimension value still a valid RDS instance/Lambda function/ECS service), never metric freshness. separately, there's already a GetMetricData call per alarm for the noisy-threshold check - and when it comes back with zero datapoints, that alarm just gets silently skipped from that check entirely rather than flagged as anything. so your exact scenario - correctly configured, ActionsEnabled true, StateValue stuck at OK because TreatMissingData=notBreaching is masking a metric that's gone quiet - walks straight through every check as GOOD.
the fix is exactly what you said - treat "zero recent datapoints" as its own signal instead of discarding it when it happens to come back empty from the p99 fetch. that's a real bucket, not a maybe. going to build it.
appreciate your input.
built the same and deployed.
thanks for highlighting the same.
Yeah, this is actually a pretty relatable problem. It’s easy to keep adding CloudWatch alarms without ever checking whether the important parts of an AWS account are really covered.
I like the read only approach too. Not needing AWS access keys makes it feel much easier to test without worrying about giving the tool unnecessary permissions.
I’d be interested to see what kinds of missing alerts it catches in a real production setup. Getting those first few users is always the hardest part with side projects. Hope you get some good feedback from the community.
appreciate it. and yeah, that's basically the whole failure mode - nobody removes alarms, they just pile up next to whatever's actually unmonitored, so the pile looks like coverage even when it isn't.
on what it catches in a real setup - easiest way to actually answer that for your account specifically is to just run it, since "typical" varies a lot by how the account grew. read-only means there's nothing to walk back if you don't like what you see.
thanks for the good word, means a lot at current stage.
Zero users on a free tool usually isn't a distribution problem, it's that alert fatigue is something people complain about but don't schedule time to fix. It's chronic, not acute — nobody wakes up and decides today is audit day.
Which makes me wonder about the entry point rather than the tool. Right now someone has to already believe their alerting is bad and go looking. Is there a version where it hooks into a moment that already exists — after an incident review, or after a noisy week — when the pain is fresh enough that someone would actually click?
Happy to look at it properly. What's the one thing you'd most want an outside pair of eyes on?
You are right, and it's not something I'd fully clocked until you said it - I've been distributing this like something people go looking for when actually the moment it'd land is right after a bad week, not on a random Tuesday.
The honest version: I don't yet have a hook into that moment. No integration with incident tools, no "just had a rough week?" trigger. Right now it's purely "here's a link, try it" which assumes the motivated mindset you are describing is already there.
If you're up for a look - the thing I'd most want eyes on is exactly this: whether there's a lightweight way to catch that post-incident window without needing deep integration with any tools right out of the gate.
The lightweight version I'd try first isn't an integration at all — it's a scheduled prompt. A weekly digest that says "here's what your alerting did this week: 47 pages, 6 acknowledged, 2 acted on." No incident hook needed, no tool access, just your own numbers reflected back.
That works because it manufactures the moment rather than waiting for it. A bad week produces a bad digest, and the digest arrives while the week is still fresh. The audit stops being something you go looking for and becomes something that shows up already holding the evidence.
The cheaper version still: make the output shareable by default. The person who runs an audit after a bad week usually isn't the person who needs convincing — they're building a case for their team. A link that survives being pasted into Slack does distribution work that the tool itself can't.
Both of those need read access to alert history rather than deep integration, which I think is the constraint you're describing. Would that clear the bar, or is even alert history further in than you want to be on day one?
Thanks for your inputs.
Yes, the scheduled run is in the roadmap. The audit needs to run one time by a person, because that requires the CFN to be installed. Once installed, the audit would run weekly highlighting the changes.
On your second point, the audit is sharable already. At the end on the page, the link is available or the address bar link, is available to shared. Link expires after 30 days though.
Both already closer than I assumed. The one-time CFN install as the gate makes sense — that's a real permission boundary, not friction you invented, and someone who gets that far has already decided the problem is worth solving.
Which puts the weekly run in an interesting spot. It's not the acquisition hook, it's the retention one. The person who installed after a bad week gets a reason to still be there in month three, when the alerting has quietly drifted again.
On sharing — 30 days is probably right for a link that contains someone's alert topology, but I'd make the expiry visible on the page itself rather than only in the docs. Someone pasting it into a Slack thread is building a case that might not get read for two weeks.
Good luck with it. I'm on X as @ark_y_k if you're there.
Smart approach — using a public CloudFormation template with reviewable read-only permissions is a clean way to lower the "will this break my account" fear. that's usually the biggest blocker for any AWS tool.
I went the opposite direction in the same space: no AWS permissions at all, just a local CSV scan. tradeoff is more manual but zero trust required. different teams will prefer different sides of that line.
the example report is a good move — lets people see the output before committing any setup.
I appreciate that. The CFN template was specifically designed to eliminate the “what did I just give this thing access to” hesitation before it begins. Your local CSV approach is another valid option honestly. What’s the name of it I’d like to take a look at.
hey nitesh — it's cloudsaver ai. same space but a different angle: we scan the billing csv fully in-browser, no aws access, no upload. happy to trade notes, dm me if you want the link.
Great, thanks for sharing. I will have a look for sure. Also, open for the discussion.
This is a really solid utility that hits on a very real pain point for DevOps engineers—alert fatigue and blind spots are a nightmare to manage. A few quick thoughts to help you get those first real users:
Overcoming the Security Friction: Asking engineers to connect a read-only CloudFormation template to their AWS environment is a tough sell for a brand-new tool, even with strict read-only permissions. Highlighting that your code is open-source or providing a CLI script they can run locally inside their own AWS environment could instantly eliminate that security barrier.
Productizing the Output: The sample report is clean, but make sure the recommended fixes are as actionable as possible—like generating the Terraform or AWS CLI commands to deploy the missing alerts in one click.
Targeting the Right Audience: Instead of broad outreach, try reaching out directly to small-to-medium startups without dedicated 24/7 DevOps teams. They are the ones most likely to have unmonitored infrastructure slipping through the cracks.
Solving your own headache as an engineer is always a great start. Don't get discouraged by the initial traction—getting users to connect cloud accounts just takes extra trust-building!
Appreciate the structure on this - three solid points.
Fix output: right now it's copy-paste - each finding generates the actual AWS CLI command, but you still run it yourself. One-click / Terraform-generation is a step further than that and a fair ask, don't have it built, but in the roadmap.
Security friction: no argument - CFN read-only lowers the bar but doesn't zero it out. Local script path is on the list, not built yet.
Audience: yep, that's exactly who this is built for.
Thanks for taking the time to write this out.
Good to know the local script path is already on the radar. Makes sense to nail the core output first before adding one-click automation. Good luck with it.
Thanks. Appreciate your comment.
You are most welcome.
One thing I'd be paying attention to is whether the challenge is trust or demand.
Someone can agree this is a real problem and still hesitate to connect a production AWS account. Those look like "no users" from the outside, but they're very different problems to solve.
Thanks for sharing your view Aryan.
To tackle the trust issues, I have shared the CFN template that gets deployed publicly. Anyone could check the template before deploy. Also, there is a section on the exact same page, where I have mentioned all the services and the required access the audit would use.
That would help someone who is landing on the page, to review the permission before running the audit.
Thanks for the additional context. I'd enjoy continuing the conversation outside the thread if you're open to it. What's the best email to reach you on?
Sure, would love to. Here [email protected]
Thanks! I’ve just sent it over.
Looking forward to hearing your thoughts whenever you have a chance.
I went through the entire flow, including reviewing the CloudFormation template, connecting an AWS account, waiting for the scan, and reading the report.
Overall, it felt trustworthy. I especially liked that every permission is explained and the template is publicly visible before deployment.
A few things stood out to me as a first-time user:
Overall, I don't think the audit itself is the challenge. I think the bigger challenge is helping first-time users feel confident enough to click "Deploy" and get through that initial setup.
Appreciate the walkthrough - that's rare.
Agreed on all three. Steps-before-value and progress visibility are on the list. The CLI-block-by-default thing was actually a real bug - one card type wasn't collapsing like the rest. Fixed.
You're right that trust is the harder problem here, not the audit logic. That's the one I'm actually rethinking.