I realized an ATS score wasn't the problem
A few months ago, I was looking at different resume tools to understand how they worked.
Almost every one of them gave me a score.
But after looking at enough resumes, I kept asking the same question.
What does that number actually tell someone?
If I'm applying for a Backend Engineer role today and an AI Engineer role tomorrow, why would my resume receive almost the same feedback? The jobs are completely different.
That was the moment I stopped thinking about building another resume scorer.
Instead, I started thinking about context.
A resume shouldn't be evaluated on its own. It should be evaluated against the job you're trying to get.
That simple idea turned into ATSAlign.
The interesting part wasn't building the interface or integrating AI models. The difficult part was figuring out what feedback would actually help someone improve their chances of getting an interview instead of just making the score look better.
Sometimes a missing keyword really matters.
Sometimes a skill is already there but written differently.
Sometimes the resume is perfectly fine and the job description is asking for an unrealistic combination of experience.
Those edge cases ended up taking far longer than I expected.
I'm still learning every week, and every piece of user feedback changes how I think about the product.
If you've built products for job seekers, recruiters, or HR teams, I'd love to know how you approached this problem. I'm sure there are things I still haven't thought about.
This is exactly the right way to think about it. A resume isn't good or bad in isolation—it's only meaningful in the context of the specific job description. Focusing on actionable, role-specific feedback instead of chasing an arbitrary ATS score solves a real problem Wishing you success with ATSAlign!
Thanks! That was probably the biggest lesson for me while building it. A resume can look great for one role and be completely wrong for another. Once I started looking at resumes in the context of the job instead of in isolation, the whole problem started to make a lot more sense.
The "skill is there but written differently" case is the whole problem for the population I work with. I run a nonprofit that helps veterans move into tech, and their resumes fail matching constantly because "led a 12-person comms team in a combat zone" never contains the phrase "project management." If ATSAlign can translate adjacent experience into a JD's vocabulary instead of just flagging gaps, that's a feature people will pay for, and nobody serves it well today.
That's a really good example. The experience is there, but the language doesn't line up with how the job describes it. That's exactly the kind of problem I wanted ATSAlign to solve not by telling someone they're unqualified, but by helping them translate their experience into the vocabulary of the role. Thanks for sharing this.
Been down a similar road building Applio (resume builder). Hit basically the same wall with generic ATS scoring, a single number doesn't tell anyone what to actually change, so people end up "optimizing" for the score and land on a worse resume for an actual human reader.
What worked better for us was less about scoring and more about diffing, showing the real gap between what's on the resume and what's in the job post, instead of one aggregate number. Sounds close to the context-per-role thing you landed on.
The harder problem for us is your exact "skill is there but written differently" case. That ends up being more of a semantic matching problem than a keyword one, synonyms, adjacent tools, seniority framing. Curious if you're doing embedding-based matching for that or leaning on the LLM to just read it holistically.
Also relate hard to the "job post itself is unrealistic" point. That's such an underrated thing to actually tell users. Most tools are incentivized to keep saying "tweak more" instead of "this one might not be winnable," since the honest answer doesn't feel as good but is way more useful.
Thanks! It's reassuring to hear someone else ran into the same challenges. We had a very similar realization that people cared more about understanding the gap than chasing a higher score.
For the wording issue, we don't rely on exact keyword matching because, as you said, that breaks down pretty quickly. We use semantic matching to recognize related skills and then focus on presenting the results in a way that's easy to understand for the user.
I also completely agree about unrealistic job descriptions. Sometimes the most honest answer isn't "optimize more," it's that the role is asking for an unusually broad mix of skills. Helping users understand that context feels more valuable than simply telling them to keep editing.
I'd be interested to see how you've approached it with Applio as well.
We went semantic too, comparing resume bullets against job requirements instead of raw keyword overlap. Honestly the harder part wasn't the matching, it was deciding how much to show the user. Showing every single gap felt overwhelming, so we ended up prioritizing the 3-4 that would actually move the needle. Curious if you ran into that same "how much to show" problem on your end. If you want to poke around Applio at some point I'd genuinely love your feedback, you clearly think about this stuff carefully.
this is the exact same problem on the skills side. we see it constantly — companies ask candidates to "rate your AI skills 1-5" and it tells them absolutely nothing. someone who scores themselves a 4 might just be confident, not competent. the context reframe you landed on is the right one. a score without a reference point is just a number.
That's a great comparison. I hadn't thought about it in the context of AI skill ratings, but it's the same underlying problem. A number on its own doesn't give people anything they can act on. Once you add context, whether that's a job description, a benchmark, or specific expectations, the feedback becomes much more meaningful. Thanks for sharing that perspective.
This resonates — I've watched the same "score theater" play out in
e-commerce. Tools spit out an optimization score, but a number with no
context just tells you the number went up, not whether it moved the thing
that matters (in my world, actual sales; in yours, actual interviews).
The context reframe is the right call. The hard question underneath it:
does your feedback change what the user does, or just what they see?
Sounds like you're already there with the "sometimes the JD is asking for
something unrealistic" cases — that honesty is probably your moat. Most
tools would never tell a user the job posting is the problem, not their
resume.
Curious how you're validating that a given piece of feedback actually
improved someone's callback rate vs. just felt more useful.
That's a really good question, and honestly I don't think we're at the point where we can say it improves callback rates.
What we can see is whether people actually change their resumes after reading the feedback, instead of just chasing a higher score. That's the behavior we're paying attention to right now.
Callback rate is definitely something I'd love to measure in the future, but it's hard because there are so many variables outside the resume itself. If someone gets more interviews, was it the resume, the market, the role, or just timing?
I think if we can consistently help people make better decisions whether that's improving their resume or realizing a particular JD just isn't realistic that's a win.
The shift from scoring resumes to evaluating them in context feels like the right direction.
A score tells someone something is wrong, but not necessarily what decision to make next. The hard part seems to be turning job descriptions into useful guidance without just creating another keyword-matching tool.
That's exactly what I found while building it. People rarely asked why their score was 72 instead of 78. They wanted to know what they were actually missing and whether fixing it would make a difference.
And I agree with your second point. The challenge isn't finding keywords it's figuring out what the job is really asking for and turning that into feedback someone can actually use.
I'm glad the questions were useful. I enjoyed thinking through it with you.
I'd be interested in comparing a few more ideas outside the thread. What's the best email to reach you on?
"why would my resume receive almost the same feedback" — this is such a good gotcha moment lol. Feels like the resume-scorer version of what I ran into building my own thing: turns out "give it a single number/generic fix" is almost always the lazy version of the real problem, and the real problem is always messier and way more context-dependent than anyone wants it to be.
Your "sometimes the skill is there but written differently" point is basically my whole week summarized in one sentence, just for a completely different product. Respect for actually sitting in those edge cases instead of shipping the score and calling it done.
Haven't built for job seekers so can't help with that specifically, but if it's any comfort — the "took way longer than expected" part never really goes away, it just becomes the job.
Haha, that's exactly how it felt. I went in thinking the hard part was building a better score, but the deeper I got into real resumes and job descriptions, the more I realized the edge cases are the product.
And yeah, "it took way longer than expected" seems to be the most universal founder experience
I like that you’re reframing the problem. In Finsight AI, I’ve seen SMBs assume cashflow forecasting is the issue, when it’s actually clarity in data. How did you discover ATS scores weren’t the real blocker?
That’s a great insight — people don’t just want a score, they want actionable clarity. I’ve seen the same with SMBs using Finsight AI: they care less about the forecast number itself and more about what changes will actually improve their cashflow. Love how you reframed the problem!
It was actually the questions people kept asking after they saw their score. Almost nobody asked how the score was calculated. They'd ask things like, "Which skills am I missing?", "Does this experience count?", or "Why doesn't this project match the role?"
That's when it clicked that the score wasn't the real problem. People were looking for clarity on what to change and whether those changes would actually matter. The score just happened to be the first thing they saw.
job matching is more important than ats score as people filtered out at this stage is much larger, however, ATS score check first so if you fail it no move to next step (job match).
That's pretty much how I see it as well. ATS compatibility still matters because if your resume can't get through the initial screening, nothing else matters.
But once you clear that first hurdle, the bigger question becomes whether your experience actually matches what the role is looking for. That's where I think a lot of existing tools fall short—they focus on improving the score instead of helping you understand the fit.
To me, the score is just a checkpoint. Job matching is what really determines whether you have a realistic chance of moving forward.