Filtering & sorting any field vs performance guarantees — where do you draw the line?
I’m building a data-driven SaaS inspired by tools like Airtable.
I’m currently dealing with ~13M records (MongoDB documents) in production, and it forced me to be very explicit about the trade-off between UX expectations and backend reality.
From a UX point of view, users expect to be able to filter and sort on any field, instantly.
From a backend point of view, that assumption breaks pretty fast once data grows.
Here’s what I’m seeing in practice (MongoDB backend):
Small datasets (<10k records):
Filtering + sorting on unindexed fields is basically free (single-digit ms).
Large datasets (millions, up to 13M+ records):
Unindexed filtering can still be acceptable if queries are properly scoped,
but unindexed sorting is where everything collapses.
So I’m leaning toward a very pragmatic rule:
Below ~10k records → open filtering & sorting on all fields (Airtable-like UX)
Above ~10k records → “performance mode”: only indexed fields are filterable/sortable
The goal is to avoid punishing small users, while being honest about what actually scales.
For those of you who’ve built similar products:
Where did you draw the line?
Did you allow unindexed sorting early on?
Any regrets or hard lessons learned as data grew?
Curious to hear real-world experiences.
Every “Airtable-like” product has this tradeoff in one way or another. ~
What worked for us was drawing the line not by record count alone, but by the visibility of query cost. At first, we allowed a lot, but slowly started introducing friction instead of hard walls.
Some designs that assisted.
It was allowed for unindexed fields for longer than sorting, so you hit this conclusion. A sort is a silent killer.
Instead of merely disabling the field, we explained why it was slow with a message like “this field isn’t indexed yet.” That only dropped complaints.
Due to the cost of using a field, power users self-selected desirable fields for indexing.
The most annoying regret: waiting too long to design the “index as a product”. When the data is large it is hard to retrofit that UX.
It seems reasonable to frame it as "performance mode." as long as that’s the graduation not the restriction.
Inquiring as to whether you’ve utilized soft warnings or delayed execution (“this may take ~X seconds”) before locking down fields? We discovered that shifted behavior without feeling punitive.
This aligns very closely with how I’m thinking about it.
The UX patterns you describe (soft warnings, cost visibility, delayed execution) are definitely planned.
Right now, before calling this a true v1, my priority is making sure the system holds up under load.
I’m intentionally spending time on the fundamentals first — DB-level optimizations, query patterns, indexing strategy, and making sure the core model scales predictably.
Once that foundation is solid, adding UX layers that guide users without feeling punitive becomes much easier (and safer).
I’d much rather ship a slightly stricter product that’s honest about its limits than mask performance issues with UI tricks too early.
Thanks for sharing your experience — the “indexing as a product” point is especially helpful.