Lookup Api for big data files

Title

Lookup Api for your big data files

Problem hypothesis

Companies small and large generate data assets (files) in CSV, Parquet, Orc, Avro, JSON formats that they would like to share with their integration products or partners, today they do it via file uploads, sftp, or likes. The problem that I've seen with that model is there are use cases where they would need to expose the contents of the file over an API probably because they need real-time access to that data or they want to use the information in those files with webhooks etc.

Challenges

• Building an in-house service comes with operational and engineering challenges around scaling, eng time investment in terms of development and operations
• Infrastructure costs and understanding the budget projections
• Might not align with companies core competency

Solution hypothesis

On a high level, the idea is to create an endpoint API that is made available to the user against such files. Similar to ngrok a customer will be provisioned with an endpoint that they could use for lookups against the data in their files. The customer could define the duration of uptime for that service and kill it when they see it as unnecessary. Customers are presented with a simpler pay-as-you-go service

Thanks for your time reading up until here, I'm looking for feedback from fellow indiehackers to learn if they have seen a similar problem space and would appreciate any pointers.