Simple data visualization - Can any of you see a real core problem here? Is this worth pursuing?

I feel like there is enough variability in data sources that a developer almost always has to come in and create the solution to present it. For example date fields sometimes come in as UNIX encoded rather than an ISO Date format. How can you deal with all of this variability in an automated fashion? I do agree that it sucks to have to always be redeveloping these features. It's an interesting problem.

This is an interesting topic that I myself have been mulling over for a while. I too reinvent these same solutions over and over while any solutions I've ever searched for are always to cumbersome to use.

At the same time, there are hundreds of data visualization solutions so I think the lack of a fit must either be due to a lack of rigor whenever I look for a solution or the variability between use-cases is just too high to have an adequate general solution.

The concept itself of data visualization, I'd say, is a bit too broad. Consider the following cases:

1. A user requires a dashboard that receives log messages with some additional metadata. The messages can fall into a fixed set of predetermined types and filtering by these types is required. The metadata can contain text, timestamps or urls to media files in a flat list, all types should be viewable within the system. The system is primarily for viewing logs on an ongoing basis and responding to real-time events so long term storage of the messages is not needed.

2. A user has a list of purchases made by customers. The purchases are represented as a list of objects and will be provided as JSON. Each object has only text and number fields. The user wishes to be able to apply aggregate functions on the data using the text fields as pivots and the number fields as the values. The app should generate a cross-table with text fields as headings and grouped number values as the table content. A slicer control should also be available to filter the data using the text field values.

3. A user requires a dashboard for responding to transient opportunities. A separate automated system will monitor the prices and availability of certain products on specific websites (e.g. shoe websites). When desired conditions are met, that system will use a web hook to post a notification to the dashboard. The dashboard will show the relevant information (e.g. price, sizes) in the notification (paired a mobile notification if this were to be a mobile app) and a button that allows the user to quickly navigate to the product page when clicked.

These three cases represent the typical level of variability between data visualization tasks. The variability can be:

• The exact source of the data
• The way it will be represented
• How it should be displayed
• The filtering options that should be available
• If it is stored or not
• How long it should be available
• What analysis tools should be available
• The type of data (text, number, media) that it should support
• Real-time notification and interactivity requirements

And so much more. All of this combines to render most generic solutions unusable for most people, not because they couldn't handle the task, but because of:

• The size of the program and how frequently it is updated (unwelcome downloads and installs for small tasks).
• The complexity of the interface, the filtering and visualization options (needlessly long learning curve for all except researchers and analysts; frequent Googling to work out how to perform specific actions; hours spent reading documentation; generally busy screen when only handfuls of features are required by most users).
• Difficulty integrating into specific workflow (a major decision maker for everyone whose trying to get work done).
• Having to pay the full cost for something large just to have a few small features.

I think if one wishes to break into the market, the wise thing to do is choose a specific workflow that you have in mind and make a tool for integrating a small number of visualizations (say, just a plot like https://www.ga-insights.com/Alerts) as well as the usual input and output streams for that type of task only.
------------------------------------------------------------------------------------------------------------------------------
I am quite interested in this realm and would be happy to have more discussions about this if you have more topics on your mind.

I would also encourage you to look into very basic machine learning that is used to perform linear regression (which is almost certainly what is being used in ga-insights) because this provides an opportunity to add a lot of extra value to anything you build, which can put any product way ahead of the competition.

Please excuse any writing errors as I really didn't feel like proofreading something this large so late in the night.

Hope this helps a bit.

There are solutions like Domo and Tableu that takes in a dataset (similar to csv or excel), and can create "cards" which are basically graphs and visual dashboards.

The issue I see here is that, these platforms are for enterprise and the licensing fee is quite expensive, so for a solo developer, startup, small business, this wouldn't be a good solution.

But, just by googling "data visualization tools," you can tell there's a crap ton of products out there.

There must already be hundreds of data visualization tools.

Are you sure you have explored the market fully, and there really isn't anything comparable?

What I often see is that prospective founders have a huge bias (often unconscious) of not exploring competitors thoroughly, because they don't want the competitors to exists. And in your case, the larger packages like Tableau vastly out-budget-and-out-market the simpler packages, so you may not have found them even if you tried to google a bit.