Data Catalog with SQL-To-Text. Will it help business users with data discovery and observability?
Hi there, my name is Igor.
I am testing different ideas in data governance space.
Sometimes business users have problems with data trust - a dashboard looks strange and they don't know whether it's a technical problem or a business one. They want to understand, how particular metric is calculated. Usually, these metrics are calculated via SQL, so I think SQL-To-Text might be usefull here to automatically generate descriptions and explanations for dashboards, reports, etc.
Do you think it's a big and important problem which is worth solving? Will natural language processing (SQL-To-Text) really help here?
Not sure if I'm 100% following the idea. If a dashboard looks strange or they can't interpret it properly, wouldn't a SQL-To-Text tool just be another way of the data to possible look "strange" as well, just in a different format? Instead of SQL-To-Text, are you referring to just being able to export or see the raw SQL data? If your app is trying to convert it to text, I see it as just another "dashboard" that can be misinterpreted as well. I personally don't see the need for another tool because I wouldn't invest in a tool whose dashboard I couldn't understand to begin with. An apps dashboard should be very self explanatory IMO; if it isn't clear, it's a no-go for me from the start of the sales pitch.
Hi, thanks for your comment!
I mean the following situation: user looks at the dasnboad and finds something strange. He wants to understand, how this dashboard is calculated. Probably, dashboard is created via SQL. My tool would read this SQL (alongside with lineage information) and give this user an overview. For instance:
"This metric is calculated based on column_1 and column_2 from table_x as an average between them". User can further examine descriptions or these columns an tables.
Does it make sense ? :)
Yes, your explanation makes sense. I guess it would get that info from the 'function'/'view' or whatever it's called in SQL that does those calculations. If the calculations are done programmatically via PHP (or another language), it wouldn't be able to diagnose the calculation. I understand what you're trying to do better now. Thanks for explaining and best of luck to you.