Sign inJoin
1
2 Comments

Tracking time series from json

by Kacper Wikieł

Multiple times I've set up a cron to download data from some API to create the chart so I have historical data for further analysis.

I've created a tool for automating this.

Looking for feedback - do you understand what the product does how to use it?

https://temporals.herokuapp.com/

Kacper Wikieł's avatarKacper Wikieł
on April 14, 2020
Say something nice to Spike…
Post Comment
  1. 2

    I saw the footnote that "This App is mostly for quant traders looking for historical data." So now I understand who the target audience is. What is the primary problem they have? Is it being able to download historical data? Or is it the ability to create a graph from historical data?

    Can you provide the following information:

    1. Who is your target audience? I understand its the quant traders. Is there a specific role or type of quant trader or is it any quant trader? I ask because from what I understand, quant traders could be dealing with different types of securities (stocks, bonds, etc.)? Are there specific endpoints that they are using or are there thousands?
    2. What is their primary problem? Is it that the JSON API doesn't return historical data and so they have to set up cron and DB to pull and store this data for further analysis?
    3. What does your solution do exactly? What benefit is your target audience deriving it? If it's time to develop and maintain the system, do you have numbers to quantity how much time will they save?
    neeravbm's avatar neeravbm
    ·
    6 years ago
    ·
    Reply
    1. 1

      Absolutely great questions. ( They are helping me write a marketing plan )

      1. Target audience: I've described them as quant traders but maybe it isn't perfectly good: (so far my 2 users are those )
      • it's a tool for either non-technical people who want to create excel or powerpoint slide with some chart from the alternative data from the internet.

      It's literally crontab + requests to download the data to postgres. it isn't complicated at all - but even for people who can code this thing DIY you have to set up a database and find data you want in the API.

      The next step would be parsing data from any website, not just the json API but it would be significantly harder from the tech side of things.

      1. Primary problem = you have some API that gives you only current data so you cannot access historical records, my hypothesis is that people would add popular data sources there so the value of historical time series would increase ( as it's sometimes impossible to replicate them )

      2. It does:

      • download the data from the url with specified frequency

      • secret sauce here << - transform nested json structure into flat paths.

      • create chart for each datapoint
      • easy csv/json export of historical data

      How much time it will save you? When I was doing this manually it's 30+ minutes per data point ( url ) + 10 minutes for chart. With temporals app it's less than 1 minute to add a record to the database and set the scraper. If you are using 4 data points a week it should save you ~9-10 hours a month

      Kacper Wikieł's avatar Kacper Wikieł
      ·
      6 years ago
      ·
      Reply