3,5K strategies added from 160 companies

Hello,

I'm still collecting and extracting data for the database of marketing strategies.

Right now, I have 164 companies and 3,519 strategies from them.

Collecting and extracting data takes time and money. Let me describe the process.

1. First, I find interesting, successful companies. Most of them are known; just because if I pick up a company that nobody knows about, it's unlikely I will find any interviews with them or an article about them.

2. I look for interviews, articles, whatever resource where they can share their stories. Most of them are YouTube videos, but also I can use a lot of podcasts and articles in blogs.

3. I manually copy the URLs of the found resources and paste them into files with the specific structure.

4. When there are 10-20 companies ready to be extracted, I run my scripts.

5. The script reads data from the files, opens URLs, and downloads transcripts of video files, then use special prompts and extracts data using OpenAI. Another script verifies the data and removes duplicates.

6. Finally, it saves the extracted information in the database.

Nothing fancy, the code looks ugly, but it's working. I spend 2-5 minutes of manual labor to collect one company, and it takes 10-15 minutes of script work to process it completely.

I hope, you find this information interesting :)