SearchTheDeck - How I search inside 15,000+ pitch deck slides
Search inside 15,000+ pitch deck slides
This is the story of how I built SearchTheDeck.com:
Klotho's pitch deck follows the classic sections everyone suggests: introduction, problem, solution, target market, market size, competition, go-to-market strategy, product, team, financials, funding, milestones, and conclusion.
When I searched for slide decks, I couldn't find the specific sections I wanted to learn from. To find that one relevant slide, I had to open tens of decks and scroll through them.
I decided to make a tool that would allow me to search inside the decks more easily.
Searching Decks
I needed to find pitch-deck slides of a certain type, like the ‘Problem’ slide or ‘Vision’ slide.
I scraped several pitch–deck sites and collected more than 15k slides, totaling 5GB of data. OCRing the images and searching the words inside the slides was the only way to find the relevant slides without manually tagging them.
Unfortunately, the resolution of the slides was too low, so tesseract couldn't recognize the text.
Using Upscayl, an open source GPU-based upscaler, I upscaled each slide to four times its original size. The result was a data set of 150GB ready for OCR.
It proved slow to run tesseract on 150 GB of images on a single machine, so I wrote a lambda-based event-driven Klotho application to speed things up.
Klotho
The application would get an image path, pass it to a function that runs tesseract on it, then passes the detected text and image path to another klotho::exec_unit that resized and optimized the image into a smaller yet still high resolution webp file.
To upload the images, I used the klotho::persist capability to create a data store backed by S3, and manually uploaded the 150 GB of images.
The event driven flow used the klotho::pubsub capability. The processed image was then written into the same klotho::persist‘ed object store but with a different path, and the path + detected text were saved into a klotho::persist‘ed key-value store.
The processed data-set was only 2GB in size.
Fast search
In order to create a fast, searchable data set, I used Algolia to index the text results from the OCR. Facets such as the startup name and the public image URL for the slide made the UI easy to construct.
For the front-end, React, NextJS, NextUI, static building, and Klotho’s klotho::static_unit capability made a great combo running on AWS’s S3+Cloudfront CDN. Due to the 15k results going over the Algolia free-tier, we decided to sponsor the project.
What I liked
-
The Upscayl GPU-based upscaler quality was impressive despite the low resolution of the sources.
-
I enjoyed the developer experience building the cloud system with Klotho. (Though being one of the founders, I’m biased). I used the open source klotho::persist and klotho::static_unit and the pro klotho::pubsub and klotho::exec_unit capabilities to construct the larger system in a few hours with virtually no infra/platform work – maximum productivity!
-
Tesseract’s OCR produced quality results and worked well in a Lambda-based environment.
-
Algolia APIs had a seamless experience and their starter React components for the front end UI worked as expected.
What I disliked
-
I couldn’t figure out how to run Upscayl in a cloud environment, so I wound up not automating it. That meant that Search the Deck isn’t fully automated (yet), and there are manual steps that have to be taken to add or update the decks.
-
The manual nature of collecting all the slides from all the web sites felt unnecessary. Similar projects pop up all the time, there’s no point in re-scraping them.
-
The developer experience for klotho::static was experimental at the time of writing, but I wanted to use it anyway. This wound up being useful input for its next iteration.
Opening the data set
I'll be releasing the image dataset as a downloadable set, or hosting it in a repository so people can contribute to it. That way the next person that wants to create a fun new version can use that central set and benefit everyone.
Open Source
I wasn’t originally planning to make this an open source project, but it seems like it would be really useful to make it available to everyone.
Help us get 1000+ Github stars within 2 weeks and we’ll prioritize the effort to open source it.
Now go and Search the Deck!
