SiteRows

Hello everyone. Welcome back to my siterows.com series.

🔹 Quick reminder of what this app does: Allows you to scrape web content with SQL, like you would query a DB. Creating a FREE account unlocks automation features and higher usage limits.

🔹 Today's example is a repeat of the previous example, except this time using the /Crawl API endpoint which is only available as part of the PRO tier. Please visit https://www.siterows.com/upgrade to see what cool features and extensions you would get in the PRO tier.

Below is a Python example/response, and I'm also saving all examples in this GitHub repo: github.com/sgt-oz/SiteRows.

What I'm doing here is first calling the /Crawl endpoint to get Metacritic's best games this year to discover game URLs, then fetch each game's data with /Scrape endpoint.

Thanks, everyone. Please don't hesitate to reach out with any questions or feedback.

🎉🎉🎉 Excited to announce that SiteRows PRO is now live. 🎉🎉🎉

Query the web with SQL.

PRO unlocks the full product:
🔹Full DOM catalog - query all standard HTML element types (FREE includes a smaller subset).
🔹Crawl - run crawls in the app or via the POST /Crawl API (not available on FREE).
🔹Higher limits - more URLs per request, longer sessions, and higher scrape/crawl rate and byte caps.

Join or upgrade here: https://www.siterows.com/upgrade

Hello everyone. Welcome back to my siterows.com series.

🔹 Quick reminder of what this app does: Allows you to scrape web content with SQL, like you would query a DB. Creating a FREE account unlocks automation features and higher usage limits.

🔹 Today's question: "On the site, what is the whole deal with this Objects box?"

The purpose of the objects box is to give it a SQL-like look and feel, and here is how it's structured:
🔹 When you expand "DOM" and further expand any element like @a or @div, you get two items: "Scraper Fields" and "Attributes".
🔹 Under "Attributes", you'll find all the standard HTML attributes for that element
🔹 Under "Scraper Fields", you'll find a couple of extra synthetic attributes I thought would be useful like Index, Parent, Children, etc...
🔹 Scrolling back to the top, you'll see a couple of "Composite" elements I also thought would be useful. For example, the @headings element represent @h1, @h2, @h3, @h4, @h5, @h6 elements combined.
🔹 Think of any element like @a, @div, or @headings as your "tables", and their attributes like href, id, class as your "fields".

Thanks, everyone. Please don't hesitate to reach out with any questions or feedback.

Hello everyone. Welcome back to my siterows.com series.

🔹 Quick reminder of what this app does: Allows you to scrape web content with SQL, like you would query a DB. Creating a FREE account unlocks automation features and higher usage limits.

🔹 Today's question: "What if I'm using something like Selenium to navigate somewhere (perhaps logging into a site), and then I want to query the page?"

In order to accommodate this, I just added the ability to pass a raw HTML string to the /Scrape API endpoint, instead of a URL. Below is a Python example/response that demonstrates this new feature, and I'm also saving all examples in this GitHub repo: github.com/sgt-oz/SiteRows.

Basically here is what's happening:
* Function fetch_logged_in_html() uses Selenium to log into a site and return the home page's HTML string
* That string is passed to the /Scrape API endpoint. Instead of the payload being: {"url": "mywebpage.com"}, you just pass:
{"html": "<body><a href=...>.....</body>"}

....and that's pretty much it. SiteRows queries your HTML and gets results as if you passed a page URL to it.

Thanks, everyone. Please don't hesitate to reach out with any questions or feedback.

Hello everyone. Welcome back to my siterows.com series.

🔹 Quick reminder of what this app does: Allows you to scrape web content with SQL, like you would query a DB. Creating a FREE account unlocks automation features and higher usage limits.

🔹 Today's question: "How can I extract some aggregated web content from a page, without any hassle? For example, count of links"

Below is a simple Python example/response, and I'm also saving all examples in this GitHub repo: github.com/sgt-oz/SiteRows. As you can see, with SiteRows this is super easy. Think of the URL as your "DB", and the tags (@a, @img, @headings, @li, @meta) as your "tables".

Thanks, everyone. Please don't hesitate to reach out with any questions or feedback.

Hello world! I'm starting this series of examples/use-cases of siterows.com, the new app I recently built. Without further ado...

🔹 Quick reminder of what this app does: Allows you to scrape web content with SQL, like you would query a DB. Creating a FREE account unlocks automation features and higher usage limits.

🔹 Today's question: "How do I call the SiteRows API so I can automate web extraction tasks?"

Below is a simple Python example/response, and I'm also saving all examples in this GitHub repo: github.com/sgt-oz/SiteRows

Thanks, everyone. Please don't hesitate to reach out with any questions or feedback.

Have you ever wondered: Why can't I query a web page like I can query a DB? That question led me to start building https://SiteRows.com

The idea is to expose web content as queryable datasets, so you can do things like:

URL: https://wikipedia.org
SQL: select * from @a where text like '%English%'
RESULT: A list of all links whose text contains "English"

There is a front-end with a SQL-like object explorer, and there is an API to set up automation.