I just launched a new Browser Automation API - looking for feedback!
I am very proud top annouce that I just released my new service which I called wibbly!
It is a browser automation API which allows developers to extract data from any website with ease. I designed to API to grab information from the actual DOM of a webpage which means an actual browser session gets spawn on every API request.
Wibbly can grab normalized metadata extracting content from HTML tags, Open Graph and JSON-LD.
Since the web is a wild place, it is possible to specify custom extraction rules ( I called it Selective Query) which allows to grab data from anywhere in a page simply providing CSS selectors and attribute values.
(see more at: https://www.wibbly.co/docs#selective-query)
I have a lot in my mind on how to improve the developer experience and add new features to the API, but it would be great to get some initial feedback which would drive my decisions for the future.
In case there is any interest, you can find more on https://www.wibbly.co
Cool product. I also like your landing page. Concise and clearly communicates the use cases.
I think you missed a typo: satructured -> structured
Do Wibbly load and execute JavaScript? And is it possible to provide a cookie jar to scrape websites that require authentication?
Thanks for your feedback!
Typo fixed!
Yes wibbly runs the webpage against a real browser session which executes Javascript out of the box.
The cookie feature is something which is on my roadmap, I have an high level idea on how to implement it, but I might need additional feedback to get into the details of it, when the time will come to imnplement it.
I imagine you'll have issues with having your IPs banned at some point. Assuming you're working with a proxy provider.