CrawlSpider SEO Monitor

I had the desire to launch on the auspicious day of Dasshera that is today. Until yesterday morning I wasn't sure if I would have everything in place to launch.

I was hoping to have documentation, videos, tutorials, artilces all ready to go for the launch.

The only thing ready was the tool itself. Tested on several wordpress installations.

I just said, the tool is ready to go! Let me just launch it.

I decided to do a very silent launch of initial 100 folks on my list. Just finished sending the email!

Now I will work on doing several such micro launches over the next several days.

#buildinpublic #seo

Here is a screenshot of the diff algorithm in action. The CrawlSpider will take snapshot at set time interval and it will compare the snapshot at extremely detail level.

It has the ability to detect
when Keywords were completed removed, NEW keywords were added or when keywords are simply replaced.

This is useful if someone in your team accidently or knowingly changed the keywords in your anchor tags, Page Titles, H1, H2 or other H3 tags.

The same diff algorithm works for script and body tags. One issue I found is that if a website javascript has a "NOUNCE" feature. This generates a unique ID everytime the snapshot is taken. This creates a false positive alert. Similarly there could be other server generated values that could be tagged to various URLs.

#buildinpublic #seo

I can say now that all of the planned checks and tests are now implemented

Custom HTML tags
Custom Class names
XPATH string
Regex string
Simple Search string

Sitemap checks
Robots.txt checks

Also, I have been playing around on my landing page
https://www.crawlspider.com/

Still think there is something missing. Let me know what you think?

I am working on a script for the explainer video and will order it soon.

For CrawlSpider SEO monitor, majority of the UI is complete. It takes in a list of URLs and starts a SEO scan. It keeps a history of all the scans and compares them to the previous to check what has changed. Depending on the severity of the changes it will send email alerts.

There is a list of default checks for root domains and individual URLs and you can also define custom checks for e.g tags that are not covered in the default checks, custom Regex, Xpath and search strings.

I have been working with several clients as a SEO consultant and I see this everytime that once the initial web design is complete, the content starts changing either intentionally or via new plugin.

Recently I finished building a website for a local manufacturing company. Once done, few days later the boss let an employee manage it and he started playing and tweaking things around and added few plugins. Next thing you know, some of the product pages dropped in ranking.

The company owner didn't like this and was curious how to detect such changes. I helped install activity log plugin but it was still lacking the SEO angle.

This led me to the idea of making SEO change monitoring tool. This will help monitor all the SEO changes for your website or your client's websites.

Please connect with me if you any interest in such tool!