TLD-List

Hi, I'm Timbo White. I've been running tld-list.com since 2015. It's a price comparison website that automatically aggregates pricing from registrars to help people find the best deals on domain names.

The website makes money from affiliate referrals and by selling direct access to the aggregated data via paid subscriptions. It gets 85K unique visitors/month.

Here are some things I've learned, ordered least important ➡️ most important.

10 - If your audience is technical, ditch Google Analytics

Many adblockers (uBlock Origin, Adblock Plus with Easylist, etc) block the Google Analytics script tag, preventing you from gaining valuable insight to your visitors.

Technical users use adblockers, and at much higher rates than non-technical users.

Consider using an analytics platform such as Plausible or Piwik, which are blocked by fewer adblockers.

Also, both of these services can be self-hosted for free, and customized to prevent your tracking script from being blocked by any adblockers.

9 - You will get scraped

You can build an API. You can offer data downloads with a non-restrictive license.

But if these options require the user to pay money, your site will get scraped by bots.

If you choose to fight them, Cloudflare is usually your best option.

Personally, I don't care for Cloudflare's prevalent presence across the web, so I use a custom defense against bots:

1. Rate limiting with fail2ban.
2. Custom bot detection and CAPTCHA display based on the request's IP's ASN, user-agent, behavior, etc.

It's not comprehensive or perfect, but it eliminates a majority scrapers without relying on a thrid party.

8 - Mask affiliate URLs with redirects

Many vendors' affiliate programs are often operated by a third party platform (Commission Junction, Impact, etc).

These platforms generate affiliate URLs for you to use for referrals, which usually contain a some weird intermediate hostname:

https://some-vendor.sjv.io/abcxyz

Unfortunately, adblockers have blacklisted all of these intermediate hostnames. This means any affiliate links you publish on your website will not even be visible to visitors with adblocker installed.

A solution is to mask your affiliate URLs with redirects:

1. Publish all affiliate links on your website with custom URLs that point back to your website at a designated path. For instance, change:

https://some-vendor.sjv.io/abcxyz

to:

https://mywebsite.com/go/some-vendor

2. On your website's backend, implement some controller code that performs redirects visitor requests made to the designated path (/go/some-vendor) to the actual affiliate link (https://some-vendor.sjv.io/abcxyz).

7 - More product details = more SEO authority

This is may be obvious, but the more information you can publish about the product, the more of an SEO boost you'll receive. For every product on your website, publish a product detail page that contains all the attributes about the product:

• table of all offers from all vendors
• price history chart or table
• product reviews
• specific product properties and features

6 - Put all the products on the landing page

It might not be the most effective layout in all cases, but I try to present as many products as possible on the landing/index page, without making visitors click through multiple categorized pages.

For instance, I make the landing/index page a searchable + filterable + sortable table of all compared products. The columns consist of:

• the product name (linked to a product detail page, see #7)
• vendor with the cheapest price (linked with affiliate URL, see #8)
• vendor/product star ratings
• extra columns for important product attributes, one column per attribute

5 - Monitor all automation

Whenever I automate a task with code, I make it email me a report when it's finished. I want the numbers: what was done, how long it took, totals, and statistics.

Most importantly, this report includes verbose details when something goes wrong, so be sure to catch and log all errors.

4 - Don't show extra banner ads

A price comparison website is already a bunch of advertisements, just normalized and mashed together. Adding extra ads can be distracting to users, and signal a bias towards those vendors that you're advertising (see #1).

3 - Sell direct access to your data

If your pricing data is accurate and up-to-date, vendors will want access to it. It allows them to establish competitive pricing and make data-driven business decisions.

Be careful how you license this data. If it's too restrictive, you'll miss out on subscribers. If it's too loose, you may be handing over your data to a competitor and allowing them to republish/resell the data on a copycat website.

2 - Listen to your users

Your users will give you the most valuable insights to your service. I get lots of useful feedback such as:

• requests to add a new vendor with competitive pricing
• data inaccuracies
• add useful feature ABC because of XYZ

User feedback is a great way to decide where to spend your time. Make it easy for your users to give feedback by prominently displaying your contact email address or a "Report Issue" button where a user can fill out a form that reports a data inaccuracy or feature request.

1 - Remain impartial

Avoid artificially promoting certain vendors, even if they pay a higher commission on referred sales.

Your users will notice this, probably question your motives, possibly publicly shame you, and then look for an alternative source of unbiased information.

By treating all vendors equally across your platform, you establish trust with your users. And this trust is extremely essential to establishing organic growth.

But nothing is absolute - sometimes a vendor is so terrible that it actually harms your reputation to even list them and their products on your website. So it's important to remain aware of your vendors' reputations, as it will directly affect your own.