Dealing With Referrer Spam

Bob Leggitt | Friday, 16 November 2012 |

In my Top Ten Most Annoying Things on the Internet piece the other day I mentioned ‘referrer spam’ within a wider rant about Web spammers in general. But what is referrer spam? Well, it’s actually a pretty clever, and often quite effective way for spammers to attract visitors to useless sites. Typically using automated bots, the spammers register numerous fake ‘hits’ on various blogs and websites. The sole aim of these fake ‘hits’ is to interface with the blog or site’s stats counter, so it looks to the receiving administrator like his/her site is getting real visits, from real people.

As well as registering these spam connections as hits, the receiving blog/site also records the spammer’s URL. That’s the important part. The spammer’s URL is recorded as a referring URL, and the receiving blog or site administrator can potentially see it in their dashboard. In theory it appears, to the blogger or administrator of the receiving site, as if another site on the Internet has posted a link to their blog, and lots of people are clicking that link.

Bloggers in particular are well known for being obsessed with their stats, and for keeping a close eye on where their visits are coming from - and for quite naturally being very curious regarding what's being said about them elsewhere on the Web. That's why referrer spam can be so effective. When bloggers spot a lot of visits coming from one place, their curiosity can rapidly take over. Who linked to their blog? Which article has this person linked to? What are they saying about the article? Is the link likely to remain visible for a long time or quickly vanish out of sight?… If the blogger thinks the spam hits are real, then there’s a very, very good chance that he or she will click on the referrer link in his/her blog dashboard to find out what the ‘buzz’ is all about. Hey presto – the spammer has successfully attracted a visitor to what normally turns out to be a total junk site – very frequently devoid of human involvement, and potentially using aggressive or malicious scripts, or perpetrating a scam, or whatever.

That’s the theory. But in practice, all but the most inexperienced bloggers and site administrators receiving this ‘referrer spam’ will normally spot straight away that the hits aren’t real. The referrer URLs will often have names like “loadz-of-dosh-4u.com” (or something equally unconnected with the subject matter of the receiving blog), and the behaviour of the spambots will not tie in with the way real visitors would use the site. That immediately suggests automation.

Some spammers do, however, try to get round the above, mainly by using URL-shortening services (like the ones people legitimately use on Twitter), and regulating the number of fake ‘hits’ so they don’t look too disproportionate against the likely numbers of real hits. URL-shortening hides the spammer’s URL, and because real people do use URL-shorteners for legitimate reasons, it can appear that the fake ‘hits’ have come from genuine referrers – like Twitter. If the number of fake ‘hits’ blend fairly closely with the number of real hits a blog tends to receive from the social networks, referrer spam can become even more convincing. For that reason, I’d suggest the following procedure for bloggers or site admins curious about shortened URLs amongst their referrer statistics…

Right click on the referrer link (DON’T LEFT CLICK IT!), and select Copy link address from the context menu which appears. Now go to the DuckDuckGo Social Goodies page on the link below…

https://duckduckgo.com/goodies/#Social

Near the top of the screen you can see the headine Social, and beneath that there’s a Computing category, with an app to expand shortened URLs. It’s highlighted in blue on the capture below…



Click that blue-highlighted area on the DuckDuckGo site, and paste your shortened URL into the box which appears to the right. Now click Go, and the app will tell you the real URL of the referring site. You may now be able to see straight away whether or not the referrer is a spam site. If not, you can click on the full (expanded) URL link provided by DuckDuckGo to check out the referrer site without connecting your responding visit with your blog. You should know, however, that you’ll still potentially be giving away your IP address and computer/device info to a spam site if you directly click the link on DuckDuckGo. If you want to completely avoid giving any info at all to the referring site, you should use a proxy such as Anonymouse…

http://anonymouse.org/anonwww.html

You can paste either the shortened or the full URL (copied from DuckDuckGo) into the Enter Website Address box on Anonymouse. Visiting the site via Anonymouse will ensure you don't give away any info regarding your whereabouts, computer system, etc. If Anonymouse can’t access the site, I wouldn’t worry too much about trying again. If you’re really that curious, and you don’t want your whereabouts identified, but Anonymous gets blocked, you could try Tor. That, however, involves downloading a program (albeit a free one, and a useful one). But realistically, most of the sites Anonymouse can’t access are either incompetently built, deliberately blocking proxies to oppress visitors’ privacy needs, or running aggressive/malicious scripts. Whichever is the case, visiting the site is probably not going to be of benefit to you.

Ultimately, spam referrers who detect that they’re getting ‘clickbacks’ from your blog/site will almost inevitably target you more frequently, and if you do try to keep an accurate picture of your site stats, that can make life a lot more difficult for you. So the message is: DON’T CLICK REFERRER SPAM LINKS! You can also of course consider reporting the spammers to any URL shortening sites they're using. I've done this and it does seem to work. It won't stop them outright there and then, obviously, but if enough people do report referrer spam, the spammers will eventually find their task too work-intensive, and give up. They don't like doing work. That's why they've become reliant on spambots.

In fact, it’s not really a good idea to directly click any link which appears in your referrer stats – even if it looks okay. It’s much better to copy and paste the link into the address bar of a separate browser tab/window, because that disassociates your blog from your responding site visit. It only takes about a second longer than directly clicking the link, too, so it’s a good habit to get into. Ideally, also use a proxy like Anonymouse to hide your IP address and computer info, and keep in mind that useful DuckDuckGo link expander for instances where you want to know the name of a link-shortened referrer before deciding whether or not you should check them out.

Referrer spam is annoying, but ignoring it is probably the easiest and most universally effective policy on dealing with it in the long term. Afterall, if everyone did that, it would cease to exist.

Planet Botch is contactable only via Twitter.