If you have a business or a website, you might see some bots. Although bot traffic can make it seem that you have more traffic than you have, it can obscure your real users who are really browsing your website and mess with your analytics. This post will discuss how to filter out bots in your Google Analytics and how to prevent the robot takeover from happening on your website.
What are bots? Are they inherently evil?
No, they’re not all evil, but I couldn’t resist a
If you’re not technical, don’t panic! I promise to include a step-by-step guide to getting rid of bots in your Analytics and you can skip ahead if you’re less interested in the
Many search engines, including Google, use spiders in order to ensure that any page shown on Google is updated when the page is updated. This is the intention of the robots.txt file in the back-end of your website. It enables Google and other websites with spiders to figure out which pages of your website should be crawled.
As someone who isn’t a general fan of spiders, I am delighted to write that there are evil spiders (and bots). These bots will crawl your website for possible security flaws, ways to spam your website, and scraping your information off your website.
Consequences of bots in your analytics
If you have more than 5% of bots in your analytics, your analytics might get skewed. If you have a lot of bots in your data, you might see strange referrals (promoting SEO services ironically) in your Referrals (Google Analytics: Acquisition -> All Traffic -> Referrals).
Similarly, you might also see a really high bounce rate as well as a low session duration (e.g. under 30 seconds). All of this can really impact your ability to understand real human users using your website and/or interested in your services if you have a business.
Another implication is bandwidth. Some people pay based on their bandwidth for their website, so if you’re paying monthly, it’s in your interest to lock out bots to reduce your bandwidth usage. I include a good solution for reducing your bandwidth at the bottom if you’re struggling with bots even after enabling the two recommendations below.
How to remove bots from your Google Analytics the simple way
We’re ready to fight the evil. This step will only filter bots out of your Google Analytics, so if your goal is to clean your analytics, this is a good first step. However, you may want to take this a step further to actually prevent bad bots from easily accessing your website.
You’ll want to go into your Google Analytics. At the bottom left corner, you’ll click Admin. From here, you’ll see a three-tiered method. You’ll head to the third column, which has to do with global filters on a specific Google Analytics account for one website. Click View Settings.
From here, you’ll see a new window slide out. From here, you’ll scroll down and simply check Bot Filtering. It will do *most* of the work for you, but it’s not perfect and I recommend going a step further (as discussed below) to actually rid your website of bots.
Additional removal of bots from your analytics
Checking the box only can do so much in terms of removing bots from your analytics. If you’re a hobbyist or you have a smaller website, it should already help. However, you might still have issues involving bots and you will want to set up additional filters in your analytics to account for any likely bots.
This can be done after doing a deep analysis of your data to understand data anomalies. ( If your eyes are glazing over, I do this for clients when requested.) From here, you can create a custom filter to take out the suspected bots.
This depends on the source of the bots, but they generally come from the same referral or include the same keywords. A few here and there won’t impact your data, but if you’re seeing a significant percentage, you will want to clean these out.
How to kill the bad bots accessing your website
If you’re frustrated with bots, you can go a step further to actually stop bots from accessing your website using a firewall or more advanced methods. 😉
How to make it harder for bots to crawl your website
If you’re on WordPress, I recommend installing Wordfence. This free tool can be installed fairly easily as a way of protecting your website against hackers, snoopers, and bots. Most importantly, it can detect bots and slow them down if they’re trying to move through your website too fast. This is a tell-tale sign of bots often on the evil side of things. Similarly, it can ban bad bots who are crawling for website insecurities.
One step further: Ban bad bots
Another larger (and pricer) step that you can take is getting Cloudflare. This service is a popular way of speeding up your website while prioritizing fast access for humans while locking out bad bots automatically. It can be pricey to set up.
If you have 100,000+ users daily and you’re struggling with bots eating through your bandwidth, you might want to consider having Cloudflare set up for you. I don’t provide this service, but your host might provide this service.