Direct ‘Spam’ Traffic in Google Analytics:
There have been numerous amount of individuals or webmasters who have seen unexplained direct traffic in their Google Analytics accounts. If you were to examine the secondary dimensions (country) of such data within Google Analytics you would see that the country is set as ‘not set‘ at times. Rendering it even more difficult acquire more specifics behind the spam and filter it our of your Google Analytics View. There are no apparent reasons behind the spam traffic — it’s just there; it can be there indefinitely. For a short amount of time or even for months. There have been a few explanations for the reason behind this spam traffic. Such reasons being ad click fraud, affiliation fraud, purpose-built crawlers that hide their identity, etc. Whatever the case webmasters or individuals need to be able to see the non-bot traffic coming to their websites so that they can take the necessary action as to not mess up their Google Analytics Data.
What can you do if you are affected?
Firstly you can perform a check on your Referral Exclusion List:
What the Referral Exclusion List does is that it prevents traffic from the listed domains or sites from initiating a new referral session. If your website links to a partner site (for example a payment gateway), this would join an initial session on your website with a returning visit from the partner site.
If you list other sites such as spam referrals, it will remove the referrals off the session, but the sessions remain. It will remain as direct visits. You can utilize the exclude filters in your view for removing spam referrals.
“Include and exclude filters
Use Include and Exclude Filters to eliminate unwanted hits.
If you apply an Exclude Filter and the pattern matches, the hit is thrown away and Analytics continues with the next hit. If the pattern does not match, the next filter is applied to that hit. You can create either a single Exclude Filter with multiple patterns separated by ‘|’ or you can create multiple Exclude Filters with a single pattern each.
Include Filters are applied with the reverse logic. When you apply an Include Filter, the hit is thrown away if the pattern does not match the data. If you apply multiple Include Filters, the hit must match every applied Include Filter in order to save the hit. To include multiple patterns for a specific field, create a single include filter that contains all of the individual expressions separated by ‘|’.
Use the Case Sensitive option to specify whether the filter is applied with or without case sensitivity.” Taken directly from Google Support
If you are not familiar with the exclude filters feature in Google Analytics you can check out this article below:
USING FILTERS & EXCLUSIONS TO IMPROVE GOOGLE ANALYTICS DATA
But what we are going to dwelve into within this article is the usage of advanced filters to weed out annoying direct spam traffic which is seemingly difficult for some to filter out due to their data being ‘not set‘. So with no delay, our first suggestion would be to:
Check City, Service Provider and Network Domain:
At times the culprit can be a single crawler, bot or service causing the problem. It could also be an ‘uptime’ monitor that was initiated whether it be by you or someone else. Uptime monitoring tests the availability of your website, applications and servers. If it all originates from one region it is quite easy to build a filter and filter it out. But as we mentioned before this can be off or not available due to the annoying ‘not set’ data.
Check Browser and Operating System Versions:
If the traffic originates from different regions, it may be caused by one of several bots that have been spread out on the internet. You can look for a spike in direct traffic from old browser versions such as Firefox 11.0, Internet Explorer 7.0, or Chrome 18.0.1015.168. Programmers at times utilize the code they found on the internet and it utilizes embedded versions of these browsers. Real people tend to upgrade their devices; bots tend to stay the same. Not sure about the particular version, you can analyze the traffic utilizing that version over the past 6 months and check to see if it is all direct traffic, or if it includes any organic search visits.
Check Browser Size:
When programmer’s design a bot, the code can spread across browser versions, and maybe even operating systems, but if they open a hidden browser window to visit your page, it is probably the same size, every time. In particular, you can go ahead and look for sizes such as 620×480 or 610×480 that have a lot of direct visits but no other sources.
The filters we suggest using:
After completing a significant analysis on a few of our affiliate sites, We here at KripiTech came up with these series of filters, and please note! They must be implemented IN ORDER. It operates based on the explanation that some of the direct spam traffic was coming from old browsers or specific browser sizes or versions, and it landed only on the home page of the web site. Sometimes with a very low bounce rate, low unique pageview count but high count of pageviews in general. Please also note that it could also be a few people operating really old computers that happen to match the profile, but they could also visit other pages of you website as well. If the session duration is really high depending on the niche of your site then it just might be spam, especially if its all on one page with a low bounce rate. We did not want to filter out this other traffic, especially if it included ecommerce transactions.
Filtering out visits from a specific service provider is quite simple actually, but how do you eliminate visits based on multiple criteria, such as visits to the home page from a specific browser and version? Google Analytics Advanced Filters gives us webmaster a solution for this.
Careful Notice: this is not a complete guide of utilizing advanced filters. Such a topic is really advanced. What we here at KrispiTech offer is as an example of what we utilized for our own clients so that others can benefit as well. Please bear in mind that this worked for us in the past…it may not work in the future.
First what you want to do is to Capture the Browser and Browser Version:
The first filter gets the browser (Firefox, Internet Explorer, Chrome, etc) and the browser version (11.0, 43.0.2357.130, etc) into a single data set that we can work with, we utilized a field called “User Defined”. After the filter runs, the User Defined field will contain data such as “Firefox 11.0”.
Tag Unwanted Traffic
Next comes a series of filters that changes the User Defined value to “Direct Spam” if it happens to use a specific browser version AND is a hit to the home page (in this case, “/Home”; yours might be “/” in which case use the expression “^/$“).
The list of browser versions I have used includes (verify on your own site before you adopt these):
Firefox 11.0
Internet Explorer 7.0
Chrome 18.0.1025.168
Chrome 39.0.2171.95
Safari 5.1
Chrome 43.0.2357.130
Internet Explorer 9.0
Internet Explorer 8.0
Firefox 3.6.28
Firefox 12.0
Tag Other Unwanted Traffic
You may have other criteria, such as browser size, that you may also want to utilize – include similar filters for them as well.
Exclude Tagged Traffic
Finally, examine the value in User Defined, and if it has been tagged as “Direct Spam”, then exclude it.
Put The Filters In Order!
This is critical! The first filter (capturing the browser and version) MUST be first, and the last filter (excluding the spam) MUST be last. Google will execute the filters in order, and that is the only way they will accomplish the task.
Enjoy
You should now see a drop in direct spam visits as the filters take effect (use a segment for historical reporting). Just remember that things change and you will need to revisit the effectiveness of the filters as time passes, especially if you see a jump in direct visits again.
This article describes a technique I used that was effective for me. Every website is different– you need to do your own analysis before implementing a solution.
Experiencing difficulties with your Device, check out our “How To” page on how to resolve some of these issues.
