Behind the Scenes: How Adblocking Lists Work

David Barton Adblocking, Uncategorized Leave a Comment

Lists are at the heart of adblocking. The adblocking community just needs to add a line of text to a list to block an ad. The largest of these lists is called EasyList. You’ve probably seen EasyList mentioned in one of the many articles about adblocking appearing recently, but you may not be sure exactly what it does and how adblocking software depends on it and other lists to function. Read on for a layman’s guide to how lists operate behind the scenes of adblocking.

Black and White
It can be helpful to think in terms of a blacklist and a whitelist to understand how adblocking works. When content on a website, such as an ad, is blocked, then it has effectively been blacklisted. If a website successfully convinces a visitor to turn off their adblocking software, then the entire website has been whitelisted.

In fact, the blacklist and whitelist exist side by side in lists such as EasyList. These lists consist of tens of thousands of rules needed to tell the adblocking software how to behave. A rule called a filter acts to block content, while another rule, called an exception, allows content to be displayed. These filters and exceptions are composed of text and special characters that tell the adblocking software everything it needs to know to block or allow content.

From Generic to Specific
A filter can be brutally simple and generic, such as /images/ads/*1, which will block any images being loaded from the “ads” directory on a website, or it can be more specific, such as ##.top-banner-ad-container2, which, for example, removes the uppermost ad slot from The Guardian website

Guardian

The screenshots below, taken from the online version of the Irish Independent newspaper, show how a handful of rules can strip several ads from the page and dramatically change both the experience for the visitor and the revenue for the publisher. In the second screenshot, Adblock Plus helpfully shows the rules (in red) actively removing content from the page.

Independent
In Firefox, click on ABP and select “Open blockable content” to view rules

While external lists like EasyList are the usual way that adblocking software receives its instructions, users can easily add custom rules. For instance, you can hide the logo image on the Adblock Plus website simply by adding the line ###logo3 to the ABP custom filters.

ABP Filtered

In Firefox, click on ABP and select “Filter Preferences” then “Custom filters” to add filters

Filters and exceptions don’t even need to be separate. They can be nested inside each other and combined into a single rule. This is useful if, for instance, the adblocker wants to remove ads coming from only a specific source on a website but wants to allow other ads through, as in this example where a user was offended by American Apparel advertising (graphic content) on Daring Fireball. In this case, another user contributed and explained the solution.

SuperUser

While adblocking rules can be complex and subtle 4, the bulk of everyday adblocking is carried out by the simpler, generic filters5. These are the lines of text that indiscriminately strip out millions of ads every day across the Internet and threaten the very existence of publishers everywhere.

Community of Lists
EasyList is predominantly aimed at English-language websites, but there are lists for other countries and regions, such as Germany and France. More aggressive lists even target what the authors call “annoyances”, such as social media buttons, extensively changing the appearance – and loading speed – of all websites. Other lists, such as the optional EasyPrivacy, are dedicated to protecting privacy or defeating malware. Some of these lists are added by default in independent alternatives to Adblock Plus. Because lists are added as subscriptions, adblocking software in the browser is automatically updated on a regular basis with new rules.

Individuals are behind the maintenance of all these lists and they depend on community interaction to identify the content to be blocked. The EasyList forums have a few thousand members and rely on a small number of official authors. While this places some human limitations on the creation of adblocking rules and suggests that adblocking could be overwhelmed with technical countermeasures or legal attacks, Adblock Plus makers Eyeo, with its no-doubt impressive war chest built up from payments by advertisers to be whitelisted, is deeply involved in supporting the EasyList community. On top of that, any well-publicised countermeasures, such as the creation of an adblock wall by WIRED, rapidly leads to new filters.

If you want to examine EasyList, you can easily view the latest rules. It’s just a long text file that you can read and search straight from your browser or in any text editor, such as Notepad or TextEdit. Even if you don’t see your website on the list, there are certain to be some filters that target ads on your pages and neatly remove your ability to monetize your content.

 

Notes

 

  1. This rule simply targets the directory images and subdirectory ads – often used in the past to store advertising content – with the wildcard asterisk character used to tell the adblock software not to show any file coming from that source. EasyList includes dozens of variations on this rule, e.g. a variety of directories are targeted with /*$image, which would block any image filetype from being loaded.
  2. ## is an instruction to select an element in the HTML code used to render a webpage, in this case an element with the class top-banner-ad-container
  3. ## selects an element on the page and the logo element is a div, or division, in the HTML code behind the Adblock Plus page.
  4. Regular expressions can be used to target elements with remarkable accuracy and control, although these are discouraged, as they slow down page rendering because the browser needs to do more work to figure out what to block.
  5. The filter examples above and in the main image behind the title of the post are taken from the current (as of March 21, 2016) rules in EasyList. Adblock Plus provides a helpful guide to writing EasyList rules, if you want to examine any rules in more detail.