Internet Security Tips and Advice

All you need to know about security leak tests

October 29th, 2008 by Igor Pankov


The term leak test, or leaktest, has become a popular buzzword among the security cognoscenti in the last couple of years. It’s in the news, it’s in the comparison charts, but what does it really mean? Why should regular PC users be concerned with leaktests and their implications in their day-to-day computer activity? How do leaktest results help you when it comes to choosing a robust security product?

These are just a few of the questions this article aims to answer. After reading this material, you’ll have all the information you need to understand leak tests and interpret their results.

What is a leaktest?

A leaktest is a tool or a set of procedures that attempts to determine a security product’s ability to prevent outbound data breaches that are designed to steal or compromise your personal information. As the name implies, leaktests are used to find out if a security product reliably protects your data against accidental or deliberate transmission through security barriers, often called data leak prevention.

Leaktests have been in existence since the appearance of the first personal firewall products more than five years ago, and their main objective has been to test the firewall’s ability to prevent unwanted applications from “phoning home” or otherwise communicating data across the Internet without the user’s consent. Leaktests have recently evolved into a broader category that includes tests using the simulated termination of a security product’s operation, controlling advanced program interactions, and other complex technologies that hackers might use to target personal information stored on users’ PCs.

If a product passes a certain leaktest, it means that the product has successfully blocked a would-be attack that’s based on a specific intrusion technique. There are many techniques known to be in use by hackers, and a robust security program should be ready to detect and rebuff all of them. New techniques are constantly being created by miscreants to get hold of personal data, so vendors of security products are constantly developing countermeasures to ensure their customers are properly protected.

When Windows XP came out in 2001, a number of malicious programs, like Trojans and spyware, already existed that could easily steal valuable user data such as login credentials or credit card information and transfer them to unauthorized third parties. In an effort to stop these breaches, security firms came up with firewall solutions that would block network activity initiated by malevolent applications by denying them outbound network access. To test firewalls’ performance, security professionals developed special tools that emulated attacks; these tools checked if the firewall was able to block such attacks from connecting by asking users to decide whether the activity should be permitted. These first leaktests were quite primitive, but they managed to expose significant deficiencies in certain firewall products.

The first leaktests used simple methods such as file name spoofing or running a trusted application with additional parameters that instructed it to send a certain text string to a target location with the goal of fooling the firewall into seeing the application as a trusted activity acting on its own behalf and consequently allowing this transfer. The earliest well-known leaktest was GRC’s Steve Gibson’s “Leak Test”, which simulated an attack in which a malicious application would rename itself to Internet Explorer (a legitimate Internet-enabled application) and determine whether the firewall was able to detect this change.

Much has changed since those days, and today’s leaktests are way more sophisticated, using advanced interaction mechanisms and network properties to simulate the data mining capabilities typical of today’s malware.

Leaktests examine the proactive protection capabilities of security solutions, checking how they respond to a particular intrusion technique, or attack vector. This is very different from the process used to examine antivirus solutions, where tests are used to determine whether a solution is resistant to a specific malware sample.

Techniques employed by leaktests

Leaktests are too varied to fall into convenient classifications for their operations, and most are based on different techniques for testing security products. These techniques are constantly evolving and improving, and the more leaketsts exist, the more rigorously security solutions are tested.

To generalize, leaktests will try to emulate one of the following techniques:

  • Impersonating a legitimate application installed on a computer or leveraging its access credentials to send information to the Internet (i.e., spoofing, trusted application launch with special parameters)
  • Interacting with a legitimate application using embedded Windows controls such as OLE Automation or DDE requests
  • Modifying active applications in memory, attaching malicious components to benign applications – for example, component injection, direct memory patching, creation of malicious threads
  • Using trusted network services and protocols to send unauthorized data in the hope that the firewall will miss the unconventional activity; such activities might include false DNS requests, BITS service exploitation, or lax ICMP filtering
  • Installing a new network adapter driver through which to route data
  • Disrupting or disabling the protective functions of an installed security application
  • Initiating system shutdown to check if the firewall monitors the activity of untrusted processes through to its completion
  • Intercepting keystrokes

Synopsys: program interactivity

The majority of leaktests were designed for Windows XP which, unlike Vista, does not verify a program’s permissions to interact with other installed programs or perform any other activity from the list above if the user has Administrator privileges. This situation creates the potential for exploitation, as any malicious program can piggy-back on a trusted, legitimate program as a means of carrying out targeted attacks. Gone are the days when malware would attempt to steal user data itself; now, it would use a legitimate application’s network access credentials with the firewall to transmit data. Security solutions need to be able to not only detect malicious programs, but be prepared to monitor the integrity of legitimate applications and the use of network resources for the advanced leak techniques used by malware.

Tradeoff: wordy alerts versus looser control

Thousands of internal interactions occur on a PC every hour. Of course, only a fraction of these are malicious. So if the firewall monitors and prompts the user regarding each operation, the user will be bombarded with alerts, making it impossible to do anything productive on the computer. As a solution to this limitation, security vendors have implemented a mechanism that “memorizes” a user’s response to a particular event so that, next time this event occurs, the previous input will be used to handle the event, and no alert is displayed. Additionally, vendors of leading security solutions such as Outpost Security Suite Pro, Kaspersky Internet Security and Comodo Firewall use online databases to automatically designate permissions for the majority of Windows applications, so the decisions are made in the background, without interrupting the user’s normal PC use. Windows Vista, with its new User Account Control (UAC) functionality, has made significant progress in stemming illegal or unauthorized activity. It does this by lowering the privileges until the user has allowed a particular operation by okaying the UAC’s foreground prompt. Unfortunately the responses given cannot be memorized, so the alert window displays start up again, frustrating the user once more.

Other vendors, such as Symantec and ESET, have chosen to control fewer events on the user’s computer, reducing the number of prompts displayed to the user. The downside of this approach is that the level of monitoring is reduced, resulting in less control over activities and the potential for some techniques actively being used in malware could bypass the protection. It’s no wonder, then, that these solutions fare quite poorly in group tests. Time will show who has the better approach, but considering that malware is becoming more sophisticated, an effective security solution really needs to control the maximum number of events on a computer, and require less user interaction.

Leaktest usage

Leaktests are safe applications designed to verify if a firewall is able to prevent an attack that uses different techniques to steal data. Leaktests can be downloaded from the Internet and executed on a user’s machine. If the security solution displays an alert when the test is run, it means that the solution successfully detected the leaktest activity and would most likely be able to deter real-world attacks based on the technique used in the leaktest.

Although successfully passing a leaktest doesn’t always mean the security solution is bullet-proof, it essentially means that it will do its best to protect a user in case a real attack strikes.

Interpretation of results

There are specialist organizations that carry out leak testing, the most active of which are Matousec Transparent Security and Firewall Leak Tester. They have vast information resources and update their leaktesting results whenever new security products come out or when updated leaktests are released. As a rule of thumb, the closer a security program to an absolute pass score of 100 percent, the more resistant to malware attack it is.

Leaktests versus other testing mechanisms

As noted earlier, leaktests test how well a security solution is equipped to combat malware that uses different intrusion techniques to bypass outbound protection. Leaktests are technique-centric, as opposed to virus testing, which is largely signature-oriented. Leaktests verify the potential of a solution to deter unknown attacks without the use of a particular threat signature.

The benefits of leaktests

It is hard to overestimate the contribution leaktests make to the security community. Leaktests are the primary techniques used to test whether a security product is capable of keeping unknown malware at bay by restricting its activity within the software environment of a PC. Leaktests have a practical value to the end user; many product reviews are complemented by leaktest results, which should give you a good indication of how well a security product is able to keep you safe from today’s widespread anonymous threats.


We hope this article has shed some light on the concept of leaktests and their relevance in testing the ability of a security product to prevent unauthorized outbound data leakage. Leaktests serve as a practical and effective tool in measuring the quality and scope of protection against the kinds of advanced breaches that are used to carry out actual attacks.

Posted in Security Insight

3 Responses

  1. Eugene

    looking forward for more information about this. thanks for sharing. Eugene

  2. da

    in the real world the “leak” will be caused by the user who cant make the right decision

  3. Dino Dubravcevic

    I have a exquisite request…
    Do I have your allowance to translate this article for studying purposes only… Please, respond to my mail…