- Date: January 24, 2008
- Author: Don Jackson, Senior Security Researcher
Computer hackers have taken advantage of the acceptance of these packers as suboptimal network optimization tactics and are using them as a way to evade and bypass security controls on the gateway and at the host. Consequently, exploits or other malicious code is delivered successfully because of the packer's ability to bypass anti-virus and IDS/IPS and directly to a user's vulnerable system.
It's important that we have some basic knowledge of the most common packers and the tools that use them.
The Dojo Toolkit and Dojo ShrinkSafe
- It does not change things that break the API, like renaming class names to something short in order to save space.
It doesn't use anything that makes detection of malicious scripting too difficult. It keeps code debuggable. It saves space by removing comments and whitespace. Because of the way it does this, it doesn't produce the very smallest of files, and it doesn't impact performance. Many web developers are oblivious to the size vs. performance equation, but not the Dojo team. They even suggest not using it at all: "Gzipping content on the wire [mod_gzip, for example] is the next obvious improvement for those deploying applications." They have an excellent write up here:
- YUI Compressor: Uses YUI Compressor to clean whitespace and rename internal variables to shorter strings. Does not use eval.
- JsMin Compression: Removes comments and whitespace.
- No Documentation: Uncompressed, removes only documentation/comments.
- Full Source: Includes documentation/comments. Recommended while testing & building.
According to Mootool, versions of their library are used on web sites for W3C, CNET, Ubuntu, Joomla, Gamespot, and Chrysler Jeep. Almost all of them use a compressed version. Gamespot, for example, uses the library as a file called mootools.js which has been packed with the Dean Edwards packer.
A build which contains all the Mootools v1.11 functionality and is not compressed at all weighs in at 180 KB.
More information on the Dean Edwards packer appears later. Let's compare the others:
The YUI Compressor
The "Yahoo! User Interface" Compressor also uses Rhino. Because it works in the Rhino parsing stream and has some sense of scope and context of variables, almost the same way as Dojo ShrinkSafe, it will not mangle the public API.
It is more aggressive on renaming local variables and classes and compressing literal values. It has more of a client performance impact than code packed with Dojo ShrinkSafe. This also makes it more difficult to audit and to decipher benign code from malicious code, especially in "Trojaned libraries" where one or more commonly used functions are used to call locally-scoped exploit code which has had it's namespace squished and jumbled.
More information can be found here: http://developer.yahoo.com/yui/compressor/
That page provides useful information such as a study on numbers of cache hits and misses, important data if one is determining impact of compression on networks. However, like all other compressor developers except the Dojo team, they ignore the fact that better options for network optimization exist.
Originally implemented in C as an MS-DOS program, it has been ported to many other languages by third parties.
Because of the way it handles whitespace and some confusing patterns, "a + ++b", which it will turn into "a+++b" and is interpreted according to the ECMAScript standard as "a++ + b", which is not what the original code intended.
Therefore, this compressor is not considered safe because it can cause unintended changes in your code. It works best for simple, well written scripting that follows one of the standard best practices coding styles. One option is to first pipe the input through JSLint (written by the same author). Using our previous example, this would produce "a + (++b)". That output could then be piped into JSMin. Ironically, the addition of the parentheses makes the linted-and-minified version of this expression exactly the same size as the original: the strings "a + ++b" and "a+(++b)" are both seven characters .
While producing relatively small files and having no client-side performance impact, the unsafe behavior of its brittle pattern substitution design makes it more likely to break code than packers that operate on a parser stream. This is typically undesirable, as output must have its functionality verified separately from the source (that is to say two testing phases are required).
JSMin does not make it much more difficult to tell benign script from malicious script. However, that is not the reason it's not used by many cyber criminals.
Dean Edwards Packer and IE7/IE8 Patches
Additionally, this packer has to include a significant amount of unpacking logic. That logic is implemented in code attached to every packed script, which means there are many common blocks of script for which the packed script is actually larger than the original.
The unpacking stub itself uses some dubious functions. The YUI Compressor developers link to information on "evil features" as described by the developer of JSLint, an ECMAScript standard-compliant tool which aims to reduce coding errors due to poor coding practices. From the JSLint documentation section titled "eval is evil":
- Douglas Crockford, 2002, http://www.jslint.com/help.html
The Dean Edwards packer makes it difficult to tell benign code from malicious code. Although it does not operate on a parser stream, its design allows it to safely pack pre-obfuscated exploits which may include patterns ambiguous to other pattern-based packers. This makes it attractive to attackers.
Because of its use of eval combined with other certain methods (such as String.fromCharCode) and layout/context of certain types of tokens, the output of the Dean Edwards packer is very similar, statistically, to other known malicious code. This would normally mean that attackers would gain nothing towards evasion by using it. However, it is now being used more in known malicious scripts than in known benign code. Why?
Because the Dean Edwards packer output is so similar to malicious code, but so widely used in benign code -- consider the popularity of IE7.js -- security companies have basically whitelisted its output based on its signature unpacking stub code which begins, cleverly:
Researchers have verified the whitelist status of this packer in at least one product by placing this stub around known exploit code. The exploit code alone generated an alert, which the (unpacked) exploit code surrounded by the packer stub sailed right through.
For years, in detecting malicious code using this packer, the risk of a false negative was very small compared to a risk of a false positive. It simply wasn't used by the cyber underground. That all changed with the success of an XSS worm which infected 700,000 users of the Orkut social networking service in November 2007.
In December 2007, a couple of security companies' products began blocking packed scripts with new signature updates. Because of complaints of false alarms from end users, both relented, leaving their customers at risk of encountering new packed exploits. In fact, shortly after re-whitelisting packed scripts, a SQL injection flaw in the web site of one of the security companies, allowed attackers to place script directing visitors to, ironically (or not?) , a packed RealPlayer exploit which installed a rootkit and backdoor from China.
Would-be attackers took notice, documented the whitelist status, and began packing their wares in order to successfully evade filters. One actual attack used both packed and unpacked versions of another RealPlayer exploit so the attackers could study the relative effectiveness of the use of this packer to evade filters. Heuristics often fail when encountering "Trojaned libraries" which are actually copies of common libraries laced with exploit code. These look and even function -- except under one unique circumstance or trigger -- identical to their benign counterparts. The files could be named the same as well. Packing increases the chance that heuristics will fail to detect previously unknown exploits, such as the packed 0-day QuickTime exploit uncovered in January 2008.
Today, there are more documented packed malicious scripts than packed benign scripts.
While the use of packers is widespread, all have drawbacks. These include:
- The inability to easily verify and audit code
- The administrative overhead of repacking code for each change
- Suboptimal compression
- The increased risk of false negatives which may lead to a site being used to spread malicious code
- The increased risk of false positives, which may lead to a site or some of its functions being blocked by security controls
- Noticeable negative impact on client-side performance.
Site owners, operators, developers and administrators can achieve intended results -- typically reducing the number of bytes downloaded from the server -- with greater degree of success and fewer side-effects using one or more alternative tactics:
- Reliance on increases in average available bandwidth
- Reliance on local and network caching
- Using only safe whitespace/comment reduction techniques
- Automatic application of safe techniques as a last step in the publishing process
- The use of mod_deflate/mod_gzip for compressing the HTTP response data
Replacing packed code is not generally an expensive proposition. Usually, libraries are available in unpacked versions. Checking the packed code out of a source code control/versioning system, copying over the unpacked code, and checking it back in to be published just like any normal code change is a quick procedure with immediate benefits.