[TIMOB-8663] R&D: Evaluate various obfuscation / minification engines for common use across platforms

GitHub Issue	n/a
Type	Sub-task
Priority	High
Status	Closed
Resolution	Fixed
Resolution Date	2012-04-17T10:27:06.000+0000
Affected Version/s	n/a
Fix Version/s	Release 2.1.0, Sprint 2012-08
Components	Tooling
Labels	core
Reporter	Marshall Culpepper
Assignee	Max Stepanov
Created	2012-04-10T15:45:13.000+0000
Updated	2012-06-15T16:29:04.000+0000

Description

We need to investigate the various JS obfuscation and minification options in the wild and decide on a common approach across platforms. Some obvious contenders: * Google Closure Compiler * YUI compressor When evaluating the various options, we should keep in mind the ability to feed in more data (i.e. an AST or other API heuristics) to further optimize the minification process

Comments

Chris Barber 2012-04-10

FWIW, the ultimate goal is mobile web's AST parser will eventually replace closure compiler. Like closure compiler, it will obfuscate through minification. There was no plan to further obfuscate the code as that would probably increase file size. Even if we did some sort of encryption or heavy obfuscation, anybody with Web Inspector or Firebug could pretty much defeat any obfuscation. Mobile web is capable of "lazy loading" code, so you could gain a level of obfuscation by pulling down sensitive data after the page loads, but we're not quite there yet with the tooling require to support this.
Max Stepanov 2012-04-11

Our options so far: - [Google Closure Compiler](https://developers.google.com/closure/compiler/) - [YUI Compressor](http://developer.yahoo.com/yui/compressor/) - [UglifyJS](https://github.com/mishoo/UglifyJS) - [/packer/](http://dean.edwards.name/packer/) - mine's favorite, but want to be objective :) - [ShrinkSafe (DOJO compressor)](http://dojotoolkit.org/reference-guide/1.7/util/shrinksafe/index.html) - [All-in-one tool (wro4j)](http://code.google.com/p/wro4j/) will go in detail over each one

Max Stepanov 2012-04-11

Original Example Code:

   var win = Titanium.UI.currentWindow;
   
   var annotation = Titanium.Map.createAnnotation({
   	latitude:42.334537,
   	longitude:-71.170101,
   	title:"Boston College",
   	subtitle:'Newton Campus, Chestnut Hill, MA',
   	animate:true,
   	leftButton:'../images/atlanta.jpg',
   	image:"../images/boston_college.png"
   });
   
   var boston = {latitude:42.334537,longitude:-71.170101,latitudeDelta:0.010, longitudeDelta:0.018};
   
   //
   // CREATE MAP VIEW
   //
   var mapview = Titanium.Map.createView({
   	mapType: Titanium.Map.STANDARD_TYPE,
   	region: boston,
   	animate:true,
   	regionFit:true,
   	userLocation:true,
   	annotations:[annotation]
   });
   
   // read in our routes from a comma-separated file
   var f = Ti.Filesystem.getFile(Ti.Filesystem.resourcesDirectory,'examples','route.csv');
   var csv = f.read();
   var points = [];
   var lines = csv.toString().split("\n");
   for (var c=0;c<lines.length;c++)
   {
   	var line = lines[c];
   	var latlong = line.split(",");
   	if (latlong.length > 1)
   	{
   		var lat = latlong[0];
   		var lon = latlong[1];
   		var entry = {latitude:lat,longitude:lon};
   		points[c]=entry;
   	}
   }
   
   // route object
   var route = {
   	name:"boston",
   	points:points,
   	color:"red",
   	width:4
   };
   
   // add a route
   mapview.addRoute(route);
   
   win.add(mapview);
   
   // when you click the logo, remove the route
   annotation.addEventListener('click',function()
   {
   	mapview.removeRoute(route);
   });
   
   // map view click event listener
   mapview.addEventListener('click',function(evt)
   {
   	var clickSource = evt.clicksource;
   	Ti.API.info('mapview click clicksource = ' + clickSource);
   });

Max Stepanov 2012-04-11

[ShrinkSafe](http://dojotoolkit.org/reference-guide/1.7/util/shrinksafe/index.html)

Part of DOJO build toolchain. Java-based, utilizes Rhino to parse code. MPL 1.1 license. Safe about public variables and APIs. Almost no configuration options.

   var win=Titanium.UI.currentWindow;
   var annotation=Titanium.Map.createAnnotation({latitude:42.334537,longitude:-71.170101,title:"Boston College",subtitle:"Newton Campus, Chestnut Hill, MA",animate:true,leftButton:"../images/atlanta.jpg",image:"../images/boston_college.png"});
   var boston={latitude:42.334537,longitude:-71.170101,latitudeDelta:0.01,longitudeDelta:0.018};
   var mapview=Titanium.Map.createView({mapType:Titanium.Map.STANDARD_TYPE,region:boston,animate:true,regionFit:true,userLocation:true,annotations:[annotation]});
   var f=Ti.Filesystem.getFile(Ti.Filesystem.resourcesDirectory,"examples","route.csv");
   var csv=f.read();
   var points=[];
   var lines=csv.toString().split("\n");
   for(var c=0;c<lines.length;c++){
   var line=lines[c];
   var latlong=line.split(",");
   if(latlong.length>1){
   var lat=latlong[0];
   var lon=latlong[1];
   var entry={latitude:lat,longitude:lon};
   points[c]=entry;
   }
   }
   var route={name:"boston",points:points,color:"red",width:4};
   mapview.addRoute(route);
   win.add(mapview);
   annotation.addEventListener("click",function(){
   mapview.removeRoute(route);
   });
   mapview.addEventListener("click",function(_1){
   var _2=_1.clicksource;
   Ti.API.info("mapview click clicksource = "+_2);
   });

In order to minimize top level scope variables the whole source code needs to be wrapped into *Immediate function* _(function(){})()_ and then it needs to be stripped of.

   var _1=Titanium.UI.currentWindow;
   var _2=Titanium.Map.createAnnotation({latitude:42.334537,longitude:-71.170101,title:"Boston College",subtitle:"Newton Campus, Chestnut Hill, MA",animate:true,leftButton:"../images/atlanta.jpg",image:"../images/boston_college.png"});
   var _3={latitude:42.334537,longitude:-71.170101,latitudeDelta:0.01,longitudeDelta:0.018};
   var _4=Titanium.Map.createView({mapType:Titanium.Map.STANDARD_TYPE,region:_3,animate:true,regionFit:true,userLocation:true,annotations:[_2]});
   var f=Ti.Filesystem.getFile(Ti.Filesystem.resourcesDirectory,"examples","route.csv");
   var _5=f.read();
   var _6=[];
   var _7=_5.toString().split("\n");
   for(var c=0;c<_7.length;c++){
   var _8=_7[c];
   var _9=_8.split(",");
   if(_9.length>1){
   var _a=_9[0];
   var _b=_9[1];
   var _c={latitude:_a,longitude:_b};
   _6[c]=_c;
   }
   }
   var _d={name:"boston",points:_6,color:"red",width:4};
   _4.addRoute(_d);
   _1.add(_4);
   _2.addEventListener("click",function(){
   _4.removeRoute(_d);
   });
   _4.addEventListener("click",function(_e){
   var _f=_e.clicksource;
   Ti.API.info("mapview click clicksource = "+_f);
   });

Max Stepanov 2012-04-11

[/packer/](http://dean.edwards.name/packer/)

JavaScript-based. [Source code](http://code.google.com/p/base2/source/browse/trunk/src/apps/packer) available. MIT License. Analyses only source code. No AST. Runs 4 phases: minifier, shrinker, privates encoder, base62 encoder. The first 3 phases are pretty ordinary comparing to others. *The most interesting phase is base62 encoder (0-9A-Za-z) which I think we should adapt regardless of chosen engine*.

   eval(function(p,a,c,k,e,r){e=function(c){return(c<a?'':e(parseInt(c/a)))+((c=c%a)>35?String.fromCharCode(c+29):c.toString(36))};if(!''.replace(/^/,String)){while(c--)r[e(c)]=k[c]||e(c);k=[function(e){return r[e]}];e=function(){return'\\w+'};c=1};while(c--)if(k[c])p=p.replace(new RegExp('\\b'+e(c)+'\\b','g'),k[c]);return p}('2 o=5.F.G;2 d=5.e.H({g:p.q,h:-r.s,I:"J K",L:\'M N, O P, Q\',t:6,R:\'../u/S.T\',U:"../u/V.W"});2 i={g:p.q,h:-r.s,X:0.Y,Z:0.10};2 3=5.e.11({12:5.e.13,14:i,t:6,15:6,16:6,17:[d]});2 f=j.v.18(j.v.19,\'1a\',\'7.k\');2 k=f.1b();2 8=[];2 l=k.1c().w("\\n");1d(2 c=0;c<l.x;c++){2 y=l[c];2 9=y.w(",");1e(9.x>1){2 z=9[0];2 A=9[1];2 B={g:z,h:A};8[c]=B}}2 7={1f:"i",8:8,1g:"1h",1i:4};3.1j(7);o.1k(3);d.C(\'m\',D(){3.1l(7)});3.C(\'m\',D(a){2 b=a.E;j.1m.1n(\'3 m E = \'+b)});',62,86,'||var|mapview||Titanium|true|route|points|latlong||||annotation|Map||latitude|longitude|boston|Ti|csv|lines|click||win|42|334537|71|170101|animate|images|Filesystem|split|length|line|lat|lon|entry|addEventListener|function|clicksource|UI|currentWindow|createAnnotation|title|Boston|College|subtitle|Newton|Campus|Chestnut|Hill|MA|leftButton|atlanta|jpg|image|boston_college|png|latitudeDelta|010|longitudeDelta|018|createView|mapType|STANDARD_TYPE|region|regionFit|userLocation|annotations|getFile|resourcesDirectory|examples|read|toString|for|if|name|color|red|width|addRoute|add|removeRoute|API|info'.split('|'),0,{}))

Max Stepanov 2012-04-11

[UglifyJS](https://github.com/mishoo/UglifyJS) (Editor's Choice)

JavaScript-based. Developed on Node.JS. BSD license. Parses AST tree then does various manipulations on it. Highly customizable. Supports variable name reserving. Can return AST as output - might be useful for API heuristics. Performs a lot of optimizations that could potentially lead to a faster code: simple constant expressions, property access as associated array, IF statements optimizations, joins var declarations etc.

   var win=Titanium.UI.currentWindow,annotation=Titanium.Map.createAnnotation({latitude:42.334537,longitude:-71.170101,title:"Boston College",subtitle:"Newton Campus, Chestnut Hill, MA",animate:!0,leftButton:"../images/atlanta.jpg",image:"../images/boston_college.png"}),boston={latitude:42.334537,longitude:-71.170101,latitudeDelta:.01,longitudeDelta:.018},mapview=Titanium.Map.createView({mapType:Titanium.Map.STANDARD_TYPE,region:boston,animate:!0,regionFit:!0,userLocation:!0,annotations:[annotation]}),f=Ti.Filesystem.getFile(Ti.Filesystem.resourcesDirectory,"examples","route.csv"),csv=f.read(),points=[],lines=csv.toString().split("\n");for(var c=0;c<lines.length;c++){var line=lines[c],latlong=line.split(",");if(latlong.length>1){var lat=latlong[0],lon=latlong[1],entry={latitude:lat,longitude:lon};points[c]=entry}}var route={name:"boston",points:points,color:"red",width:4};mapview.addRoute(route),win.add(mapview),annotation.addEventListener("click",function(){mapview.removeRoute(route)}),mapview.addEventListener("click",function(a){var b=a.clicksource;Ti.API.info("mapview click clicksource = "+b)})

Has the same issue with shortening _global_ variables as ShrinkSafe. Use of *Immediate function* is encouraged.

   var a=Titanium.UI.currentWindow,b=Titanium.Map.createAnnotation({latitude:42.334537,longitude:-71.170101,title:"Boston College",subtitle:"Newton Campus, Chestnut Hill, MA",animate:!0,leftButton:"../images/atlanta.jpg",image:"../images/boston_college.png"}),c={latitude:42.334537,longitude:-71.170101,latitudeDelta:.01,longitudeDelta:.018},d=Titanium.Map.createView({mapType:Titanium.Map.STANDARD_TYPE,region:c,animate:!0,regionFit:!0,userLocation:!0,annotations:[b]}),e=Ti.Filesystem.getFile(Ti.Filesystem.resourcesDirectory,"examples","route.csv"),f=e.read(),g=[],h=f.toString().split("\n");for(var i=0;i<h.length;i++){var j=h[i],k=j.split(",");if(k.length>1){var l=k[0],m=k[1],n={latitude:l,longitude:m};g[i]=n}}var o={name:"boston",points:g,color:"red",width:4};d.addRoute(o),a.add(d),b.addEventListener("click",function(){d.removeRoute(o)}),d.addEventListener("click",function(a){var b=a.clicksource;Ti.API.info("mapview click clicksource = "+b)})})()

Max Stepanov 2012-04-11

[YUI Compressor](http://developer.yahoo.com/yui/compressor/)

Java-based, utilizes Rhino to parse code. MPL 1.1/BSD license. Optional private variables shortening. Pretty much the same as ShrinkSafe.

   var win=Titanium.UI.currentWindow;var annotation=Titanium.Map.createAnnotation({latitude:42.334537,longitude:-71.170101,title:"Boston College",subtitle:"Newton Campus, Chestnut Hill, MA",animate:true,leftButton:"../images/atlanta.jpg",image:"../images/boston_college.png"});var boston={latitude:42.334537,longitude:-71.170101,latitudeDelta:0.01,longitudeDelta:0.018};var mapview=Titanium.Map.createView({mapType:Titanium.Map.STANDARD_TYPE,region:boston,animate:true,regionFit:true,userLocation:true,annotations:[annotation]});var f=Ti.Filesystem.getFile(Ti.Filesystem.resourcesDirectory,"examples","route.csv");var csv=f.read();var points=[];var lines=csv.toString().split("\n");for(var c=0;c<lines.length;c++){var line=lines[c];var latlong=line.split(",");if(latlong.length>1){var lat=latlong[0];var lon=latlong[1];var entry={latitude:lat,longitude:lon};points[c]=entry}}var route={name:"boston",points:points,color:"red",width:4};mapview.addRoute(route);win.add(mapview);annotation.addEventListener("click",function(){mapview.removeRoute(route)});mapview.addEventListener("click",function(b){var a=b.clicksource;Ti.API.info("mapview click clicksource = "+a)});

With *Immediate function* trick:

   var h=Titanium.UI.currentWindow;var d=Titanium.Map.createAnnotation({latitude:42.334537,longitude:-71.170101,title:"Boston College",subtitle:"Newton Campus, Chestnut Hill, MA",animate:true,leftButton:"../images/atlanta.jpg",image:"../images/boston_college.png"});var b={latitude:42.334537,longitude:-71.170101,latitudeDelta:0.01,longitudeDelta:0.018};var o=Titanium.Map.createView({mapType:Titanium.Map.STANDARD_TYPE,region:b,animate:true,regionFit:true,userLocation:true,annotations:[d]});var i=Ti.Filesystem.getFile(Ti.Filesystem.resourcesDirectory,"examples","route.csv");var g=i.read();var n=[];var q=g.toString().split("\n");for(var k=0;k<q.length;k++){var p=q[k];var e=p.split(",");if(e.length>1){var j=e[0];var a=e[1];var m={latitude:j,longitude:a};n[k]=m}}var l={name:"boston",points:n,color:"red",width:4};o.addRoute(l);h.add(o);d.addEventListener("click",function(){o.removeRoute(l)});o.addEventListener("click",function(f){var c=f.clicksource;Ti.API.info("mapview click clicksource = "+c)})

Max Stepanov 2012-04-11

[Google Closure Compiler](https://developers.google.com/closure/compiler/)

Java-based, utilizes Rhino to parse code and build AST. Apache License 2.0 Only SIMPLE_OPTIMIZATIONS could be applied to source code. ADVANCED_OPTIMIZATIONS involves aggressive renaming which will break app if performed w/o declaring externs (literally _all Titanium APIs_). Also ADVANCED_OPTIMIZATIONS removes _dead code_ like functions defined but never explicitly called. See [dangers](https://developers.google.com/closure/compiler/docs/api-tutorial3#dangers) section for the full list of limitations. Could print AST for analysis.

   for(var win=Titanium.UI.currentWindow,annotation=Titanium.Map.createAnnotation({latitude:42.334537,longitude:-71.170101,title:"Boston College",subtitle:"Newton Campus, Chestnut Hill, MA",animate:!0,leftButton:"../images/atlanta.jpg",image:"../images/boston_college.png"}),boston={latitude:42.334537,longitude:-71.170101,latitudeDelta:0.01,longitudeDelta:0.018},mapview=Titanium.Map.createView({mapType:Titanium.Map.STANDARD_TYPE,region:boston,animate:!0,regionFit:!0,userLocation:!0,annotations:[annotation]}),
   f=Ti.Filesystem.getFile(Ti.Filesystem.resourcesDirectory,"examples","route.csv"),csv=f.read(),points=[],lines=csv.toString().split("\n"),c=0;c<lines.length;c++){var line=lines[c],latlong=line.split(",");if(1<latlong.length){var lat=latlong[0],lon=latlong[1],entry={latitude:lat,longitude:lon};points[c]=entry}}var route={name:"boston",points:points,color:"red",width:4};mapview.addRoute(route);win.add(mapview);annotation.addEventListener("click",function(){mapview.removeRoute(route)});
   mapview.addEventListener("click",function(a){Ti.API.info("mapview click clicksource = "+a.clicksource)});

Marshall Culpepper 2012-04-12

Thanks Max.. lots of good info here :) It sounds like ADVANCED_OPTIMIZATIONS in Closure might be a good bet if we can have an up-front declared list of Titanium APIs. Can you try manually creating the list for Closure, and use ADVANCED_OPTIMIZATIONS to see what the output is like?
Marshall Culpepper 2012-04-12

Also, since many of these use Rhino for parsing the Javascript, would that stop us from using newer ECMA 5 (or eventually Harmony) features in our platform?
Max Stepanov 2012-04-12

Marshall, I would say that using UglifyJS would be the best option. It should be easy to integrate with Node.JS build system if we switch to it. Plus it has configuration options for individual optimizations, rather than simple/advanced in Closure. Plus, since it's all in JavaScript, it will be easier for us to customize/change code. And given your concern about ECMA 5, UglifyJS seems only the option right now.
Max Stepanov 2012-04-12

Marshall, I don't see reasons to test Closure with ADVANCED_OPTIMIZATIONS since in addition to exports list requirement and does other undesirable stuff like dead code elimination and inconsistent property names mess.
Max Stepanov 2012-04-17

Conclusion: *UglifyJS running on top of Node.JS* Also it works with Rhino, but very slow.

JSON Source