CTPI Benchmark & Analysis – There is no other mainstream software category that evolves as fast the web browser. Recent announcements by Mozilla to upgrade its JavaScript and graphics engine were reason enough for us to compare the developer versions of upcoming IE10, Firefox 9, Chromium 15, Opera 12 and Safari 5.1. And boy, were we surprised!
It has been some time since our last browser performance test and since our last update to our CT Performance Index (CTPI). One of the reasons simply was that the browser speed race in JavaScript performance was largely declared over at the time Firefox 4 was released earlier this year and Google indicated that there would be only be marginal improvements in the future, even if JavaScript execution remains the most critical component in future web apps as JavaScript will be the interface to hardware acceleration in technologies such as WebGL and WebCL.
Last week, Mozilla announced that it has found new ways to accelerate Firefox in JavaScript again – we found that claim to be accurate as Firefox 9 was up to 32% faster than Firefox 6 in a quick benchmark run. However, new versions of Firefox (starting with version 7) will also get Mozilla’s new Azure graphics backend that promises much better graphics and Canvas 2D performance that can challenge the class-leading graphics engine in IE9 and IE10. Our JavaScript and HTML5 benchmark index revealed that browser performance has shifted within half a year.
A few notes
Our performance index does not compare absolute benchmark result against absolute benchmark result. In the end, what does a score in WebViz of “2823″ really mean? Our index compares the performance of a browser to the performance of Firefox 3.6.11, which we use as our baseline benchmark with a performance level of 100% in each category. A performance of, for example, 150% would mean that this specific browser delivers 50% more performance than Firefox 3.6.11. Rather than giving our readers absolute numbers, we believe that a relative performance index is much more useful. we are providing the absolute performance numbers in a separate table as well.
Keep in mind that all test results heavily depend on the software and hardware test platform. All our browser tests use an off-the-shelf Dell Studio XPS 7100 PC that Dell has donated to us for the purpose of browser performance evaluation. The hardware includes an AMD Phenom II X6 1055T processor with six CPU cores. The PC has 8 GB of memory and integrates an AMD Radeon 5870 graphics card and runs on an up to date Windows 7 OS. Depending on the hardware in your PC, your benchmark results may vary. However, our benchmark parcours is designed to test a wide variety of browser features and balance the strengths of each browser.
Our JavaScript run includes Sunspider 0.9.1 (50 iterations, weighed at 60%), Mozilla Kraken (20%) and Google V8 (20%). Our HTML5 evaluation includes Asteroids (Canvas 2D effects, 20%), WebViz Bench (25%), Mandelbrot Canvas 2D (15%), GUIMark 2 Chart (10%), GUIMark 2 Gaming (10%), GUIMark 2 (10%), Microsoft Pysychedelic Browsing (5%) and Microsoft Mr. Potato Gun (5%).
The browsers included in this test are:
Firefox 3.6.11 (baseline browser)
IE10 PP2 Firefox 9 Nightly (August 31 build) Chromium 15.0.871.0 Build 99507 Safari 5.1Safari 5.1 stands out as it is a stable and not a final version. Apple does not provide easy access to developer versions of its browser and the raw Webkit build does not truly reflect what Apple’s next browser may be. However, we thought it would be interesting to see the difference of this browser in comparison to the developer version of the other upcoming Webkit browser, Chromium. That said, it is no secret that Safari not a competitive browser on Windows anymore and that is in desperation need of a new installation process, much memory management as well as JavaScript and HTML5 enhancements. So think of Safari as a complementary extra, but not as critical information in this test.
JavaScript: Firefox Catches Up
There is little denying that Chromium owns this discipline. It is more than 7.3x faster than Firefox 3.6 across Sunspider, V8 and Kraken. Firefox 9 follows with a 5.2x advantage, Opera with 3.5x, IE10 with 3.0x and Safari with 2.3x.
Chromium wins all three disciplines with a significant margin, while we notice that Firefox 9 is slowly catching up with Google and may be able to match Chrome with its new IonMonkey JavaScript engine. IE10 has fallen behind again: Chromium is 33% faster in Sunspider, 74% in Kraken and 272% in V8. Opera, which dominated the JavaScript discipline in the past, is just ahead of IE10 and placed third twice and placed fourth once in this run. Both Opera 12 and IE10 need JavaScript enhancements desperately as this discipline is driven by Google and Mozilla these days. There is no reason to use Safari 5.1 on Windows these days and Apple may want to think about removing the claim as world’s fastest browser, as long as Apple considers Windows a part of this world.
Firefox 9, which is scheduled for a December 20 release showcases the impressive improvement Mozilla has made in just one year. Compared to Firefox 3.6, Firefox 9 is 4.2x faster in Sunspider. 4.3x faster in Kraken and 9.4x times ahead in Google V8.
Wolfgang Gruener in Products on September 03
No comments:
Post a Comment