Latest Shocking Web Browser Benchmarks

[Pae]

Check out these latest web browser benchmarks from ZDNet.

Proving itself a staggering 42 times faster at rendering JavaScript than IE 7, our benchmarks confirm Apple's Safari 4 browser, released in beta Tuesday, is the fastest browser on the planet.

I am one of those apple fanboys BUT I have always HATED Safari with a passion. After this though, I must say my curiousity is rising.

[Rilthyn]

I am absolutely shocked by these latest web browser benchmarks.

[Rosuav]

It's patently unfair to compare IE with anything else.

* IE does not conform to the standards. Ask any HTML/CSS/JS developer, especially anyone who's tried to make something look really pretty AND work across browsers, and they'll tell you that making something work in Firefox, Safari, Opera, etc, etc, etc, is easy, but making it work in both IE and any-of-the-above is hard.

* IE, well, is slow. And sluggish on low memory computers. This is unlikely to change in the near future.


With all the other browsers, you can reasonably compare their rendering engines, their Javascript engines, their extensions, etcetera, etcetera. One test will show that Safari beats all the others; another will show that Chrome does; still another will hint that Firefox's next version will beat both of them in real-world situations. In fact, they're all just suited to different things. Chrome's V8 engine is massively superior to just about everything else when it comes to recursion, but Firefox's TraceMonkey, if I am remembering this right, is better at memory management.

The other consideration is that every new version of every browser (and that includes IE) is touting performance improvements. It's the current "big thing", because web applications are getting huge, and performance IS a problem sometimes. People want faster browsers, so people are writing faster browsers. This is a good thing, but it does mean that media releases like this aren't really anything special or unusual.

I'm not sure what it is you're describing as "shocking". That Safari 4 outdoes IE 7 at Javascript? Not shocking in the least; proves little about Safari, and lots about IE. (Actually, to be fairer, Safari 4 should be compared against IE 8, 6 times faster than IE 7. But that's a minor consideration.) Or that Safari's latest offering beats all the other current offerings? Interesting, but still not shocking. It is all too easy to code for a specific benchmark - even subconsciously, without even realising it, if you use that same benchmark suite for your own development testing.

By the way, the language is slightly misleading. Safari is described as being "1.2 times faster" than Chrome... it's actually 1.2 times the speed of Chrome - or, to be more accurate, it took about 77% of the time Chrome did. It'd be more fair to say that Safari is 25% faster, not 1.2 times faster.

The REAL information to be learned from this data is:

* Safari 4, Moz Minefield 3.2a1, and Google Chrome (no version number given) are all about the same, with minor variance between them that can be the difference between one benchmark and another;

* Opera is significantly slower;

* Older versions of most browsers underperform compared to newer ones;

* IE sucks.

None of this is surprising. (Opera's never boasted superior Javascript performance. It lags behind Firefox and Chrome.) However, it IS interesting, and the third point is one that seems to be keeping up quite nicely. Moore's Law seems to work in reverse here; which probably just means that for the last X years, Javascript performance has been so neglected that there's room for fairly stunning improvements. That's largely because of changes to what Javascript is being made to do these days... but that's a topic for a separate discussion.

[Aristotle]

Is Mozilla Minefield 3.2a1 the upcoming version of Firefox? If so, then it would appear Firefox, Chrome, and Safari will be virtually identical.

This only tested the rendering of javascript? Is that really all they do to test browsers?

How about rendering actual pages?

[Rosuav]

Originally posted by Aristotle
Is Mozilla Minefield 3.2a1 the upcoming version of Firefox? If so, then it would appear Firefox, Chrome, and Safari will be virtually identical.

This only tested the rendering of javascript? Is that really all they do to test browsers?

How about rendering actual pages?

SunSpider is specifically a Javascript benchmark. It's not the only thing done to test browsers; there's also Acid 2/Acid 3 (although I think that's more a yes-no than a performance test), and some others. However, in the context of web applications, Javascript is the biggest thing. If the JS engine wastes memory, the entire system will bog down, and if it can't efficiently handle recursion, then pages that allow you to drag things around will suffer. (Apparently that requires immense levels of recursion. Why? I dunno either.)

[Gesslar]

I don't think it's unfair to test IE against other browsers, at all. It's a browser. When deciding which one you're going to use, how much sense does it make to eliminate the stemming reason why we have so many browser alternatives as we do. If they (MS) do something differently, then they have to prove that what they're doing is more effective and resourceful. If they (MS) simply do not care, then it will be reflected in the result.

People who base their decisions on lab benchmarks are missing out anyway. Min/maxing by definition means losing flavour for the sake of the ultimate numbers and also don't operate in benchmark testing environments.

Sidenote: Safari's HTML rendering engine is based off WebKit, WebKit is based off KHTML which is the HTML rendering engine created by the KDE open source project for *NIX. KHTML is fast but it appears its strengths were not tested in this benchmark.

[Rosuav]

Originally posted by Gesslar
I don't think it's unfair to test IE against other browsers, at all. It's a browser. When deciding which one you're going to use, how much sense does it make to eliminate the stemming reason why we have so many browser alternatives as we do. If they (MS) do something differently, then they have to prove that what they're doing is more effective and resourceful. If they (MS) simply do not care, then it will be reflected in the result.

I was being semi-sarcastic... saying that comparing IE with {Firefox|Opera|Chrome|Safari} was like comparing a Formula 1 racing car with a Volkswagen Beetle... and I don't mean Herbie. IE7 will start in the 10AM race and then come third in the 2PM race. IE8 will at least finish before the next race starts.

People who base their decisions on lab benchmarks are missing out anyway. Min/maxing by definition means losing flavour for the sake of the ultimate numbers and also don't operate in benchmark testing environments.

Right, which was my point about the difference between almost all the non-IE browsers being negligible.