Macenstein

Posted by Dr. Macenstein

In the online technology forums, there is perhaps no greater battle than the one that inevitably ensues when tech geeks get together to decide which is better, Macs or PCs. A close second to that battle, however, is which is the better web browser.

On the Mac side, it essentially comes down to Firefox vs. Safari (yes, I know there are others, but let’s be realistic). The main arguments most people use in defending or criticizing a browser are 1) its ability to accurately render a page, and 2) the speed at which it does so. For the most part, I do not notice a big difference in the page rendering accuracy between Firefox and Safari. Most sites I go to, including banking sites, work just fine in both. I’m sure there are exceptions, but for 95% of my surfing, it hasn’t been an issue. As for speed, well both browsers perform very snappy on both my home and work machines thanks to high speed broadband connections, and while one may render certain pages faster than the other, it has never been noticeable enough to bother me. But what I have noticed (and what made me write this article) is there are 2 ways to look at speed when talking about web browsers.

Redefining “speed”

While how fast a browser can render a web page is certainly an important weapon in the browser war arguments, the difference is usually a matter of seconds at the most. To my mind, a more important measure of speed is how a browser affects the overall speed of your SYSTEM.

I noticed this quite by accident the other day while rendering out an After Effects animation I had done. I had made a spelling change in one of the graphics, and re-rendered a composition I had rendered the day before. When it was finished, I noticed in the Render Queue that it had taken 15 minutes longer to render than it had the day before. This seemed odd to me, as like I said, all I had done was changed the spelling of one word in the animation.

So I began to think why this would be. I had not restarted the computer since the day before, nothing had changed hardware wise. The only thing different was that I had been surfing the web a bit while the render was going on that day, where the day before I had not. “Surely surfing the web on a mulit-processor machine shouldn’t add 15 minutes to a render”, I thought. Well, yes it does actually, if you’re using Safari.

The tests

I decided to test out my theory. I began by restarting my system (a dual 2GHz G5 PowerMac with 2GB of RAM), then rendering out my After Effects movie again, with no other applications open. I got an almost identical render time as the first render. I then opened Safari, and opened 5 random sites in tabs (digg.com, cnn.com, myspace.com, macenstein.com (of course) and virb.com). I figured these represented a good mix of the type of sites I visit, as well as sites with differnet types of layouts and memory requirements. I clicked around a bit in each site to build up a little cache action, then re-opened After Effects and rendered the scene again. The render time was almost exactly the same 15 minutes longer that I experienced the first time.

The next step was to figure out if this slowdown was only due to Safari, or just web browsing in general. So I made a stripped down test project for After Effects (so I didn’t have to spend 30 minutes waiting for each test to complete) and did the same “fresh reboot, open After Effects, render, quit, open Safari, open After Effects, and render” test as before to get my baseline times. Using After Effects sans ANY browser scored a render time of 154 seconds. After Effects with Safari running got a time of 271 seconds.

Next, I repeated the test with Firefox instead of Safari, and was very surprised to see that Firefox added only 6 seconds to After Effect’s render time. (See Below)

Above: Results on a PowerMac rendering an After Effects project, then again with Firefox and Sarafi open. The PowerMac ran almost 76% slower with Safari open.

I decided to grab the ol’ stopwatch and then try some Photoshop tests. I recently had to make a series of banners for a project I was working on, so I used one of them (a 1.72GB 50 x 32.5-inch Photoshop file at 300 dpi) for my test. First, I tested how long it took Photoshop to launch and open the file via double-clicking. Next, I timed how long it took to do a “Save As” PSD. I then quit Photoshop and repeated the tests with both Safari and Firefox. Results are below.

Above: Results on a PowerMac opening and saving a 1.72 GB file, then again with Firefox and Sarafi open. The system took 40% longer to open the document and 49% longer to save it with multiple sites open in Safari.

As you can see, Photoshop gave a similarly dismal performance while Safari was running, particularly during the “Open Document” test, where it added a full 97 seconds to the process.

I did one final test in QuickTime, exporting a 2 minute full screen video clip to iPod. I was pleasantly surprised (and yet somewhat baffled) to find that Safari had no adverse effect at all on the export time. I got the exact same 3 minute 44 second time for all 3 tests. I was sort of surprised as I figured compressing a movie would use similar function calls resources as rendering an animation from After Effects. So apparently this issue is somewhat application specific (perhaps even exclusive to Adobe apps? Insert conspiracy theory here).

For the record, all tests were done twice, and the average times were used. My Safari installation should be fairly clean, as I have not installed any odd plug-ins or such. Pretty much the factory default settings.

A “Universal” problem

As one final test (and because everyone loves graphs!) I repeated all tests again on a Quad-core 2.66 GHz MacPro, with 2 GB of RAM. Of course both Adobe apps had to run under Rosetta, as neither is a Universal app yet. While in some cases the difference is not AS great as on the PowerPC model, you can see that there is still a sizable performance hit when using Safari (and remember, the Mac Pro has 2 extra processors available to handle the “demands” of Safari).

Above: Results on a Mac Pro rendering an After Effects project, then again with Firefox and Sarafi open. The Mac Pro ran 21% slower with Safari open.

Above: Results on a Mac Pro opening and saving a 1.72 GB file, then again with Firefox and Sarafi open. Performance hits of 37% and 29% for the “Open” and “Save As” tests.

It was nice to see that despite both After Effects and Photoshop being PowerPC apps relying on Rosetta, After Effects actually rendered slightly faster on the Mac Pro than the PowerMac, and wasn’t that out done in the Photoshop “Open” test.

Conclusion

The interesting thing about these results (to me at least) is that both Firefox and Safari were simply open during their tests. I was not actively “surfing”, ie. clicking on things, moving windows, etc. It seems to me that a background application, especially one that should not really be doing anything all that processor-intensive even when in the foreground, should not hog system resources the way Safari apparently does. If Firefox can play nice, why not Safari?

I suppose as a final disclaimer I should say that while I did everything I could think to keep the tests fair (using the same web sites, running multiple tests and so forth) it is possible that both my work and home machines (with different processors, graphics cards, software installations, etc.) are both somehow uniquely wacky, and I have the only 2 machines on the planet that will bear out these results. It could also be my choice of sites, although the fact remains Firefox had no problem handling them.

To that end, I would encourage anyone with these apps, a stop watch, and too much free time to conduct their own tests. I would also love feedback on performance hits on other apps when using Safari, such as some 3D applications. The fact that both affected apps were Adobe apps, and QuickTime (an Apple app) was NOT affected is a little strange to me. For those of you looking to conduct your OWN tests, I would like to point out that simply opening Safari and conducting tests will not yield too much of a difference; you need to have at least a few open tabs (I used 5), and I would suggest visiting some “intense” sites. I chose MySpace as one of my test sites because I consider it to be the worst coded page out there, with tons of flash ads and other gunk that can stop up a browser.

So what does this mean? Well, for most people, not a lot. If you are just using your computer for Word, e-mail, and web browsing, I doubt there is much of a noticeable performance hit. It really seems that only “Power Users” (if I may be so bold as to lump myself in that group) will be affected, and possibly not even all “power” apps will be affected (as shown in the QuickTime test).

For ME, however, these results means that while Safari is still my browser of choice, if I am going to kill time while large, time-critical projects are rendering, I will be using Firefox. Or better yet, another computer.

UPDATE: OK, at the request of faithful Macenstein reader Richard Neal, I have run some quick tests on OmniWeb and Camino. I did not do as thorough a test, just one round each on the PowerMac, but based on my previous tests the differences in times should be negligible.

Above: Time in Seconds. Shorter bars are better. (Sorry the colors on this graph don’t match the above graphs, I have no idea how to change graph bar colors in Keynote!)

I’m not sure what these results really prove, except that Safari is consistently the worst performer out of the group, and Firefox seems the best (except for Camino’s After Effects score).

OmniWeb (which should be similar to Safari according to Richard) performed well on the After Effects test, and in the middle on the Photoshop ones, and even beat Camino on the “Save As” and After Effects rendering tests. So if there is a problem with the underlying architecture of Safari’s webkit, I don’t quite see it.

[UPDATE 2:] OK, For any who care, here is an activity monitor screenshot of 5 web browsers (Safari, Firefox, Opera, OmniWeb, and Camino) sorted by amount of RAM used.

Click to enlarge

All 5 browsers have the exact same web pages open. You can see Safari is using over 2.5 times the amount of memory as Opera and OmniWeb, and about 3 times as much memory as Camino and Firefox. You can also see that while it is using the most RAM, Safari is actually tied for the lowest CPU load. Conversely, Firefox, using the least amount of RAM ,is using the most CPU power (yet Safari has claimed the 2nd highest amount of threads). Safari is also using twice the Virtual RAM (1.22GB ) of any other browser). This seems to bear out the memory-leak/RAM issues theories.

The only other “real” application open at the moment (aside from Activity Monitor) is Audio Hijack, which is only using 27 MB of RAM, 5% of the CPU power, and only 7 threads, which is funny to me since it is actively recording 2-hour long AAC audio files at the moment, and all 5 web browsers are simply “background” applications (in fact, Safari is the “back-most background app at the moment).

These results seem to prove that the Photoshop and After Effects tests were severely hampered by having Safari open due to the amount of RAM Safari hordes and does not release. While I still think over 120MB is too much RAM for ANY web browser to want, I can’t think of any good reason Safari would need 3 times as much. So while buying more RAM would likely lessen the problem for me in my tests, the fact that out of the 5 browsers only Safari is making buying more RAM a necessity is somewhat disappointing.

[UPDATE 3:] A happy ending
OK, well, many of you will be happy to know we have run the tests again at the request of David Hyatt, the architect behind Safari, using the latest WebKit build (WebKit-SVN-r19919) and there is a world of improvement (see below).

Above: Results of the After Effects render on the dual G5 PowerMac, now including results for using the latest Webkit build. As you can see, it is more or less tied with Firefox.

Above: Results of the Photoshop “Open” and “Save As” tests on the dual G5 PowerMac, now including results for using the latest Webkit build. As with the After Effects test, the latest Webkit builds almost tie Firefox in the “Open” test, and beat it in the “Save As” test.

David would be better able to tell you what has changed the most between the shipping version of Safari and this latest WebKit build that could conceivably account for these results, but suffice it to say we are pleased as punch to know the next version of Safari (which will be released when, David? :)) seemingly puts things back on track. Of course, we have no idea how stable this release is (although so far it seems fine to us) so use at your own risk. However, for the moment, WebKit is now Macenstein’s default browser of choice. Good work, Mr Hyatt!

-The Doc