After Upgrades, Vista Eats XP's Dust

In a benchmarking comparison between a beta version of XP Service Pack 3 and Vista with Service Pack 1, Devil Mountain Software of Florida found that the SP3-tweaked XP left Vista in the dust.

The tests showed that XP, modified with the forthcoming service pack, had twice the performance of Vista modified with SP1, the new OS's first upgrade that is going to be released early next year.

Vista was more than twice as slow as XP with SP3 in the tests using DMS Clarity Studio OfficeBench test script "to drive Microsoft Office 2007 through a scripted set of productivity tasks, including creating a compound document and supporting workbooks and presentations materials," according to Devil Mountain Software.

The Vista SP1 Dud

In a prior report, published Nov. 18, Devil Mountain expressed disappointment with Vista SP1 even without comparing it to the speedy XP with SP3.

"With the initial performance characteristics of Windows Vista leaving much to be desired ... many IT organizations have put off deploying the new OS until the first service pack (SP1) is released by Microsoft early next year," said the testers. "The thinking goes that SP1 will address all of these early performance issues and somehow bring Windows Vista on par with -- or at least closer to -- Windows XP in terms of runtime performance. Unfortunately, this is simply not the case."

The bottom line, when it comes to Vista is clear, the company said: "If you've been disappointed with the performance of Windows Vista to date, get used to it. SP1 is simply not the panacea that many predicted. In the end, it's Vista's architecture -- not a lack of tuning or bug fixes -- that makes it perform so poorly on systems that were 'barn-burners' under Windows XP."

Give and Take

Craig Barth, chief technology officer at Devil Mountain Software, was surprised by the test results, particularly because he believed SP1 for Vista was going to dramatically improve the new OS's performance.

"We were as shocked as everybody is," Barth told TechNewsWorld. "They talked up the whole thing saying, 'If you're not happy with Vista, don't make any comments until SP1 comes out.'"

Barth stressed he is not trying to denounce Microsoft in any way.

"I use Vista every day," he said. "I'm actually a big fan of Vista from a user interface standpoint." The fact that it is slower than XP is "a trade-off" that Barth said he is "willing to make for some of the additional features" that come with Vista. "In order to get some of the advantages, there's going to be some heft," he reasoned.

An Ulterior Motive

As Devil Mountain expected, the publication of its tests unleashed the usual firestorm of online bickering. Reactions posted on Devil Mountain's blog covered the gamut from trashing Devil Mountain and/or Microsoft to thanking the researchers for their effort.

Barth's "ulterior motive" in publishing the report was to convince people to sign up with Devil Mountain's "exo.performance.network," a free, community-based effort to gather information about Windows-based systems around the world, he told TechNewsWorld.

"We got 500 people signed up in the past three days," said Barth.

No Biggie

Few people will notice the performance difference, especially while running Microsoft Office, said Jack Gold, founder and principal analyst of J. Gold Associates.

"I'm not all that surprised or shocked," said Gold. "Vista is a relatively new piece of software while XP is six years old, or something like that. I would hope that, in six years, Microsoft would learn how to optimize something they'd written."

Even without Devil Mountain's findings, most enterprises are seeing "no compelling reasons" to upgrade to Vista their systems that are running perfectly well on XP, said Gold. "This is not a major gotcha," he said. "If it was a security flaw they discovered, one where people could get into the operating system, that's one thing. But if it just says, 'XP runs faster,' then who cares?"