IBM GETTING 190 SPECMARK IN THE LABS, BUT MICROSOFT EXCEL RUNS SLOW

(July 3rd 1995) In the last edition we reported that the Specmarks of
IBM's new Power Series machines were well below the estimated
SPECmarks of the chips. Well it seems that the Power Personal Systems
Division has demo systems 133MHz with 1 MB of level 2 cache and 60
nanosecond interleaved RAM that hit 190 Specmarks. The big cache and
fast memory account for much of the difference, however there is also
a suggestion that IBM's 'Dakota' PCI memory controllers still has
some rough edges to be knocked of.

Meanwhile, IBM's benchmarking team have been doing their own
measurements. The nutshell claim from the division is that at the
same clock-speed the PowerPC systems run Bytemarks benchmarks more
than twice as fast (2.15x - 2.17x), as a top-of-the-line Compaq and
NSTL's InterMark nearly twice as fast (1.94x). The Spec figures
look less impressive: 1.19x for integer and 1.38x for floating point,
but these are compared against Intel's own figures presented to the
SPEC commitee.

The benchtests were carried out under Windows NT, and IBM makes a
point of noting that the Motorola compiler under NT are "still in
their infancy and as it matures significant performance improvements
are expected."

Apart from the synthetic benchmarks the company has been testing a
mixture of 32bit NT applications, 16bit-32bit conversions and 16 bit
Windows applications running under emulation. The most notable aspect
of the results here is the dire performance shown by some aspects of
Microsoft's 32 bit version of Excel. IBM used the National Software
Testing Laboratory's (NSTL) test-script and is designed to produce a
representative workload. With most of the functions - insert/delete;
scaled cut and paste; file I/O; charting etc, the PowerPC appears to
outstrip the Pentium machine comfortably. However, when it comes to
measuring financials; statistical maths; addition and exponentials,
speed drops way below the results for the Pentium.

A number of functions in 32bit Word suffer too - search and replace;
block copy; and page down are all slower on the PowerPC. Why? the
suggestion from within IBM is that Microsoft either originally
implemented some of these functions as chunks of Intel assembler
which it then converted, or that the functions are optimised for the
Intel architecture - fair enough, considering that the vast majority
of NT installations are on the Intel-compatible platform, but less
good as an example of Microsoft's cross-platform pretensions. It
would be interesting to see the same tests run on Alpha and MIPs
platforms.

The same problems are even more pronounced when 16-bit Excel is run
under emulation on Windows NT, though 16-bit Word runs consistently
faster on PowerPC than Pentium.

as always, with benchmarks carried out by one vendor, its worth being
cautious about the results. IBM is pondering whether to go to NSTL
for a truely independent test.