BYTE Benchmarks

Benchmarks and Test Methodologies

BYTE rates products based on technology, implementation, performance, features, and value. We depend on the following benchmarks because of their applicability to the task, ability to produce relevant metrics, and reliability.

BYTEmark

• download BYTEmark
• read the BYTEmark documentation
• read the new BYTEmark FAQ

Release 2 of BYTE Magazine's BYTEmark benchmark program (previously known as BYTE's Native Mode Benchmarks) is designed to test raw CPU speed across a variety of platforms. The BYTEmark documentation discusses the tests at the algorithm level. BYTEmark is designed to expose the capabilities of a system's CPU, FPU, and memory system.

BAPCo SYSmark 32 for Windows 95 and Windows NT

To measure a system's real-world performance when running typical business applications, we use BAPCo's SYSmark 32 for Windows 95 and Windows NT ver 1.0. This benchmark suite comprises the retail versions of eight application programs and measures the speed with which the system under test executes pre-determined scripts of user tasks typically performed when using these applications.

All programs run in native 32-bit mode and each application script is performed three times. The performance times of the individual applications are weighted and combined into both category-based performance scores as well as a single overall score.

The application programs employed by SYSmark 32 are:

• WORD PROCESSING
  • Microsoft Word 7.0
  • Lotus WordPro 96
• SPREADSHEET
  • Microsoft Excel 7.0
• DATABASE
  • Borland Paradox 7.0
• DESKTOP GRAPHICS
  • CorelDraw 6.0
• DESKTOP PRESENTATION
  • Lotus Freelance Graphics 96
  • Microsoft Powerpoint 7.0
• DESKTOP PUBLISHING
  • Adobe Pagemaker 6.0

IMPORTANT NOTE: The BYTE/SYSmark Benchmark Disk

If you are experiencing problems using the BYTE/SYSmark Benchmark Disk, please go to http://www.bapco.com/misc.htm, scroll down to the "SYSmark32" section and read "Symptom #3" and the solution. If this information is insufficient to resolve your problems, you can reach BAPCO at 1-800-321-0457.

BAPCo SYSmark for Windows NT ver 4.0

To compare the performance of cross-platform systems running common business applications exclusively under Windows NT, BYTE has used BAPCo SYSmark for Windows NT. This benchmark suite comprises five application programs and measures the speed with which the system under test executes pre-determined Scripts of user tasks typically performed when using these applications.

The application programs used are:

• Word Processing: Microsoft Word 6.0
• Spreadsheet: Microsoft Excel 5.0
• Presentation Graphics: Microsoft PowerPoint 4.0
• Computer-Aided Design: OrCAD Layout for Windows 7.0
• Project Management: Welcom Software Texim Project 2.0e

All programs run in native 32-bit mode except PowerPoint, which runs the application code in 16-bit emulation mode.

SPEC OPC Viewperf

Makers of high-end 3D graphics accelerators typically quote performance as the number of primitives (such as triangles) drawn per second. Numbers that, in the absence of additional information such as the context, size, shading, color depth, and the smoothing method used to draw those triangles, make direct comparisons difficult.

Given the growing importance of OpenGL on the desktop, we characterize 3D performance using the Viewperf benchmark, developed by the Standard Performance Evaluation Corporation (SPEC) Graphics Performance Characterization (GPC) Group's OpenGL Performance Characterization (OPC) Project. Viewperf is a portable benchmark and the current industry standard for evaluating OpenGL performance. Viewperf does not benchmark individual primitives, but measures how well a system accelerates actual applications data sets called viewsets.

• The CDRS viewset is derived from Parametric Technology's modeling and rendering software for computer-aided industrial design. The test measures seven different operations on a model of a lawnmower.
• The DX viewset is based on IBM's Visualization Data Explorer, a general purpose scientific data visualization and analysis package. The DX benchmark draws a set of particle traces thorough a vector flow field.

Viewperf measures frames-per-second for each component test. The single result for a each viewset is a weighted geometric mean.

Intel Media Benchmark

To measure MMX performance, BYTE uses the Intel Media Benchmark that puts the systems through a gauntlet of multimedia tasks. The algorithms used are coded for both scalar and MMX-enabled processors.

The automated test suite simulates the following activities that may exploit MMX-enabled CPUs.

• Home Photo Editing with Adobe Photo Deluxe
• Intel Indeo multimedia video clip performance
• MPEG-1 video file performance
• Direct 3D gaming
• Business Photo Manipulation with Adobe Photoshop

NSTL InterMark

NSTL's InterMark benchmark tool profiles applications by capturing the system calls and actions. InterMark then replays these actions to test a system's components without the need for third-party applications. A precision event timer is used to measure the response time of the system for each task. NSTL utilizes the video, hard disk, CPU, and CD-ROM components of the InterMark suite to test these Pentium MMX PCs.

Video tests are broken into three categories: Windows draw, bit-mapped images, and non-bit-mapped images. The Windows draw tests use the Windows API calls to draw images, including text, a picture in the Windows Metafile format, horizontal and vertical lines, rectangles, and ellipses. The bit-mapped images and non bit-mapped image tests display an image on the screen. This image is then stretched to one and one-half, and then twice its original size.

Hard Disk tests measure the speed of the hard disk for reading and writing. The test simulates sequential, constant rate sequential, random, localized random, and segmented activity in varying block sizes. The tests measure the average response time, the sustained rate, the burst rate, the random access time, and CPU utilization of the hard disk.

CPU tests measure the performance of the processor for both integer and floating-point calculations. These tests provide an index of the computing power of the processor chip. CD-ROM tests examine several different aspects of CD-ROM performance: random service time at 500 and 1,000 milliseconds, cached service time at 13.3 milliseconds; and CPU utilization at 550KB per second.

---