High-performance sorting on networks of workstations

1 June 1997

journal article
Published by Association for Computing Machinery (ACM) in ACM SIGMOD Record

Vol. 26 (2) , 243-254
https://doi.org/10.1145/253262.253322

Abstract

We report the performance of NOW-Sort, a collection of sorting implementations on a Network of Workstations (NOW). We find that parallel sorting on a NOW is competitive to sorting on the large-scale SMPs that have traditionally held the performance records. On a 64-node cluster, we sort 6.0 GB in just under one minute, while a 32-node cluster finishes the Datamation benchmark in 2.41 seconds. Our implementations can be applied to a variety of disk, memory, and processor configurations; we highlight salient issues for tuning each component of the system. We evaluate the use of commodity operating systems and hardware for parallel sorting. We find existing OS primitives for memory management and file access adequate. Due to aggregate communication and disk bandwidth requirements, the bottleneck of our system is the workstation I/O bus.

Keywords

This publication has 21 references indexed in Scilit:

Fast parallel sorting under LogP: experience with the CM-5
IEEE Transactions on Parallel and Distributed Systems, 1996
Assessing fast network interfaces
IEEE Micro, 1996
Myrinet: a gigabit-per-second local area network
IEEE Micro, 1995
A case for NOW (Networks of Workstations)
IEEE Micro, 1995
Cooperative shared memory
ACM Transactions on Computer Systems, 1993
Parallel programming in Split-C
Published by Association for Computing Machinery (ACM) ,1993
FastSort: a distributed single-input single-output external sort
ACM SIGMOD Record, 1990
The Gamma database machine project
IEEE Transactions on Knowledge and Data Engineering, 1990
Prototyping Bubba, a highly parallel database system
IEEE Transactions on Knowledge and Data Engineering, 1990
Operating system support for database management
Communications of the ACM, 1981