Efficient barrier using remote memory operations on VIA-based clusters
- 25 June 2003
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
Abstract
Most high performance scientific applications require efficient support for collective communication. Point-to-point message-passing communication in current generation clusters are based on Send/Recv communication model. Collective communication operations built on top of such point-to-point message-passing operations might achieve suboptimal performance. VIA and the emerging InfiniBand architecture support remote DMA operations, which allow data to be moved between the nodes with low overhead, they also allow to create and provide a logical shared memory address space across the nodes. In this paper, we focus on barrier, one of the frequently-used collective operations. We demonstrate how RDMA write operations can be used to support inter-node barrier in a cluster with SMP nodes. Combining this with a scheme to exploit shared memory within a SMP node, we develop a fast barrier algorithm for cluster of SMP nodes with cLAN VIA inteconnect. Compared to the current barrier algorithms using Send/Recv communication model, the new approach is shown to reduce barrier latency on a 64 processor (32 dual nodes) system by up to 66%. These results demonstrate that high performance and scalable barrier implementations can be delivered on current and next generation VIA/Infiniband-based clusters with RDMA support.Keywords
This publication has 10 references indexed in Scilit:
- Active Messages: A Mechanism for Integrated Communication and ComputationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2005
- Global combine on mesh architectures with wormhole routingPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Performance and experience with LAPI-a new high-performance communication library for the IBM RS/6000 SPPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Global reduction in wormhole k-ary n-cube networks with multidestination exchange wormsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Efficient virtual interface architecture (VIA) support for the IBM SP switch-connected NT clustersPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- EMPPublished by Association for Computing Machinery (ACM) ,2001
- Realizing the performance potential of the virtual interface architecturePublished by Association for Computing Machinery (ACM) ,1999
- U-NetPublished by Association for Computing Machinery (ACM) ,1995
- Myrinet: a gigabit-per-second local area networkIEEE Micro, 1995
- Optimum broadcasting and personalized communication in hypercubesIEEE Transactions on Computers, 1989