Remote queues

Abstract
We introduce Remote Queues (RQ), a communication model that integrates polling with selective interrupts to support a wide range of applications and communication paradigms. We show that polling is desirable for a range of applications for both perfor-mance and atomicity. Polling enables optimizations that are essen-tial for fine-grain applications such as sparse-matrix solution. Polling also improves flow control for high-level communication patterns such as transpose. We use RQ to implement active messages, bulk transfers, and fine-grain applications on the MIT Alewife, Intel Paragon and Cray T3D using extremely different implementations of RQ. RQ improves performance on all of the machines, and provides atomicity guar-antees that greatly simplify programming for the user. RQ also sep-arates handler invocation from draining the network, which simplifies deadlock avoidance and multiprogramming. We also introduce efficient atomicity mechanisms on Alewife to integrate polling with interrupts, and discuss how to exploit inter-rupts on Alewife and the Intel Paragon without forfeiting the atom-icity and optimization advantages of RQ.