Benchmarking GPUs to tune dense linear algebra
Top Cited Papers
- 1 November 2008
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 21674329,p. 1-11
- https://doi.org/10.1109/sc.2008.5214359
Abstract
We present performance results for dense linear algebra using recent NVIDIA GPUs. Our matrix-matrix multiply routine (GEMM) runs up to 60% faster than the vendor's implementation and approaches the peak of hardware capabilities. Our LU, QR and Cholesky factorizations achieve up to 80-90% of the peak GEMM rate. Our parallel LU running on two GPUs achieves up to ~540 Gflop/s. These results are accomplished by challenging the accepted view of the GPU architecture and programming guidelines. We argue that modern GPUs should be viewed as multithreaded multicore vector units. We exploit blocking similarly to vector computers and heterogeneity of the system by computing both on GPU and CPU. This study includes detailed benchmarking of the GPU memory system that reveals sizes and latencies of caches and TLB. We present a couple of algorithmic optimizations aimed at increasing parallelism and regularity in the problem that provide us with slightly higher performance.Keywords
This publication has 9 references indexed in Scilit:
- A compiler framework for optimization of affine loop nests for gpgpusPublished by Association for Computing Machinery (ACM) ,2008
- Optimization principles and application performance evaluation of a multithreaded GPU using CUDAPublished by Association for Computing Machinery (ACM) ,2008
- Efficient gather and scatter operations on graphics processorsPublished by Association for Computing Machinery (ACM) ,2007
- Memory---A memory model for scientific algorithms on graphics processorsPublished by Association for Computing Machinery (ACM) ,2006
- LU-GPU: Efficient Algorithms for Solving Dense Linear Systems on Graphics HardwarePublished by Institute of Electrical and Electronics Engineers (IEEE) ,2005
- Understanding the efficiency of GPU algorithms for matrix-matrix multiplicationPublished by Association for Computing Machinery (ACM) ,2004
- A set of level 3 basic linear algebra subprogramsACM Transactions on Mathematical Software, 1990
- An adaptive blocking strategy for matrix factorizationsPublished by Springer Nature ,1990
- LAPACK: A portable linear algebra library for high-performance computersPublished by Institute of Electrical and Electronics Engineers (IEEE) ,1990