Efficient organization of large multidimensional arrays
- 17 December 2002
- proceedings article
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 328-336
- https://doi.org/10.1109/icde.1994.283048
Abstract
Large multidimensional arrays are widely used in scientific and engineering database applications. The authors present methods of organizing arrays to make their access on secondary and tertiary memory devices fast and efficient. They have developed four techniques for doing this: (1) storing the array in multidimensional “chunks” to minimize the number of blocks fetched, (2) reordering the chunked array to minimize seek distance between accessed blocks, (3) maintaining redundant copies of the array, each organized for a different chunk size and ordering and (4) partitioning the array onto platters of a tertiary memory device so as to minimize the number of platter switches. The measurements on real data obtained from global change scientists show that accesses on arrays organized using these techniques are often an order of magnitude faster than on the unoptimized dataKeywords
This publication has 10 references indexed in Scilit:
- The POSTGRES next generation database management systemCommunications of the ACM, 1991
- Linear clustering of objects with multiple attributesPublished by Association for Computing Machinery (ACM) ,1990
- A new vector quantization clustering algorithmIEEE Transactions on Acoustics, Speech, and Signal Processing, 1989
- A virtual memory system for picture processingCommunications of the ACM, 1984
- The Grid FileACM Transactions on Database Systems, 1984
- Picture paging for efficient image processingPublished by Springer Nature ,1980
- An Algorithm for Vector Quantizer DesignIEEE Transactions on Communications, 1980
- Storage reorganization techniques for matrix computation in a paging environmentCommunications of the ACM, 1979
- Preserving Proximity in ArraysSIAM Journal on Computing, 1975
- Organizing matrices and matrix operations for paged memory systemsCommunications of the ACM, 1969