Cumulus
- 8 June 2011
- conference paper
- Published by Association for Computing Machinery (ACM)
Abstract
Amazon's S3 protocol has emerged as the de facto interface for storage in the commercial data cloud. However, it is closed source and unavailable to the numerous science data centers all over the country. Just as Amazon's Simple Storage Service (S3) provides reliable data cloud access to commercial users, scientific data centers must provide their users with a similar level of service. Ideally scientific data centers could allow the use of the same clients and protocols that have proven effective to Amazon's users. But how well does the S3 REST interface compare with the data cloud transfer services used in today's computational centers? Does it have the features needed to support the scientific community? If not, can it be extended to include these features without loss of compatibility? Can it scale and distribute resources equally when presented with common scientific the usage patterns? We address these questions by presenting Cumulus, an open source implementation of the Amazon S3 REST API. It is packaged with the Nimbus IaaS toolkit and provides scalable and reliable access to scientific data. Its performance compares favorably with that of GridFTP and SCP, and we have added features necessary to support the econometrics important to the scientific community.Keywords
This publication has 7 references indexed in Scilit:
- The Hadoop Distributed File SystemPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2010
- The Eucalyptus Open-Source Cloud-Computing SystemPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2009
- Eventually ConsistentQueue, 2008
- Amazon S3 for science gridsPublished by Association for Computing Machinery (ACM) ,2008
- Flying Low: Simple Leases with Workspace PilotPublished by Springer Nature ,2008
- The Globus Striped GridFTP Framework and ServerPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2005
- Virtual Workspaces: Achieving Quality of Service and Quality of Life in the GridScientific Programming, 2005