On correlated availability in Internet-distributed systems
- 1 September 2008
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 21521085,p. 276-283
- https://doi.org/10.1109/grid.2008.4662809
Abstract
As computer networks rapidly increase in size and speed, Internet-distributed systems such as P2P, volunteer computing, and Grid systems are increasingly common. A precise and accurate characterization of Internet resources is important for the design and evaluation of such Internet-distributed systems, yet our picture of the Internet landscape is not perfectly clear. To improve this picture, we measure and characterize the time dynamics of availability in a large-scale Internet-distributed system with over 110,000 hosts. Our characterization focuses on identifying patterns of correlated availability. We determine scalable and accurate clustering techniques and distance metrics for automatically detecting significant availability patterns. By means of clustering, we identify groups of resources with correlated availability that exhibit similar time effects. Then we show how these correlated clusters of resources can be used to improve resource management for parallel applications in the context of volunteer computing.Keywords
This publication has 6 references indexed in Scilit:
- RidgePublished by Association for Computing Machinery (ACM) ,2007
- The Computational and Storage Potential of Volunteer ComputingPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2006
- Resource Discovery Techniques in Distributed Desktop Grid EnvironmentsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2006
- BOINC: A System for Public-Resource Computing and StoragePublished by Institute of Electrical and Electronics Engineers (IEEE) ,2005
- Characterizing and evaluating desktop grids: an empirical studyPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2004
- Feasibility of a serverless distributed file system deployed on an existing set of desktop PCsPublished by Association for Computing Machinery (ACM) ,2000