Applying Chimera Virtual Data Concepts to Cluster Finding in the Sloan Sky Survey
- 1 January 2002
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 10639535,p. 56
- https://doi.org/10.1109/sc.2002.10021
Abstract
In many scientific disciplines — especially long running, data- intensive collaborations — it is important to track all aspects of data capture, production, transformation, and analysis. In principle, one can then audit, validate, reproduce, and/or re-run with corrections various data transformations. We have recently proposed and prototyped the Chimera virtual data system, a new database-driven approach to this problem. We present here a major application study in which we apply Chimera to a challenging data analysis problem: the identification of galaxy clusters within the Sloan Digital Sky Survey. We describe the problem, its computational procedures, and the use of Chimera to plan and orchestrate the workflow of thousands of tasks on a data grid comprising hundreds of computers. This experience suggests that a general set of tools can indeed enhance the accuracy and productivity of scientific data reduction and that further development and application of this paradigm will offer great value.Keywords
This publication has 13 references indexed in Scilit:
- Chimera: a virtual data system for representing, querying, and automating data derivationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2003
- GriPhyN and LIGO, building a virtual data Grid for gravitational wave scientistsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2003
- Condor-a hunter of idle workstationsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2003
- Supporting fine-grained data lineage in a database visualization environmentPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Condor-G: a computation management agent for multi-institutional gridsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- The World-Wide TelescopeScience, 2001
- Tracing the lineage of view data in a warehousing environmentACM Transactions on Database Systems, 2000
- Designing and mining multi-terabyte astronomy archivesACM SIGMOD Record, 2000
- ZooPublished by Association for Computing Machinery (ACM) ,1997
- CONCEPTUAL SCHEMAS: MULTI-FACETED TOOLS FOR DESKTOP SCIENTIFIC EXPERIMENT MANAGEMENTInternational Journal of Cooperative Information Systems, 1992