Recording and using provenance in a protein compressibility experiment
- 24 October 2005
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
Abstract
Very large scale computations are now becoming routinely used as a methodology to undertake scientific research. In this context, 'provenance systems' are regarded as the equivalent of the scientist's logbook for in silico experimentation: provenance captures the documentation of the process that led to some result. Using a protein compressibility analysis application, we derive a set of generic use cases for a provenance system. In order to support these, we address the following fundamental questions: what is provenance? How to record it? What is the performance impact for grid execution? What is the performance of reasoning? In doing so, we define a technology-independent notion of provenance that captures interactions between components, internal component information and grouping of interactions, so as to allow us to analyze and reason about the execution of scientific processes. In order to support persistent provenance in heterogeneous applications, we introduce a separate provenance store, in which provenance documentation can be stored, archived and queried independently of the technology used to run the application. Through a series of practical tests, we evaluate the performance impact of such a provenance system. In summary, we demonstrate that provenance recording overhead of our prototype system remains under 10% of execution time, and we show that the recorded information successfully supports our use cases in a performant manner.Keywords
This publication has 8 references indexed in Scilit:
- From Sandbox to Playground: Dynamic Virtual Environments in the GridPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2005
- A case for grid computing on virtual machinesPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2004
- Chimera: a virtual data system for representing, querying, and automating data derivationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2003
- A block coding method that leads to significantly lower entropy values for the proteins and coding sections of Haemophilus influenzaeProceedings. IEEE Computer Society Bioinformatics Conference, 2003
- Archiving scientific dataPublished by Association for Computing Machinery (ACM) ,2002
- Tracing the lineage of view data in a warehousing environmentACM Transactions on Database Systems, 2000
- Protein is incompressiblePublished by Institute of Electrical and Electronics Engineers (IEEE) ,1999
- Design of a Lineage-Based Meta-Data Base for GISCartography and Geographic Information Systems, 1991