Towards long term data quality in a large scale biometrics experiment

Quality of data plays a very important role in any scientific research. In this paper we present some of the challenges that we face in managing and maintaining data quality for a terabyte scale biometrics repository. We have developed a step by step model to capture, ingest, validate, and prepare data for biometrics research. During these processes, there are many hidden errors which can be introduced into the data. Those errors can affect the overall quality of data, and thus can skew the results of biometrics research. We discuss necessary steps we have taken to reduce and eliminate the errors. Steps such as data replication, automated data validation, and logging metadata changes are necessary and crucial to improve the quality and reliability of our data.

• Type: Conference
• Year: 2010
• Where: HPDC
• Authors: Hoang Bui, Diane Wright, Clarence Helm, Rachel Witty, Patrick J. Flynn, Douglas Thain
• Keywords: Biometrics Research, Data Quality, Distributed And Parallel Computing, HPDC 2010, Terabyte Scale Biometrics