Advanced Information Criterion for Environmental Data Quality Assurance : Volume 8, Issue 1 (07/05/2012)

By Düsterhus, A.

Book Id: WPLBN0003991387
Format Type: PDF Article :
File Size: Pages 6
Reproduction Date: 2015

Title:	Advanced Information Criterion for Environmental Data Quality Assurance : Volume 8, Issue 1 (07/05/2012)
Author:	Düsterhus, A.
Volume:	Vol. 8, Issue 1
Language:	English
Subject:	Science, Advances, Science
Collections:	Periodicals: Journal and Magazine Collection (Contemporary), Copernicus GmbH
Historic Publication Date:	2012
Publisher:	Copernicus Gmbh, Göttingen, Germany

Member Page:	Copernicus Publications
Citation APA MLA Chicago Hense, A., & Düsterhus, A. (2012). Advanced Information Criterion for Environmental Data Quality Assurance : Volume 8, Issue 1 (07/05/2012). Retrieved from http://gutenberg.cc/

Description
Description: Meteorological Institute of the University of Bonn, Auf dem Hügel 20, 53121 Bonn, Germany. A new method for testing time series of environmental data for internal inconsistencies is presented. The method divides the dataset into several disjunct blocks. By means of a comparison of the blocks' estimated probability density distributions, each block is compared with the others. In order to judge the differences, four different measures are used and compared: Kullback-Leibler Divergence, Jensen-Shannon Divergence, Earth Mover's Distance and the Root Mean Square. By looking at the resulting patterns, conclusions on possible inconsistencies in the data can be drawn.

This paper shows some sensitivitiy tests and gives an example for an application to real data. Furthermore, it is shown, in which cases of errors (shift in mean, shift in variance and rounding), which measure performs best.

Summary
Advanced information criterion for environmental data quality assurance

Excerpt
Ducr{é}-Robitaille, J.-F., Vincent, L. A., and Boulet, G.: Comparison of techniques for detection of discontinuities in temperature series, Int. J. Climatol., 23, 1087–1101, 2003.; Durre, I., Menne, M. J., Gleason, B. E., Houston, T. G., and Vose, R. S.: Comprehensive Automated Quality Assurance of Daily Surface Observations, J. Appl. Meteorol. Clim., 49, 1615–1633, 2010.; Endres, D. M. and Schindelin, J. E.: A new metric for probability distributions, IEEE T. Inform. Theory, 49, 1858–1860, 2003.; Gandin, L. S.: Complex Quality Control of Meteorological Observations, Mon. Weather Rev., 116, 1137–1156, 1988.; Furrer, R., Nychka, D., and Sain, S.: Fields: Tools for spatial data, http://CRAN.R-project.org/package=fields, R package version 6.6.2, 2011.; Graybeal, D. Y., DeGaetano, A. T., and Eggleston, K. L.: Complex Quality Assurance of Historical Hourly Surface Airways Meteorological Data, J. Atmos. Ocean. Tech., 21, 1156–1169, 2004.; Hubbard, K. G., Goddard, S., Sorensen, W. D., Wells, N., and Osugi, T. T.: Performance of Quality Assurance Procedures for an Applied Climate Information System, J. Atmos. Ocean. Tech., 22, 105–112, 2005.; Jim{é}nez, P. A., Gonz{á}lez-Rouco, J. F., Navarro, J., Mont{á}vez, J. P., and Garcia-Bustamante, E.: Quality Assurance of Surface Wind Observations from Automated Weather Stations, J. Atmos. Ocean. Tech., 27, 1101–1122, 2010.; Kullback, S. and Leibler, R. A.: On Information and Sufficiency, The Annals of Mathematical Statistics, 22, 79–86, 1951.; Levina, E. and Bickel, P.: The Earth Mover's distance is the Mallows distance: some insights from statistics, Eighth IEEE International Conference on Computer Vision, 2001, ICCV 2001, Proceedings, 251–256, 2001.; Lin, J.: Divergence measures based on the Shannon entropy, IEEE T. Inform. Theory, 37, 145–151, 1991.; Mathes, A., Friederichs, P., and Hense, A.: Towards a quality control of precipitation data, Meteorol. Z., 17, 733–749, 2008.; Meek, D. W. and Hatfield, J. L.: Data Quality Checking for Single Station Meteorological Databases, Agr. Forest Meteorol., 69, 85–109, 1994.; Owen, A. B.: Nonparametric Likelihood Confidence Bands for a Distribution Function, J. Am. Stat. Assoc., 90, 516–521, 1995.; Peterson, T. C., Easterling, D. R., Karl, T. R., Groisman, P., Nicholls, N., Plummer, N., Torok, S., Auer, I., Boehm, R., Gullett, D., Vincent, L. A., Heino, R., Tuomenvirta, H., Mestre, O., Szentimrey, T., Salinger, J., Førland, E. J., Hanssen-Bauer, I., Alexandersson, H., Jones, P., and Parker, D.: Homogeneity adjustments of in situ atmospheric climate data: a review, Int. J. Climatol., 18, 1493–1517, 1998.; R Development Core Team: R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing, R Foundation for Statistical Computing, http://www.R-project.org/, ISBN 3-900051-07-0, 2011.; Rabin, J., Delon, J., and Gousseau, Y.: Circular Earth Mover's Distance for the comparison of local features, 19th International Conference on Pattern Recognition, 2008.; Rubner, Y., Tomasi, C., and Guibas, L. J.: The Earth Mover's Distance as a Metric for Image Retrieval, Int. J. Comput. Vision, 40, 99–121, 2000.; Zahumensky, I.: Guidelines on Quality Control Procedures for Data from Automatic Weather Stations, World Meteorological Organization, WMO-No. 488, Appendix VI.2, 2007.