research-article

CurrentClean: Interactive Change Exploration and Cleaning of Stale Data

Authors:

Mostafa MilaniAuthors Info & Claims

CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management

Pages 2917 - 2920

https://doi.org/10.1145/3357384.3357839

Published: 03 November 2019 Publication History

Get Access

Abstract

Enterprises often assume their data is up-to-date, where the presence of a timestamp in the recent past qualifies the data as current. However, entities modeled in the data experience varying rates of change that influence data currency. We argue that data currency is a relative notion based on individual spatio-temporal update patterns, and these patterns can be learned and predicted. We develop CurrentClean, a probabilistic system for identifying and cleaning stale values, and enables a user to interactively explore change in her data. Our system provides a Web-based user-interface, and a backend infrastructure that learns update correlations among cell values in a database to infer and repair stale values. Our demonstration provides two motivating scenarios that highlight change exploration, and cleaning features using clinical, and sensor data from a data centre enterprise.

References

[1]

C. Batini and M. Scannapieco. 2016. Data and Information Quality: Dimensions, Principles, and Techniques. Springer-Verlag New York, Inc.

Google Scholar

[2]

H. Chen, W. Ku, H. Wang, and M. Sun. 2010. Leveraging Spatio-temporal Redundancy for RFID Data Cleansing. In SIGMOD. 51--62.

Google Scholar

[3]

P. J. Denning. 2005. The Locality Principle. Commun. ACM 48, 7 (2005), 19--24.

Digital Library

Google Scholar

[4]

W. Fan, F. Geerts, N. Tang, and W. Yu. 2013. Inferring data currency and consistency for conflict resolution. In ICDE. 470--481.

Google Scholar

[5]

D. Koller and N. Friedman. 2009. Probabilistic Graphical Models. The MIT Press.

Google Scholar

[6]

C. Mayfield, J. Neville, and S. Prabhakar. 2010. ERACER: a database approach for statistical inference and data cleaning. In SIGMOD. 75--86.

Google Scholar

[7]

M. Milani, Z. Zheng, and F. Chiang. 2019. CurrentClean: Spatio-temporal Cleaning of Stale Data. In ICDE. 172--183.

Google Scholar

[8]

K. Nodira, B. Magdalena, and S. Dan. 2006. Towards correcting input data errors probabilistically using integrity constraints. In MobiDE. 43--50.

Google Scholar

[9]

T. Rekatsinas, X. Chu, I. F. Ilyas, and C. Ré. 2017. HoloClean: Holistic Data Repairs with Probabilistic Inference. PVLDB 10 (2017), 1190--1201.

Digital Library

Google Scholar

[10]

J. Shin, S. Wu, F. Wang, C. De Sa, C. Zhang, and C. Ré. 2015. Incremental Knowledge Base Construction Using DeepDive. PVLDB 8, 11 (2015), 1310--1321.

Digital Library

Google Scholar

[11]

S. Song, A. Zhang, J. Wang, and P.S. Yu. 2015. SCREEN: Stream Data Cleaning Under Speed Constraints. In SIGMOD. 827--841.

Digital Library

Google Scholar

[12]

A. Zhang, S. Song, J. Wang, and P.S. Yu. 2017. Time Series Data Cleaning: From Anomaly Detection to Anomaly Repairing. PVLDB 10 (2017), 1046--1057.

Digital Library

Google Scholar

Cited By

View all

Nilforoushan SWu QMilani M(2022)Entity Matching with AUC-Based Fairness2022 IEEE International Conference on Big Data (Big Data)10.1109/BigData55660.2022.10020293(5068-5075)Online publication date: 17-Dec-2022
https://doi.org/10.1109/BigData55660.2022.10020293
Yin SYi YWang H(2020)Data Cleaning About Student Information Based on Massive Open Online Course SystemData Science10.1007/978-981-15-7981-3_3(33-43)Online publication date: 20-Aug-2020
https://doi.org/10.1007/978-981-15-7981-3_3

Index Terms

CurrentClean: Interactive Change Exploration and Cleaning of Stale Data
1. Information systems
  1. Data management systems
    1. Database design and models
      1. Data model extensions
        Temporal data

Recommendations

An Enhanced Technique to Clean Data in the Data Warehouse
DESE '11: Proceedings of the 2011 Developments in E-systems Engineering

Data quality is a critical factor for the success of data warehousing projects. Improving the quality of data is important in data warehouse, because it is used in the process of decision support, which requires accurate data. There are many errors and ...
A Comparative Study of Data Cleaning Tools

In the information era, data is crucial in decision making. Most data sets contain impurities that need to be weeded out before any meaningful decision can be made from the data. Hence, data cleaning is essential and often takes more than 80 percent ...
A data quality metamodel extension to CWM
APCCM '07: Proceedings of the fourth Asia-Pacific conference on Comceptual modelling - Volume 67

The importance of metadata has been broadly referred in the last years, mainly in the field of data warehousing and decision support systems. Contemporarily, in the adjacent field of data quality, several approaches and tools have been set out for the ...

Comments

Information & Contributors

Information

Published In

CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management

November 2019

3373 pages

ISBN:9781450369763

DOI:10.1145/3357384

General Chairs:
Wenwu Zhu
Tsinghua University, China
,
Dacheng Tao
University of Massachusetts, USA
,
Xueqi Cheng
Institute of Computing Technology, CAS, China
,
Program Chairs:
Peng Cui
Tsinghua University, China
,
Elke Rundensteiner
Worcester Polytechnic Institute, USA
,
David Carmel
Amazon Research, USA
,
Qi He
LinkedIn, USA
,
Jeffrey Xu Yu
Chinese University of Hong Kong, China

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 November 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CIKM '19

Sponsor:

CIKM '19: The 28th ACM International Conference on Information and Knowledge Management

November 3 - 7, 2019

Beijing, China

Acceptance Rates

CIKM '19 Paper Acceptance Rate 202 of 1,031 submissions, 20%;

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
191
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)0

Reflects downloads up to 14 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Nilforoushan SWu QMilani M(2022)Entity Matching with AUC-Based Fairness2022 IEEE International Conference on Big Data (Big Data)10.1109/BigData55660.2022.10020293(5068-5075)Online publication date: 17-Dec-2022
https://doi.org/10.1109/BigData55660.2022.10020293
Yin SYi YWang H(2020)Data Cleaning About Student Information Based on Massive Open Online Course SystemData Science10.1007/978-981-15-7981-3_3(33-43)Online publication date: 20-Aug-2020
https://doi.org/10.1007/978-981-15-7981-3_3

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

An Enhanced Technique to Clean Data in the Data Warehouse

A Comparative Study of Data Cleaning Tools

A data quality metamodel extension to CWM