Application of data mining techniques to identify data anomalies: a case study in the oil and gas industry

Jenifer S. McCormack; Brian Wohlschlaeger; Bryan Lanier

doi:10.1117/12.460241

12 March 2002 Application of data mining techniques to identify data anomalies: a case study in the oil and gas industry

Jenifer S. McCormack, Brian Wohlschlaeger, Bryan Lanier

Author Affiliations +

Proceedings Volume 4730, Data Mining and Knowledge Discovery: Theory, Tools, and Technology IV; (2002) https://doi.org/10.1117/12.460241
Event: AeroSense 2002, 2002, Orlando, FL, United States

Abstract

This paper presents the application of the AgentMiner^TM tool suite to improve the efficiency of detecting data anomalies in oil well log and production data sets, which have traditionally been done by hand or through the use of database business rules. There was a need to verify the data sets, once cleansed and certified to ensure that the existing data certification process was effective. There was also a need to identify more complex relational data anomalies that cannot be addressed by simple business rules. Analysis techniques including statistical clustering, correlation and 3-D data visualization techniques were successfully utilized to identify potential complex data anomalies. A data-preprocessing tool was also applied to automatically detect simple data errors such as missing, out of range, and null values. The pre-processing tools were also used to prepare the data sets for further statistical and visualization analyses. To enhance the discovery of data anomalies two different data visualization tools for the data clusters were applied.

Citation Download Citation

Jenifer S. McCormack, Brian Wohlschlaeger, and Bryan Lanier "Application of data mining techniques to identify data anomalies: a case study in the oil and gas industry", Proc. SPIE 4730, Data Mining and Knowledge Discovery: Theory, Tools, and Technology IV, (12 March 2002); https://doi.org/10.1117/12.460241

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available