Missing data exploration in air quality data set using r-package data visualisation tools

• Published in: Bulletin of Electrical Engineering and Informatics
• Main Author: Ghazali S.M.; Shaadan N.; Idrus Z.
• Format: Article
• Language: English
• Published: Institute of Advanced Engineering and Science 2020
• Online Access: https://www.scopus.com/inward/record.uri?eid=2-s2.0-85083047189&doi=10.11591%2feei.v9i2.2088&partnerID=40&md5=909ffdb37449535c3d93fd96c88de6ec

Missing values often occur in many data sets of various research areas. This has been recognized as data quality problem because missing values could affect the performance of analysis results. To overcome the problem, the incomplete data set needs to be treated using imputation method. Thus, exploring missing values pattern must be conducted beforehand to determine a suitable method. This paper discusses on the application of data visualisation as a smart technique for missing data exploration aiming to increase understanding on missing data behaviour which include missing data mechanism (MCAR, MAR and MNAR), distribution pattern of missingness in terms of percentage as well as the gap size. This paper presents the application of several data visualisation tools from five R-packages such as visdat, VIM, ggplot2, Amelia and UpSetR for data missingness exploration. For an illustration, based on an air quality data set in Malaysia, several graphics were produced to illustrate the contribution of the visualisation tools in providing insight on the pattern of missingness. Based on the results, it is shown that missing values in air quality data set of the chosen sites in Malaysia behave as missing at random (MAR) with small percentage and long gap sizes of missingness. © 2020, Institute of Advanced Engineering and Science. All rights reserved.