Missing data exploration in air quality data set using r-package data visualisation tools

Missing values often occur in many data sets of various research areas. This has been recognized as data quality problem because missing values could affect the performance of analysis results. To overcome the problem, the incomplete data set needs to be treated using imputation method. Thus, explor...

Full description

Bibliographic Details
Published in:Bulletin of Electrical Engineering and Informatics
Main Author: Ghazali S.M.; Shaadan N.; Idrus Z.
Format: Article
Language:English
Published: Institute of Advanced Engineering and Science 2020
Online Access:https://www.scopus.com/inward/record.uri?eid=2-s2.0-85083047189&doi=10.11591%2feei.v9i2.2088&partnerID=40&md5=909ffdb37449535c3d93fd96c88de6ec
Description
Summary:Missing values often occur in many data sets of various research areas. This has been recognized as data quality problem because missing values could affect the performance of analysis results. To overcome the problem, the incomplete data set needs to be treated using imputation method. Thus, exploring missing values pattern must be conducted beforehand to determine a suitable method. This paper discusses on the application of data visualisation as a smart technique for missing data exploration aiming to increase understanding on missing data behaviour which include missing data mechanism (MCAR, MAR and MNAR), distribution pattern of missingness in terms of percentage as well as the gap size. This paper presents the application of several data visualisation tools from five R-packages such as visdat, VIM, ggplot2, Amelia and UpSetR for data missingness exploration. For an illustration, based on an air quality data set in Malaysia, several graphics were produced to illustrate the contribution of the visualisation tools in providing insight on the pattern of missingness. Based on the results, it is shown that missing values in air quality data set of the chosen sites in Malaysia behave as missing at random (MAR) with small percentage and long gap sizes of missingness. © 2020, Institute of Advanced Engineering and Science. All rights reserved.
ISSN:20893191
DOI:10.11591/eei.v9i2.2088