The challenges of enterprise record keeping and data quality
There are two elements to getting pay right in Australia, one side is the employment requirements that describe the entitlements that an employee should receive and the other is the maintenance of correct business records that are used to calculate what the employee should be paid based on the applicable legal requirement.
Business records can be a challenge for all companies, there are multiple data points that have to be collected for every shift, every employee, every day. Data collection can be especially challenging for companies with thousands of employees across the country.
The high level of data collected and stored by large enterprises can cause increased challenges when remunerating staff, which is one of the key reasons that employee underpayments exist. It can also create numerous challenges when the company undertakes an external review of their pay processes.
Key business record challenges for businesses
Data quality is key to understanding how enterprise business records can be used to increase operational efficiency. Data quality can be assessed on six dimensions; completeness, accuracy, consistency, validity, uniqueness and integrity. When reviewing roster and payroll data the most frequent data quality issues we see at PaidRight are around completeness, consistency and volume of data.
It’s expected that over a period of time errors will occur and not all datasets will be totally complete. There may be some missing roster or timesheet data that not only could have caused an incorrect payment but also makes it difficult for a third party to review.
One of the main contributors to missing data is incorrect or missing data on break times. Given the changing nature of the way business is conducted and employees work, we regularly see employees failing to punch in and out of breaks, which means that not all the required data points have been collected to know exactly when the employee had their break. Did the break occur in ordinary hours or during overtime?
Consistency in enterprise data has proven to be a challenging aspect as sometimes not all payroll data received is congruous. Companies may have used different payroll systems over a period of time, which is one of the main reasons for inconsistencies. This could lead to difficulty in identifying data points from different payroll systems that have the same context or even as basic as date formats changing with some presenting as dd/mm/yyyy and some mm/dd/yyyy.
One of the biggest challenges of working with enterprise business records is the huge volume of data. The bigger the company, the bigger the dataset. When dealing with a big volume of data there is firstly a greater possibility for incompleteness and inconsistency but also the sheer volume of data alone means that there may be constraints on time and speed of cleaning and reviewing the data.
PaidRight and the use of Google Cloud Platform
At PaidRight we handle very large customer datasets securely by leveraging the enormous scalability of Google Cloud Platform, particularly BigQuery and Cloud Storage which are serverless and auto-scaling systems which enable our large data storage and processing needs.
Our use of Google Cloud Platform (GCP) will be further explored in our next blog, where we will discuss how PaidRight addresses key record keeping challenges like missing and inconsistent data as well as managing large sets of enterprise data.