Data Preparation in CPMAI

Data Preparation in CPMAI
Table of Contents

Phase 3: Data Preparation in CPMAI

Data Preparation is the third of six stages of the CPMAI methodology. This is where a team prepares data by cleansing, transformation, labelling, and validation. This stage consumes approximately 80% of time in typical AI projects. In the context of the CPMAI methodology, this stage is more important than selecting the appropriate algorithm in the subsequent stage. A good algorithm when used on poor quality data results in a poor model. On the other hand, using a less sophisticated algorithm on data that has been prepared, will yield a better model. Therefore, the quality and work done in the Data Preparation stage directly impacts the quality of the modeling process.

What Makes Data Preparation Distinct from Other Stages of CPMAI?

The goal of the Data Preparation stage is to take the raw data confirmed to be available in the preceding Data Understanding stage and convert it to clean and ready to use data for the modeling process. The Data Understanding stage confirms the availability of data. Many teams incorrectly assume that the Data Understanding stage also confirms the usability of data. This leads to a significant underestimation of this stage of the process, as the work required for data to be available and usable is substantially different.

Data Preparation Includes What Activities?

In the process of Data Preparation, five activities focus on: cleaning data, transforming data, labeling data, checking the quality of data, and documenting compliance. Each activity progressively bridges the gap between the data assessment in Phase 2 of the process and a data set that is appropriate for training a model.

Activity

Output

Data Cleaning

No duplicate data, no unprocessed missing values, and no data errors

Data Transformation

Conversion of data to a form and structure that is comprehensible to the model

Data Labeling

Data that has been consistently labeled for supervised learning

Quality Check

Checking that the prepared data meets the criteria defined in Phase 2

Compliance Documentation

Documentation and data that shows traceability related to compliance and privacy

What Is Data Cleaning In This Phase?

Issues related to data quality can manifest themselves in behavioral issues for the model that are unpredictable and without notice. In data cleaning, duplicate data is removed, data with errors is edited, and a decision is made about dealing with missing data. Data cleaning is a repetitive process, since the decision made regarding missing data can influence significantly what the model learns.
Remove bias in data by eliminating records that have been represented an unrealistically high number of times.
Fix incorrect data. This can mean data that contains invalid values, has data that is formatted incorrectly, or data that contains an error.
Decide consistently on the handling of missing data, and document your decision. Removal, imputation, or treating missing data as missing are all possible decisions.
Unreconcilable records from different sources can be resolved by mapping during Data Understanding.

What Does It Mean To Transform Data Into Model-Ready Form?

Data must be shaped to fit modeling tasks, as modeling tasks require data in a particular form. Since raw data, which is data that has been cleaned, is typically deficient in a form that models can work with, we have to perform category encoding, perform data normalization, or create additional features in the data that help the model interpret hidden data.
Convert categorical data to a format that can be meaningfully processed.
Numeric attributes should be normalized or data should be scaled to maintain a balance in the importance of all the features in the model.
Create new features by combining or transforming existing features.
Model-specific data architecture should be used to shape time series, text, and image data.

How do you label data for supervised learning?

Using supervised learning, one must correctly assign values to target positions for training data samples, confirm that the quality of the labels is adequate, and adjust the labels if the quality of the labels is not consistent and fulfills the criteria that sets the model’s expected accuracy. Label noise (labeling agents applying different standards or criteria to the same or similar objects) will increase the noise that a model will not be able to distinguish from a real signal. This will result in an increase in the model’s error regardless of the algorithm’s complexity in the fourth phase.
Labeling standards must be set and given to labeling agents before labeling work begins to ensure consistency.
The accuracy of the labels must be confirmed by a review. This is especially needed for subjective classification tasks.
Labeling consistency for ambiguous data classes must be described.

What Are Some Examples of Validating Prepared Data Before Model Development?

Before developing a model, prepared data can be validated by checking if the data meets the requirements specified during the Data Understanding phase. It is done to avoid performance issues caused by gaps in the preparation of data and is done long before we send prepared data to the Model Development phase.
We need to analyze the prepared data in the context of the original Data Understanding requirements in terms of quantity and scope.
We need to perform summary statistics and distributions, and assess if the data fits our preconceived notions.
We need to perform checks for data leakage and assess if there are elements in our prepared data that provide our predicted model with information in order for the model to “learn” beyond what it should, thus allowing it to “cheat” during the predicted test.
We also need to verify that a validation set is separated from the training data and has been held out prior to the commencement of Model Development.

What Are Some Examples of Documenting Data Lineage and Compliance?

To document data lineage and compliance, we need to capture the source of each dataset, the transformations done to the dataset, and confirm the datasets meet the captured privacy and regulatory requirements in Data Understanding. Model Evaluation and Model Operationalization necessitate this documentation to perform their respective functions; this is the rationale behind this documentation.
We need to document the source and the method of each dataset.
We need to confirm compliance after transformations and verify privacy.
We need to document the reasons for each data exclusion.
Keep this documentation available for Model Evaluation and Operationalization as an ongoing record.

What Result Do We Get from Data Preparation?

There are two key results of the Data Preparation process. The first and most important result is a dataset that satisfies the requirements of the model. The other equally important result is a description of the processes performed to clean, transform, and label the data. The model development phase begins with this result and any data quality issues that are not resolved will carry over to the model training phase.

Output

Description

Clean, Model-Ready Dataset

Cleaned, transformed, and labeled data along with data lineage and compliance documentation.

What If We Hurry and Skip Data Preparation?

It is the Data Preparation phase of most AI projects that suffers the most from scheduling constraints. In an effort to meet a deadline, Data Preparation is often rushed resulting in faster, less efficient model development. These issues become apparent only during the model evaluation stage and by the time they are discovered it is even more expensive to remediate the data.
This phase of work is typically the most time consuming phase of a project, therefore this phase is most often shortened when a project falls behind schedule.
Data lineage that is inconsistent and unstructured passes unnoticed and will limit the accuracy of the model, resulting in the incorrect assumption that the model is faulty, when in fact the data lineage is at fault.
Data leakage occurs when the model data that was used to train the model is made available during the performance of the model in the real world. Model performance that is good during testing is indicative of data leakage.
Governance suffers and compliance documentation is not performed during the operationalization of models.

.

How Does Data Preparation Relate to the PMI-CPMAI Exam?

Data preparation falls under Domain III of the PMI-CPMAI exam in the area of Identifying Data Needs. Approximately 26% of the exam covers Identifying Data Needs and Data Understanding. Multiple questions on data preparation will likely assess the quality of data, how data is labeled, and the existence of compliance documents. The PMI Exam Content Outline combines all the activities of the readiness of the data in one domain; therefore, separation of activities by phases will not occur.

Where Does Data Preparation Fall in the Full CPMAI Methodology?

The CPMAI methodology has six phases, and data preparation is one of them. For more detail and to see where this phase falls in Business Understanding, Data Understanding, Model Development, Model Evaluation, and Model Operationalization, see The CPMAI Methodology: 6 Phases Explained.

Where Can You Find Opportunities to Apply Data Preparation?

For training opportunities on the CPMAI methodology, including data preparation, PMTI offers live instructor-led training through its CPMAI Certification Training.

Phase 3: Data Preparation FAQs

Why does Data Preparation take up so much of an AI project’s time?

Most teams usually underestimate the amount of work involved in the cleaning and the judgment required in the data transformations and the subsequent labeling and validation of the data. Furthermore, it is common to find serious quality issues that require data to be iteratively cleaned and transformed.

What does data leakage mean in the context of the steps of Data Preparation?

Data leakage describes a situation where training data incorporates information that a model should not use when making predictions. As a result, it yields excellent test performance and poor actual performance. Data Preparation’s validation step is the correct step to address this situation.

Data cleaning versus data transformation, what’s the difference?

Data cleaning comprises the elimination of errors, removal of duplicates, and filling of missing values. Data transformation is the restructuring of clean data so that it conforms to the requirement of the model’s architecture. Both have to be done, and in that order.

Which PMI-CPMAI exam domain includes Data Preparation?

Domain III: Identify Data Needs, which is the same Domain that includes Data Understanding performed as a prerequisite to the next step in Data Preparation, and comprises 26% of the questions likely to appear on the exam.

.

Yad Senapathy
Yad Senapathy

Your project managers will be trained on the PMI PMBOK Guide's best practices and ethics. They'll understand the framework of a successful project from initiating to close.

Share this article
Twitter
Facebook
Linkedln
whatsapp
telegram
pinterest
Get in Touch With Us