Software runs the world, and data is everywhere. But not all data is equally valuable. Data comes in many different forms, shapes, sizes, and structures. It can be structured and ordered, or it can be unstructured, badly duplicated, and altogether raw in its base form.
When migrating any SAP environment, there are many factors & topics to consider. One major topic is data. What to do with legacy data and what are the implications of what we do, are relevant questions. Whether legacy data is migrated or not, it still must be taken into consideration. The impact on legacy data and the future use of it after being migrated to Target must be well thought out.
This blog will explain about getting insight into SAP database growth and controlling the growth by Data cleaning, cleansing, archiving etc. to maintain the high-end Data quality in the organization.
Data quality is the foundation of everything that is built on an organization’s data assets. Poor data quality destroys business value. As per the Gartner 2020 Survey it has been found that Organizations estimate the average cost of poor data quality at $12.8 million per year. Also, another recent survey found that 60% organizations feel that the need to strive for data quality is the biggest challenge in data management practice.
Reasons of Data Growth
Majorly the Data Growth in SAP systems are due to some statutory and Legal requirements, still there are multiple other reasons as well which are responsible to increase the size of the Database some are –
- Duplicate records
- Huge count of work order processing
- Lack of Data Housekeeping
- No SAP Standard procedure followed
- Lack of archiving strategy
- Number of articles or stores of a company
- Orphaned entries or files
Impact of Data Growth
Due to the Data growth there are several negative impacts on the SAP system, some are
- System Performance
- System backup runtime
- System needs to Scale
- New Database Space allocation
- Increased cost for the Infrastructure
- Readiness for future modernization of SAP application
Data Management Guidelines and Priorities
1. Data prevention/data avoidance
Data Prevention or avoidance means avoiding creation of unnecessary data which removes the requirement to leverage data reduction method/exercises. This means less work for the storage layer which results in more available front-end IO to service the virtual machines.
Technically, it is possible to deactivate updating for certain types of data. If it is not required for business, then you should deactivate updating.
Example: Switch off updating for table ACCT*
2. Data aggregation/data summarization
Data aggregation is the process where raw data is gathered and expressed in a summary form for statistical analysis.
In some cases, raw data can be aggregated over a given time period to provide statistics such as average, minimum, maximum, sum, and count.
Example: For retail customers, line items are usually not necessary because their data volumes are too high for reporting.
3. Data Deletion
You can delete a lot of data that you do not want to archive soon after it has been created in your system. Many technical Tables hold the unwanted data like Logs, Change docs, Processed records, History etc. There are many SAP Standard reports are also available to perform the deletion using the selection criterion. In most cases we can reduce the Database size by 20-30% via performing the Cleanup following SAP’s recommendations.
Example: Spool data.
Note: Before you delete data records from the system, make sure that they are no longer referenced to any other data that requires these records remain in the system. If so, do not delete the data records.
4. Data Archiving
Archiving business application data must be discussed with the relevant business departments to ensure that all statutory and fiscal requirements are met and can continue to be met.
Data archiving handles data that cannot be prevented or easily deleted. You should examine archiving possibilities as early as possible in the implementation process (blueprint project preparation), and long before you go live. Check how long you want to retain your data in your system. You should only archive data that you no longer require for live operations. Archiving can only, therefore, be used in a limited context when reducing the amount of data in your system.
Archiving reduces the costs of memory, disk and administration costs. Ensures cost-efficient system upgrades and future readiness for S/4 migration. Improved system performance due to shorter response time. Reduces the cost of maintenance and run of growing application infrastructure.
Example: Archiving of accounting documents using archiving object FI_DOCUMNT. These archives header data (table BKPF) and items data (cluster RFBLG).
Data Volume Management (DVM)
SAP Data Volume Management (DVM) is a framework that helps the solution operations team to balance the need of business access to a wealth of data and IT efforts to maintain storage, databases, and applications. SAP Data Volume Management (DVM), which is an SAP tool-based approach powered by SAP Solution Manager.
What is Data Cleansing?
Data cleansing, data cleaning or data scrubbing is the process of detecting and correcting (or removing) corrupt or inaccurate records from a record set, table, or database. Technical IT or business users in their day-by-day work and enabling them to get better understanding or insight into the data they are using.
Data Correction is the process of identifying and cleansing “dirty data” using sophisticated algorithms and rules in conjunction with custom and SAP referential data.
Example: Make the System settings in the IMG of the SAP Business Partner Select Basic Settings, Go to Data Cleansing and select Activate Data Cleansing.