Delete data from Data Lake only for one company

HI All.

In our company in environment TST (CLOUD) we have 3 company 790, 090 and 092 – all with division AAA.

790AAA – clear company

090AAA – test company

092AAA – full copy 090AAA

Sometimes we do full copy from 090AAA to 092AAA and copy 790AAA (clear company) to 090AAA with clear all database in 090AAA. After that we have in Dala Lake old information for old 090AAA and we want clear information in DataLake ONLY for 090AAA (not for ALL companies)

I tried to do it in Data Fabric -> Purge - > Purge by filter and after select Data range from, Data Range To, Data Objects I have position Account entity. After wrote in this position 090_AAA (090AAA or 90AAA) and I had information that no data for purge.

But if I stay clear position Account entity I can purge data from Data Lake but for all companies.

I check in Atlas information for MITBAL – Object ID and saw that Data Object don't have any Account entity.

What I do wrong?

Is it possible clear Data Lake ONLY for one company (with division)?

And if YES what will I need to do next with Compass (maybe clear cache for all data object) and how I can do this.

Best regards

Aleksiej

Find more posts tagged with

Infor Data Fabric

Accepted answers

All comments

unknown

I've seen several customers need this type of functionality as well. Often on the LN ERP implementations, they may create a company for configuration testing, and eventually wipe it out and restart with it. The old company data does not get a delete record published to the data lake, so we have a scenario where the old 'orphan' data is in the data lake. They don't want to remove all history for a given table, just the history for a specific company.

unknown

Hi Aleksiej,

Data Lake is considered "immutable" meaning the data cannot be edited or removed at the line-level using SQL DML statements like DELETE. The purge fields in Data Lake correspond to payload headers that source applications can share & publish. However, it's often the case that applications are replicating data from multiple companies, accounting entities, etc. and data within the data objects is striped across functional companies. While purge is working "as designed" here, it unfortunately won't be of assistance in this particular case.

Recently, we've adding our new streaming ingestion platform to Data Fabric and are investigating options to add data organization features so that administrators could create conditions to organize events by data attributes like "company." We're interested in knowing if and what kind of rules and features users might want to have available them to configure data organization such that when it lands in Data Lake, it's better collated for downstream management.

I'm likewise curious if everyone's use cases here are more aligned with non-production environments (e.g. TST, TRN).

Copyright © 2025 Infor. All rights reserved.