Bulk data transfer from cloud db to Datalake

srkndnsn

Dear all,

We are migrating from an 107 on-premises environment to the cloud and wishes to continue using our existing local BI reporting tool.

Thats why, we would like to bulk data transfer fom cloud to local db system for our customer. Let me explain using an example so it’s easier to understand:

For instance, I’m going to create an exact copy of the tccom100 table in the local database and I want to update it every night. If we do this via a stored procedure(db connection in ION), it involves BOD parsing, which is a lot of work, and there’ll also be a high traffic every time the BP is updated. As we need 15–20 tables, this would be a rather time-consuming solution.

Is there a better way to transfer the updated data just once a day? Would Data Lake be a suitable solution for this? I would like to hear your best practices.

Thanks in advance!

Best regards,

Serkan

Kavya

We completed an LN On-Prem to LN Cloud migration in April where the customer had a very similar requirement. Instead of replicating data to an on-premise reporting database, they chose to build their reports in Infor Birst, which is Infor's BI solution and integrates well with LN. The customer was satisfied with this approach as it reduced their BI licensing and maintenance costs while providing a scalable reporting platform.

That said, if the requirement is specifically to replicate 15–20 tables to an on-premise database and continue using an existing BI tool, I would recommend the following architecture:

LN Cloud → Data Lake → On-Premise Database via ETL Tool

This approach is scalable, easier to maintain, and can be expanded in the future as reporting requirements grow.

I would not recommend using BODs for this scenario. In addition to the overhead of message processing and parsing, there are some limitations:

• Standard BODs may not contain all the fields required for reporting.
• Custom fields (CDFs) are often not included in standard BODs.
• In some cases, custom BODs need to be developed to expose the required data.
• Event-driven integrations can generate unnecessary traffic when the requirement is only for a nightly refresh.

With Data Lake, you have much better control over the data that is published and extracted, including custom fields, making it a more suitable solution for bulk data synchronization and reporting purposes.