What latency can we expect from the datalake API's?

Hello!

Our team is trying to sync Infor M3 tables with lots of records and we were wondering what would be the latency if we expect to pull more than 10,000 records in one call?
Would datalake be the ideal solution to go with in such a scenario? This would be happening in an M3 multi-tenant environment.

Appreciate your attention to this topic.

Thanks,

Suhas

Find more posts tagged with

Infor Data Fabric

Accepted answers

Legacy Contributor

Hi Suhas,

I'd recommend trying to experiment with this as it's relatively easy to do. If you haven't already, I'd also recommend taking a look at M3's new capabilities around streaming data to Data Fabric - as compared to using ION for delivery.

You can initialize some loads from M3 for delivery to Data Lake and experiment with Compass SQL, using the UI or APIs, to see query processing overhead. Generally speaking, you shouldn't expect database-like performance as this is a data platform that leans towards scaling volume of data retention but retrieving 10,000 records isn't especially challenging. Query platforms are highly dependent on various factors and in the case of Data Lake, you benefit from optimizing for column selection. Limiting the number of columns needed for a use case delivers faster performance, as does utilizing analytical (aggregation) functions on extraction.

As well, using features like infor.lastModified() to manage incremental load processing helps manage workload retrieval and overall query performance.

All comments

Rene Bottern

Is replication to Datalake sufficient, or do you need it somewhere else (i.e. local SQL instance) in the end?

Suhas Kalasapura

Replication is sufficient. To clarify, I want to load this into datalake as M3 API's have restrictions with respect to the amount of records that can be pulled in one call. So after we replicate the database, I wanted to know what the latency would look like when we try to pull the data from the datalake.

Legacy Contributor

Hi Suhas,

As well, using features like infor.lastModified() to manage incremental load processing helps manage workload retrieval and overall query performance.

Rene Bottern

After the initial load has been taken care of, the 'latency' according to Infor is 10+ minutes.

Copyright © 2025 Infor. All rights reserved.