The Infor OS 2022.08 release featured a number of enhancements to the ION ETL Client for Data Lake, which is a utility that can be used for data extraction, upserting data and replicating data from Infor Data Lake on a Windows Client.
This video below provides a general overview of the ION ETL August 2022 update including a refresher on the improvements introduced since December 2020:
[View:
Click here to download the slides.
Here is a summary of the changes introduced:
QueryAll function changed in Data Lake Input transformation step
The QueryAll function is changed in the Data Lake Input transformation step so that it uses the data object’s Indexed Date (dl_document_indexed_date) timestamp property. The Indexed Date property and value are used in the Data Lake API filter query and in the dlDocumentDate field values. When you upgrade, you are not required to change the existing transformations that use QueryAll. The existing transformations automatically use the Indexed Date timestamp.
In the previous version, the QueryAll function was using the Stored Date (dl_document_date) property. In specific scenarios, when indexing delays occurred in Data Lake, the incremental load mechanism could miss data objects, so clients were required to reload data from the past.
ETL Tool: retry on errors and timeouts
Enhancements were made to the ETL Tool robustness by including retries on error responses when calling the Data Lake API, Data Catalog API and the API Gateway. Retries are performed when these errors occur:
- A 5xx error
- A ZLIB error
- A generic connection time-out error
ETL Tool: increased connection time-out
The connection time-out limit for the ETL Tool has been increased to 1 minute to avoid frequent timeout errors.
QueryAllOld function deprecated in Data Lake Input step
The QueryAllOld function in the Data Lake Input step is deprecated. The function uses the Data Lake Payloads Stream By ID API, which is not optimized for extractions with the ETL Tool.
If your solutions use the QueryAllOld function, rework your transformations to use the QueryAll function. If you need help, please reach out to Infor Services.
Additional resources
