Trigger ION API after Data Lake ingestion completes?

Is it possible to trigger an ION API call after a Data Lake Ingestion activity completes?

For example, I have a d/EPM load query that loads data from the Data Lake that I want to execute when relative data is replicated to the Data Lake.

I've successfully created a ION API Data Flow that triggers a d/EPM load query with the same IMS document as the Data Lake flow, but the API load query completes before the Data Lake ingestion completes so the data in d/EPM is not up to date.

My next thought was to modify the Data Lake ingestion flow with a filter and ION API activity after the Data Lake ingestion activity, instead of having a separate data flow, but that doesn't appear to be possible since the Data Lake ingestion activity must be at the end of the flow.

Any thoughts or ideas?

Thanks!
Andy

Find more posts tagged with

Infor Data Fabric

Accepted answers

All comments

antonio-caria

Manually?

Kevin Heiman

a couple of thoughts:.

If the data being loaded has the batch sequence in it then you can put a filter on the flow to take action on the last file of the batch. ie BatchSeq = BatchSize https://docs.infor.com/ion/2022.x/en-us/ioncedg_cloud_osm/default.html?helpcontent=dqn1652777923110.html

I have used this in a couple of scenarios. The producer has to publish the Batch properties. You could put a nn second delay on the last one before calling the API if you really wanted to be safe.

If there are not Batch headers however there is something in the data that marks the last row, you could use Scripting to detect and then route to a API call .

Legacy Contributor

Another option would be to delay the load query with an 'Wait' activity in your flow before the API step; But its not a fool proof mechanism.

andy

Thanks Vignesh, I actually implemented this in an IPA flow (triggered by the rep set) with a 3 minute wait, as the data lake load can take up 2 1/2 minutes, even though it's only a single record.

I'll probably end up implementing a Data Lake check before the load query, but that feels like a little overkill since I have to loop to check the Compass query is done, and then loop the check if the new data in the data lake has been loaded.

It would be nice if the Data Lake ingestion flow in ION could publish a confirmation when completed, or allow activities after the Data Lake activity.

andy

Thanks for the insight Kevin. It's actually only a single AnalyticCube record, so no batch. I think I've got what I need by using and IPD flow triggered the relative replication set.

Kevin Heiman

If you are simply wanting to verify a file is there, then consider using the Data Fabric object APIs. Just loop looking to see if the file instance is there using whatever you mark the file with, such as Document ID, or Message ID, or simply 'in the last 5 minutes'.., where the result count >1. It would be a little less work that using the Compass API, but depends on what you are attempting to do.

However if the is in a rep set, thenit woudl have batch sequence in it. Note that a batch of 1 will not be a batch, so you can just check that in the Data Flow and progress. Check the Batch Seq to be equal to the BatchSize should still work as we do that in some of our solutions.

Again I am a bit vaugue on what you are sending, but it sounds like you have some ideas and ways forward.

Copyright © 2025 Infor. All rights reserved.