Can I use my existing ION integrations to send data to the Data Lake?

Legacy Contributor

Editor's note: this Q&A topic features a real sample question to demonstrate the new Q&A features of the Data Lake forum.

"I currently use Document Flows in ION to model data to be exchanged across multiple systems. Is all of this data already available in the Data Lake or do I need to do something in order to publish my integration data to the Data Lake?"

Legacy Contributor

Sending content to the Data Lake is an explicit decision - not all of your integration data moving around ION today becomes immediately available in the Data Lake. However, it's extremely simple to use the work you've already done in setting up your integrations and configure that transactional data to make it over to the Data Lake whenever your integrations fire off.

Data Lake Flows

The Infor OS February release (Technology Blog announcement) featured a similar-but-different flow modeler named "Data Lake Flows," an almost identical flow modeler if you've already designed your own Document Flows for integrations. The only difference between the two is that Data Lake Flows always ends in a data object being sent to the Data Lake.

You can create, model, save, and activate a Data Lake Flow much like you would in an ordinary Document Flow. Today, this functionality is currently embedded in ION's Connect navigation menu under Document Flows. Note that activating a Data Lake Flow means that any new data picked up by ION will now be captured to the Data Lake as well. If you have a backlog of integration content sent in the past that would be useful in the Data Lake, see the OneView section below.

We also introduced some additional icons in the Tiles overview page to make it easier to identify what's a Document Flows vs. Data Lake Flow.

OneView

If you've been using ION for a long time now, you'll know that ION stores a copy of your integration transactions as well as details, events, and other logged impressions in OneView, ION's troubleshooting and lineage application. Rather than having to resync all of your data from your source systems, you may instead wish to publish content to the Data Lake directly from OneView.

Simply run a OneView query to return a group of messages based on your filter parameters. You'll see a new option named "Publish to Data Lake" the triggers a background process that immediately starts running to make your historical integration data available in the Data Lake and exposed to our Data Lake services tier.