Greetings;
I have a need to generated a document from the Data Lake on a daily schedule. While this is similar to the question Venky did, "How to run a Compass API as a Job", I'm required to make this work completely in ION. *I have written a C# program to export a Data Lake query to a file, but the customer doesn't want anything not in the Infor Cloud.
My question is this, I've setup ION Desk so I can schedule a Data Lake query and write the results to a file using ION Desk. However, the solution took a little bit of duct tape and bailing wire to get working.
Question 1: Does Infor has plans to release an official method of doing something like this? A way to schedule an extract to run in ION?
Question 2: The V2 Compass APIs are marked Beta. Since V1 doesn't support JSON, but V2 does, I had to start the query with a V2 call to get JSON output. I use the V1 APIs for everything else. The V2 work, but they are definitely throw odd errors at times, but a retry seems to resolve it. Any timeline on those becoming Gold?
For the curious, the current way I'm testing this process is as follows:
- Setup an IONAPI connection to the Data Lake
- Download the Swagger for the Data Lake APIs
- Edit the Swagger document to allow V2 API calls (ION Desk won't call deprecated)
- Upload the Swagger to the IONAPI connection you created for the Data Lake
Test in ION API at this point to make sure your connection works
Setup an ION Desk Connection point to the ION API Connection for the Data Lake
Create BODs for Job Status and the BOD for the data you want to extract
Set the schedule for the Job to Run, I chose 2 hours for development testing the Item Catalog
Create a Document Flow to receive the document from the connection point, extract the query ID with a Python script, and pass the value into the workflow to read the query status in a loop.
Create a workflow to check the status in a loop. (This is one of the hackish things I don't like, the V2 APIs are marked Beta by Infor). I set the Loop to 60 times with 1 minute delay. ***EDIT*** I ended up using V1 API calls for everything except the initial call. You can start the query with a V2 call to set the return data type to something ION Desk / Connect with accept, then use the V1 API calls to check the status, they are much more reliable. ***End Edit***
When the status returns FINISHED, the Document Flow writes the file to the SFTP Server for Processing (I created a File Connector for the Item Catalog)
This was a bear to figure out, and I hope it helps folks get started in the right direction. I still think an external program is best for extracting data, but sometimes we're required to do it all in cloud.
Billy
