Python scripts JSON output mangled by data flow. Specifically the double quotes are replaced with single quotes.
I am working on a Data Flow in Cloud MT that grabs a bod, takes one value from the bod, sends that value in a call to CRS610MI and then the response from the call is filtered down to a JSON object with 6 Key/values.
An example of the output from the last above-described step.
{
"DIVI": "ABC",
"ORNO": "123456789",
"CUNO": "324433",
"ORTP": "A01",
"CUOR": "12-20-22-4",
"OREF": "noreply@infor.com"
}
I wrote a python script that takes those values, puts them into a dictionary, and then creates a new dictionary from a JSON request body template.
The request body template:
{
"emailMessage": {
"to": "string",
"cc": "string",
"bcc": "string",
"subject": "string",
"body": "string",
"template": "string",
"attachments": {
"file": [
{
"pid": "string",
"filename": "string",
"mimetype": "string",
"base64": "string",
"xquery": "string"
}
]
}
}
}
Then the script blanks out the unnecessary fields and decorates the required/used fields with data from the first dictionary.
So you would get something like
And I test this inside of the ION Desk scripting editor.
I get:
The data in the example is not actual data. But I take an example with actual data from our tenant that is output from that test result in the ION Desk script editor and run it in Postman or the IDM API Documentation. The request body, as the output generated in the above image, is valid and working.
However, something after the scripting element in the dataflow is modifying the JSON.
It's removing the single quotes and replacing them with Double quotes.
I presume this happens after the output leaves the scripting element and before the Java-based FastXML Jackson JSON Parsing library validates the output JSON.
I get this error when I run the data flow:
<?xml version="1.0" encoding="UTF-8"?>
<ConfirmBOD xmlns="
">schema.infor.com/.../2" xmlns:xsd="
">www.w3.org/.../XMLSchema" xmlns:xsi="
">www.w3.org/.../XMLSchema-instance" xsi:schemaLocation="
schema.infor.com/.../2 ">schema.infor.com/.../ConfirmBOD.xsd" versionID="2" releaseID="9.2"><ApplicationArea><Sender><LogicalID>infor.scripting.buildjsonforcall_4259_ecomorderplacementdocumentredirect</LogicalID><ComponentID>channel.queue.MessageDistributor/Qcc27fecc-ff89-491f-8a01-84eea9276ca1</ComponentID></Sender><CreationDateTime>2022-12-22T16:12:42.570Z</CreationDateTime><BODID>eoid://WAUSAU_TRN/BOD?Verb=Confirm</BODID></ApplicationArea><DataArea><Confirm><TenantID>WAUSAU_TRN</TenantID><AccountingEntityID>100_WSC</AccountingEntityID></Confirm><BOD><BODFailureMessage><ErrorProcessMessage><Description>com.infor.ion.container.common.exception.ServiceContainerException: Json parsing error occurred while resolving headers from script response. Reason: Unexpected character (''' (code 39)): was expecting double-quote to start field name
at [Source: (String)"{'emailMessage': {'to': 'noreply@infor.com', 'cc': '', 'bcc': '', 'subject': 'WSC Confirmation of Order #123456789 Customer PO# 12-20-22-4', 'body': 'WSC Confirmation of Order #123456789 Customer PO# 12-20-22-4', 'template': 'WSCecom1', 'attachments': {'file': [{'pid': 'M3_OrderConfirmation-3680-1-LATEST', 'filename': '', 'mimetype': 'application/pdf', 'base64': 'UTF-8', 'xquery': "/M3_OrderConfirmation[
@M3_OrderNumber ='123456789'"}]}}}"; line: 1, column: 3]
at com.infor.ion.broker.scripting.utils.ScriptingUtils.createNormalizedMessage(ScriptingUtils.java:302)
at com.infor.ion.broker.scripting.internal.ScriptingAdapter.processMessage(ScriptingAdapter.java:304)
at com.infor.ion.broker.scripting.internal.ScriptingAdapter.handleMessage(ScriptingAdapter.java:369)
at com.infor.ion.container.element.base.adapter.BaseElement.handleMessage(BaseElement.java:409)
at com.infor.ion.container.element.base.adapter.BaseElement.process(BaseElement.java:204)
at com.infor.ion.broker.process.channel.queue.md.MDDontOwnQueuePollOneQueue.handleMessage(MDDontOwnQueuePollOneQueue.java:1328)
at com.infor.ion.broker.process.channel.queue.md.MDDontOwnQueuePollOneQueue.bodyOfReceivingLoop(MDDontOwnQueuePollOneQueue.java:868)
at com.infor.ion.broker.process.channel.queue.md.MDDontOwnQueuePollOneQueue$1.run(MDDontOwnQueuePollOneQueue.java:737)
at com.infor.ion.shared.tenant.ThreadLocalRunnable.run(ThreadLocalRunnable.java:193)
at com.infor.ion.grid.common.util.ObservableSingletonThread.lambda$start$0(ObservableSingletonThread.java:114)
at com.infor.ion.shared.tenant.ThreadLocalRunnable.run(ThreadLocalRunnable.java:193)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.fasterxml.jackson.core.JsonParseException: Unexpected character (''' (code 39)): was expecting double-quote to start field name
at [Source: (String)"{'emailMessage': {'to': 'noreply@infor.com', 'cc': '', 'bcc': '', 'subject': 'WSC Confirmation of Order #123456789 Customer PO# 12-20-22-4', 'body': 'WSC Confirmation of Order #123456789 Customer PO# 12-20-22-4', 'template': 'WSCecom1', 'attachments': {'file': [{'pid': 'M3_OrderConfirmation-3680-1-LATEST', 'filename': '', 'mimetype': 'application/pdf', 'base64': 'UTF-8', 'xquery': "/M3_OrderConfirmation[
@M3_OrderNumber ='123456789'"}]}}}"; line: 1, column: 3]
at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:2391)
at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:735)
at com.fasterxml.jackson.core.base.ParserMinimalBase._reportUnexpectedChar(ParserMinimalBase.java:659)
at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._handleOddName(ReaderBasedJsonParser.java:1860)
at com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:734)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:176)
at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:322)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4674)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3629)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3597)
at com.infor.ion.broker.scripting.utils.ScriptingUtils.filterHeadersFromJson(ScriptingUtils.java:600)
at com.infor.ion.broker.scripting.utils.ScriptingUtils.createNormalizedMessage(ScriptingUtils.java:299)
... 11 more
</Description><Type>GENERAL</Type><ReasonCode>GENERAL</ReasonCode></ErrorProcessMessage></BODFailureMessage><OriginalBOD><MessageHeader><MessageHeaderProperty xmlns="" listID="MessageHeader"><NameValue name="CREATED_DATE_TIME">2022-12-22T16:12:31.177Z</NameValue></MessageHeaderProperty><MessageHeaderProperty xmlns="" listID="MessageHeader"><NameValue name="BODType">optiEmailIDMCall</NameValue></MessageHeaderProperty><MessageHeaderProperty xmlns="" listID="MessageHeader"><NameValue name="TenantId">WAUSAU_TRN</NameValue></MessageHeaderProperty><MessageHeaderProperty xmlns="" listID="MessageHeader"><NameValue name="AccountingEntity">100_WSC</NameValue></MessageHeaderProperty><MessageHeaderProperty xmlns="" listID="MessageHeader"><NameValue name="BODId">731f941eca6742aa991f104d6457d79f</NameValue></MessageHeaderProperty><MessageHeaderProperty xmlns="" listID="MessageHeader"><NameValue name="VariationId">50412</NameValue></MessageHeaderProperty><MessageHeaderProperty xmlns="" listID="MessageHeader"><NameValue name="Encoding">UTF-8</NameValue></MessageHeaderProperty><MessageHeaderProperty xmlns="" listID="MessageHeader"><NameValue name="FromLogicalId">infor.api.ecomorderacknowledgements_4093_EcomOrderPlacementDocumentRedirect</NameValue></MessageHeaderProperty><MessageHeaderProperty xmlns="" listID="MessageHeader"><NameValue name="ToLogicalId">infor.scripting.buildjsonforcall_4259_ecomorderplacementdocumentredirect</NameValue></MessageHeaderProperty><MessageHeaderProperty xmlns="" listID="MessageHeader"><NameValue name="MessageId">e8a8bfbf-6bf6-43cc-aab7-eddce574cd38:-ION-:005ef0d1245348e78d755a10dc571cd5</NameValue></MessageHeaderProperty></MessageHeader><MessageContent xmlns="">
{"DIVI":"WSC","ORNO":"123456789","CUNO":"323280","ORTP":"A01","CUOR":"12-22-2022-1","OREF":"noreply@infor.com"}
</MessageContent></OriginalBOD></BOD></DataArea></ConfirmBOD>
Basically, the FastXML.Jackson.JSON parse library is trying to validate the payload as JSON, but it can't because the double quotes were replaced with single quotes by some process after the script output but before the Jackson.JasonParse action.
So I tried fixing this by adding "" before the double quotes suggested in several places where people described a similar problem with this JSONParse library.
But that does not work either.
The results are the same. Single quotes where the double quotes should be.
Any suggestions? Has anyone here encountered this sort of behavior before? Am I doing something wrong?
Python is an excellent tool for manipulating string data, is this normal behavior from ION Desk Data Flows? If so that is a shame because it would prevent users from building JSON payloads easily with Python in ION Desk.
I will file a ticket with Infor, but I welcome any input or suggestions.
