compass select doesn't match the data catalog

We have an issue (we just went into product) where the select from a Data Lake Table does not match the data catalog layout. I am assuming these are new fields. But what drives the columns returned when we do a select *. It obviously is not the data catalog. Is it headers on the JSON files? What will happen when we repush down data from Optiva into the DL and they have the structure of the new layouts while old json files have structures of maybe an old layout?

Tina

Find more posts tagged with

Infor Data Fabric

Accepted answers

All comments

prabodhaa

Please check below option as well.

tdouglass

This is our last resort. How is information_schema maintained?

tdouglass

HI, Should i say remove formatted date true or retain formatted data (false)?

Bojan Rafajloski

Hi Tina,

Compass does not automatically update when there are changes in the Data Catalog object schema. To update Compass cached object schema you need to manually run the clear_table command.

I advise against using the 'true' option in the clear_table command, as this would also delete ALL of the Compass cached data. Meaning, the next time you execute query it will trigger caching all the data in Data Lake for this table. In most cases this wouldn't be necessary as you want to delete the Compass cached data only from the time when the schema changed.

Instead, after using clear_table to just update the schema, you should follow that by running the clear_data command. In the clear_data function, specify the date of the schema change. Next time you run a query, it will only have to cache the new data that was stored after the schema change.

This documented here: docs.infor.com/.../default.html

Copyright © 2025 Infor. All rights reserved.