Notes
This was our first full sprint in the new production Workday environment. All datasets are now available, with only two exceptions:
- Billing schedules
- Tax codes
Tax codes have a slightly different data model in the production environment than they did in QA, so we’re working with our EAP partners to adapt our models and bring in the new data.
We’ve received some reports of data that’s been entered in Workday not appearing in the Finance API in a timely fashion. We’re working with our upstream data sources to determine the cause of these. It’s our goal that no data in the API should be more than 24 hours out of date with Workday, and ideally much less. The way data flows to us is:
- Data is entered or generated in Workday
- Workday scheduled integrations deliver data to EAP every evening, Monday through Friday
- EAP’s ETL process makes data available in the operational data store (ODS) tables
- The Finance API’s ETL updates from the ODS tables in the data lake every 24 hours
Most of our time this sprint was spent fixing small bugs that popped up now that we have production data. We fixed a bug related to how worker IDs display in the API when they’re related to worktags, like the accountants that are assigned to awards. They should always appear as zero-padded strings of numbers, like “00123456”. We found an error that was causing them to display with decimal points, but this is fixed now.
We corrected some inconsistencies in the content of different sections of the response. Each record should have four parts: an ID, a type, an “attributes” element that contains the properties of that record, and a “meta” section that shows when the record was last updated. Some of our endpoints had redundant elements, like a lastModified date that was included in the attributes in addition to the meta section. We’ve fixed that as well.
We updated our specification to ensure the difference between journal entries and journal entry lines was called out. We corrected an ETL issue that was causing duplicate customer addresses to appear, and we fixed a customer invoice that went missing.
We did some experimentation to determine what kind of limits we might need to put on our ETL process if we start running it more frequently. We found that if two ETL processes run at the same time, the one that started first always completes successfully and the second one always crashes. This doesn’t do any damage, though, and the next process will run sucessfully, so we’re now more confident that we can increase our ETL frequency without causing problems.
API changes
n/a