Incident Description After our deployment on Friday, 17 March 2023 we started seeing a huge increase in an error reporting ids cookies required in the request.
Actions Taken An immediate assessment of affected services and products was conducted. Immediate measures were explored by our engineering team by adjusting how our back-end services extract the headers contained in the response from IDS (on login calls) to append to upstream requests.
Root Cause Upgrading the node version on some of our back-end services resulted in using an upgraded implementation of the NextJs be nodejs native implementation which changed how response headers are retrieved.
Resolution Using node-fetch instead will allow us to keep our back-end services on the same version of node as the other services.
Preventative Measures Using node-fetch instead will allow us to keep back-end services on the same version of node as the other services without being concerned with any bugs possibly introduced by changes to NextJs’s implementation.
Conclusion The issue resulted in some clients not being able to make any payments specifically in cases where MFA and OTPs were required. The issue has been resolved without clients reporting it although users were negatively affected.