Eptura Workplace Status - S2 - Inability to Access iOffice (some users)

S2 - Inability to Access iOffice (some users)

Incident Report for Eptura Workplace

Postmortem

On April 3rd 2023 we began having reports of an issue with the inability to access iOffice, receiving reports of an error that on attempt of logging they were met with an error (javax.servlet.ServletException: javax.servlet.ServletException: java.lang.NoClassDefFoundError: Could not initialize class com.iofficeconnect.api.yoga.SplitConfig)

Type of Event:

Service Disruption

Services\Modules Impacted:

Production

Remediation:

Engineering has implemented a try/catch around the split issue in our login page. This will allow the login to load/work despite Split issues arising.

Timeline: (Times are in MST)

At 10:44am Devops alerts engineering of increased errors and intermittent downtime with status cake. Engineering beings their investigation of why test sites and status cake are showing intermittent downtime.

Around 11:15 Support notifies engineering of more increased issues with logging and updates status page 11:19. Engineering identifies a solution so that login page can load, although the split error is still occurring at 12:09pm.

Engineering notifies support that the investigation continues for the split error. The Login page solution is built and deployed to our test site, confirming that logging can happen, but split error root cause continues.

QA has done a full test confirming logging works and at 15:36pm Production build with login changes to split error has been found and implemented, Subsequent hot fix deployed 16:23pm and Status page updated to Monitoring.

7:36am after monitoring (Following day) Status page changed to reflect incident resolved.

Total Duration of Event:

7Hrs In monitoring for 15 hrs. 2 mins

Root Cause Analysis:

An error in a third-party library caused the login page to not load. This only affected the login page, and only for some users when attempting to log in. We have put in place measures to make sure that if there is a similar third-party error in the future, the login page will continue to load as expected.

Preventative Action:

Engineering has implemented a catch for split issues that could arise on our login page. We’re currently working on more data gathering available for logging.

Posted Apr 13, 2023 - 19:16 UTC

Resolved

After monitoring, we are marking this issue as resolved. We appreciate your patience as our team worked to find a resolution for the "S2 - Inability to Access iOffice (some users)". We will have a completed RCA in the next 10 Business Days.

Posted Apr 04, 2023 - 13:36 UTC

Update

We have moved into a monitoring phase. Our engineering team has identified a change and has deployed a fix to production. As our support team begins to reach out to you and confirm resolution, we will continue to monitor. We appreciate your patience through this incident.

Posted Apr 03, 2023 - 22:34 UTC

Update

We continue to investigate this issue with the inability to access iOffice for some users. Our next update will be 7pm MST. We appreciate your continued patience as we work towards a resolution.

Posted Apr 03, 2023 - 21:05 UTC

Investigating

We are currently investigating the issue where some users have the inability to access iOffice. We will share more on the status page as we learn next steps. Our next update will be at 3pm MST or sooner.

Posted Apr 03, 2023 - 17:19 UTC

This incident affected: System Status.