Skip to main content Skip to navigation

Lecture Capture Status

Echo360 Status
March 28, 2022 1:49AM UTC
Identified - We are investigating an issue with Live playback on Echo360 Active Learning Platform. Please stay tuned to status.echo360.com; we'll have another update within the next hour.

March 28, 2022 2:58AM UTC
Identified - The Echo360 Engineering Team are currently investigating the issue and an update will be provided as soon as possible.

March 28, 2022 4:30AM UTC
Identified - The Echo360 Engineering Team continues to investigate an issue occurring with live streaming in the APAC region.

March 28, 2022 5:13AM UTC
Monitoring - Echo360 has made a change to our Live infrastructure and is continuing to monitor the Live streams within the region.

March 28, 2022 7:21AM UTC
Resolved - The issue has been identified and resolved and our teams are continuing to monitor the platform. Echo360 will update this status page with a full incident report soon.

March 9, 2022 12:13AM UTC
Investigating - We have received reports that some LMS/VLE embedded videos are not playable in our Asia Pacific region. We are investigating this issue at this time and will have another updated within an hour.

March 9, 2022 1:13AM UTC
Investigating - We are continuing to investigate reports of users not being able to play back content embedded in an LMS in our Asia Pacific region and will have another update here within one hour.

March 9, 2022 2:13AM UTC
Identified - We have identified the root cause of this issue and are now investigating a solution. We will have another update within an hour.

March 9, 2022 3:16AM UTC
Identified - We are continuing to work on a solution and will have another update within the next hour.

March 9, 2022 4:04AM UTC
Resolved - Due to a playback issue discovered following this week's Echo360 deployment (undertaken on 9 March 2022), we have rolled back this platform update and will target a 22-23 March 2022 deployment, pending further testing. A full incident report about the issue will be published here on our status page (status.echo360.com) in the next few business days.

Should you have any questions regarding this update or Echo360's deployments in general, please contact your Echo360 representative.

March 14, 2022 10:21PM UTC
Resolved - Echo360 Incident Report 20210309

Incident Date:
Start:
Asia Pacific Region: 2021-03-09 04:04 AM AEDT
Europe & Middle East Region: 2021-03-08 10:05 PM GMT
End:
Asia Pacific Region: 2021-03-09 02:49 PM AEDT
Europe & Middle East Region: 2021-03-09 03:49 AM GMT

Region(s) Affected:
Echo360 Asia Pacific Region
Echo360 Europe & Middle East Region

Service(s) Affected:
Playback of content in our Asia Pacific and Europe & Middle East Regions.

User Impact:
For several hours, a subset of Echo360 content embedded within an LMS or VLE was not accessible for playback from within the LMS/VLE.

Timeline:
2021-03-09 04:04 AM AEDT: Deployment was completed in our Asia Pacific region and the playback of some content embedded in the LMS/VLE was impacted.
2021-03-08 10:05 PM GMT: Deployment was completed in our Europe & Middle East region and the playback of some content embedded in the LMS/VLE was impacted.
2021-03-09 09:50 AM AEDT: Echo360 support received reports of some content not playing back and began investigating the issue with Echo360 engineering.
2021-03-09 12:30 PM AEDT: Echo360 engineering and support were able to reproduce the issue and identify a root cause.
2021-03-09 01:00 PM AEDT: Echo360 engineering determined the most expedient path to resolution was to roll back our recent deployment in the Asia Pacific and Europe & Middle East regions.
2021-03-09 01:15 PM AEDT: Echo360 engineering completed rolling back systems in our QA environment and began testing the solution.
2021-03-09 02:23 PM AEDT: Testing and validation of the solution was completed in the Echo360 QA environment.
2021-03-09 02:23 PM AEDT: The deployment in the Echo360 Asia Pacific region was confirmed fully rolled back and functionality was restored.
2021-03-09 03:23 AM GMT: The deployment in the Echo360 Europe & Middle East region is confirmed fully rolled band functionality is restored.

Summary, Root Cause, and Resolution:
Following the standard deployment of the Echo360 platform in our Asia Pacific and Europe & Middle east regions, some content embedded with an LMS/VLE became unplayable. Specifically, if Echo360 LTI secure links had been embedded into multiple LMS/VLE courses or if courses those links had been embedded into were then cloned multiple times, then it was possible that requests to view embedded Echo360 content could return a 403 Unauthorized error for users after this deployment, preventing those users from viewing the content. This issue was caused by a change to how the Echo360 embedded video player interacted with secure links within the LMS/VLE and did not impact any other types of content access links. After investigating the issue and determining the root cause, our engineering team determined the most expedient resolution was to roll back the recent deployment which contained these changes and cancel this deployment in our other regions. After testing the process in our QA environment and then completing rolling back the deployment in our Asia Pacific and Europe & Middle East regions, we confirmed the issue was fully resolved. We then immediately developed and fully tested a solution to this playback issue and were able to deploy this release in all four regions on March 9 and 10.

Corrective Action:
Echo360 is adjusting our QA process to ensure we test playback of content embedded in multiple LMS/VLE courses. As noted, we have already addressed this issue in production and deployed this update to all regions.


January 26, 2022 2:22PM UTC
Identified - We have identified an issue with capture processing in our US region causing a delay to capture availability. We are currently working on a solution which will be deployed as soon as possible and will have another update on this status page within the next hour.

January 26, 2022 3:21PM UTC
Identified - We are still working on a solution to the issue causing processing delay in the US region. We will have another update here within an hour.

January 26, 2022 4:19PM UTC
Identified - We are continuing to to work on a solution for our processing delay in the US region and will have another update within the next hour.

January 26, 2022 5:15PM UTC
Monitoring - We have resolved the cause of our processing delay and are actively working on our existing processing jobs. Content is completing processing rapidly and we expect to be fully recovered soon. We'll have another update within the next hour

January 26, 2022 6:14PM UTC
Resolved - This issue is fully resolved an our processing times have returned to normal in our US region.