Microsoft Teams
December 30, 11:02pm
Collaboration & Communication
Microsoft Teams
Dec-30, 11:02pm
Title: Users can't make relayed calls via the Microsoft Teams web client
User Impact: Users are unable to make relayed calls via the Microsoft Teams web client.
Current status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.
User Impact: Users are unable to make relayed calls via the Microsoft Teams web client.
Current status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.
Dec-30, 11:31pm
More info: This issue is specific to calls placed on the Microsoft Teams web client that are relayed or not a direct connection. Calls placed in a network environment that doesn't allow direct connectivity will be affected.
Users can make calls as expected via the Microsoft Teams desktop or mobile client.
Current status: We've determined that a recent service update to authentication components contained a misconfiguration, causing impact to relayed calls made via the Microsoft Teams web client. We've developed and initiated deployment of a fix for this issue.
Scope of impact: This issue could potentially affect any of your users if they are attempting to make relayed calls via the Microsoft Teams web client.
Root cause: A recent service update to authentication components contained a misconfiguration, causing impact to relayed calls made via the Microsoft Teams web client.
Next update by: Friday, December 31, 2021, at 7:00 AM UTC
Users can make calls as expected via the Microsoft Teams desktop or mobile client.
Current status: We've determined that a recent service update to authentication components contained a misconfiguration, causing impact to relayed calls made via the Microsoft Teams web client. We've developed and initiated deployment of a fix for this issue.
Scope of impact: This issue could potentially affect any of your users if they are attempting to make relayed calls via the Microsoft Teams web client.
Root cause: A recent service update to authentication components contained a misconfiguration, causing impact to relayed calls made via the Microsoft Teams web client.
Next update by: Friday, December 31, 2021, at 7:00 AM UTC
Dec-31, 12:52am
More info: This issue is specific to calls placed on the Microsoft Teams web client that are relayed or not a direct connection. Calls placed in a network environment that doesn't allow direct connectivity will be affected.
Users can make calls as expected via the Microsoft Teams desktop or mobile client.
Current status: We initially encountered an issue with the fix deployment, but we've restarted the deployment process and we're monitoring it to ensure it completes as quickly as possible.
Scope of impact: This issue could potentially affect any of your users if they are attempting to make relayed calls via the Microsoft Teams web client.
Start time: Friday, December 31, 2021, at 2:09 AM UTC
Root cause: A recent service update to authentication components contained a misconfiguration, causing impact to relayed calls made via the Microsoft Teams web client.
Next update by: Friday, December 31, 2021, at 9:00 AM UTC
Users can make calls as expected via the Microsoft Teams desktop or mobile client.
Current status: We initially encountered an issue with the fix deployment, but we've restarted the deployment process and we're monitoring it to ensure it completes as quickly as possible.
Scope of impact: This issue could potentially affect any of your users if they are attempting to make relayed calls via the Microsoft Teams web client.
Start time: Friday, December 31, 2021, at 2:09 AM UTC
Root cause: A recent service update to authentication components contained a misconfiguration, causing impact to relayed calls made via the Microsoft Teams web client.
Next update by: Friday, December 31, 2021, at 9:00 AM UTC
Dec-31, 1:10am
Title: Users can't make relayed calls via the Microsoft Teams web client
User Impact: Users were unable to make relayed calls via the Microsoft Teams web client.
More info: This issue was specific to calls placed on the Microsoft Teams web client that were relayed or not a direct connection. Calls placed in a network environment that doesn't allow direct connectivity would have been affected.
Users could make calls as expected via the Microsoft Teams desktop or mobile client.
Final status: Our fix has saturated affected service environments, and we've confirmed via monitoring that this has resolved impact.
Scope of impact: This issue could have potentially affected any of your users if they were attempting to make relayed calls via the Microsoft Teams web client.
Start time: Friday, December 31, 2021, at 2:09 AM UTC
End time: Friday, December 31, 2021, at 7:10 AM UTC
Root cause: A recent service update to authentication components contained a misconfiguration, causing impact to relayed calls made via the Microsoft Teams web client.
Next steps: - We're reviewing our deployment procedures to identify options to prevent any future misconfigurations from affecting the service.
This is the final update for the event.
User Impact: Users were unable to make relayed calls via the Microsoft Teams web client.
More info: This issue was specific to calls placed on the Microsoft Teams web client that were relayed or not a direct connection. Calls placed in a network environment that doesn't allow direct connectivity would have been affected.
Users could make calls as expected via the Microsoft Teams desktop or mobile client.
Final status: Our fix has saturated affected service environments, and we've confirmed via monitoring that this has resolved impact.
Scope of impact: This issue could have potentially affected any of your users if they were attempting to make relayed calls via the Microsoft Teams web client.
Start time: Friday, December 31, 2021, at 2:09 AM UTC
End time: Friday, December 31, 2021, at 7:10 AM UTC
Root cause: A recent service update to authentication components contained a misconfiguration, causing impact to relayed calls made via the Microsoft Teams web client.
Next steps: - We're reviewing our deployment procedures to identify options to prevent any future misconfigurations from affecting the service.
This is the final update for the event.
Dec-31, 1:10am
Resolved
December 07, 1:48pm
Collaboration & Communication
Microsoft Teams
Dec-7, 1:48pm
Title: Users may be intermittently unable to join group calls in Microsoft Teams from multiple devices
User Impact: Users may be intermittently unable to join group calls in Microsoft Teams from multiple devices.
More info: As an example of impact, a user can connect through the desktop client. However, when the user attempts to connect on a separate connection method such as a mobile device, this may intermittently fail. Users can circumvent this issue by connecting through a Public Switched Telephone Network (PSTN) or only leveraging a single device.
Current status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.
Scope of impact: Your organization is affected by this event, and any user attempting to join a group call in Microsoft Teams from multiple devices is affected.
User Impact: Users may be intermittently unable to join group calls in Microsoft Teams from multiple devices.
More info: As an example of impact, a user can connect through the desktop client. However, when the user attempts to connect on a separate connection method such as a mobile device, this may intermittently fail. Users can circumvent this issue by connecting through a Public Switched Telephone Network (PSTN) or only leveraging a single device.
Current status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.
Scope of impact: Your organization is affected by this event, and any user attempting to join a group call in Microsoft Teams from multiple devices is affected.
Dec-7, 2:29pm
More info: Our investigation has determined that a recent update to the group calling API intended to streamline endpoint communication processes caused a compatibility issue, resulting in impact. We've developed and validated a fix and we're aiming to have a deployment timeline by our next update.
Current status: Our investigation has determined that a recent update to the group calling API intended to streamline the endpoint transfer process contained a compatibility error, resulting in impact. We've developed and validated a fix which we believe will resolve this impact, and we're aiming to have a deployment timeline in our next update.
Scope of impact: Your organization is affected by this event, and any user attempting to join a group call in Microsoft Teams from multiple devices is affected.
Root cause: A recent update to the group calling API intended to streamline endpoint communication processes caused a compatibility issue, resulting in intermittent endpoint transfer failures.
Next update by: Tuesday, December 7, 2021, at 10:30 PM UTC
Current status: Our investigation has determined that a recent update to the group calling API intended to streamline the endpoint transfer process contained a compatibility error, resulting in impact. We've developed and validated a fix which we believe will resolve this impact, and we're aiming to have a deployment timeline in our next update.
Scope of impact: Your organization is affected by this event, and any user attempting to join a group call in Microsoft Teams from multiple devices is affected.
Root cause: A recent update to the group calling API intended to streamline endpoint communication processes caused a compatibility issue, resulting in intermittent endpoint transfer failures.
Next update by: Tuesday, December 7, 2021, at 10:30 PM UTC
Dec-7, 3:10pm
More info: As an example of impact, a user can connect through the desktop client. However, when the user attempts to connect on a separate connection method such as a mobile device, this may intermittently fail. Users can circumvent this issue by connecting through a Public Switched Telephone Network (PSTN) or only leveraging a single device.
Current status: We've begun deployment of the fix to the affected environment. We anticipate the deployment will complete on Friday, November 10, 2021, after which we'll monitor to ensure impact is remediated. Some users may begin to experience relief as the fix progresses.
Scope of impact: Your organization is affected by this event, and any user attempting to join a group call in Microsoft Teams from multiple devices is affected.
Estimated time to resolve: We anticipate deployment of the fix to complete on Friday, November 10, 2021.
Root cause: A recent update to the group calling API intended to streamline endpoint communication processes caused a compatibility issue, resulting in intermittent endpoint transfer failures.
Next update by: Friday, December 10, 2021, at 8:00 PM UTC
Current status: We've begun deployment of the fix to the affected environment. We anticipate the deployment will complete on Friday, November 10, 2021, after which we'll monitor to ensure impact is remediated. Some users may begin to experience relief as the fix progresses.
Scope of impact: Your organization is affected by this event, and any user attempting to join a group call in Microsoft Teams from multiple devices is affected.
Estimated time to resolve: We anticipate deployment of the fix to complete on Friday, November 10, 2021.
Root cause: A recent update to the group calling API intended to streamline endpoint communication processes caused a compatibility issue, resulting in intermittent endpoint transfer failures.
Next update by: Friday, December 10, 2021, at 8:00 PM UTC
Dec-9, 6:06pm
Title: Users may be intermittently unable to join group calls in Microsoft Teams from multiple devices
User Impact: Users may have been intermittently unable to join group calls in Microsoft Teams from multiple devices.
More info: As an example of impact, a user could have connected through the desktop client. However, when the user attempted to connect on a separate connection method such as a mobile device, this may have intermittently failed. Users could circumvent this issue by connecting through a Public Switched Telephone Network (PSTN) or only leveraging a single device.
Final status: Our deployment of the fix has completed, and we've confirmed after monitoring telemetry that impact is remediated.
Scope of impact: Your organization was affected by this event, and any user attempting to join a group call in Microsoft Teams from multiple devices was affected.
Start time: Monday, November 15, 2021, at 5:00 PM UTC
End time: Friday, December 10, 2021, at 12:06 AM UTC
Root cause: A recent update to the group calling API intended to streamline endpoint communication processes caused a compatibility issue, resulting in intermittent endpoint transfer failures.
Next steps: - We're reviewing the affecting update and our testing and validation procedures to identify how impact was missed and better prevent similar future occurrences.
This is the final update for the event.
User Impact: Users may have been intermittently unable to join group calls in Microsoft Teams from multiple devices.
More info: As an example of impact, a user could have connected through the desktop client. However, when the user attempted to connect on a separate connection method such as a mobile device, this may have intermittently failed. Users could circumvent this issue by connecting through a Public Switched Telephone Network (PSTN) or only leveraging a single device.
Final status: Our deployment of the fix has completed, and we've confirmed after monitoring telemetry that impact is remediated.
Scope of impact: Your organization was affected by this event, and any user attempting to join a group call in Microsoft Teams from multiple devices was affected.
Start time: Monday, November 15, 2021, at 5:00 PM UTC
End time: Friday, December 10, 2021, at 12:06 AM UTC
Root cause: A recent update to the group calling API intended to streamline endpoint communication processes caused a compatibility issue, resulting in intermittent endpoint transfer failures.
Next steps: - We're reviewing the affecting update and our testing and validation procedures to identify how impact was missed and better prevent similar future occurrences.
This is the final update for the event.
Dec-9, 6:06pm
Resolved
December 07, 3:16am
Collaboration & Communication
Microsoft Teams
Dec-7, 3:16am
Title: Some users may be unable to add others to meetings within the Microsoft Teams service
User Impact: Users may be unable to add others to meetings within the Microsoft Teams service.
Current status: We're reviewing service monitoring telemetry to identify what is causing the issue and establish a fix.
Scope of impact: Impact is specific to a portion of users served through the affected infrastructure in the Asia-Pacific region.
Next update by: Tuesday, December 7, 2021, at 11:30 AM UTC
User Impact: Users may be unable to add others to meetings within the Microsoft Teams service.
Current status: We're reviewing service monitoring telemetry to identify what is causing the issue and establish a fix.
Scope of impact: Impact is specific to a portion of users served through the affected infrastructure in the Asia-Pacific region.
Next update by: Tuesday, December 7, 2021, at 11:30 AM UTC
Dec-7, 4:52am
Title: Some users may have been unable to join or add others to meetings within the Microsoft Teams service
User Impact: Users may have been unable to join or add others to meetings within the Microsoft Teams service.
More info: Whilst we have mitigated the immediate impact, some users may still face the issue and meeting organizers can utilize the "End meeting" button, to allow them to create a new meeting.
Final status: We've successfully optimized traffic and performance parameters on the affected infrastructure and confirmed impact was mitigated after a period of monitoring.
Scope of impact: Impact was specific to any user joining a meeting that was created in the Asia-Pacific region.
Start time: Tuesday, December 7, 2021, at 8:13 AM UTC
End time: Tuesday, December 7, 2021, at 10:52 AM UTC
Root cause: A recent change made to our maintenance detection parameters was unexpectedly resulting in traffic being rerouted incorrectly.
Next steps: - We're reviewing our update procedures to better identify similar issues during our development and testing cycles.
This is the final update for the event.
User Impact: Users may have been unable to join or add others to meetings within the Microsoft Teams service.
More info: Whilst we have mitigated the immediate impact, some users may still face the issue and meeting organizers can utilize the "End meeting" button, to allow them to create a new meeting.
Final status: We've successfully optimized traffic and performance parameters on the affected infrastructure and confirmed impact was mitigated after a period of monitoring.
Scope of impact: Impact was specific to any user joining a meeting that was created in the Asia-Pacific region.
Start time: Tuesday, December 7, 2021, at 8:13 AM UTC
End time: Tuesday, December 7, 2021, at 10:52 AM UTC
Root cause: A recent change made to our maintenance detection parameters was unexpectedly resulting in traffic being rerouted incorrectly.
Next steps: - We're reviewing our update procedures to better identify similar issues during our development and testing cycles.
This is the final update for the event.
Dec-7, 4:52am
Resolved
Dec-7, 5:05am
More info: Some users may also experience issues joining meetings.
Current status: We've identified that a portion of infrastructure responsible for detecting service issues is behaving unexpectedly, causing impact. We're proceeding to scale up the affected infrastructures to provide short-term relief while we develop a long-term fix.
Scope of impact: Impact is specific to a portion of users served through the affected infrastructures across the Asia-Pacific region, Europe and North America.
Root cause: A portion of infrastructure responsible for detecting service issues is behaving unexpectedly, causing impact.
Next update by: Tuesday, December 7, 2021, at 5:00 PM UTC
Current status: We've identified that a portion of infrastructure responsible for detecting service issues is behaving unexpectedly, causing impact. We're proceeding to scale up the affected infrastructures to provide short-term relief while we develop a long-term fix.
Scope of impact: Impact is specific to a portion of users served through the affected infrastructures across the Asia-Pacific region, Europe and North America.
Root cause: A portion of infrastructure responsible for detecting service issues is behaving unexpectedly, causing impact.
Next update by: Tuesday, December 7, 2021, at 5:00 PM UTC
December 06, 10:17am
Collaboration & Communication
Microsoft Teams
Dec-6, 10:17am
Title: Some users are seeing incorrect presence status in Microsoft Teams
User Impact: Users are seeing incorrect presence status in Microsoft Teams.
More info: Users may see presence status as 'offline' or 'unknown.'
Current status: We're analyzing service telemetry showing presence availability so we can better understand the origin of impact and determine our next troubleshooting steps.
Scope of impact: Your organization is affected by this event, and a subset of users may experience impact.
Next update by: Monday, December 6, 2021, at 5:30 PM UTC
User Impact: Users are seeing incorrect presence status in Microsoft Teams.
More info: Users may see presence status as 'offline' or 'unknown.'
Current status: We're analyzing service telemetry showing presence availability so we can better understand the origin of impact and determine our next troubleshooting steps.
Scope of impact: Your organization is affected by this event, and a subset of users may experience impact.
Next update by: Monday, December 6, 2021, at 5:30 PM UTC
Dec-6, 11:34am
More info: Users may see presence status as 'offline' or 'unknown.'
Current status: Our analysis of service telemetry in parallel with assessment of newly implemented changes made to the service suggests that a recent update intended to optimize presence functionality may inadvertently be contributing to impact. We're continuing to verify this theory and are simultaneously determining our next steps to provide alleviation.
Scope of impact: A subset of users hosted out of the affected infrastructure may experience impact.
Next update by: Monday, December 6, 2021, at 8:00 PM UTC
Current status: Our analysis of service telemetry in parallel with assessment of newly implemented changes made to the service suggests that a recent update intended to optimize presence functionality may inadvertently be contributing to impact. We're continuing to verify this theory and are simultaneously determining our next steps to provide alleviation.
Scope of impact: A subset of users hosted out of the affected infrastructure may experience impact.
Next update by: Monday, December 6, 2021, at 8:00 PM UTC
Dec-6, 12:56pm
More info: Users may see presence status as 'offline' or 'unknown.'
Current status: We've identified through further investigation that the aforementioned presence update caused a discrepancy between the information needed to propagate users' presence and the cache information currently available, consequently inhibiting users from seeing correct presence across all Microsoft Teams connection methods. To alleviate impact, we've developed, tested, and initiated the deployment of an update to resolve the cache issue by eliminating the impacting discrepancy. We'll provide additional information regarding the fix's deployment status and estimated time to resolution via our next scheduled update.
Scope of impact: A subset of users hosted out of the affected infrastructure may experience impact.
Start time: Saturday, December 4, 2021, at 12:00 PM UTC
Root cause: A recent update intended to enhance presence functionality caused a discrepancy between the information needed to propagate users' presence and the information currently available in the cache, consequently inhibiting users from seeing correct presence across all Teams connection methods.
Next update by: Monday, December 6, 2021, at 10:00 PM UTC
Current status: We've identified through further investigation that the aforementioned presence update caused a discrepancy between the information needed to propagate users' presence and the cache information currently available, consequently inhibiting users from seeing correct presence across all Microsoft Teams connection methods. To alleviate impact, we've developed, tested, and initiated the deployment of an update to resolve the cache issue by eliminating the impacting discrepancy. We'll provide additional information regarding the fix's deployment status and estimated time to resolution via our next scheduled update.
Scope of impact: A subset of users hosted out of the affected infrastructure may experience impact.
Start time: Saturday, December 4, 2021, at 12:00 PM UTC
Root cause: A recent update intended to enhance presence functionality caused a discrepancy between the information needed to propagate users' presence and the information currently available in the cache, consequently inhibiting users from seeing correct presence across all Teams connection methods.
Next update by: Monday, December 6, 2021, at 10:00 PM UTC
Dec-6, 2:27pm
More info: Users may see presence status as 'offline' or 'unknown.'
Current status: We've confirmed via service telemetry that our change addressing the cache issue has already brought alleviation to a large portion of formerly affected users. We're continuing to monitor its deployment to confirm resolution of persisting impact, and we anticipate full remediation for all users by our next scheduled update.
Scope of impact: A subset of users hosted out of the affected infrastructure may experience impact.
Start time: Saturday, December 4, 2021, at 12:00 PM UTC
Root cause: A recent update intended to enhance presence functionality caused a discrepancy between the information needed to propagate users' presence and the information currently available in the cache, consequently inhibiting users from seeing correct presence across all Teams connection methods.
Next update by: Monday, December 6, 2021, at 10:00 PM UTC
Current status: We've confirmed via service telemetry that our change addressing the cache issue has already brought alleviation to a large portion of formerly affected users. We're continuing to monitor its deployment to confirm resolution of persisting impact, and we anticipate full remediation for all users by our next scheduled update.
Scope of impact: A subset of users hosted out of the affected infrastructure may experience impact.
Start time: Saturday, December 4, 2021, at 12:00 PM UTC
Root cause: A recent update intended to enhance presence functionality caused a discrepancy between the information needed to propagate users' presence and the information currently available in the cache, consequently inhibiting users from seeing correct presence across all Teams connection methods.
Next update by: Monday, December 6, 2021, at 10:00 PM UTC
Dec-6, 2:57pm
Title: Some users are seeing incorrect presence status in all Microsoft Teams connection methods
User Impact: Users were seeing incorrect presence status in all Microsoft Teams connection methods.
More info: Users may have seen presence status as 'offline' or 'unknown.'
Final status: Our investigation is complete, and we've verified through continued monitoring of service telemetry that our update to address the cache issue has resolved the problem for all previously impacted users.
Scope of impact: A subset of users hosted out of the affected infrastructure may have experienced impact.
Start time: Saturday, December 4, 2021, at 12:00 PM UTC
End time: Monday, December 6, 2021, at 8:57 PM UTC
Root cause: A recent update intended to enhance presence functionality caused a discrepancy between the information needed to propagate users' presence and the information available in the cache, consequently inhibiting users from seeing correct presence across all Teams connection methods.
Next steps: - We're assessing potential optimizations within our testing methodology to isolate and resolved future impact scenarios prior to deployment.
This is the final update for the event.
User Impact: Users were seeing incorrect presence status in all Microsoft Teams connection methods.
More info: Users may have seen presence status as 'offline' or 'unknown.'
Final status: Our investigation is complete, and we've verified through continued monitoring of service telemetry that our update to address the cache issue has resolved the problem for all previously impacted users.
Scope of impact: A subset of users hosted out of the affected infrastructure may have experienced impact.
Start time: Saturday, December 4, 2021, at 12:00 PM UTC
End time: Monday, December 6, 2021, at 8:57 PM UTC
Root cause: A recent update intended to enhance presence functionality caused a discrepancy between the information needed to propagate users' presence and the information available in the cache, consequently inhibiting users from seeing correct presence across all Teams connection methods.
Next steps: - We're assessing potential optimizations within our testing methodology to isolate and resolved future impact scenarios prior to deployment.
This is the final update for the event.
Dec-6, 2:57pm
Resolved
December 04, 4:17am
Collaboration & Communication
Microsoft Teams
Dec-4, 4:17am
Title: Some users may not see up to date presence information in Microsoft Teams
User Impact: Users may not see up to date presence information in Microsoft Teams.
Current status: We're reviewing service monitoring telemetry to isolate the source of the issue and establish a fix. Scope of impact: Impact is specific to some users who are served through the affected infrastructure. Next update by: Saturday, December 4, 2021, at 12:30 PM UTC
User Impact: Users may not see up to date presence information in Microsoft Teams.
Current status: We're reviewing service monitoring telemetry to isolate the source of the issue and establish a fix. Scope of impact: Impact is specific to some users who are served through the affected infrastructure. Next update by: Saturday, December 4, 2021, at 12:30 PM UTC
Dec-4, 5:05am
Title: Some users may not have seen up to date presence information in Microsoft Teams
User Impact: Users may not have seen up to date presence information in Microsoft Teams.
Final status: We've determined that a scheduled maintenance to the infrastructure, responsible for processing user presence results, unexpectedly caused impact. We've halted the maintenance and confirmed after monitoring the affected environment that impact was remediated.
Scope of impact: Impact was specific to some users who were served through the affected infrastructure.
Start time: Saturday, December 4, 2021, at 7:30 AM UTC
End time: Saturday, December 4, 2021, at 11:05 AM UTC
Root cause: A scheduled maintenance to the infrastructure, responsible for processing user presence results, unexpectedly caused impact.
Next steps: - We're reviewing our maintenance procedures to better identify similar issues during our development and testing cycles.
This is the final update for the event.
User Impact: Users may not have seen up to date presence information in Microsoft Teams.
Final status: We've determined that a scheduled maintenance to the infrastructure, responsible for processing user presence results, unexpectedly caused impact. We've halted the maintenance and confirmed after monitoring the affected environment that impact was remediated.
Scope of impact: Impact was specific to some users who were served through the affected infrastructure.
Start time: Saturday, December 4, 2021, at 7:30 AM UTC
End time: Saturday, December 4, 2021, at 11:05 AM UTC
Root cause: A scheduled maintenance to the infrastructure, responsible for processing user presence results, unexpectedly caused impact.
Next steps: - We're reviewing our maintenance procedures to better identify similar issues during our development and testing cycles.
This is the final update for the event.
Dec-4, 5:05am
Resolved
December 02, 4:22am
Collaboration & Communication
Microsoft Teams
Dec-2, 4:22am
Title: Users may have experienced issues accessing live events within Microsoft Teams
User Impact: Users may have experienced issues accessing live events within Microsoft Teams.
Current status: We've investigated a problem with the Microsoft Teams service in which users may have experienced issues accessing live events, and we’ve confirmed that service is now restored. We'll provide additional information in a closure summary within the “History” section of the “Service health” dashboard shortly.
Start time: Thursday, December 2, 2021, at 9:42 AM UTC
User Impact: Users may have experienced issues accessing live events within Microsoft Teams.
Current status: We've investigated a problem with the Microsoft Teams service in which users may have experienced issues accessing live events, and we’ve confirmed that service is now restored. We'll provide additional information in a closure summary within the “History” section of the “Service health” dashboard shortly.
Start time: Thursday, December 2, 2021, at 9:42 AM UTC
Dec-2, 4:22am
Final status: While we were in the process of analyzing system logs to isolate the source of the problem, system monitoring indicated that the service had returned to normal health. We've confirmed that the problem is no longer occurring, and we'll continue to monitor the service to ensure that the problem does not happen again.
Scope of impact: Impact was specific to some users who were served through the affected infrastructure.
Start time: Thursday, December 2, 2021, at 9:42 AM UTC
End time: Thursday, December 2, 2021, at 10:22 AM UTC
This is the final update for the event.
Scope of impact: Impact was specific to some users who were served through the affected infrastructure.
Start time: Thursday, December 2, 2021, at 9:42 AM UTC
End time: Thursday, December 2, 2021, at 10:22 AM UTC
This is the final update for the event.
December 01, 7:06pm
Collaboration & Communication
Microsoft Teams
Dec-1, 7:06pm
Title: Admins may be unable to submit port and transfer requests for Public Switched Telephone Network (PSTN) numbers
User Impact: Admins may be unable to submit port and transfer requests for PSTN phone numbers in the Microsoft Teams admin center.
Current status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.
Scope of impact: This issue affects any admins attempting to port and transfer PSTN numbers in the Microsoft Teams admin center.
User Impact: Admins may be unable to submit port and transfer requests for PSTN phone numbers in the Microsoft Teams admin center.
Current status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.
Scope of impact: This issue affects any admins attempting to port and transfer PSTN numbers in the Microsoft Teams admin center.
Dec-1, 7:39pm
Title: Admins may be unable to submit manual requests to the Telephone Number Services (TNS) Service Desk
User Impact: Admins may have been unable to submit manual requests to the TNS Service Desk in the Microsoft Teams admin center.
More info: - While we were focused on remediation, admins could go to the "Phone Numbers" page in the Microsoft Teams admin center and then select "Get phone number support".
- Manual actions outside of the United States and Canada included submitting port and transfer requests for Public Switched Telephone Network (PSTN) phone numbers.
- Manual actions within the United States and Canada included contacting the TNS Service Desk when having trouble purchasing net new phone numbers.
Final status: We've determined that an upstream telemetry update inadvertently resulted in impact to Microsoft Teams due to an unforeseen incompatibility. We've reverted the update and monitored the environment to confirm service is restored.
Scope of impact: This issue affected any admins attempting to submit manual requests to the TNS Service Desk in the Microsoft Teams admin center.
Start time: Wednesday, December 1, 2021, at 7:05 PM UTC
End time: Thursday, December 2, 2021, at 1:39 AM UTC
Root cause: An upstream telemetry update inadvertently resulted in the impact to Microsoft Teams due to an unforeseen incompatibility.
Next steps: - To help prevent similar impact in the future, we're further reviewing how we can improve our validation procedures to catch such compatibility issues before deployment.
This is the final update for the event.
User Impact: Admins may have been unable to submit manual requests to the TNS Service Desk in the Microsoft Teams admin center.
More info: - While we were focused on remediation, admins could go to the "Phone Numbers" page in the Microsoft Teams admin center and then select "Get phone number support".
- Manual actions outside of the United States and Canada included submitting port and transfer requests for Public Switched Telephone Network (PSTN) phone numbers.
- Manual actions within the United States and Canada included contacting the TNS Service Desk when having trouble purchasing net new phone numbers.
Final status: We've determined that an upstream telemetry update inadvertently resulted in impact to Microsoft Teams due to an unforeseen incompatibility. We've reverted the update and monitored the environment to confirm service is restored.
Scope of impact: This issue affected any admins attempting to submit manual requests to the TNS Service Desk in the Microsoft Teams admin center.
Start time: Wednesday, December 1, 2021, at 7:05 PM UTC
End time: Thursday, December 2, 2021, at 1:39 AM UTC
Root cause: An upstream telemetry update inadvertently resulted in the impact to Microsoft Teams due to an unforeseen incompatibility.
Next steps: - To help prevent similar impact in the future, we're further reviewing how we can improve our validation procedures to catch such compatibility issues before deployment.
This is the final update for the event.
Dec-1, 7:39pm
Resolved
Dec-1, 7:39pm
More info: - Manual actions outside of the United States and Canada include submitting port and transfer requests for PSTN phone numbers.
- Manual actions within the United States and Canada include contacting the TNS Service Desk when having trouble purchasing net new phone numbers.
Current status: We're analyzing service logs to determine the source of the issue.
Scope of impact: This issue affects any admins attempting to submit manual requests to the TNS Service Desk in the Microsoft Teams admin center.
Next update by: Thursday, December 2, 2021, at 2:30 AM UTC
- Manual actions within the United States and Canada include contacting the TNS Service Desk when having trouble purchasing net new phone numbers.
Current status: We're analyzing service logs to determine the source of the issue.
Scope of impact: This issue affects any admins attempting to submit manual requests to the TNS Service Desk in the Microsoft Teams admin center.
Next update by: Thursday, December 2, 2021, at 2:30 AM UTC
December 01, 7:51am
Collaboration & Communication
Microsoft Teams
Dec-1, 7:51am
Title: Some users may be unable to get location information for incoming emergency calls within Microsoft Teams
User Impact: Users may be unable to get location information for incoming emergency calls within Microsoft Teams.
Current status: We're analyzing service monitoring telemetry to identify what is causing the issue and establish a fix.
Scope of impact: Impact is specific to a portion of users located across Europe.
Next update by: Wednesday, December 1, 2021, at 4:00 PM UTC
User Impact: Users may be unable to get location information for incoming emergency calls within Microsoft Teams.
Current status: We're analyzing service monitoring telemetry to identify what is causing the issue and establish a fix.
Scope of impact: Impact is specific to a portion of users located across Europe.
Next update by: Wednesday, December 1, 2021, at 4:00 PM UTC
Dec-1, 9:25am
Current status: We've identified that an unexpectedly high volume of user traffic is resulting in resource contention, causing requests to timeout. We've redirected user traffic to alternate infrastructure and confirmed, after a period of monitoring, that the service has restored for impacted users in Europe. We're analyzing service logs to determine why impact has not been alleviated across all regions, specifically for users based in North America.
Scope of impact: Impact is specific to a portion of users located across Europe and North America.
Start time: Thursday, December 2, 2021, at 1:36 PM UTC
Root cause: A unexpected surge in user activity caused a CPU spike, causing impact.
Next update by: Wednesday, December 1, 2021, at 5:30 PM UTC
Scope of impact: Impact is specific to a portion of users located across Europe and North America.
Start time: Thursday, December 2, 2021, at 1:36 PM UTC
Root cause: A unexpected surge in user activity caused a CPU spike, causing impact.
Next update by: Wednesday, December 1, 2021, at 5:30 PM UTC
Dec-1, 11:29am
Current status: We're continuing to redirect user traffic to an alternate subsection of service infrastructure which we're expecting will remediate impact.
Scope of impact: Impact is specific to a subsection of users located across North America, and previously impacted users located in Europe.
Start time: Wednesday, December 1, 2021, at 1:36 PM UTC
Next update by: Wednesday, December 1, 2021, at 7:30 PM UTC
Scope of impact: Impact is specific to a subsection of users located across North America, and previously impacted users located in Europe.
Start time: Wednesday, December 1, 2021, at 1:36 PM UTC
Next update by: Wednesday, December 1, 2021, at 7:30 PM UTC
Dec-1, 11:45am
Title: Some users may be unable to get location information for emergency calls within Microsoft Teams
User Impact: Users may have been unable to get location information for emergency calls within Microsoft Teams.
Final status: We've confirmed that a subsection of service infrastructure that facilitates location information for emergency calls with Microsoft Teams had fallen below our manageable service performance thresholds, resulting in impact. We've rerouted traffic to an alternate subsection of service infrastructure to remediate impact.
Scope of impact: Impact was specific to a subsection of users located across North America and Europe.
Start time: Wednesday, December 1, 2021, at 1:07 PM UTC
End time: Wednesday, December 1, 2021, at 5:45 PM UTC
Root cause: A subsection of service infrastructure that facilitates location information for emergency calls with Microsoft Teams dropped below our manageable service performance thresholds, resulting in impact.
Next steps: - We're analyzing performance data and trends on the affected systems to determine why levels dropped below our manageable thresholds, and to help prevent this problem from happening again.
This is the final update for the event.
User Impact: Users may have been unable to get location information for emergency calls within Microsoft Teams.
Final status: We've confirmed that a subsection of service infrastructure that facilitates location information for emergency calls with Microsoft Teams had fallen below our manageable service performance thresholds, resulting in impact. We've rerouted traffic to an alternate subsection of service infrastructure to remediate impact.
Scope of impact: Impact was specific to a subsection of users located across North America and Europe.
Start time: Wednesday, December 1, 2021, at 1:07 PM UTC
End time: Wednesday, December 1, 2021, at 5:45 PM UTC
Root cause: A subsection of service infrastructure that facilitates location information for emergency calls with Microsoft Teams dropped below our manageable service performance thresholds, resulting in impact.
Next steps: - We're analyzing performance data and trends on the affected systems to determine why levels dropped below our manageable thresholds, and to help prevent this problem from happening again.
This is the final update for the event.
Dec-1, 11:45am
Resolved
November 25, 3:12pm
Collaboration & Communication
Microsoft Teams
Nov-25, 3:12pm
Title: Producers in Microsoft Teams live events are unable to preview a presenter’s screen
User Impact: Users who are producing Microsoft Teams live events are unable to preview a presenter’s screen.
Current status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.
Scope of impact: All users producing Microsoft Teams live events are impacted.
User Impact: Users who are producing Microsoft Teams live events are unable to preview a presenter’s screen.
Current status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.
Scope of impact: All users producing Microsoft Teams live events are impacted.
Nov-25, 3:31pm
Current status: We're reviewing similar instances of impact in parallel with support case information, including screenshots of impact and Microsoft Teams logs, to identify potential sources of impact.
Scope of impact: All users producing Microsoft Teams live events are impacted.
Next update by: Thursday, November 25, 2021, at 11:00 PM UTC
Scope of impact: All users producing Microsoft Teams live events are impacted.
Next update by: Thursday, November 25, 2021, at 11:00 PM UTC
Nov-25, 4:41pm
More info: Producers are able to see a presenter’s content or video in the source tray; however, once they are clicked to queue up the presenter, the preview in the source tray goes black.
The preview will remain if the presenters are not clicked on.
Impact is specific to the source tray. As a workaround, producers experiencing impact can click on the blank thumbnail in the source tray to receive a detailed, larger thumbnail in the top half of the live event screen to preview presenter content and video.
Current status: We've confirmed that a recent update introduced a code regression which impacts source tray rendering during live events. We're preparing a fix which will undergo testing and validation as a potential remediating action to resolve the issue.
Scope of impact: All users producing Microsoft Teams live events are impacted.
Start time: Wednesday, November 17, 2021, at 8:00 PM UTC
Root cause: A recent update introduced a code regression which impacts source tray rendering during live events.
Next update by: Tuesday, November 30, 2021, at 12:30 AM UTC
The preview will remain if the presenters are not clicked on.
Impact is specific to the source tray. As a workaround, producers experiencing impact can click on the blank thumbnail in the source tray to receive a detailed, larger thumbnail in the top half of the live event screen to preview presenter content and video.
Current status: We've confirmed that a recent update introduced a code regression which impacts source tray rendering during live events. We're preparing a fix which will undergo testing and validation as a potential remediating action to resolve the issue.
Scope of impact: All users producing Microsoft Teams live events are impacted.
Start time: Wednesday, November 17, 2021, at 8:00 PM UTC
Root cause: A recent update introduced a code regression which impacts source tray rendering during live events.
Next update by: Tuesday, November 30, 2021, at 12:30 AM UTC
Nov-29, 4:38pm
More info: Producers are able to see a presenter’s content or video in the source tray; however, once they are clicked to queue up the presenter, the preview in the source tray goes black.
The preview will remain if the presenters are not clicked on.
Impact is specific to the source tray. As a workaround, producers experiencing impact can click on the blank thumbnail in the source tray to receive a detailed, larger thumbnail in the top half of the live event screen to preview presenter content and video.
Current status: We've completed our development and validation, and have confirmed that the fix will be a viable solution for resolving impact. We've begun deployment of the fix which will progressively resolve impact for affected users as it reaches their environment. We're expect the fix deployment to complete and for impact to be resolved by Thursday, December 2, 2021, at approximately 10:00 PM UTC.
Scope of impact: All users producing Microsoft Teams live events are impacted.
Start time: Wednesday, November 17, 2021, at 8:00 PM UTC
Estimated time to resolve: Based on current deployment progress, we expect this problem to be resolved within the next three days.
Root cause: A recent update introduced a code regression which impacts source tray rendering during live events.
Next update by: Friday, December 3, 2021, at 12:00 AM UTC
The preview will remain if the presenters are not clicked on.
Impact is specific to the source tray. As a workaround, producers experiencing impact can click on the blank thumbnail in the source tray to receive a detailed, larger thumbnail in the top half of the live event screen to preview presenter content and video.
Current status: We've completed our development and validation, and have confirmed that the fix will be a viable solution for resolving impact. We've begun deployment of the fix which will progressively resolve impact for affected users as it reaches their environment. We're expect the fix deployment to complete and for impact to be resolved by Thursday, December 2, 2021, at approximately 10:00 PM UTC.
Scope of impact: All users producing Microsoft Teams live events are impacted.
Start time: Wednesday, November 17, 2021, at 8:00 PM UTC
Estimated time to resolve: Based on current deployment progress, we expect this problem to be resolved within the next three days.
Root cause: A recent update introduced a code regression which impacts source tray rendering during live events.
Next update by: Friday, December 3, 2021, at 12:00 AM UTC
Dec-2, 12:44pm
Title: Producers in Microsoft Teams live events are unable to preview a presenter’s content in the source tray
User Impact: Users who were producing Microsoft Teams live events were unable to preview a presenter’s content using the source tray.
More info: Producers were able to see a presenter’s content or video in the source tray; however, once they were clicked to queue up the presenter, the preview in the source tray went black.
The preview would remain if the presenters were not clicked on.
Impact was specific to the source tray. As a workaround, producers experiencing impact could click on the blank thumbnail in the source tray to receive a detailed, larger thumbnail in the top half of the live event screen to preview presenter content and video.
Final status: We've completed deployment of the fix to the affected environment and confirmed with users that impact is remediated.
Scope of impact: All users producing Microsoft Teams live events were impacted.
Start time: Wednesday, November 17, 2021, at 8:00 PM UTC
End time: Thursday, December 2, 2021, at 6:44 PM UTC
Root cause: A recent update introduced a code regression which impacted source tray rendering during live events.
Next steps: - We're reviewing our update procedures to better identify similar live event issues during our development and testing cycles.
This is the final update for the event.
User Impact: Users who were producing Microsoft Teams live events were unable to preview a presenter’s content using the source tray.
More info: Producers were able to see a presenter’s content or video in the source tray; however, once they were clicked to queue up the presenter, the preview in the source tray went black.
The preview would remain if the presenters were not clicked on.
Impact was specific to the source tray. As a workaround, producers experiencing impact could click on the blank thumbnail in the source tray to receive a detailed, larger thumbnail in the top half of the live event screen to preview presenter content and video.
Final status: We've completed deployment of the fix to the affected environment and confirmed with users that impact is remediated.
Scope of impact: All users producing Microsoft Teams live events were impacted.
Start time: Wednesday, November 17, 2021, at 8:00 PM UTC
End time: Thursday, December 2, 2021, at 6:44 PM UTC
Root cause: A recent update introduced a code regression which impacted source tray rendering during live events.
Next steps: - We're reviewing our update procedures to better identify similar live event issues during our development and testing cycles.
This is the final update for the event.
Dec-2, 12:44pm
Resolved
November 22, 6:12am
Collaboration & Communication
Microsoft Teams
Nov-22, 6:12am
Title: Some users can't see images in chats
User Impact: Users may be unable to view images in within Microsoft Teams' chats.
Current status: We’re looking into your reported issue and checking for impact to your organization. We'll provide an update within one hour.
User Impact: Users may be unable to view images in within Microsoft Teams' chats.
Current status: We’re looking into your reported issue and checking for impact to your organization. We'll provide an update within one hour.
Nov-22, 6:29am
Current status: We've determined that an authentication configuration issue is resulting in impact. We're deploying an fix to correct the configuration, which we anticipate will resolve the issue.
Scope of impact: Impact is specific to users who are served through the affected infrastructure.
Next update by: Monday, November 22, 2021, at 2:30 PM UTC
Scope of impact: Impact is specific to users who are served through the affected infrastructure.
Next update by: Monday, November 22, 2021, at 2:30 PM UTC
Nov-22, 7:32am
More info: As a potential workaround, users are able to upload the image as a file and other users could download the image to view.
We've confirmed the storage of the images is unaffected, and once the issue is resolved the previously posted images will render as expected.
Current status: We've determined that an internal service to service authentication configuration, used in the rendering of Teams chat and channel images, is not performing as expected. We're continuing to deploy the fix and estimate that it should complete within the next hour.
Scope of impact: Any user could be affected by this issue.
Start time: Monday, November 22, 2021, at 11:30 AM UTC
Root cause: An internal service to service authentication configuration, used in the rendering of Teams chat and channel images, is not performing as expected.
Next update by: Monday, November 22, 2021, at 3:00 PM UTC
We've confirmed the storage of the images is unaffected, and once the issue is resolved the previously posted images will render as expected.
Current status: We've determined that an internal service to service authentication configuration, used in the rendering of Teams chat and channel images, is not performing as expected. We're continuing to deploy the fix and estimate that it should complete within the next hour.
Scope of impact: Any user could be affected by this issue.
Start time: Monday, November 22, 2021, at 11:30 AM UTC
Root cause: An internal service to service authentication configuration, used in the rendering of Teams chat and channel images, is not performing as expected.
Next update by: Monday, November 22, 2021, at 3:00 PM UTC
Nov-22, 8:25am
Title: Users can't view images in chats and channels
User Impact: Users may have been unable to view images within Microsoft Teams' chats and channels.
More info: Additionally, some other scenarios were impacted by this issue: - Users may have experienced delays with calls or meeting recordings being available within SharePoint Online and OneDrive for Business. Users may have received the error 'The recording could not be uploaded and will expire.' - Verbiage that was within the same message as an impacted image would not have returned within Microsoft Teams search.
As a potential workaround, users were able to upload the image as a file and other users could download the image to view.
We've confirmed that the storage of the images was unaffected, and once the issue was resolved, the previously posted images rendered as expected.
Final status: We've deployed the fix and confirmed that the issue is resolved after a period of monitoring. Users may need to switch between chats or scroll up and down within a chat to trigger the re-rendering of the images.
Scope of impact: Any user could have been affected by this issue.
Start time: Monday, November 22, 2021, at 11:30 AM UTC
End time: Monday, November 22, 2021, at 2:25 PM UTC
Preliminary root cause: An internal service to service authentication configuration, used in the rendering of Teams chat and channel images, was not performing as expected.
Next steps: - We're investigating why the authentication configuration started to experience impact and stop the issue from reoccurring. - We're analyzing performance data and trends on the affected systems to help prevent this problem from happening again.
We'll publish a post-incident report within five business days.
User Impact: Users may have been unable to view images within Microsoft Teams' chats and channels.
More info: Additionally, some other scenarios were impacted by this issue: - Users may have experienced delays with calls or meeting recordings being available within SharePoint Online and OneDrive for Business. Users may have received the error 'The recording could not be uploaded and will expire.' - Verbiage that was within the same message as an impacted image would not have returned within Microsoft Teams search.
As a potential workaround, users were able to upload the image as a file and other users could download the image to view.
We've confirmed that the storage of the images was unaffected, and once the issue was resolved, the previously posted images rendered as expected.
Final status: We've deployed the fix and confirmed that the issue is resolved after a period of monitoring. Users may need to switch between chats or scroll up and down within a chat to trigger the re-rendering of the images.
Scope of impact: Any user could have been affected by this issue.
Start time: Monday, November 22, 2021, at 11:30 AM UTC
End time: Monday, November 22, 2021, at 2:25 PM UTC
Preliminary root cause: An internal service to service authentication configuration, used in the rendering of Teams chat and channel images, was not performing as expected.
Next steps: - We're investigating why the authentication configuration started to experience impact and stop the issue from reoccurring. - We're analyzing performance data and trends on the affected systems to help prevent this problem from happening again.
We'll publish a post-incident report within five business days.
Nov-22, 8:25am
Resolved
Nov-22, 8:27am
We've deployed the fix throughout the affected environments and telemetry is showing signs of recovery.
This quick update is designed to give the latest information on this issue.
This quick update is designed to give the latest information on this issue.