Microsoft has identified the root causes of the global outage that paralysed Outlook, Exchange and a number of other key office applications last week. The company published an initial report on Monday citing three factors that led to the 24-hour outage of its services: a bug dating back several months, faulty internal links and server capacity.
Each of these factors could likely have been resolved individually, but taken together they led to a major outage that was difficult to mitigate, Microsoft executives said. The outage began on Thursday when US users reported difficulty accessing their Outlook accounts. As it continued the following day, millions of people around the world reported being unable to access email or other Office 365 services at all. Prisons could not release inmates, hospitals reverted to paper documentation, schools lost contact with parents, and police departments were unable to receive criminal reports. Microsoft stated that the problem was resolved and services restored on Saturday, issuing an apology. The company did not attribute the outage to a security incident.
In a post on X on Monday, Charles Lamanna, executive vice president for Copilot, agents and platform at Microsoft, said that a latent bug or defect from July caused Microsoft's servers to become "overloaded and unresponsive". Lamanna wrote that the company usually has two available network gateways, or entry and exit points that connect separate systems and enable data transfer between them. However, one of them was not working for Microsoft Teams, which overloaded the primary gateway. Subsequently, more than 14 per cent of the company's server capacity was also non-functional due to an unrelated issue from the previous day.
"Taken together, the increased load and reduced server capacity led to the network gateway encountering the latent bug from July", Lamanna wrote. "This caused requests to accumulate until the servers eventually stopped responding. This happened during peak traffic hours in the United States." The company also had difficulty mitigating the error because it had been created months earlier, Lamanna said, making it not easy to remove by simply reverting a recent change.
"Recovery took longer than usual because the servers, as they returned to service, were immediately flooded with waiting traffic and became unresponsive again", he wrote. "Our usual automated recovery processes were not designed to restore service from such a widespread unresponsive state. In addition, because the main code defect was a latent bug several months old, our usual rollback, testing and features could not quickly undo the change." The company plans to publish a final report containing more information, he said.
Analysts say Microsoft's global outage highlights the need for companies to reassess how much they rely on a single operating system. "This is a prime example of the importance of not just cyber, but business resilience", said Matt Hull, vice president for cyber intelligence and response at NCC Group. "Reports that hospitals, schools and other key services were affected show the wide impact a system outage can have, regardless of the cause."













