A few days ago, a historic global outage affected Facebook, Instagram, Messenger and other Meta services, rendering them completely inaccessible. The company has finally explained the origin of the problem.
Big heat on social networks last Tuesday, March 5! Around 4 p.m., it became impossible to access Meta’s social networks and instant messaging! Many users of Facebook, Instagram, Messenger, WhatsApp and Threads deprived of access to their favorite communication tools immediately rushed to the Internet in search of information, reporting connection problems on sites like DownDetector, which lists outages at major operators and major online services, but also on X – after all, it’s the social network for complaining. Even on the How It Works forum, posts from anxious Internet users were arriving from all sides! Everything finally returned to normal after 5 p.m. But then, what happened?
Global outage at Meta: panic on board!
Tens of millions of users around the world have been logged out of their Facebook and Messenger accounts, unable to log back in. Worse still: a message indicated that “the password you entered is incorrect”. And it was impossible to reset it, because the code for double authentication did not arrive correctly, and the platform did not recognize the new identifiers either. On the Instagram side, users faced a message “An error has occurred”. However, the problem only affected people logged in to their account and those operating offline had no problem. For their part, Threads and WhatsApp seemed to be less affected, even though several reports had been made.
The situation finally began to return to normal from 5 p.m., Meta having visibly succeeded in repairing the outage, as the company had indicated on its service status page, obviously apologizing. And, as we noted, access to the various services was well restored in the early evening, at 7 p.m. In the end, the outage lasted two and a half hours, making it the most significant outage for the group in four years.
Global outage at Meta: a technical error with serious consequences
Don’t panic though! Contrary to what many believed, this was not an account hacking problem, but rather an outage that affected users around the world. OnAndy Stone, communications manager at Meta, confirmed the news: “We are aware that our users are having difficulty accessing our services. We are currently working on the matter”he explained.
Mark Zuckerberg’s company gave more details about what happened in a blog post. Apparently, this was an issue with the handling of an error by an automated system for checking configuration values within the group’s infrastructure. To put it simply, this system is responsible for identifying and correcting invalid configuration values in the cache and replacing them with updated values from permanent memory. However, a modification that was made to a persistent configuration value was incorrectly interpreted as invalid. As a result, each customer attempted to correct this error at the same time. The result: the database cluster was quickly overwhelmed by hundreds of thousands of queries per second.
To make matters worse, every time a client got an error while trying to query one of the databases, it interpreted it as an invalid value and deleted the corresponding cache key. This meant that even after the initial problem was resolved, the query flow continued and the databases failed to recover. In short, the automated system did more damage than it repaired. To resolve this problem, Meta had no choice but to cut off all traffic to this group of databases, i.e. to temporarily disable the different platforms. Their access was then gradually restored.
Global outage at Meta: forced disconnection and loading error
The last major outage of Meta services dates back to May 2023 and concerned Messenger, which was unavailable for more than an hour. Meta then faced a giant outage, affecting all of its services, in October 2021, which lasted almost six hours (see our article).
As usual when Meta’s social networks go down, all users took refuge on X (formerly Twitter) to share their distress. “We know why you’re here.”ironically published X’s official account. “If you can read this tweet, it’s because our servers are working”had for his part commented Elon Musk, quickly forgetting the many setbacks encountered by the platform since its takeover. When the cat’s away the mice will play…