It has been a controversial couple of days for Facebook. For nearly six hours on Monday night, Facebook, Instagram and WhatsApp remained completely inaccessible to people from around the world. The development comes a mere day after the Facebook whistleblower accused the social media giant of choosing itself whenever there was a conflict between public good and what benefited the company. And while some continue to make memes and joke about the nearly six hour outage, others see conspiracy theories and hidden meaning behind the 'blackout'.
Facebook insists that the root cause of this outage "was a faulty configuration change" and that the issue has been addressed. "Our engineering teams have learned that configuration changes on the backbone routers that coordinate network traffic between our data centers caused issues that interrupted this communication. This disruption to network traffic had a cascading effect on the way our data centers communicate, bringing our services to a halt," the engineers said in an official statement.
Put more simply, Facebook and its sites had managed to send an update that effectively locked it out of the internet, unable to undo the situation. According to researchers at Cloudflare, at around 10:30 pm (IST) Facebook's "DNS names stopped resolving, and their infrastructure IPs became unreachable". With DNS servers unavailable, Cloudflare’s 1.1.1.1 DNS resolver could no longer respond to queries asking for the IP address of facebook.com or instagram.com.
"It was as if someone had "pulled the cables" from their data centers all at once and disconnected them from the Internet," a report from Cloudflare notes. While other Facebook IP addresses remained routed, this was not of any use as they were unavailable without DNS Facebook and related services.
Alex Hern, the UK Technology Editor for The Guardian simplified this further in a lengthy Twitter thread on Tuesday. As he put it, Facebook had sent an update to a "deep level routing protocol on the internet" suggesting that they no longer had any servers. The company, he contends, found it difficult to correct course because "Facebook runs EVERYTHING through Facebook".
Cloudflare tracks all Border Gateway Protocol (a mechanism to exchange routing information between autonomous systems on the Internet) updates and announcements seen in their global network. "A BGP update message informs a router of any changes you’ve made to a prefix advertisement or entirely withdraws the prefix. We can clearly see this in the number of updates we received from Facebook when checking our time-series BGP database," the report explains.
But while this chart is normally quiet, things changed abruptly at the beginning of the outage with a peak of routing changes from Facebook. "Routes were withdrawn, Facebook’s DNS servers went offline, and one minute after the problem occurred, Cloudflare engineers were in a room wondering why 1.1.1.1 couldn’t resolve facebook.com and worrying that it was somehow a fault with our systems. With those withdrawals, Facebook and its sites had effectively disconnected themselves from the Internet," the article adds.
Excerpt from blog.cloudflare.com report |
Many on Twitter have also shared online listings that suggest that "facebook.com is for sale". The glitch also prompted a response from Twitter CEO Jack Dorsey who wondered "how much" it would cost.
But as the controversy dies down, some are convinced that yesterday's events were far from "accidental". With the outage coming a day after Frances Haugen was identified as the anonymous whistleblower who filed complaints with federal law enforcement conspiracy theories suggest that Facebook had been using this time to "destroy evidence". Haugen was identified in a "60 Minutes" interview on Sunday and elaborated further on the way company's own research shows how it magnifies hate and misinformation.
