BlackBerry's Thorny Problem Chokes Service in North America
The Great BlackBerry Blackout of 2011 has spread to cover North America, bringing the days-long problem to even more users. RIM says the messaging and browsing problems impacting BlackBerry subscribers were due to a core switch failure within its infrastructure. Apparently, the system's designed to failover to a backup switch, but the switch didn't work as it should have, resulting in a large backlog of data.
10/12/11 11:51 AM PT
Research In Motion's continuing BlackBerry service disruptions hit customers in North America Wednesday.
This makes the disruptions, which first emerged in Europe, the Middle East and Africa on Monday, a worldwide phenomenon.
RIM states that subscribers in the Americas may be experiencing intermittent service delays this morning, that it's working to resolve the situation as quickly as possible, and that it will provide a further update when it can.
"I don't know why, but RIM won't say anything other than that they know what the problem is and they're working on it," Craig Mathias, a principal at the Farpoint Group and a mobile communications expert at Focus.com, told TechNewsWorld.
Asked whether the outage has affected BlackBerry Enterprise Server or is limited to BlackBerry Internet Services, Mathias indicated the situation is unclear.
"Nobody seems to know anything," he said.
RIM's public relations agency Brodeur Partners declined to provide TechNewsWorld with additional comment beyond a statement posted on the company's website.
The service interruptions didn't hit everyone in North America, although it did impact some major corporate RIM customers.
"I was watching MSNBC and this reporter said the last email he received was at 3 a.m.," Farpoint's Mathias said.
"Everyone on the business channels on TV depends on BlackBerry, so this is a very serious outage," Mathias added.
On the other hand, "Our internal email working off our BES server wasn't down," said Michael Morgan, a senior analyst at ABI Research.
BES stands for BlackBerry Enterprise Server.
"Everything that goes through RIM's servers is at risk, but it affects different people differently," Morgan told TechNewsWorld. "I'm sure everyone at least experienced a slowdown, and it affected Web browsing."
Corporate users often access the Web using the BlackBerry browser, Morgan said.
What Probably Happened
RIM said on Tuesday that the messaging and browsing problems impacting BlackBerry subscribers were due to a core switch failure within its infrastructure. Apparently, the system's designed to failover to a backup switch, but the switch didn't work as it should have, resulting in a large backlog of data that needs to be cleared.
That's quite possible, ABI's Morgan suggested.
"Equipment never works in real life the way lab tests show it will," Morgan said. "I can see a cascading failure happening -- the standard computer networking stuff. One part of the network fails, another part takes over but gets overloaded."
Or it could be a software crash.
"RIM likely updates its internal software quite often, to add features or find bugs," speculated Allen Nogee, a research director at In-Stat.
Failed software upgrades have impacted the BlackBerry service twice previously, Farpoint's Mathias pointed out.
If the software had some bug in it that was replicated across the entire network, that could account for the worldwide system problems RIM is facing, Nogee suggested.
RIM's process for handling its services could exacerbate system problems.
"For the most part, everything has to travel through RIM's servers, and if there's a problem, entire countries or more could lose service for hours," Nogee told TechNewsWorld. "It's happened many times before."
More Skull Sweat's Needed
RIM's system may have set itself up for a single point of failure, something IT assiduously tries to avoid.
"They can't have a single point of failure," Farpoint's Mathias said.
"This does point out that mission-critical systems need some rethinking -- and BlackBerry service is mission-critical in many organizations," Mathias added. "We need to address this as an industry."
Black Eye for RIM, or a TKO?
Between this days-long service problem and various other obstacles to the business, RIM has taken a battering in the past year or so.
The iPhone and Android smartphones are hacking away at its market share; its recently released BlackBerry PlayBook tablet isn't faring especially well in the market, and its Q2 results were dismal.
Some analysts have speculated that RIM's on its last legs.
Will the intermittent failure of BlackBerry services worldwide over the past three days put the kibosh on RIM?
"That depends on how quickly the service is restored," Farpoint's Mathias said.
"It's a black eye for the company both technically and from the financial perspective, but people have short memories, and switching from one IT system to another is not a simple matter," Mathias added.
"I'm one of the rare analysts who believes RIM will survive," Mathias remarked. "They have a huge installed base that's not going away any time soon."