Why Lumen's water leak caused such widespread phone, internet outages
KCCI EIGHT NEWS AT SIX STARTS NOW. FIRST AT SIX. MOST IOWANS AFFECTED BY AN INTERNET AND PHONE OUTAGE YESTERDAY NOW HAVE SERVICE BACK. BUT QUESTIONS REMAIN ABOUT JUST HOW THIS HAPPENED. A BROKEN WATER PIPE IN THE LUMEN BUILDING IN DOWNTOWN DES MOINES LED TO OUTAGES FOR CENTURYLINK AND AT&T CUSTOMERS. KCCI IS MARCUS MCINTOSH IS LIVE IN DOWNTOWN TO EXPLAIN WHAT HAPPENED AND IF THE PROBLEM IS FIXED, MARCUS. WELL, WE’RE LIVE IN FRONT OF LUMEN, WHICH IS DOWNTOWN ON NINTH STREET, AS YOU MENTIONED. LUMEN ALSO OPERATES AS AS YOU MENTIONED ALREADY, CENTURYLINK. NOW YOU CAN SEE BEHIND ME, THERE’S STILL WORK TRUCKS HERE, WHICH IS A LIKELY INDICATION THAT THERE’S STILL WORK BEING DONE TO RESTORE SERVICES, WHETHER IT’S LANDLINE, LANDLINE, 911 SERVICES OR INTERNET SERVICES TO PEOPLE WHO ARE STILL OUT AND TODAY WE GOT OUR FIRST LOOK INSIDE THE COMPANY TO SEE WHERE THE PROBLEM STARTED. THESE ARE PICTURES FROM INSIDE LUMEN AND THE AREA WHERE THE BROKEN WATER PIPE LED TO A SYSTEM FAILURE, SHUTTING DOWN LANDLINE PHONE SERVICE AND PUTTING A ROADBLOCK ON THE INFORMATION SUPERHIGHWAY. IOWA STATE UNIVERSITY PROFESSOR GUY HELMER HAS NO INSIDE KNOWLEDGE OF LUMEN, BUT FROM OUR PICTURES CAN TELL US WHAT LIKELY HAPPENED TO SCRAMBLE THE SYSTEM AND THE BACKUP PLAN. WATER GETTING IN THE WRONG SPOT AND TAKING OUT THOSE POWER SYSTEMS. EVEN THE REDUNDANT BATTERY AND GENERATOR SUPPORT THAT THEY HAVE. UM, THE ALL THE ALL THE TAKES, ALL THE SYSTEMS DOWN. HELMER HAS 30 YEARS EXPERIENCE IN DATA CENTERS AND INTERNET SERVICES. HE TELLS US WHEN THERE’S A CENTRAL OFFICE FAILURE, IT IS DIFFICULT TO STOP THE INEVITABLE. A LOT OF OUR COMMUNICATIONS FOR EVEN EVEN CELL PHONES AND ESPECIALLY LANDLINES AND SUCH GO THROUGH CENTRAL OFFICES. ARION IS A TELECOMMUTING SERVICE PROVIDER. SIMILAR TO BUT SMALLER THAN LUMEN, THE BROKEN WATER PIPE AFFECTED. IT’S 911 AND LONG DISTANCE SERVICE. ARION WAS ABLE TO REACT ON THE FLY WHEN THEY REALIZED IT WAS NOT AN ISSUE WITH THEIR NETWORK. HAVING THE TEAM STEP UP AND CONVERT THOSE CUSTOMERS QUICKLY TO DIFFERENT PATHS THROUGHOUT THE NETWORK, THROUGHOUT THE STATE, DIFFERENT PROVIDERS FELT REALLY GOOD THAT WE COULD TAKE CARE OF OUR CLIENTS IN A QUICK MANNER. ARION SENIOR VICE PRESIDENT SCOTT JOHNSON SAYS. YOU CAN’T PREDICT EVERY FAILURE SCENARIO, BUT IT’S IMPORTANT TO BE TRAINED AND EXPERIENCED WHEN THINGS GO WRONG, SO YOU ARE NOT STUCK IN ONE PLACE. YOU TRY TO LOOK AT ALL SCENARIOS AND TRY TO HAVE A PLAN IN PLACE. MY LAST COMMUNICATION WITH LUMEN WAS ABOUT 330 THIS AFTERNOON, WHERE I REQUESTED TO FIND OUT EXACTLY WHEN THEY EXPECT EVERYONE TO HAVE THEIR SERVICES RESTORED, THEIR RESPONSE WAS MOST PEOPLE HAD THEIR SERVICES RESTORED ABOUT 1230 THIS MORNING. WE’RE LIVE IN DOWNTOWN DES MOINES. I’M MARCUS M
Advertisem*nt
Why Lumen's water leak caused such widespread phone, internet outages
KCCI Anchor, Reporter
A mess inside the Lumen building in downtown Des Moines caused a telecommunication issue this week that left tens of thousands of Iowans without phone and internet service, including a landline 9-1-1 outage.Pictures showing puddles on the floor inside the building long after the leak was detected help tell the story. A broken water pipe Thursday morning led to the massive system failure, which was restored for most customers in the early hours Friday.ISU professor Guy Helmer has no inside knowledge of Lumen's leak. But those pictures and the resulting outage give him an idea of what happened. "Water getting in the wrong spot and taking out the systems. Even the redundant battery and generator support. It takes all the systems down," Helmer said. Helmer has 30 years of experience in data centers and internet services. He tells KCCI that when there is a central office failure, it is difficult to stop the inevitable."A lot of our communications for even cell phones, and especially landlines and such, go through central offices," Helmer said. We paid a visit to Aureon, a telecommunications service provider that's similar to but smaller than Lumen. The broken water pipe affected its 911 and long-distance service.Aureon was able to react on the fly when they realized it was not an issue with their network."Having the team step up and convert those customers quickly to different paths through the network, throughout the state, different providers really felt good that we could take care of our clients in a quick manner," said Scott Johnson, Aureon senior vice president. Johnson says you can't predict every failure scenario. But it is important to be trained and experienced when things go wrong you are not stuck in one place."You try to look at all the different scenarios and try to have a plan in place," Johnson said.
DES MOINES, Iowa —
A mess inside the Lumen building in downtown Des Moines caused a telecommunication issue this week that left tens of thousands of Iowans without phone and internet service, including a landline 9-1-1 outage.
Pictures showing puddles on the floor inside the building long after the leak was detected help tell the story. A broken water pipe Thursday morning led to the massive system failure, which was restored for most customers in the early hours Friday.
Advertisem*nt
ISU professor Guy Helmer has no inside knowledge of Lumen's leak. But those pictures and the resulting outage give him an idea of what happened.
"Water getting in the wrong spot and taking out the systems. Even the redundant battery and generator support. It takes all the systems down," Helmer said.
Helmer has 30 years of experience in data centers and internet services. He tells KCCI that when there is a central office failure, it is difficult to stop the inevitable.
"A lot of our communications for even cell phones, and especially landlines and such, go through central offices," Helmer said.
We paid a visit to Aureon, a telecommunications service provider that's similar to but smaller than Lumen. The broken water pipe affected its 911 and long-distance service.
Aureon was able to react on the fly when they realized it was not an issue with their network.
"Having the team step up and convert those customers quickly to different paths through the network, throughout the state, different providers really felt good that we could take care of our clients in a quick manner," said Scott Johnson, Aureon senior vice president.
Johnson says you can't predict every failure scenario. But it is important to be trained and experienced when things go wrong you are not stuck in one place.
"You try to look at all the different scenarios and try to have a plan in place," Johnson said.
Lumen says 'vast majority' of broadband services restored after outage, landline 911 and phone service work continues
