Campus Web pages suffered “degraded service” Thursday after IT Services staff said they experienced problems with a network traffic-shaping device.
The problems began at 7:37 a.m., when a software patch applied overnight to correct other problems with the device’s performance caused an “unexpected degradation of network performance,” according to Matthew Ricks, the executive director of IT Services, in an e-mail to The Daily. As a result, the University’s Web traffic came to a “crawl.”
“During the disruption — University Web pages would not load and in some cases e-mail service was disrupted,” Ricks said. “The disruption effected [sic] those attempting to access Stanford Web pages from campus as well as off-campus computers.”
Visitors to some Stanford Web sites on Thursday encountered an error message stating that the site was operating on “emergency mode” and that the page was not available. IT Services posted a message stating that it was working to resolve the problem, and that there was no emergency situation on campus.
The network-shaping device is used to fairly and evenly allocate bandwidth to support Stanford’s Internet traffic needs.
“The WWW servers have a lot of traffic coming in and coming out, so they’ve got to go through several pieces of networking equipment to get to them,” said Ethan Rickleen, a Residential Computing senior network administrator. “You can update the software that’s running on them, and here it had unintended side effects.”
Network engineers worked with the device vendor to determine and implement a short-term fix for the issues, which returned services to normal at 12:18 p.m., according to an e-mail from the IT Client Alerts mailing list. The e-mail said that engineers will continue to pursue a long-term solution.
-Ivy Nguyen