Planned Down Time
Duration: Emerging Issue Description: Network Engineers, working in tandem with the hardware vendor, have
identified an issue on customer routers which provides connectivity for
your server(s) and back-end services. To prevent an unplanned outage
resulting in significant downtime, Engineers will be performing firmware
upgrades during a scheduled maintenance window.
During this maintenance, all servers in this datacenter will initially
experience a disruption in back-end services on the private network, such
as DNS resolution, iSCSI, NAS, data backup, and SSL/PPTP/IPSec managed VPN
access to servers while the MBR (master back-end router) is rebooted for
upgrade. Servers in the first "Pod" (behind fcr01/bcr01) will also
experience a disruption of back-end private and front-end public network
connectivity, and back-end private network connectivity to servers in
other pods will also be disrupted the initial step of this maintenance.
Afterward, servers in other Pods (behind the respective router pairs
listed below) will experience a disruption of both back-end private and
front-end public, network connectivity as those routers are rebooted in
the following sequence:
mbr01.wdc01 + fcr01.wdc01 + bcr01.wdc01
fcr02.wdc01 + bcr02.wdc01
fcr03(a/b).wdc01 + bcr03(a/b).wdc011
In addition to the router maintenance, SAN-based CCI and VPX instances
will be shutdown/paused in order to maintain data integrity due the fact
that they rely on storage subsystems that reside on the backend network.
Once the routers have returned to service, all CCIs will be booted up and
brought back online.
While the maintenance window is set for 6 hours from 23-Jan-2014 5:00 - 11:00 UTC
(12:00AM - 6:00AM 1/23/2014 datacenter local time), we expect the initial
services and first Pod disruption to last less than 30 minutes, and
disruption to each subsequent pod to last less than 30 minutes, as well.
We expect a disruption of up to 120 minutes for SAN-based CCI and
VPX instances.
Register To Reply
Originally Posted by maynard
