Past Incidents

Our Seattle network is currently offline. It's being looked into.

UPDATE @ 23:43 BST - Data centre has informed us that the cause is a power issue despite our router being on redundant power, we're awaiting more information.

UPDATE @ 23:47 BST - ATS switch (which transfers power from mains to generator) failed during a mains power supply outage causing both of our A + B redundant circuits to go offline. The data centre has said they're on the phone to the vendor to resolve the issue at the moment.

UPDATE @ 00:11 BST - Mains power has been restored but power isn't yet restored to our servers due to the failed ATS switch causing the UPS's to not pass power. Vendor is working on it.

UPDATE @ 00:36 BST - Power has been partially restored, we're checking everything to get it fully restored.

UPDATE @ 00:53 - Power is fully restored. A switch in one of our cabinets is still offline and is being physically checked therefore some servers will still be offline.

UPDATE @ 01:15 - One of our switches is not responding, it's being looked into.

UPDATE @ 02:26 - All servers are now online. A configuration issue in a switch caused one of our cabinets to be offline for an extended period. This will be corrected incase anything causes it to be rebooted in future.

All Telia routes to Dallas appear to be suffering high packet loss. It's being looked into.

UPDATE @ 06:05 BST - This was resolved at 05:44 BST. Issue was a ddos attack against the data centre which was eventually mitigated.

Our Dallas Bi16 cabinet is currently experiencing network issues. It's being looked into.

UPDATE @ 09:37 - Network issues were resolved at 09:11. The cause was the carrier GTT carrying out maintenance on our upstream's circuit without prior warning. Most servers affected were in a single /24 prefix which was single homed to GTT, we are getting this announced to Telia now also to limit the impact of a single carrier having issues in future.