Our Dallas DDoS protection provider, CosmicGuard, is suffering global network issues due to very large ddos attacks coming from a known 40Tb/s botnet, they're working on resolving the situation as soon as possible, no exact ETA currently available.
Stability returned at 12:01 UTC, CosmicGuard has posted a RFO here: https://status.as30456.net/cmiivbxm6004wt518aafiq5dd
We're seeing ongoing long outages, we're still awaiting an update from CosmicGuard.
CosmicGuard saw a brief global outage from 09:49 - 09:53 UTC, issues with specific POPs are ongoing.
CosmicGuard started global maintenance at 17:58 UTC and it appears it caused a global outage rather than only in Los Angeles, things have since came back starting at 18:04 UTC.
Our Seattle DDoS protected network saw instability starting at 20:37 UTC and ending at 20:45 UTC, this was caused by one of our DDoS mitigation providers, Neoprotect, suffering a full network outage, our backup network automatically took over within minutes and this will remain active until Neoprotect is stable again.
See the update below for full details:
Neoprotect has stated that they are not going to continue their service. We are investigating alternative DDoS mitigation providers and hope to have one available soon, this will also return latency to normal levels.
Neoprotect is still offline, they use datapacket/CDN77 as a sole upstream and they disabled Neoprotect's service which resulted in the full global outage of Neoprotect's network. You can read more about the cause here: https://status.neoprotect.net
We are currently routing DDoS protected inbound traffic through Cnservers Los Angeles so there will be a latency increase for locations that are geographically close to Seattle, if this is a problem for you in the short term, please contact us and we'll discuss the options available with you.
Longer term, we will onboard another ddos mitigation provider, potentially GSL, that has a local Seattle POP, this will ensure latency is normal.
No incidents reported
One of the two PDUs in the Seattle cabinet IGW1.13.3 has gone offline, this will mean some dedicated servers are offline (if your dedicated server ID starts with IGW1.13.3) and VPS server SEABKVM5.
We're getting the PDU checked currently.
The equipment that went offline is currently powered by a temporary PDU so things were online quicker, we will schedule a maintenance window in the coming days to swap back to the permanent PDU.
Everything is now back online, as of 05:20 UTC, at least temporarily. We're awaiting further updates from the data centre.
The data centre has informed us that they're working on getting temporary power to the affected PDU so hopefully the downtime will not be the full time mentioned.
We received an update from the data centre that the power outlet on their side has failed and they need to re-run a new cable to a new outlet. This process will unfortunately take 2 - 3 hours but they are expediting it as much as possible.
No incidents reported
No incidents reported
No incidents reported
Our Seattle AS4837 is no longer being used for routing due to a fault. We have contacted our network provider and are awaiting an update.