We're observing intermittent network outages for our Seattle network, lasting up to around 1 minute, multiple times per day. The cause is issues with our network transport provider, we've been discussing it with them but unfortunately they haven't found the cause yet, in part due to how short the outages are. We are continuing to discuss it with them so we can reach a resolution as soon as possible.
Our Seattle DDoS protected network saw instability starting at 20:37 UTC and ending at 20:45 UTC, this was caused by one of our DDoS mitigation providers, Neoprotect, suffering a full network outage, our backup network automatically took over within minutes and this will remain active until Neoprotect is stable again.
See the update below for full details:
Neoprotect has stated that they are not going to continue their service. We are investigating alternative DDoS mitigation providers and hope to have one available soon, this will also return latency to normal levels.
Neoprotect is still offline, they use datapacket/CDN77 as a sole upstream and they disabled Neoprotect's service which resulted in the full global outage of Neoprotect's network. You can read more about the cause here: https://status.neoprotect.net
We are currently routing DDoS protected inbound traffic through Cnservers Los Angeles so there will be a latency increase for locations that are geographically close to Seattle, if this is a problem for you in the short term, please contact us and we'll discuss the options available with you.
Longer term, we will onboard another ddos mitigation provider, potentially GSL, that has a local Seattle POP, this will ensure latency is normal.
We're currently seeing significant instability in Seattle due to issues with our transit provider, Cogent, we're working on it currently.
This was fully resolved at 09:10 UTC on 30th September and has been stable since except for 01:38 - 01:41 UTC on 1st October.
We've determined the issue is related to an issue with Ceranetworks' Los Angeles POP, we're awaiting them resolving the issue, in the meantime, bursts of packet loss will be seen for traffic coming from China. If it's not resolved shortly, we'll enable alternative routes temporarily.
All non China traffic returned to normal at 06:40 UTC, we're still seeing some packet loss with China connectivity due to an issue with our connection to Ceranetworks, we're still investigating and monitoring.
No incidents reported
We're currently seeing network instability in Dallas due to issues with CosmicGuard's network, they are investigating.
No incidents reported
No incidents reported
No incidents reported
We saw intermittent network disruption in Seattle starting at 09:20 UTC, this was caused by a T1 carrier (Cogent) having major packet loss in the United States between east and west coast. We have since re-routed away from Cogent so there should be no packet loss to/from most destinations. Our China connectivity has seen some disruption as a result however we have seen this stabilise but there is still the potential for a small amount of intermittent disruption.