Our Seattle DDoS protected network saw instability starting at 20:37 UTC and ending at 20:45 UTC, this was caused by one of our DDoS mitigation providers, Neoprotect, suffering a full network outage, our backup network automatically took over within minutes and this will remain active until Neoprotect is stable again.
See the update below for full details:
Neoprotect has stated that they are not going to continue their service. We are investigating alternative DDoS mitigation providers and hope to have one available soon, this will also return latency to normal levels.
Neoprotect is still offline, they use datapacket/CDN77 as a sole upstream and they disabled Neoprotect's service which resulted in the full global outage of Neoprotect's network. You can read more about the cause here: https://status.neoprotect.net
We are currently routing DDoS protected inbound traffic through Cnservers Los Angeles so there will be a latency increase for locations that are geographically close to Seattle, if this is a problem for you in the short term, please contact us and we'll discuss the options available with you.
Longer term, we will onboard another ddos mitigation provider, potentially GSL, that has a local Seattle POP, this will ensure latency is normal.
No incidents reported
Our DDoS protected Seattle network saw an outage starting at 23:59 UTC caused by Cnservers/Ceranetworks going offline, our backup network took over within 2 minutes. We're still talking to them about a solution being implemented on their end for this.
We are currently seeing degraded network performance to China using our AS4837 route, this is caused by a cut in the APG submarine fiber cable AS4837 uses.
Degraded performance may continue until this fiber cut is fixed.
China Unicom has said the APG Marine cable S3 has been repaired as of 06:45 UTC, this means all submarine cables are now back online. We're monitoring the network, let us know if you see any issues.
China Unicom has confirmed that the APG Marine cable S3 repair is scheduled to start on 1st September and is expected to be completed on 8th September. We believe that this submarine cable carries a lot of capacity so hopefully significant improvements will be seen when it's back online.
One of the offline submarine cables has been repaired but at least one, including the APG Marine cable S3, is still offline with no estimated restoration date, we believe this submarine cable being offline is responsible for most of the congestion. We'll update when we receive an estimated restoration date.
The submarine cable that was due to be fixed on 20th August has been rescheduled to 23rd August.
We've been informed that one of the currently offline submarine fiber cables is due to be fixed on 20th August but there is still one other offline with no estimated fix date so performance may still not improve to previous performance levels.
Throughput at peak time has dropped to very low levels, less than 5Mb/s multithreaded. Ceranetworks has confirmed they're seeing significantly lower traffic levels meaning the fault is still within China Unicom's network.
Our Seattle network is currently suffering an outage due to issues with our router, we're monitoring.
No incidents reported
No incidents reported
No incidents reported