We're currently seeing significantly elevated latency for our Seattle DDoS protected network, of around 66ms. This is caused by multiple fiber cuts in the transport network we use to San Jose. We've been given an ETA for one of the cuts to be resolved on 13th April by 13:00 UTC, but we're hopeful the separate fiber cut that occurred at 10:36 UTC today (12th April) will be resolved long before this.
Our Seattle DDoS protected network saw instability starting at 20:37 UTC and ending at 20:45 UTC, this was caused by one of our DDoS mitigation providers, Neoprotect, suffering a full network outage, our backup network automatically took over within minutes and this will remain active until Neoprotect is stable again.
See the update below for full details:
Neoprotect has stated that they are not going to continue their service. We are investigating alternative DDoS mitigation providers and hope to have one available soon, this will also return latency to normal levels.
Neoprotect is still offline, they use datapacket/CDN77 as a sole upstream and they disabled Neoprotect's service which resulted in the full global outage of Neoprotect's network. You can read more about the cause here: https://status.neoprotect.net
We are currently routing DDoS protected inbound traffic through Cnservers Los Angeles so there will be a latency increase for locations that are geographically close to Seattle, if this is a problem for you in the short term, please contact us and we'll discuss the options available with you.
Longer term, we will onboard another ddos mitigation provider, potentially GSL, that has a local Seattle POP, this will ensure latency is normal.
No incidents reported
No incidents reported
No incidents reported
Our Dallas network is currently offline, as two independently networked cabinets are offline, the problem is likely with our upstream provider but we're investigating.
The network came back up and has been stable since 14:28 UTC after our upstream replaced their switch hardware. We apologise greatly for the inconvenience caused during this extended network disruption event.
The Dallas network has been completely offline for 47 minutes so far, our upstream is already aware of the current incident and simply tells us they're working on it.
No meaningful updates to provide yet, our upstream is still saying they're working on stabilising the malfunctioning switch.
Our upstream has confirmed the outages have been caused by a malfunctioning switch of theirs, they're still working to stabilize it. Continuing to monitor the issue.
We have seen one further outage within the last 5 minutes, there may be further short outages until our upstream resolves the issue. We are currently monitoring the issue.
Network came back up at 01:35 UTC. We'll speak to our upstream about what the cause was.
No incidents reported
No incidents reported
No incidents reported