We're seeing intermittent network outages for our Dallas and Ashburn network due to issues with CosmicGuard's network, they're working on a resolution.
This has now been stable for over 24 hours.
CosmicGuard has started seeing global packet loss of up to 10%, affecting our Dallas and Ashburn locations
Issues have started again, CosmicGuard has said they have a plan to resolve the disruption, caused by large attacks leaking past Cloudflare.
We haven't seen any disruption for the past 35 minutes so it's likely this has now been resolved.
CosmicGuard has now updated their status page: https://status.as30456.net/en-gb/cmhhpokrs00s1omomxdm6ppa4
CosmicGuard has confirmed they're actively working on a resolution with Cloudflare to stabilise the network.
The network instability has been continuing, with disruptions every ~5 minutes affecting almost the whole of CosmicGuard's global network. Our current assumption is that ddos attacks are likely the cause, we haven't yet received an update from CosmicGuard.
Our Seattle DDoS protected network saw instability starting at 20:37 UTC and ending at 20:45 UTC, this was caused by one of our DDoS mitigation providers, Neoprotect, suffering a full network outage, our backup network automatically took over within minutes and this will remain active until Neoprotect is stable again.
See the update below for full details:
Neoprotect has stated that they are not going to continue their service. We are investigating alternative DDoS mitigation providers and hope to have one available soon, this will also return latency to normal levels.
Neoprotect is still offline, they use datapacket/CDN77 as a sole upstream and they disabled Neoprotect's service which resulted in the full global outage of Neoprotect's network. You can read more about the cause here: https://status.neoprotect.net
We are currently routing DDoS protected inbound traffic through Cnservers Los Angeles so there will be a latency increase for locations that are geographically close to Seattle, if this is a problem for you in the short term, please contact us and we'll discuss the options available with you.
Longer term, we will onboard another ddos mitigation provider, potentially GSL, that has a local Seattle POP, this will ensure latency is normal.
No incidents reported
Our Dallas Path.net DDoS protected IPs are currently seeing intermittent high packet loss and very short outages. We're already in contact with Path about this and hope it's resolved shortly.
We have now removed Path.net from our network. Path has not shown any willingness to work with us to resolve major issues on their end therefore we have decided to permanently switch to CosmicGuard. We do not currently have a customer control panel developed due to the quick switch to CosmicGuard, it is being worked on and will be released soon. If you need rules/filters created in the meantime, please contact us via support ticket.
We have moved our IP prefixes to CosmicGuard ddos protection until Path.net resolves their issues. If you require any specific filters made for CosmicGuard, let us know, as we do not currently have a customer control panel available for this yet.
No incidents reported
No incidents reported
The switch in one of our Seattle cabinets has gone offline, IGW1.10.2, this will affect all dedicated servers prefixed with this ID.
VPS servers SEAKVM18, SEAKVM19, SEABKVM24 and SEABKVM25 are also offline as a result.
We're investigating.
Our Minecraft node 17 in Lenoir, North Caroline has been offline since 11:28 UTC as the data centre, Dacentec, appears to be offline. We're awaiting an update.
The data centre came back online on 29th September at 06:26:24 UTC.
The network outage is caused by a storm that impacted the region, taking down all network connectivity to the data centre. Updates can be followed here.