At around 3pm UK time on Monday, October 20 our monitoring systems detected a complete service outage at our Netherlands location at Worldstream datacenter, which took offline over 500 customers, as well as our own website. We immediately contacted the datacenter for an update on the issue and were informed that this is because of a copyright infringement. Worldstream mentioned about 10 IP addresses which were used by 3 customers.
Calling them on the phone didn't bring any results - we were told that phone support only deals with technical issues, and we have to send an email to their abuse team, which we did without hesitation. We argued with Worldstream that they have overstepped a boundary, as disabling access to our entire infrastructure due to a few abusive customers is unacceptable.
About an hour later we received a reply stating that their decision is final and the services will not be reinstated and backups will not be provided. We immediately contacted our solicitor for advice. We then sent another email to Worldstream explaining that we will seek damages and will proceed with legal action if they don't provide access to our data.
About another hour later they replied that they will enable the services for us to back up and download the data, which has now been done, and is currently being transferred to our other location. The other location is also in the Netherlands, and we have a lot more control over it, as we only use our own network and hardware there - the datacenter only provides rack space. Such a sudden requirement to move meant that we had to increase capacity of our network on short notice, and we're still making sure that everything is set up properly before reactivating all services.
If you have been affected, please do not open a ticket for an update:
- We know about the issue
- We are working round the clock to bring everything back online
- We can't take time off to reply to tickets regarding this issue, as that would slow down the actual recovery. Furthermore, we do not know how long it will take to bring everything back up.
Thank you for your patience. Please accept our apologies.
Tuesday, October 20, 2020