ddos + hardware failure = HoN issues
Quote
This thread is intended to be a more comprehensive update as to what is going on and what we are doing to fix the ongoing problems:
Issue #1: DDoS Attacks, we are still experiencing these on an almost daily frequency, the issue with these attacks is that it takes anywhere from 5-10 minutes for the mitigation to begin filtering the correct good traffic from the bad traffic, obviously an an online game this is very bad and interrupts whatever is going on that is good traffic for that time frame while things are sorted out.
Solution: We are working with engineers at our mitigation firm to create workarounds for this problem.
Issue #2: Massive hardware failures at one of our major NOC's. The master servers that control the game servers are experiencing massive hardware issues and have been for the last 3 days, we are working around the clock to keep these up and running but as you may have noticed yesterday (2x) and today, once so far, the service has still had failures. If one of these boxes fails it causes a ripple affect rendering match making inoperable and a general inability to connect to games.
Solution: We have new boxes up in a new datacenter where the Client master servers are located, HOWEVER, because of latency between the two datacenters we cannot put these boxes live until the databases are moved to the new datacenter, this is the scheduled maintenance that is happening tomorrow morning.
Issue #3: Stats not recording: This is also a function of the above issues we are aware of this and are working on a fix for this going forward as well as retrieving any non-recorded stats by the database from game servers directly and should have this implemented very soon.
We apologize for the issues and are working diligently to get the service back to the high standard of availability everyone expects.
This thread will stay open for questions, and will be heavily moderated.