It's no secret that Torchlight III has had a rough go of it so far in Early Access. The team took to their Steam Community page to discuss those woes.
In an extremely legnthy and long-winded post, Guy Somberg, a lead programmer with Echtra, goes into exactly what went wrong during Torchlight III's initial launch and the problems that have surfaced since. The post is long, so grab a coffee/tea/soda - really anything to sip on while you enjoy the read.
The post goes into the myriad issues players experienced in the opening hours and days of Torchlight III, as well as the team's steps to solve them. It's interesting because while the post is technical in nature, you can clearly see the excitement in Somberg's voice as he describes what the team went through, especially during the initial launch as well as some unexpected issues along the way.
Our live team got together early on Saturday morning. Or, at least, early for game developers. At 10am, we had our build ready to go, and being white-gloved by QA. A white-glove test is a test that QA does on a build when there are no other players on it to “check for dust” - that is, to make sure that there are no surprise issues. That process was complete, and at 10:30am - with no fanfare - the game was available for purchase on Steam.Just after 11am, Max went on the PC Gaming Show to talk about Torchlight 3, and told people that it’s live now. So much for no fanfare. Thus opened the floodgates, and people started playing. For a little while, things seemed to go well. Our concurrent users (CCU) numbers went from zero up to a few thousand in a very short timeframe.
A little further down, he talks about what has now become known as "Zombie Zones," with those zombie processes counting towards server capacity.
Although we did not realize it at the time that we found the zombies, we later discovered these zombie processes were actually counting toward the capacity of the servers. We would ask a server if it had more capacity to spin up a new zone, and it would say “no, I’m full up”. But it was full of zombies, not actual zones.At this point, we had to do two things:
- We had to kill all of the zombies.
- We had to figure out why the zombies were appearing in the first place.
In order to kill the zombies for now, we actually just rebooted each of the servers in order to free up the capacity. Players could then start taking up that space, but we knew that the zombies would be coming back, so we needed a fix for the underlying issue. It was like playing Whack-a-Mole - as soon as we had cleaned up one set of servers, another one would get infested with zombies again.
It's well worth the full read to get a full understanding of exactly what the team went through dealing with the issues from its surprise Early Access launch. The team is actively working to improve the experience for players, releasing a recent patch as well as adding Echonok, the game's third act earlier this week.
