The D2Moddin Project

D2Moddin was a service I wrote to bring Dota 2 custom game modes to the public from May to June 2014. The game modes were based on the game’s seasonal game modes, and were created initially without the support of the developer, Valve. After the massive demand demonstrated by the service, Valve ultimately released the Workshop Tools Alpha far earlier than expected, including an unannounced Source 2 distribution, much to the surprise of the Source and Dota communities. Their in-game custom game system eventually went on to replace D2Moddin.

There was some work towards a system to host the new custom game lobbies on the D2Moddin server network, but I stopped working on it due to the massive resource requirement of the optimized Source 2 engine.

History and People

I began the project after seeing the CustomDOTA video about Fight or Flight. I was looking for a project to work on at the time, and decided to create a platform to bring these game modes to the public. I never expected it to grow to the scale it eventually reached.

The initial release was developed using Meteor which offered rapid prototyping and reactive development at the price of insane performance problems at scale. Read on to find out the technical background of the project.

The initial release was made under the Masters of Development banner, a group run by Jicyphex. I later left the group along with the developers of the original game modes for various reasons. I was the sole developer at that time. The overwhelming response to the release resulted in a complete crash of the optimized Meteor app.

I spent around a week attempting to scale the app, rewriting several parts to no real avail. The result was incredibly expensive over-the-top hosting solutions for what was ultimately a somewhat simple app at that point. It quickly became clear that a full rewrite was necessary. Using different technologies that allowed me to develop every single component of the system from the ground up, I re-wrote D2Moddin in a three day period, while streaming my development, often in nine to ten hour periods.

The re-release went far smoother than the initial release of the platform. Over time I added more features to the system, and several contributors joined the project.

Finally Valve’s system was announced and implemented in game, in their Dota 2 Workshop Tools announcement.

Contributors

I was the lead developer for the project, but members of the community contributed with GitHub issue reports, and in particular, I would like to thank the two contributors who joined the project later in its development:

Statistics

About halfway through its lifetime, D2Moddin added a stats API, which jimmydorry used to create the D2Moddin Stats page of the GetDotaStats website. It is a good representation of the popularity of each mod and our overall uptime and player count over time.

Some general stats:

We had around 13 different servers that were on the network over time. The system load balanced players across the network to support the playerbase.

Reddit Posts

Here are the posts I made on Reddit regarding the project:

The initial announcement was published by Jicyphex and backed by DotaCinema. The combined Reddit hype and DotaCinema followers resulted in a massive amount of traffic. See the details below on the flawed design decisions and my resulting rewrite.

Stream Clips

Sadly there are only two stream clips from the three days of streams.

Technical Details

In every implementation I used MongoDB as the database. This was my first time working with NoSQL and Mongo, but I enjoyed it and will be using it for future projects.

I tried just about every hosting PaaS out there, from Heroku to OpenShift to Modulus. Nothing worked better than bare-metal hosting. I used DigitalOcean droplets for hosting the web and master server.

Meteor.JS Implementation

The original implementation was written in Meteor.JS. It hosted both the website and the master server. It never worked properly for more than 100 players, and discouraged me from using Meteor for any large scale projects until it matures some more in the future. After being forced to learn other web frameworks, however, I enjoy the level of control granted by other frameworks more than Meteor’s start-to-finish easy solution. Too much is abstracted here for finely tweaking the system in production.

.NET Master Server Implementation

I wrote the re-write of the server implementation using C#. Originally I wrote the system using WebSockets.NET but later discovered issues with it. After rotating through a few other options I landed on XSockets.NET which worked very nicely for low level communication. The team has recently released XSockets 4 which I am now using for other projects. It supports RPC, WebSockets, WebRTC, and long polling fallback and is a great solution for communication between web browsers and .NET apps, as well as between .NET and .NET apps.

The system tracked users, lobbies, browsers, and servers. It kept state on everything and was responsible for authentication, distributing updates, interfacing with Amazon S3 for file storage and download authentication, and more. It also tracked load across the system and balanced players onto the available server hardware.

Angular.JS / Node Web Implementation

I needed a quick replacement for Meteor’s reactive DOM and easy templating. Angular was a natural choice. I was already working in Node.JS with Meteor, and stepped down to a basic Express.JS and Passport.JS setup for the server. Passport handled the authentication, using MongoDB for the backend.

The web server served the Angular app and managed authentication. It did not interact with the server network like the old Meteor system did.

The Angular app used the simple ng-route router, and Bootstrap for basic styling. I also used PNotify for notifications.

D2MPServer: The Server Manager

While the networking behind the server manager changed between master server implementations, the core code changed very little from the initial release.

The server manager was responsible for:

This part of the system worked so well that we could just extract the base compiled program into a folder and run it. It would build the entire setup automatically as well as download the latest addon versions. This allowed us to quickly add new servers and scale up without restarting the system.

The Mod Manager

Note: the source can be found on GitHub.

D2Moddin was a step forward from previous services like D2Ware. While D2Ware allowed the Dota 2 client to simply connect to modded Dota 2 servers, it never allowed for client-side file changes or additions. D2Moddin allowed for custom models, maps, textures, animations, characters, allowing for rich experiences rather than simple game modifications.

This required a client-side program that would move files around for the players. As I designed the system with the intent of ease-of-use and simplicity, the manager would have to do everything for the user automatically, and require little to no manual actions to work properly.

Furthermore I wanted to dispel fears of a virus inside the program, so I built it to be as temporary as possible. The launcher EXE was very lightweight and deleted itself after being run. It simply downloaded/updated/ran the actual manager which sat in the user’s temporary directory. This allowed it to run with no administrator privileges and could be removed with a single delete operation.

The program sat in the system tray and would exit if the user was disconnected from the site for more than a few minutes.

It would perform the following operations:

When a user joined a lobby the manager would:

When the game started the manager would:

Originally everything was done through the Web UI but later Ilian added a in-app UI for managing mods.

Conclusion

This was my first foray into large-scale service development. I learned a lot about server infrastructures, public relations, scaling networks, developing with speed in mind, and web frameworks. All in all, I enjoyed the experience and will be looking to do fun projects like this again in the future! Thanks again to the community for supporting my work.

Quantum / Paralin

 
3,103
Kudos
 
3,103
Kudos

Read this next

ODROID Setup for Robotics with ROS

This will be documentation of my process for setting up a 14.04 Ubuntu image with ROS Indigo on an ODROID device for robotics processing. This particular setup will focus on a U2 but the same procedure could be used for any system. The... Continue →

Get updated when Christian Stewart publishes

Don’t worry; we hate spam with a passion.
You can unsubscribe with one click.

VTL4iPjJQ60q1sB4IFk8