Don’t kid yourself.  Writing a multiplayer game that runs over the Internet can be tricky. If you are using Unity you have the entire .NET network library at your disposal. Simply making a TCP connection or sending UDP packets is pretty trivial. Why on earth would you bother using someone else’s library?

The debate between TCP and UDP for MMOs is as strong as ever and I’m certainly not going to talk about it here. For Ghostees, I am using UDP and the Lidgren library. It’s open source, free but best of all it solves a lot of the issues revolving around using UDP as your transport.

The types of things you can encounter with UDP can be a pain but since it is low level, you get incredible flexibility and performance. Think of UDP as a message passing protocol as opposed to TCP which is a stream based protocol.  TCP will ensure that your data makes its way across the trenches of the internet in tact and in order.  UDP on the other hand makes no guarantees.  Some messages may go missing.  You might even get two of the same or get messages out of order. Tricky stuff.  Lidgren takes care of all that stuff for your but only where you want it.  It will definitely let you shoot yourself in the foot also.

The (very) Basic Server

By the way, Lidgren builds in Mono so you can run your server on Linux. I’m pretty sure it works but I haven’t done enough testing to say 100% at this point and I’m pretty much a Windows guy. Enough yapping, lets get coding.

Start a new project for your server and create your server class. I prefer to add the Lidgren project to my solution but either way, add a reference to Lidgren.Network.

Your class will need two private members to start: NetServer and NetBuffer, both in the Lidgren.Network namespace.

class MyServer
{
    private NetServer server;
    private NetBuffer buffer;

    public MyServer()
    {
        var config = new NetConfiguration("hello-world");
        config.Port = 42948;

        server = new NetServer(config);
        buffer = new NetBuffer();
        server.Start();
    }
}

Lets go through this quickly.  On line 3 the NetServer is the class from Lidgren that will listen for incoming traffic.  The NetBuffer on the next line is what we read our messages into.  We can reuse it so it is declared as a class member.

We create an instance of NetConfiguration to pass configuration information into the NetServer constructor. NetConfiguration takes a single string argument that is our application identifier.  This can be anything you like as long as both the client and server use the same string.

DOH! you wouldn’t believe how many times I’ve had projects where I change the application identifier on either the client or server but not the other and then subsequently spend w-a-a-a-y too long trying to figure out why I can’t connect.

On line 9 I set the port that the server will listen on for incoming messages. Choose pretty much any number you like that is less than 65535 as long as it doesn’t conflict with another application listening on the same port. Only one port per network card can be in use at any time. You are best off choosing a number somewhere over 32768 to avoid “well known” ports.

Lastly, we just call Start() on the server to have it bind to the port and buffering messages.

The Pump

When the server receives a message, after some processing it places it into a queue. This all happens behind the scenes in another thread. You need to write code to retrieve them.  I usually have a single thread reading and distributing incoming messages and you might as well make it your main thread.  Add this method to your server class:

    public void Process()
    {
        NetMessageType type;
        NetConnection source;

        while (server.ReadMessage(buffer, out type, out source))
        {
            HandleMessage(buffer, type, source);
        }
    }

Your server host, whether its a console application or Windows service, will be responsible for pumping the messages by calling Process. Avoid the temptation of over complicating your server with a thread for the message pump. You won’t see any speed or efficiency increase putting more threads in here.

Each time through the while loop, the server will hand you one message. It will also give you the type of message and which client connection it came from.

The Mail Room

Now we need to decide what’s in the message we just got, and what to do with it. You will receive messages not only from your connected clients but also from Lidgren.  Lets cover Lidgren’s messages first.

Among the several types of messages Lidgren can send you, debug messages are one of the two basic ones you definitely want to handle. They will simply be strings that you can output to the console or debug stream.

Another  message you want to handle is the StatusChanged message. This message tells you when someone has connected or disconnected. I’ve created another method called HandleMessage which is where I will switch on the message type.

The last of the most basic messages is the Data message. These are messages sent to you from your connected clients.

I would suggest you throw an exception or at least log a warning if a message comes through that you didn’t handle. I’ve caught many weird bugs this way simply because I wasn’t aware I was receiving a BadMessageReceived or other message.

    public event Action<NetConnection> ConnectionStatus;

    private void HandleMessage(NetBuffer buffer, NetMessageType type, NetConnection source)
    {
        switch (type)
        {
            case NetMessageType.DebugMessage:
                {
                    Debug.WriteLine(buffer.ReadString());
                    break;
                }

            case NetMessageType.StatusChanged:
                {
                    if (ConnectionStatus != null)
                        ConnectionStatus(source);
                    break;
                }

            case NetMessageType.Data:
                {
                    if (DataReceived != null)
                        DataReceived(source, buffer);
                    break;
                }

            default:
                throw new NotImplementedException("Message type not handled: " + type.ToString());
        }
    }

You can see in the snippet above on the first two lines, I’ve declared two public events. Other classes in your server will want to know when a client comes and goes so its wise to publish this information. The other event gets fired when we receive data from clients.

HandleMessage works its magic by simply switching on the message type. In the case of a debug message, we simply write it to the debug stream.  In the case of a ConnectionStatus or Data message, we fire the appropriate events.

OK so here is the code so far:

using System;
using System.Diagnostics;
using Lidgren.Network;

namespace BakedOn
{
    class MyServer
    {
        private NetServer server;
        private NetBuffer buffer;

        public event Action<NetConnection> ConnectionStatus;
        public event Action<NetConnection, NetBuffer> DataReceived;

        public MyServer()
        {
            var config = new NetConfiguration("hello-world");
            config.Port = 42948;

            server = new NetServer(config);
            buffer = server.CreateBuffer();
        }

        public void Process()
        {
            NetMessageType type;
            NetConnection source;

            while (server.ReadMessage(buffer, out type, out source))
            {
                HandleMessage(buffer, type, source);
            }
        }

        private void HandleMessage(NetBuffer buffer, NetMessageType type, NetConnection source)
        {
            switch (type)
            {
                case NetMessageType.DebugMessage:
                    {
                        Debug.WriteLine(buffer.ReadString());
                        break;
                    }

                case NetMessageType.StatusChanged:
                    {
                        if (ConnectionStatus != null)
                            ConnectionStatus(source);
                        break;
                    }

                case NetMessageType.Data:
                    {
                        if (DataReceived != null)
                            DataReceived(source, buffer);
                        break;
                    }

                default:
                    throw new NotImplementedException("Message type not handled: " + type.ToString());
            }
        }
    }
}

In the next part, I will show you how to create a client that will work within Unity. It’s not tough at all. And if you deep-tech readers are concerned what we have done here is far too simplistic, not to worry. We will be going much deeper. After the client, we will start talking about message formats, serialization and message envelopes.

If this topic is useful or interesting to you, shoot me a quick comment and let me know so I’ll keep going with it.

• Bookmark on Delicious
• Digg this post
• Recommend on Facebook
• share via Reddit
• Share with Stumblers
• Tweet about it
• Subscribe to the comments on this post

I’ve also been adding Facebook integration.  So far the integration solutions for Unity3D that I’ve found so far have had huge security holes so I’ve had to come up with something myself. Not to say it doesn’t have holes either but at least I tried to keep people from hacking the app

What I’ve done is use the .NET Facebook API. I validate the Facebook user and look up their details in SimpleDB and put it all together in a class.  I serialize the class, RSA encrypt the bits salt and sign the data.  I send this as a cookie that gets picked up in the Facebook page and forwarded to Unity3D through JavaScript.

Unity3D takes this blob of data when establishing communication with the UDP server.  The server can then validate the signature, decrypt the data and viola.

• Bookmark on Delicious
• Digg this post
• Recommend on Facebook
• share via Reddit
• Share with Stumblers
• Tweet about it
• Subscribe to the comments on this post

The server culls down how many entities the game client will see so not to overload it. On the server I have masks that are used to categorize and prioritize what needs to be seen on any client. Click the video to head on over to youtube.com and watch it in HD.

As of this writing, I’ve had the server up to 400 concurrent connections @ 50% cpu but in the next build I think I can double that performance. There are many optimizations I know I can do at this point. Load has a lot to do with the number of entities that can see each other at any one time which corresponds to the number of messages I need to generate.

The chart below is with 95 connected entities that can all literally see each other.

As it is in that video, every entity can see every other entity within a vision radius. To achieve the 400 connections, I reduced the vision radius of NPCs and left the larger radius for actual players. This is where the masking comes in. NPCs don’t really care about a lot of entities. They may not care about any entities of their same race for instance. They may only care about players or perhaps even entities of a race they are at war with. The mask is simply a set of flags that determine who they care about seeing. Combine this with an appropriate vision radius. For instance, NPC’s don’t have to see as far as players. They only need to see as far as their maximum aggro radius. On the other hand, players need to see further so that characters don’t magically warp in.

World design also plays a part in controlling this. If you can see a great distance in front of you at any one time, you could potentially see a lot of entities. Have a look carefully at the design of Orgimmar or Iron Forge in World of Warcraft. Also think about how dungeon instances are designed. These areas are designed either to manage a high density of players or to keep game latency down to a minimum by carefully managing your visual distance. This will keep both the client and server snappy.

If you have other ideas for managing this, I’d love to hear them. Please post a comment!

• Bookmark on Delicious
• Digg this post
• Recommend on Facebook
• share via Reddit
• Share with Stumblers
• Tweet about it
• Subscribe to the comments on this post

The server technology is interesting. Our background is primarily in high performance serverdevelopment. We wanted a design that could scale to support a very large number of players. Who doesn’t? Server-side tasks are logically divided into services and architected so that they can run and live independently on remote hardware. This way we can distribute the load based on capacities any particular service can handle.

Servers are auto discoverable and communicate with each other efficiently over UDP. This is a huge benefit. As servers come on and offline, we don’t need to do any reconfiguring of dependent servers. They are designed to locate the appropriate service by dropping a service discovery packet on the network. All servers that provide that service respond with server information and current server load. This way the seeking server can choose among servers with more available capacity.

Servers are auto discoverable and communicate with each other efficiently over UDP

For instance we have a combat service. All it does is create and maintain a combat encounter for a set of entities (players and NPCs). A Ghosteesplayer initiating combat will cause the player’s gateway server to request a combat encounter instance on an available combat server. Other players and NPCs can join a combat encounter even if they are not on the same gateway.

A gateway is the server a player’s browser communicates directly with. It takes care of the core RTMP / AMFstuff that Flash communicates with natively. It is also responsible for persisting player information to the database and also for facilitating access to the other servers in the system. It does not do chat, movement, routing or even quests or NPC AI. Those are all different services potentially running elsewhere.

A zone server is responsible for simply tracking the entrance/exit and movement of entities. When an entity enters a zone (player or NPC), the zone broadcasts to all connected gateways that they need to subscribe to this new entity. It also sends messages back to the connecting player information about the zone, such as graphic asset information, music, collision layers and other existing entities.

To a zone server, an NPC looks just like any other player.

On each simulation step the zone server updates the positions of entities it knows about. When a player or NPC requests a route to a new destination, the zone server facilitates the route calculation, obstacle avoidance and potential zone transfers. Once the route is calculated it is sent to the requesting entity. This overrides any predicted route done by the client.  It’s pretty cool actually!

NPC AI runs on their own gateway server. To a zone server, an NPC looks just like any other player. NPCs can be scripted to request routes and respond to player interactions among other things just like a player.

I think your getting the idea by now. We’re really enjoying working with this design. It’s quite flexible and powerful.

• Bookmark on Delicious
• Digg this post
• Recommend on Facebook
• share via Reddit
• Share with Stumblers
• Tweet about it
• Subscribe to the comments on this post

The most obvious piece is the game client. This is the part that everyone sees. It is a Flash based application that runs in your browser written with the Flex 2 SDK. Some of us use Flex Builder and some like to just build from the command line and use other text editors.

The client itself is made up of many pieces that get downloaded on demand to reduce the initial download size. The Ghostees world is very large and would take a very long time to download if you had to get the whole thing at startup. Static content, such as artwork and images, is delivered from CacheFly. CacheFly is a content delivery system designed for very high speed and 100% uptime availability.

One distictive feature of Ghostees is the smooth scrolling of the world rather than going “screen-to-screen” as is more common in other virtual worlds. The way we accomplish this is by downloading the pieces of the world that are nearby only when they are needed and we can discard pieces that are no longer needed to save memory. The browser caches the previous downloads for us so subsequent requests to download world pieces are much faster.

The client talks to a variety of servers to not only get content but also take direction and negotiate transactions. Each server has a distinct job so that the overall application can be spread out. One of the most important servers that I’ll talk about in another post is the Map Server. This server is responsible for any given portion of the world. The Map Server is where everything “lives” and the client is just a viewport into it, responding to direction given by the server. In all cases the Map Server is the authority to what is going on in the world.

To take care of the inevitable network latency, one of the client’s jobs is to smooth this out. The client makes predictions of what the server is going to say and will initiate actions based on those predictions and uses the server’s message as confirmation. If the server issues a different message than expected, it is up to the client to smooth out the discrepancy.

Other servers include the Authentication, Chat, Auditing, Exchange (not mail but game currency), the API, Inventory/Shopping and a Mini-Game Server per Mini Game. All of these could run on a single box if that’s all we needed but the system was designed to scale out based on demand. The goal is to be able to pack as many people into the world as we can by distributing the work and keep the experience as fluid as possible.

• Bookmark on Delicious
• Digg this post
• Recommend on Facebook
• share via Reddit
• Share with Stumblers
• Tweet about it
• Subscribe to the comments on this post