May 10th, 2010

Network services, part 1

I've finally had a chance to do some of the foundational work for the networked TADS I've mentioned a couple of times.  This initial work involves the low-level plumbing for TCP/IP and HTTP support, and that's now basically done.  I think this makes TADS the first IF-specific language in which you can write a Web server.

There's an example at the end of this message showing a very basic, but complete, Web server written with the new infrastructure.  This server simply accepts file path requests and sends back the requested files.  Apache it's not, but it should give you an idea of how the new infrastructure works at the nuts-and-bolts level.

The example uses a couple of new intrinsic classes: HTTPServer, which encapsulates the code that binds to a port and listens for incoming connections; and HTTPRequest, which represents an HTTP request from a connected client.  An HTTPServer object contains a background thread that runs autonomously once started, so starting a Web server is simply a matter of creating the object.  An HTTPRequest is where most of your program interaction takes place.  This object contains the information the client sent with the request (and provides a method to parse a query-style URL string, with "?"  parameters), and has methods that let you send a reply, including headers, a status code, and the content body.

To keep the example short and to the point, I've left the loadFile() function as an exercise to the reader.   That's not a new intrinsic; it's just some mundane file I/O code written with the existing File class.  I'm not leaving out anything interesting there.

Now, I said this example gives you an idea of the nuts-and-bolts level of this stuff, but don't worry that game programming is going to turn into Web server programming.  It's not.  All of this HTTPxxx stuff will be buried in the library; most games won't have to even know it's there.  The library will provide a UI-event-oriented API on top of its network code, so most games will see a UI API that looks a lot like the current one.  But games that do want to get down into the networking details will be able to, which will give them a degree of control that's fairly unprecedented (in IF systems).  I can imagine using this to create extensions for things like collaborative play and multi-player games.

But the main motivation and initial use for this is obviously single-player, browser-based games that simply replicate the current playing experience in a browser across a network.  I'll talk more later about how we build that on top of this foundation.

If you know anything about network servers, you're probably wondering how a single-threaded system like TADS is going to handle a multi-threaded job like HTTP serving.  In fact, the new infrastructure is multi-threaded: an HTTPServer starts an autonomous thread to listen for new connections, and that thread launches session threads when clients connect.  This is all hidden away in the native code, though.  The byte-code program remains blissfully single-threaded.  Eric Eve should be particularly happy about this, because it means he won't be adding sections to the Getting Started about mutexes and race conditions.  (I don't think that would really help the infamous TADS 3 learning curve.)  The cross-thread coordination is achieved via a message queue; you'll notice in the example that the Web server is basically an event loop that reads messages from the new netEvent() API.  This design allows for efficient handling of the low-level networking, by assigning a separate thread to handle the network I/O on each connection, while keeping game code single-threaded.

So here's the example: presenting the world's first IF-language Web server (as far as I know)...

main(args)
{
    /* set up the server */
    local ip = getLocalIP();
    local l = new HTTPServer(ip, nil, nil);
    "HTTP server listening on <<ip>>:<<l.getPortNum()>><br>";

    /* handle network events */
    for (;;)
    {
        try
        {
            /* get the next network event */
            local evt = netEvent();
           
            /* see what we have */
            if (evt.evType == NetEvRequest
                 && evt.evRequest.ofKind(HTTPRequest))
            {
                /* get the request, and parse the resource name */
                local req = evt.evRequest;
                local res = req.parseQuery()[1].substr(2);

                /* try opening the file */
                local f = loadFile(res);
                if (f != nil)
                {
                    /* found a file - send the contents to the client */
                    req.sendReply(f.body, f.mimeType);
                }
                else
                {
                    /* not found - send a 404 error */
                    req.sendReply('File not found', 'text/plain', 404);
                }
            }
        }
        catch (SocketDisconnectException sdx)
        {
            /*
             *   ignore these - they just mean that the client closed its
             *   connection before we could send the reply; their loss
             */
        }
        catch (NetException nx)
        {
            "<<nx.displayException()>><br>";
        }
    }

    "Shutting down the HTTP server...<br>";
    l.shutdown(true);
}