May 20th, 2010

Network services, part 2

I don't have any big updates to report right now; I'm just slogging away at getting a core Web UI built on top of the network plumbing I described last time.  In the process I've been adding to the http infrastructure as I run into things useful for the web UI, so it's in that "last 20% that takes 80% of the time" stage right now.

For example, I know some people will be concerned about the security implications of making network connections possible in an IF system, so one of the things I've added is a "network safety" option setting, basically parallel to the "file safety" options.  People coming to TADS from the neo-BSG universe who know about the apocalyptic dangers of hooking up even an RS-232 connection between two computers will be able to shut everything off if they so choose.

The Web UI part is far from done, but I know how it's *going* to work, so I can at least give you an outline of the design.  The basic idea is that the game program acts as a Web server, and serves up the UI through an HTTP connection.  When I run through this design with most people, they scratch their head and try to figure out where the Apache server or whatever fits in.  They picture an HTML page with the game's initial text sitting on a hosting server somewhere, and an Apache server serving up that page to the client browser.  Then somehow you get the game involved in sending text across a socket mumble mumble...

So the important thing to understand is that it's all self-contained in the game program.  The game itself is the Web server.  There's no separate Apache server and no separate HTML page to be installed on a hosting service; all of the HTML comes out of the game via its integrated HTTP server.  The HTML might be static or it might be entirely generated - in practice, it'll be a mix.  Static HTML pages, along with static JPEGs, PNGs, etc., can be served directly out of the resource bundle.  (One of the helper classes I've already built is a little "web server plug-in" (plugging in to the in-game web server) that serves resource files - so with couple of lines of object declaration you can tell your game server to serve up any subset of your bundled resources.)  This is important when it comes to deploying games, because it means that a .t3 file is a complete Web deployment package, just like it's a complete download deployment package today.  That is, to publish your game to the Web, you just upload the .t3 file to a suitable location.  Depending on what I can arrange in terms of engine hosting, it might turn out that the .t3 file can be just about anywhere, so that a generic engine on my site can load your game from your site (or the IF Archive) via a URL.

Let's go a little more into the technical details of how the UI actually works.  The first step is that we have to load some kind of HTML into the browser.  This initial page can be anything you want it to be - that's one of the great benefits of having the whole thing integrated in the game engine - but there'll be a core Web UI start page in the library that sets up an initial window layout a la HTML TADS, with the browser interior devoted to an HTML transcript window.  So when you launch the game, the browser connects to the game server and requests that core UI start page, and the game server reads the page out of its resource bundle and sends it back to the client:

Browser -> Game:  GET ""
Game -> Browser: <html>...core UI page contents...</html>

The browser then loads up this page.  The default Web UI page has some on-load javascript that sets up the transcript window and sends an asynchronous XML request to the server for an "event".  This is known as an AJAX request - Asynchronous Javascript and XML - and it's the technology that sites with dynamic content use to get server updates without reloading the page.  For example, when you're typing a search string into Google, and it shows its list of suggestions as you type, that's AJAX at work: the page is sending your keystrokes to the Google server, the server sends back matching search terms, and the browser shows the reply data in a popup under the search box.

Browser -> Game:  GET

The game engine receives this request and holds onto it.  When there's something interesting to tell the UI, such as when there's some text to display in the transcript window or it's time to ask the user for a command line, the game sends back a reply with the event information.

Game -> Browser:  <?xml version="1.0"?><event><writeText>Welcome to TADS Online!</writeText></event>

The browser parses the XML and sends it back to the Javascript that sent the request in the first place.  The javascript processes it and carries out whatever the event says to do, in this case adding text to the transcript window.  The javascript finishes by queuing up the next event request.

Browser -> Game:  GET

When the game finishes displaying the intro text and wants to ask for input, it replies to the last getEvent with an input-line event.  Or if it wants a character, it sends an input-char event.  Etc.   The core UI page handles an input-line event by displaying an input editor and letting the user type in a command line.  (One of the little details that's taken some time is tweaking that command-line handler to look seamless on the different browsers.  This kind of thing is fiddly, so it's good to have in a reusable library.)  The user types in the command and presses enter...

Browser -> Game GET

The game server gets the input line text, sends an acknowledgment to the client, and passes ithe text back to the game for processing via the normal parser mechanisms.

Game -> Browser: <?xml version="1.0?><inputLine><ok/></inputLine>

The game processes the text as normal, sends its response, and asks for another input line.   (Recall that we still have a getEvent request from the game pending - that's what we're replying to with our first item below.)

Game -> Browser:  <?xml version="1.0"?><event><writeText>box: Taken.&lt;br&gt;ball: Taken.&lt;p&gt;&gt;</writeText></event>
Browser -> Game:  GET
Game -> Browser: <?xml version="1.0?><inputLine><ok/></inputLine>

So you can see how the basic flow works.  It's one thing to see on paper, but quite another to see it actually working - I'm at that point in the lab, and hopefully I'll have something to demonstrate publicly before too long.

There are a couple of big-picture things I want to point out.  First is that, as with the network-level plumbing, games won't have to deal first-hand with any of this, because the core Web UI library will take care of it all.  Games that just want the traditional transcript-and-banners type of UI will get a traditional print/inputLine/inputChar API for that.  The second thing is that games that want more control can tweak, revamp, or replace that library layer and write as close to the "metal" as they want, right down to the HTTP conversation with the browser, and right down to every line of HTML and Javascript that goes across the wire.  This is an opening up of the UI technology that's comparable to the evolution of the parser and execution engine going from TADS 2 to TADS 3 - what was once wrapped up tight inside the VM is moving out into the daylight of the library.  Granted, the "metal" in this case an interpreted language (javascript/html dom) running inside a sandboxed app frame (a browser), but it's a heck of a leap in power nonetheless.

Next time: the experienced Web programmers among us have probably noticed that I'm leaving out a key piece of the puzzle for Web deployment, which is how you hook a browser up to this thing in the first place.  The answer isn't "run it on port 80", as that would never be scalable - every game would need its own IP address, and the world is already short on those as it is!  I had to work out the same details in an earlier unrelated project, so I have a proven solution.  The solution is also conducive to multi-player and collaborative systems, which is in fact how it was used originally.