April 2nd, 2010

Welcome

Welcome to the TADS 3 System Development forum!

To those who were on the old tads3 mailing list, welcome back.  This forum is essentially the new incarnation of the list.  The server where the list was hosted could no longer accommodate us, so I had to find a replacement.  Rather than just replacing it with the same setup on a different server, though, I thought it was a good opportunity to think about new approaches.  The mailing list served us fairly well, but it seemed kind of clunky and outdated in this era of social media and blogs.

This new setup is essentially a collaborative blog.  It has the look and feel of a blog, but like the old mailing list, it's open to anyone to join and post messages.  I've been intending for some time to set up a tads 3 blog of my own anyway, to communicate what I'm planning and what I'm working on - the mailing list didn't seem appropriate for that kind of chatter, since most people's inboxes are stuffed with enough detritus as it is.  This forum should serve as my tads 3 blog, but it's better than a regular blog because it won't just be one-way - it'll also serve the same group discussion functions that the mailing list did.

So, welcome to the new forum, and on to the TADS 3 technical talk!

TADS 3.1

The next update is shaping up to be a fairly big one, so much so that I'm considering calling it TADS 3.1 rather than just another point release (which would be 3.0.19).

There are two main themes that seem to be emerging.  The first is more "dynamic" features - this is roughly the area covered by reflection and similar services in the current system, and big new upgrades are coming in this area.  The second is coding convenience features.  There are a bunch of little nits that have been bugging me for a while, and I've been in the process of cleaning up many of those, but I've also come up with a number of thing I've been wanting to add based on what I've gotten accustomed to from doing a lot of php and javascript coding lately.

I'll go into more detail on specific features in the days ahead - I want to pace myself on this new blogging stuff, after all...

Performance tuning

Another area I've been working on for the next release is performance enhancements.  This is mostly out of concern that some of the new dynamic features introduce new overhead, so I've been trying to compensate by finding things to make faster in the existing code.  So far I've managed to net an overall speed improvement, so the system should actually run faster after all the new overhead-inducing features are added.

Many of the performance changes are really small tweaks that gain a fraction of a percent, but I did find one thing that made a surprisingly big difference.  I did some experimentation with the garbage collection parameters and found that I could get about a 20% throughput gain in typical adv3 games by choosing the right parameters.  Basically, the key parameter in GC is how frequently you run collection cycles.  If you run GC very frequently, you typically get fast *individual* GC passes, because the short time between passes means that you haven't generated a lot of garbage to collect since the last pass; but the trade-off is that there's a fixed amount of overhead on each pass, so the more frequently you run, the more times you repeat that fixed overhead, which slows down overall throughput.  On the other hand, if you run GC very infrequently, each pass can take a noticeable amount of time to run because it has to collect a large amount of garbage; but you don't repeat that fixed overhead as often, so overall system throughput is higher.

The conventional wisdom is that you need to lean toward the frequent-GC end of the spectrum for an interactive system, because you don't want "jerky" behavior - i.e., you don't want the system to intermittently freeze up during those long GC passes.  So instead you run GC frequently so that each pass is too quick for the user to perceive a freeze, at the cost of slowing things down in the long run.

I followed this thinking in setting the original GC parameters, since TADS 3 is purpose-built for a particular kind of extremely interactive application.  What I didn't do before was make comparative measurements at various frequency points; I really just kind of guessed at what a reasonable frequency would be.  I finally made the measurements, and discovered that my guess was way off.  GC was running much too frequently - way past the point of diminishing returns.  The time per pass is basically inversely proportional to the frequency, but there's also that fixed cost component added on, so at a certain point the curve flattens out and the time per pass is basically all the fixed portion.  Measuring it empirically turned up the sweet spot where the per-pass time is still about the same as it's always been, but the overall throughput cost is minimized for that desired per-pass time.  So there's no trade-off in making this adjustment - you still get imperceptibly fast (about 5 ms) per-pass times, so the user doesn't ever see a GC freeze, but the overall throughput is significantly improved.

There won't be a night-and-day difference.  My main goal was really to make sure that the new features didn't slow things down horribly, so it's pretty pleasing to have been able to manage a net improvement.

Mercury?

The other, other area I've been working on - and this is a heck of a lot more speculative than the system features I've been talking about - is a new library.  This isn't a new adv3 version, and it's not a replacement for adv3.  It's a whole new library that I intend to sit alongside adv3, as a different option for developing games.

The "high concept" for the new library is pretty simple and compelling: it's the tads 3 library for tads 2 users.  My feeling is that there's a portion of the community that's been kind of left out in the cold as TADS and Inform have evolved.  These are the people who want a relatively simple and traditional system, something on the level of TADS 2 or Inform 6 in terms of learning curve and overall complexity and programming style.  But they also want to know that what they're using is still being actively developed, so that they can expect ongoing new features and bug fixes and so on, and that there are other game writers out there for technical support.  Adv3 is off the deep end in terms of complexity - not gratuitously, since you get a lot of power in return for the work of learning it, but for people who don't need all that power that's not an appealing bargain.  And Inform 7 is a radical departure in its own way; while the bulk of Inform 6 users and a lot of newbies have embraced the new approach enthusiastically, there are some people out there who'd be happier with a "normal" programming language. 

Hypothetical user base aside, the real impetus that got me working on this is that I count myself in that disenfranchised tads 2 camp.  When you're working for free I've found it's best to work on the stuff you'd like to use yourself.  Adv3 is really cool in a lot of ways, but whenever I use it I find myself battling with the layers of details and wanting something simpler.

So I'm thinking of calling it something like Mercury: the smaller, lighter, faster option, compared to the Mega-Jupiter that is Adv3.  Any other ideas for names?  It's still very much a WIP and an experiment, so I don't know if it'll ever see the light of day; but I really hope it does because there are (in my opinion, anyway) some cool new ideas in it.

Browser-Based Interpreter

Someone (I think it was Zarf) made a comment on the newsgroup today to the effect that the IF player community will grow at least 10x and easily as much as 50x thanks to browser-based interpreters. Seems like an accurate assessment to me. It's still early days in this phase of an IF rebirth ... a Glulx terp doesn't exist yet. But when it does, T3 is going to fall even further behind in the popularity contest.

Since I'm a huge fan of T3, I will hate watching this happen.

I'm not competent to develop a browser terp myself, unfortunately. But I hope others agree that it's urgently needed.

What's really needed, of course, is One True Interpreter that can play IF in any format in the browser of your choice. There are, I'm sure, several ways that that could be developed, and even more ways that it could fail to be developed.

Glulx itself is in a sort of moribund transitional state. It's far from being desirable as a de facto standard, as it won't do nearly as much as HTML TADS. So I'm reluctant to suggest that compiling to Glulx is something that should be considered for T3. But what I would hate to see would be for the interest in TADS to decline any further due to the burgeoning popularity of Inform.

I have no solutions to offer. I just think it's a topic that would be worth discussing.

--Jim Aikin