An in-depth look at the five core principles of API design.
Ten years ago, I gave an API design lecture at the (now-defunct) GameTech conference. To the best of my knowledge, it was the first (and last?) time anyone enumerated concrete, specific characteristics of APIs that could be directly observed and evaluated to determine whether an API would be effective in practice. It introduced a mental model for visualizing how APIs are used during project development, and introduced the term “integration discontinuity” to describe the primary pitfall that APIs must strive to avoid.
I never published any supporting materials, and the conference was not video recorded, so it has always been difficult to access the information in the talk if you didn’t attend the conference. I always thought this was unfortunate, because whenever I am forced to use other people’s APIs, it quickly becomes clear to me that they either don’t know or don’t care about a lot of the things that make APIs work well. API design, for whatever reason, seems stuck in the past, regardless of how far the underlying technologies underneath advance.
So, as a bookend to my discussion of programming principles used to develop The Witness’s Lister Panel, I decided to finally go back and resurrect my GameTech 2004 lecture by taking a surviving recording of the audio and manually syncing it up to an exported set of slides. Even with ten intervening years of experience, I think it’s all just as true now as it was then. There are a few things I might present differently today, but for the most part, although I am biased, I feel like I can wholeheartedly recommend the practices set forth in this lecture to anyone who works on code that needs to be reused.
API design, despite the lack of emphasis we tend to place on it as an industry, is increasingly one of the most important aspects of programming. From operating system APIs to commercial middleware to public domain libraries to bits of reusable code completely internal to a project, the more thought and care that people put into an API’s design up front, the more work is saved across the potentially hundreds of thousands of programmers who subsequently use it.
The API design principles from this lecture were formed and refined by my five-year experience designing and maintaining Granny 3D at RAD Game Tools, the first widely-adopted licensable character animation package. When I began work on the project in 1999, a lot of my friends in the industry thought the project was doomed to failure. At the time, the prevailing wisdom was that the only kinds of libraries that could succeed in the world of high-performance game development were what we called “leaf technologies” — so named because they existed as little leaves in the “tree” of your game architecture, and never had any non-trivial feedback. In the lecture, I refer to these as “layers”. A classic example for RAD at the time was Smacker, the very successful precursor to the then-still-in-development Bink Video, the industry’s current dominant middleware for video playback. Character animation was definitely not a leaf technology, because it was a core element of how a game’s rendering, physics, and gameplay code worked, so it always had to be tightly integrated with a number of interdependent systems.
When I shipped the first version of Granny 3D in late 1999, it appeared that the prevailing wisdom was indeed correct. Although we did have a smattering of customers for the 1.x branch, the library largely failed to be integrated into any high-profile, high-end game projects, despite Granny often possessing technology that would clearly have been useful. Not wanting to admit defeat, Jeff Roberts and I spent over a year working out how we might design a new API for the 2.x branch that would finally crack the reusable component problem and get us out of the world where only layer (“leaf”) technologies could be licensed.
With a lot of hard work, and probably more than a little luck, we succeeded. The API design of the Granny 2.x branch debuted in 2002, and it has endured with relatively minor modifications for 12 years. It has shipped in over 2600 different SKUs worldwide on everything from first person shooters to real-time strategy games. It’s one of the only bodies of code I’ve ever written that I am truly proud to be associated with.
Looking back at the API today, there are definitely a number of things I think we got wrong, largely because we were in uncharted territory and learning as we went. So I’ve always been a little sad I never did a Granny 3.x, just to have one version that was designed from the very start with the complete understanding of what makes a great API. But on the whole, I think the Granny 2.x API went a long way towards demonstrating how an API can be thoughtfully designed to allow smoothly integrating one complex system with another in a way that doesn’t sacrifice performance or squander resources.
The 2004 GameTech lecture was an attempt to look back on the Granny 2.0 development process and codify what we actually did that made such a substantial difference in (re)usability. In developing the slides, I identified four primary characteristics that I thought were most relevant. Since I was staying at Chris Hecker’s house while in California for the conference, I gave him a trial run of the lecture and he said, “you have to talk about flow control! Flow control is implicit in all of this.” He was right, and so the four characteristics became five characteristics for the final version.
As suggested by the final slide of the talk, I reserved time in the lecture for answering questions from the audience. I’d be happy to field questions here on the blog as well, so if you have anything you want to drill down on, please feel free to send it to firstname.lastname@example.org. If I get enough relevant questions, I’ll dedicate space next week to answering everything in detail.