One of the things about working with data at 37signals is that I end up interacting with a lot of different APIs—I’ve used at least ten third-party APIs in the last few months, as well as all of our public APIs and a variety of internal interfaces. I’ve used wrappers in a couple different languages, and written a few of my own. It’s fair to say I’ve developed some strong opinions about API design and documentation from a data consumer’s perspective.
From my experience, there are a few things that really end up mattering from an API usability perspective (I’ll leave arguments about what is truly REST, or whether XML or JSON is actually better technically to someone else).
Tell me more: documentation is king
I have some preferences for actual API design (see below), but I will completely trade them for clear documentation. Clear documentation includes:
- Examples that show the full request. This can be a full example using
curl
like we provide in our API documentation, or just a clear statement of the request like Campaign Monitor does for each of their methods.
- Examples that show what the expected response is. One of the most frustrating things when reading API documentation is not knowing what I’m going to get back when I utilize the API—showing mock data goes along way towards this. Really good API documentation like this would let you write an entire wrapper without ever making a single request to the API. Campaign Monitor and MailChimp both have good, but very different takes on this.
- A listing of error codes, what they mean, and what the most common cause of receiving them is. I’m generally not the biggest fan of the Adwords API in many ways, but they are a great example of exhaustively documenting every single response code they return.
- A searchable HTML interface. Whether it’s visually appealing doesn’t really matter much, and Google indexing it is plenty of search. What doesn’t work for me is when the API documentation is in PDF, or I have to authenticate to get access to it.
- Communication of versioning and deprecation schedules. There’s some debate about whether versioning is better than gradual evolution, but regardless, anytime you’re changing something in a way that might break someone’s existing code, fair warning is required, and it should be on your documentation site. Sometimes you have to make a change for security reasons that don’t allow much advance notice, but wherever possible, providing a couple of weeks notice goes a long way. The Github API clearly shows what will be removed when and shows the differences between versions clearly.
Let me in: all about authentication
Most APIs either use OAuth/OAuth2, user/password or API token via HTTP basic or digest authorization, or a special parameter included in each request. These all have advantages and disadvantages:
- OAuth and the still developing OAuth2 are becoming the defacto standard for any API that primarily expects the API consumer to be a third-party application. It’s secure, relatively consumer-friendly, and relatively easy to use. The downsides: implementations vary slightly across service providers (I often hear that “the documentation is the implementation”), and it’s a lot of overhead for something where you just want a single piece of data.
- User/password or special API token via HTTP basic or digest authorization is fast, doesn’t require anything more than
curl
, and you can even make some request from a browser. The downside here is that it’s not especially “friendly” to ask an end-user to go find an API token.
- User/password or API token as a URL parameter fortunately isn’t very common. It’s more confusing than basic authentication, has all of the same downsides, and no real benefit.
There’s clearly a tradeoff here—OAuth is great for delegating authorization for a third-party, but API tokens are better for quick access to your own data. My preference is for an API to offer multiple methods of authentication so you can choose the best method for what you’re trying to do.
Underlying design: REST or something like it
There’s a formal definition of REST, and endless debates have erupted over whether a given API is truly RESTful or not. As a consumer of APIs, I don’t really care whether it’s technically RESTful, but there are a few “RESTlike” attributes that do matter:
- Not SOAP. I know there are legitimate reasons to have a SOAP API, and I can imagine that translating a massive API from a legacy SOAP to REST interface is quite difficult, but it’s incredibly difficult to consume SOAP if you have to develop a client library from scratch or do anything non-standard.
- Use HTTP verbs to mean something. Any API consumer is capable of sending
GET
,POST
,PUT
, andDELETE
verbs, and they greatly enhance the clarity of what a given request does. It’s terrifying to use an API where aGET
request can change the underlying data.
- Sensible resource names. Having sensible resource names/paths (e.g.,
/posts/23
instead of/api?type=posts&id=23
) improves the clarity of what a given request does. Using URL parameters is fantastic for filtering, but if everything is based on a single API endpoint and tons of parameters, the mental model required to use it gets too complex very quickly. I really like the way Assistly’s API has implemented resource paths and uses HTTP verbs.
- XML and JSON. I like JSON. Some people like XML. Unless the costs of offering both are staggering, offer both. Ideally, you’ll let me switch between just by changing an extension from
.xml
to.json
.
- Use abstraction where it’s helpful. Your API implementation does not have to mimic your underlying application architecture. For example, the Adwords API has 25 distinct “services”, each of which seem like one module of the underlying implementation. This is logical if you implemented it, but that doesn’t make it particularly friendly to use—if you wanted to retrieve your click through rate for each ad you’re running, you’d end up making (literally) dozens of distinct requests. On the other hand, the Google Analytics data export API lets you define reports so that you rarely have to make more than one request, but you’re still only fetching what you need. If you think there’s some common action that people will go to the API for, you should make that action easy, even if it means compromising on ideology.
These are my preferences, and there are no hard and fast rules of API design. Regardless of whether you apply any of these principles in your next API, I would absolutely encourage you to look at your API from the standpoint of a consumer. Maybe try building something small using your own API – it can be an eye opening experience.
Brandon Hansen
on 28 Sep 11The API that drives me most insane is that of Akamai. First, you have to authenticate. Then you have to go and try to find it. Then you get to decide what version you are using (and they have their own set of “keywords” to try to decipher). Then the whole thing is a PDF.
Thank you for making your docs clear.
Ryan Eccles
on 28 Sep 11Nice post.
I think software architecture should be taught and practiced from this perspective. As Alan Cooper put it recently:
The defining component of “architecture” is interaction with people and their motives, not with bricks and boards.
TVD
on 28 Sep 11Something about this just screams standardization. But is it that simple? Is API design similar in nature to language design? Is it more art than science?
I hope we can all agree SOAP needs to burn in the fiery depths of hell. But why are semantics like Resource Naming Conventions and XML versus JSON still up for debate?
I believe /posts/23 in combination with proper HTTP Verbs is both beautiful and synaptically pleasing.
Both JSON and XML are well supported server-side. The win for JSON comes down to versatility and it’s out-of-box support client-side. Consuming JSON client-side is clean and feels natural. Valid JSON is valid JavaScript Object Literal Notation. Because of this, it seems logical to make JSON the default API response.
Kelly Miyashiro
on 28 Sep 11Klout’s Docs use Mashery’s I/O Docs, which is based on Wordnik’s API docs. They allow you to read about methods and each parameter, then make a live call directly from the documentation. You must sign up for an API key first, but it’s free.
This might work for smaller APIs, bigger/complex APIs might still require traditional documentation.
NL
on 28 Sep 11@Kelly - thanks for the tip on the Mashery style documentation - that’s super innovative and neat. Not totally sure how it would work on a bigger API, but I’d love to see more documentation like this.
John Sheehan
on 28 Sep 11I have a clear bias now, but before I worked here one of the biggest reasons I wanted to work at Twilio was because of our API docs. I felt respected as a developer and customer every time I used then. I never had to sign up to access them and they’re extremely clear and complete. they’ve turned out to be good for recruiting too since many of our employees were attracted to the company because of the docs as well.
Erin
on 28 Sep 11Kelly, thanks for the shout-out for Wordnik’s docs—we open-sourced our API framework, and you can find it here: swagger.wordnik.com.
Greg Kilwein
on 28 Sep 11Gowalla’s API explorer”> is a pretty cool tool as well. I’m hoping to get similar functionality into my company’s API documentation site at some point.
I think that by creating an API design and development process that’s focused on and responds to the API’s users (the developers who use the API), many of these items mentioned here will naturally occur: robust documentation, searchable documentation, steady communication of changes, and a design that’s friendly to developers. An API really is its own product, distinct from a company’s other product(s) and should be managed as such.
Web programmer
on 28 Sep 11APIs are meaningless on the web. They only make sense if one is dealing with proprietary 3rd party applications that are built/deployed on non-standard technological platforms. If a resource is hosted on the web (which is a de facto standard technological platform), that said resource is discoverable and consumable. What’s the need for an API?
Henrik Hodne
on 28 Sep 11@Web programmer, I disagree. Yes, if something is on the web it is discoverable and consumable, but it is far from the best way of doing things. Let’s say I want to list your Gowalla checkins in my own app for some reason. I would first have to get your username and password (both of which I need to store securely somehow, but don’t get me started on that), log in to the website, store the cookies, get the list of checkins and parse the HTML into something I can use in my program. With a (good) API I can use OAuth, and not have to store the passwords anywhere. I can also send one requests without having to deal with cookies, and I get easily parseable code without any arbitrary information (such as styling) with it.
So while an API may not be strictly necessary, it makes things much, much easier for a programmer.
Scott Baird
on 28 Sep 11I’m finishing/maintaining a API documentation package right now. Thanks for the tips! As a non-developer writing for a developer audience, this is incredibly helpful.
Brandon Hansen
on 28 Sep 11@Web programmer
You can’t really be serious? More than likely you were using some sort of RSS reader to read the post, no? API.
The benefit of the API is that, in theory, it creates a contract between the application and those consuming the resources. That contract should never change (though it may be versioned). So as a developer I can be sure that my app is not going to break simply because a third party decided to change the structure of their HTML.
Ferenc Mihaly
on 28 Sep 11I love it when API users speak out about what they like or dislike about APIs. Getting user feedback can be sometimes challenging for the API designer.
I don’t believe poor APIs are built intentionally that way. Developing great APIs takes time and effort and designers often need to push back against the pressure to just hack it together and ship. Posts like this are very helpful to prove that users care just as much about API quality than functionality.
I’m definetly bookmarking this post!
Ferenc
David Zuelke
on 28 Sep 11I’d recommend reading up on the part of REST that is really exciting but barely anyone knows about, which is HATEOAS. In REST, Hypermedia types drive the interactions from the server side and clients mostly follow links. It is fundamentally different from an RPC approach like SOAP.
It also means that properly RESTful services (there aren’t a lot of them out there because simply labeling something “RESTful” doesn’t make it RESTful) are not only easy to learn, but also easy to evolve over time without breaking clients, and they enable interoperability to a point where several services can implement the same media type to allow clients to talk to them without assumptions about URL structure, or that /v1/ means API version one (a common mistake), or that a product with ID “1234” can be retrieved at “http://api.example.com” + ”/products/” + “1234”...
REST is not a standard for data exchange. It is an architectural style. It is not about pretty URLs that make sense from a semantic point of view (although that’s always helpful), or about HTTP status codes. The significant part is the Hypermedia constraint.
A good book on this whole topic is, IMHO, “REST in Practice”.
Jonathan Kressaty
on 28 Sep 11I don’t understand – many of the things mentioned in the post are things that 37Signals fails to do in their own API. In fact, I often times dread working with the 37Signals API due to the lack of valuable documentation. 37Sig products are great and I use them every day, but as far as the API and docs go, things could be spruced up a bit.
That said, I dig the post.
Andy Markham
on 28 Sep 11In your section entitled “Communication of versioning and deprecation schedules”, it seems like we can do a lot better than just keeping documentation available and up-to-date. Polling sucks, let’s push instead.
How about suggesting that APIs provide a mechanism (er, API) for you to submit a contact email? Given the date the email was provided will tell you when they started using it and should be adequate information (versioning can also do the same thing, if applicable) to let them know when they are in jeopardy of breaking as a result of an update to the API contract.
Kelly Miyashiro
on 29 Sep 11@NL: FanFeedr’s Docs are an example of I/O Docs for a larger API.
MattO
on 29 Sep 11OO design is key for API’s. If an object is inside another object send it back with the rest of the response. xml and json are really good at showing these kind of relationships. don’t make me do 3 calls to get all of one object.
totally agree on providing an example request and return format examples. if you allow xml and json show both xml and json return examples
one thing that I didn’t see mentioned. please keep to your naming convention consistent. you would think this is a no brainier, but it doesn’t always happen. decide if you are using – or _ to delimit words. don’t switch back and forth.
DougReeder
on 29 Sep 11May I suggest you add a list of error codes and more timely communication of API changes for Basecamp?
catwell
on 29 Sep 11Excellent post and examples. I like the pragmatic approach of REST.
Regarding REST / OO design, I tend to think in terms of data APIs and service APIs.
If the purpose of your API is to expose the data of your Web application, REST design makes sense. If it is to provide a service (eg. “convert a document from format A to format B”), sometimes it does not. If you wanted to implement this RESTfully you would actually have to make your API stateful, which defeats the purpose.
IMO the best resource about the design of modern Web APIs is the RESTful Web Services Cookbook by Subbu Allamaraju.
@Web programmer
on 29 Sep 11“The benefit of the API is that, in theory, it creates a contract between the application and those consuming the resources.”
That’s not the benefit of the API, that’s a huge liability. That’s precisely what prudent software developers are trying with all their might to avoid.
“That contract should never change (though it may be versioned).”
Obviously spoken by someone who hasn’t been around the block a few times. ALL contracts change, all the time. That’s the nature of the beat. Betting the farm on the naive assumption that ‘the contract should never change’ is how many people in this industry lose their jobs.
“So as a developer I can be sure that my app is not going to break simply because a third party decided to change the structure of their HTML.”
HTML? You’re signing contracts built on HTML? No wonder the state of software is in such a terrible shambles today.
@Web programmer
on 29 Sep 11“I disagree. Yes, if something is on the web it is discoverable and consumable, but it is far from the best way of doing things.”
Why? I think it’s the absolute best way of doing things.
“Let’s say I want to list your Gowalla checkins in my own app for some reason. I would first have to get your username and password (both of which I need to store securely somehow, but don’t get me started on that), log in to the website, store the cookies, get the list of checkins and parse the HTML into something I can use in my program. With a (good) API I can use OAuth, and not have to store the passwords anywhere. I can also send one requests without having to deal with cookies, and I get easily parseable code without any arbitrary information (such as styling) with it.”
HTML, cookies? Are you serious? Are you sure you’re not some graphic-designer-turned-web-developer?
“So while an API may not be strictly necessary, it makes things much, much easier for a programmer.”
Say no more. You’re obviously not a programmer, if you can say such a thing.
Findns
on 30 Sep 11@Web programmer Back in my days trolls were better… I’ll give you 2/10 cause i still laughed a bit.
Brian Hazzard
on 01 Oct 11@Findns actually I’m not so sure he is a “troll” as you say… His tone is a little contrarian to be sure, but his bit about contracts being a liability is spot on.
“That’s not the benefit of the API , that’s a huge liability. That’s precisely what prudent software developers are trying with all their might to avoid.”
Absolutely. This is also what HATEOAS (the hypertext constraint) is intended to avoid.
In my opinion, what should be documented are the media types and relations in the service. What should absolutely NOT be documented is URL templates for resources.
Building a client that relies on well known URL templates tightly couples your client to the server in a way that isn’t resilient to change. That service provider now has a choice: slow progress to a crawl or break your app.
A better way is to build your client to follow links based on understood (versioned) media types with understood relations.
Randall
on 02 Oct 11“Use abstraction where it’s helpful” leads to another thing with Web APIs in particular: don’t make me do lots of requests if just one could do it. Waiting for a chain of five or six HTTP requests is enough to slow down an interactive app, and if someone wants to do actions on hundreds of items at a time but your API only accepts one, it gets to be a pain to work around latency.
Saeed Neamati
on 03 Oct 11Thanks for the article. It was fine and informative. However, I’d like to mention that a good API should also be able to support JSONP (JSON with Padding).
Because lots of websites today use ajax heavily, and when you want to use ajax with an API exposed by a third party via a different domain, definitely XMLHttpRequest is not enough and JSONP should get into work.
Thanks.
Jonathan Dickinson
on 03 Oct 11Just keep in mind that REST (Web API) does not mean API. Web API is an element of API (mathematically [Web API] E [API]); which means “Tell me more: documentation is king” is the only part of your article that actually has anything to do with APIs – although it is very good advice none-the-less. You should consider correcting the title.
Anonymous
on 04 Oct 11Have you looked at your Basecamp API docs in the past couple years? JSON and XML feeds don’t match. Terrible documentation. plzfix kthx
This discussion is closed.