Designing robust web API

Many companies over the past years have been exposing their services via web service API. Some are doing a better job than the other. Once the API s publicly available, people will write code based on it, every change will break the client and make them unhappy. It is very important to get the first version of your API right.

In this post, I summarize some good practices that I have used successfully in my previous projects.

Use a RESTful Model
Design the web service API model on HTTP, REST and JSON format.
There has been a lot of comparison between SOAP and REST that I don't want to repeat here. The winner is surely REST. In this "resource oriented model", every resource type will be represented as a collection. On the other hand, HTTP verbs will be used to manipulate the resources.

Restful URL typically comes with the collection URI as well the individual URI. Here are some common examples for a resource type Person:

1. List all persons. GET /persons
2. Find a person with a particular id. GET /persons/123
3. Get partial fields. GET /persons/123?fields=(name,age)
4. Find a person's particular friend. GET/persons/123/friends/456
5. Find all persons named John. GET /persons/search?q=(name,eq,John)
6. Find all dogs whose master is John. GET /persons/search?q=(name,eqJohn)/dogs
7. Create a person with a server assigned id. POST /persons?name=Dave&age=10
8. Create a person with a client assigned id. PUT /persons/123?name=Dave&age=10
9. Ask the person to perform an action. POST /persons/123/action/travel?location=Euro
10. Remove a person. DELETE /persons/123
11. Return a page of result. GET /persons/search?q=(name,eq,John)&offset=1&limit=25

It is a good practice to categorize the API based on whether it is cacheable and whether it is idempotent. Cacheable API is typically implemented using HTTP GET, whose response can be cached somewhere and can be reused in subsequent request. Notice that cacheable API can help reducing the workload on the server, but on the other hand, not accurately tracking the actual usage of the client. Idempotent API can be invoked multiple times with the same effect of being invoked once, which is important in unreliable network where retry is very common. POST is the only mechanism to implement non-idempotent API.
For input parameters, I tend to put everything into the URL and not using the HTTP headers, which is used for OAuth headers.
In case of any error happens, it should fail fast and communicate the error clearly to the calling user. HTTP error code should be used for this purpose; response code 200 is typically used if the request is processed successfully; response code 400 is used if the server runs into some application logic problem; response code 500 is used if the API request send by the client is wrong. A human readable error message, together with the hint to fix that, should be sent back in the HTTP body response.
For those who is interested in seeing more examples, here is an earlier blog that I wrote.
Stabilize your APIAPI is public and hence must be stable. To achieve the stability, the API should expose as minimal as possible (notice it is much easier to add stuff later but very difficult to remove anything that you have expose). Also, the API should only expose the function semantics but nothing about its implementation details, which allows the implementation to continuously evolve without breaking the client interface. There is actually a development practice that I advocate about having the architect to write API interface that confines the functional expectation of every system component. API designed in this way usually ends up with a higher level of abstraction.

A good API should focus to do one thing well, rather than multiple things of different purposes. Each API must be self-contained and not relying on any specific call sequence to work correctly. The only exception is the authentication call which must the first call to make and precede any other application API calls. As far as API security, App level key, with OAuth2.0 protocol should be used for authentication and authorization purpose.

It should also be very intuitive to use and difficult to make undetected mistakes without needing to read through the documentation. This usually requires the API to have a high degree of consistency (such as parameter ordering, naming convention etc) as well as short but clear documentation with simple usage example code.

On the development process side, we should let more people to review the API design, and perhaps run some pilot client projects before exposing it to the general public.

Version your APIEven you try to stablise your API, change will still be needed over time. This can be due to new requirements that you don't originally expected has arrived.

If you follow the minimal API design approach, the newer version is usually about adding parameters to your original API rather than removing parameters. A good practice is to support both API interface (the older version as well as the newer version) behind the same URL. (e.g. http://xyz.com/v1/path/...). On the implementation side, you only have the implementation that takes the latest version API parameters as input. In other words, you are prepared to receive request of the older version as well as the latest version. But you substitute the default value of the parameters of the newer version that is missing in the older version. And then send this request (with all the parameters filled) to the latest implementation.

Notice that by keeping the original endpoint URL unchanged, you are committed to provide backward compatibility (able to handle an older version request).

If your later version of API have an incompatible change (e.g. having a new parameter that is mandatory). The general practice is to use a different URL endpoint for the new version (e.g. http://xyz.com/v2/path/...). You also keep the corresponding implementation (v1 and v2) behind those endpoints. Depends on your decision whether to keep supporting the older version, you may want to introduce a deprecation process. Unfortunately there is no standard way to indicate an API will be deprecated in the response. One possible way is to put a flag in the HTTP header of the response to indicate when the API will be deprecated.

Asynchronous Invocation
Most API tend to be synchronous which means the client makes a call, wait until the server finishes the processing and return the result. But if the server takes a longer time to finish its processing, then an asynchronous invocation mechanism is important.

One simple mechanism is the client request is immediately returned with a receipt, while the server queue up the request and processing it at a later time. The receipt will be used by the client later to poll the server with the processing status and claim the result once it is available.

Another mechanism is to have the client provide a callback address in its request, which gets return immediately. After the server finishes the processing at a later time, it will invoke the callback address to push the result back to the client. While this is more resource efficient than the polling mechanism above, it involves more setup on the client side, such as running the server to listen for callbacks, and make sure the server can be authenticated in its callback.