Writing Web API Clients

The Application

To help explain this law I will use an example of a tiny web (HTTP) API. This API is for a web application for saving web pages that you might want to view later. it will have 3 endpoints.

One endpoint for saving a new web page:

POST /web-pages

Parameters:

• body (required)

{
    "url": "string"
}

Responses:

{
   "pageId": "ObjectId"
}

one for getting the content of a saved web page:

GET /web-pages/{pageId}

one for deleting a saved web page:

DELETE /web-pages/{pageId}

This documentation seems minimal but it is actually complete. This is an HTTP API, and HTTP is well documented.

I will keep the API tiny but this law applies for any size API. You will easily see why that is the case soon.

The law

First off, an API client can have many purposes. Chief among those will be to help the application users consume the service easily. Not among those is to interpret the application responses.

An API client should never interpret the results it gets. It should only ever return them to the calling application.

-~ Lungelo Ximba

Below is a Python example of a client for the above API:

import requests
import json

class WebPagesClient(object):
    def __init__(self):
        self.api_host = "string"

    def createWebPage(self, url):
        body = {"url": url}
        return requests.post(self.api_host + "/web-pages", json=json.dumps(body))

    def getWebPage(self, page_id):
        assert isinstance(page_id, ObjectId)
        return requests.get(self.api_host + "/web-pages/" + page_id)

    def deleteWebPage(self, page_id):
        assert isinstance(page_id, ObjectId)
        return requests.delete(self.api_host + "/web-pages/" + page_id)

The important thing to note about this client is how slim it is. It doesn't even assign responses to variables, it just returns them.

Obviously, improvements can be made to this client. For example, it can be made to use persistent connections, handle api keys if necessary, etc. However, those improvements should never be for interpreting the result. That is the job of the calling application.

I have seen API clients that interpret responses and in doing so end up forcing a programming style on the calling application. I will illustrate this with an example:

class WebPagesClient1(object):

    def getWebPage(self, page_id):
        assert isinstance(page_id, ObjectId)
        response = requests.get(self.api_host + "/web-pages/" + page_id)
        response.raise_for_status()
        return response.json()

To the inexperienced eye the above client looks harmless, and possibly better than the first version. However, I assert that this client is a horrible piece of code. Here is why. In the application code, if I want to check if a web page exists in the application I can use the getWebPage endpoint to check. If the page does not exist then I can decide what I want to do. If I use the second client I will have to try and catch exceptions like a Java developer who just discovered exceptions, like this:

client = WebPagesClient1()
try:
    client.getWebPage(5)
except requests.exceptions.HTTPError as e:
    if e.status_code == 404:
        ...
except requests.exceptions.ConnectionError:
    ...

The problem is that I actually expect that a web page may not exist, so that is not an exception. With the first client I will be able to do something like this

client = WebPagesClient()
try:
    response = client.getWebPages(5)
except requests.exceptions.ConnectionError:
    ...

if response.status_code == 404:
   ....

The flow of the second call is better because I am not adding logic for expected flow in exceptions. The client does not enforce the terrible style of programming by exception to the calling application.

Sidenote

For anyone wondering what I have against exceptions. Nothing. I do not have anything against exceptions. I take issue with developers who use them where they were not meant to be used. So I will share some wisdom of when to use exceptions.

Your code should only ever raise an exception in the case where it can no longer perform its contract. e.g the contract of getWebPage is to take a web page id and return a response from the API. If a connection cannot be established then that is an exception, and the requests library raises the exception. Not finding the intended web page does not prevent returning a response so that is not an exception.

Also, your code should only ever catch an exception if it has the responsibility to recover from it. You can catch exceptions to collect stats but then after re-raise them and let the calling application recover how it wants. e.g:

try:
    ...
except requests.exceptions.HttpError as e:
    stats_client.log(e.status_code)
    raise

Can this law be bent?

The simple answer is no. If the calling application wants to raise an exception for a non-200 response it can do so. A client should never interpret responses, it should just return them to the calling code.

Echo

When writing library code ask yourself with every line of code if you are not doing anything that will unnecessarily influence the calling application. Ensure that you are not raising or catching exceptions where you need not. Also, API clients are not meant to interpret responses. It is up to the calling application how to interpret the responses.

Note that this is an opinion that I think should be law. If you have a different opinion, please share it with me on Twitter

Ciao for now!