Since the Last Post: Context is Key

In the previous platform update I added subscriptions. All a user would get as feedback for subscribing was an alert above the form confirming they were subscribed. Notice that I said “was”. That's because this has changed and in this post I will talk about how. This post will be slightly more technical than previous posts, featuring code snippets. But it will be easy to follow even if you are not technical.

Up until about a week ago, the platform had no outgoing emails. After getting quite a number of subscriptions, emails seemed like the best thing to tackle. Emails are a big part of many software systems and I have a lot of experience with them.

Context is key

When designing software systems it is important to keep in mind that software is always executed with and within a context; and that context is key. That sentence sounds like a lot of nonsense, and it might be, but it's important nonsense. Even if you were to stop reading now I'd urge you to remember that nonsense.

I could go on and write a book explaining what that means. Instead, I will use a short practical example to illustrate this.

When you loaded this page into your browser, you initiated a request to the server hosting this blog and you got a response that you are reading now. We will call that context A. Now, if you fill in the subscription form and submit it (go ahead, do it now) you will get a response that confirms your subscription. You will also get an email from me thanking you for your trust. Call that context B.

The set of actions and events that the platform must perform and go through to respond to your request is different for context A and context B. But the most important thing is that neither of them include sending you that email! Sending you that email is context C and is only initiated in the background by context B. The reason for this is simple: to generate the response for context B, I do not need anything from the result of sending you the email.

To summarise, do not do in the request-response cycle actions that are not necessary to generate the response to a request. If the result of the action does not change the response, put it in the background (read: different context). No matter how small, put it in the background.

Background tasks

Different programming languages provide different ways of doing background tasks. This platform is built in Django and I use Celery for background tasks. However, the concepts illustrated below apply to any other tool for background tasks.

Note that the code below has been oversimplified for illustration purposes. It is not optimally structured and it does not handle errors or exceptions.

Let us assume you use a function like the following to send new subscribers a welcome email.

def send_welcome_email(user_id):
    user = User.objects.get(pk=user_id)
    if user.is_subscriber:
        subject = "Welcome {}".format(user.first_name)
        # load message body from template
        body = template.render({'user': user})
        email = EmailMessage(
            subject=subject, body=body,
            from_email="Admin<no-reply@email.com>", to=user.email
        )
        email.send()

And you call this function from somewhere in your request handling code.

def subscribe(self, request):
    # load POST data into form and validate
    form = ...
    ...
    user = form.save()
    send_welcome_email(user.id)
     # construct  response
    ...
    return response

At first glance, there is nothing wrong with doing things this way. And, most likely, in your development environment you will not notice the problem with the code above.

However, in a production environment there is a lot that can go wrong in the send_welcome_email function. The template rendering can take a while, especially if you are using HTML emails which are notorious for being a nuisance to get right. The actual sending of the email depends on network conditions and how fast your email provider responds (Yes, you should be using a mail provider). This will make your user wait longer than they need to for a response. It gets worse. Any of the email sending code can fail or timeout after making your user wait for a while. That could leave your user thinking their subscription failed when it didn't. The point is, the above code is bad.

A better alternative is to background the email sending code. Using Celery, you can do:

from celery import shared_task

@shared_task
def send_welcome_email(user_id):
    ...

What this does is declare that function as a Celery task; a function that you can execute in the background.

And then in your request handling code you background it like:

def subscribe(self, request):
    ...
    user = form.save()
    send_welcome_email.delay(user.id)
    ...

With this approach, even if the email sending code takes 5 seconds your user does not need to wait 5 seconds for a response.

Even though I use Celery for examples, this pattern is possible in any language that has task queue libraries. If you are a Python developer and you are not familiar with Celery, change that today. Ruby has Sidekiq and Resque. Java has Quartz. I suggest you familiarise yourself with a task queue library for your programming language. If your language does not have a task queue library, then that should be your next open-source project :)

Echo

Always keep the context of any function you write narrow. Do not synchronously execute actions that do not alter the response of your function in the context of that function. And remember, a function call doesn't always change context. Background all the unnecessary actions.

Reach me on Twitter if you have any thoughts or questions.