Frontend cache invalidator

Many websites use a frontend cache such as Varnish, Squid, Cloudflare or CloudFront to gain extra performance. The downside of using a frontend cache though is that they don’t respond well to updating content and will often keep an old version of a page cached after it has been updated.

This document describes how to configure Wagtail to purge old versions of pages from a frontend cache whenever a page gets updated.

Setting it up

Firstly, add "wagtail.contrib.frontend_cache" to your INSTALLED_APPS:

INSTALLED_APPS = [
  ...

  "wagtail.contrib.frontend_cache"
]

The wagtailfrontendcache module provides a set of signal handlers which will automatically purge the cache whenever a page is published or deleted. These signal handlers are automatically registered when the wagtail.contrib.frontend_cache app is loaded.

Varnish/Squid

Add a new item into the WAGTAILFRONTENDCACHE setting and set the BACKEND parameter to wagtail.contrib.frontend_cache.backends.HTTPBackend. This backend requires an extra parameter LOCATION which points to where the cache is running (this must be a direct connection to the server and cannot go through another proxy).

# settings.py

WAGTAILFRONTENDCACHE = {
    'varnish': {
        'BACKEND': 'wagtail.contrib.frontend_cache.backends.HTTPBackend',
        'LOCATION': 'http://localhost:8000',
    },
}
WAGTAILFRONTENDCACHE_LANGUAGES = []

Set WAGTAILFRONTENDCACHE_LANGUAGES to a list of languages (typically equal to [l[0] for l in settings.LANGUAGES]) to also purge the urls for each language of a purging url. This setting needs settings.USE_I18N to be True to work. Its default is an empty list.

Finally, make sure you have configured your frontend cache to accept PURGE requests:

Cloudflare

Firstly, you need to register an account with Cloudflare if you haven’t already got one. You can do this here: Cloudflare Sign up

Add an item into the WAGTAILFRONTENDCACHE and set the BACKEND parameter to wagtail.contrib.frontend_cache.backends.CloudflareBackend.

This backend can be configured to use an account-wide API key, or an API token with restricted access.

To use an account-wide API key, find the key as described in the Cloudflare documentation and specify EMAIL and API_KEY parameters.

To use a limited API token, create a token configured with the ‘Zone, Cache Purge’ permission and specify the BEARER_TOKEN parameter.

A ZONEID parameter will need to be set for either option. To find the ZONEID for your domain, read the Cloudflare API Documentation

With an API key:

# settings.py

WAGTAILFRONTENDCACHE = {
    'cloudflare': {
        'BACKEND': 'wagtail.contrib.frontend_cache.backends.CloudflareBackend',
        'EMAIL': 'your-cloudflare-email-address@example.com',
        'API_KEY': 'your cloudflare api key',
        'ZONEID': 'your cloudflare domain zone id',
    },
}

With an API token:

# settings.py

WAGTAILFRONTENDCACHE = {
    'cloudflare': {
        'BACKEND': 'wagtail.contrib.frontend_cache.backends.CloudflareBackend',
        'BEARER_TOKEN': 'your cloudflare bearer token',
        'ZONEID': 'your cloudflare domain zone id',
    },
}

Amazon CloudFront

Within Amazon Web Services you will need at least one CloudFront web distribution. If you don’t have one, you can get one here: CloudFront getting started

Add an item into the WAGTAILFRONTENDCACHE and set the BACKEND parameter to wagtail.contrib.frontend_cache.backends.CloudfrontBackend. This backend requires one extra parameter, DISTRIBUTION_ID (your CloudFront generated distribution id).

WAGTAILFRONTENDCACHE = {
    'cloudfront': {
        'BACKEND': 'wagtail.contrib.frontend_cache.backends.CloudfrontBackend',
        'DISTRIBUTION_ID': 'your-distribution-id',
    },
}

Configuration of credentials can done in multiple ways. You won’t need to store them in your Django settings file. You can read more about this here: Boto 3 Docs

In case you run multiple sites with Wagtail and each site has its CloudFront distribution, provide a mapping instead of a single distribution. Make sure the mapping matches with the hostnames provided in your site settings.

WAGTAILFRONTENDCACHE = {
    'cloudfront': {
        'BACKEND': 'wagtail.contrib.frontend_cache.backends.CloudfrontBackend',
        'DISTRIBUTION_ID': {
            'www.wagtail.io': 'your-distribution-id',
            'www.madewithwagtail.org': 'your-distribution-id',
        },
    },
}

Note

In most cases, absolute URLs with www prefixed domain names should be used in your mapping. Only drop the www prefix if you’re absolutely sure you’re not using it (e.g. a subdomain).

Advanced usage

Invalidating more than one URL per page

By default, Wagtail will only purge one URL per page. If your page has more than one URL to be purged, you will need to override the get_cached_paths method on your page type.

class BlogIndexPage(Page):
    def get_blog_items(self):
        # This returns a Django paginator of blog items in this section
        return Paginator(self.get_children().live().type(BlogPage), 10)

    def get_cached_paths(self):
        # Yield the main URL
        yield '/'

        # Yield one URL per page in the paginator to make sure all pages are purged
        for page_number in range(1, self.get_blog_items().num_pages + 1):
            yield '/?page=' + str(page_number)

Invalidating index pages

Pages that list other pages (such as a blog index) may need to be purged as well so any changes to a blog page are also reflected on the index (for example, a blog post was added, deleted or its title/thumbnail was changed).

To purge these pages, we need to write a signal handler that listens for Wagtail’s page_published and page_unpublished signals for blog pages (note, page_published is called both when a page is created and updated). This signal handler would trigger the invalidation of the index page using the PurgeBatch class which is used to construct and dispatch invalidation requests.

# models.py
from django.dispatch import receiver
from django.db.models.signals import pre_delete

from wagtail.core.signals import page_published
from wagtail.contrib.frontend_cache.utils import PurgeBatch

...

def blog_page_changed(blog_page):
    # Find all the live BlogIndexPages that contain this blog_page
    batch = PurgeBatch()
    for blog_index in BlogIndexPage.objects.live():
        if blog_page in blog_index.get_blog_items().object_list:
            batch.add_page(blog_index)

    # Purge all the blog indexes we found in a single request
    batch.purge()


@receiver(page_published, sender=BlogPage)
def blog_published_handler(instance, **kwargs):
    blog_page_changed(instance)


@receiver(pre_delete, sender=BlogPage)
def blog_deleted_handler(instance, **kwargs):
    blog_page_changed(instance)

Invalidating URLs

The PurgeBatch class provides a .add_url(url) and a .add_urls(urls) for adding individual URLs to the purge batch.

For example, this could be useful for purging a single page on a blog index:

from wagtail.contrib.frontend_cache.utils import PurgeBatch

# Purge the first page of the blog index
batch = PurgeBatch()
batch.add_url(blog_index.url + '?page=1')
batch.purge()

The PurgeBatch class

All of the methods available on PurgeBatch are listed below:

class wagtail.contrib.frontend_cache.utils.PurgeBatch(urls=None)

Represents a list of URLs to be purged in a single request

add_url(url)

Adds a single URL

add_urls(urls)

Adds multiple URLs from an iterable

This is equivalent to running .add_url(url) on each URL individually

add_page(page)

Adds all URLs for the specified page

This combines the page’s full URL with each path that is returned by the page’s .get_cached_paths method

add_pages(pages)

Adds multiple pages from a QuerySet or an iterable

This is equivalent to running .add_page(page) on each page individually

purge(backend_settings=None, backends=None)

Performs the purge of all the URLs in this batch

This method takes two optional keyword arguments: backend_settings and backends

  • backend_settings can be used to override the WAGTAILFRONTENDCACHE setting for just this call
  • backends can be set to a list of backend names. When set, the invalidation request will only be sent to these backends