:doc:`DiskCache <index>` provides a Django-compatible cache API in :class:`diskcache.DjangoCache`. A discussion of its options and abilities are described in the :doc:`tutorial <tutorial>`. Here we try to assess its performance compared to other Django cache backends.

A survey of repositories on Github showed a diversity of cached values. Among those observed values were:

1. Processed text, most commonly HTML. The average HTML page size in 2014 was 59KB. Javascript assets total an average of 295KB and images range dramatically but average 1.2MB.
2. QuerySets, the building blocks of the Django ORM.
3. Numbers, settings, and labels. Generally small values that vary in how often they change.

The diversity of cached values presents unique challenges. Below, keys and values, are constrained simply to short byte strings. This is done to filter out overhead from pickling, etc. from the benchmarks.

Django ships with four cache backends: Memcached, Database, Filesystem, and Local-memory. The Memcached backend uses the PyLibMC client backend. Included in the results below is also Redis provided by the django-redis project built atop redis-py.

Not included were four projects which were difficult to setup and so impractical for testing.

1. uWSGI cache backend.

   https://pypi.python.org/pypi/django-uwsgi-cache
2. Amazon S3 backend.

   https://pypi.python.org/pypi/django-s3-cache
3. MongoDB cache backend.

   https://pypi.python.org/pypi/django-mongodb-cash-backend
4. Cacheops - incompatible filebased caching.

   https://pypi.python.org/pypi/django-cacheops

Other caching related projects worth mentioning:

5. Request-specific in-memory cache.

   http://pythonhosted.org/johnny-cache/localstore_cache.html
6. Cacheback moves all cache store operations to background Celery tasks.

   https://pypi.python.org/pypi/django-cacheback
7. Newcache claims to improve Django's Memcached backend.

   https://pypi.python.org/pypi/django-newcache
8. Supports tagging cache entries.

   https://pypi.python.org/pypi/cache-tagging

There are also Django packages which automatically cache database queries by patching the ORM. Cachalot has a good comparison and discussion in its introduction.

Django's filesystem cache backend has a severe drawback. Every set operation checks whether a cull operation is necessary. This check requires listing all the files in the directory. To do so a call to glob.glob1 is made. As the directory size increases, the call slows linearly.

Above, the count regards the number of files in the directory and the time is the duration of the function call. At only a hundred files, it takes more than five milliseconds to construct the list of files.

The concurrent access workload starts eight worker processes each with different and interleaved operations. None of these benchmarks saturated all the processors. Operations used 1,100 unique keys and, where applicable, caches were limited to 1,000 keys. This was done to illustrate the impact of the culling strategy in locmem and filebased caches.

Under heavy load, :class:`DjangoCache <diskcache.DjangoCache>` gets are very low latency. At the 99th percentile they are on par with the Memcached cache backend.

Not displayed above is the filebased cache backend. At all percentiles, the latency exceeded five milliseconds. Timing data is available below. Though :doc:`DiskCache <index>` is the slowest its latency remains competitive.

Like sets, deletes require writes to disk. Though :class:`DjangoCache <diskcache.DjangoCache>` is the slowest, it remains competitive with latency less than five milliseconds. Remember that unlike Local-memory, Memached, and Redis, it persists all cached data.

Not all data is easily displayed in the graphs above. Miss rate, maximum latency and total latency is recorded below.

Notice the high cache miss rate. This reflects the isolation of local memory caches from each other. Also the culling strategy of local memory caches is random.

Memcached performance is low latency and very stable.

Redis performance is roughtly half that of Memcached. But notice also the maximum latency is a magnitude larger.

:class:`DjangoCache <diskcache.DjangoCache>` defaults to using eight shards with a 25 millisecond timeout. The total cache time for all operations is only 20% slower than Memcached. Cache access is in aggregate twice as fast.

Notice the higher cache miss rate. This reflects the cache's random culling strategy.