Bug summary

Whenever a file b.rst is included in a.rst via the RST include directive, Sphinx rebuilds the a.html page; but in plot_directive, a.rst is not considered 'out of date' (as judged by the out_of_date() function), because the modification time of a.rst was not changed.

This discrepancy is part of the reason why #17860 exists; and while working on it (at PR #20374) I discovered also that figures with the :context: option set will get confused. A specific example is shown below. It's quite hard to encounter in real life but it is directly relevant to matplotlib's tests as there is a very similar construct in plots 6-9 of matplotlib's test_sphinxext.py.

Code for reproduction

conf.py

extensions = ['matplotlib.sphinxext.plot_directive']
exclude_patterns = ['_build']

index.rst

Index
=====

.. toctree::
   
   a
   b

a.rst

File A
======

It's important that the first plot produces an image, and also sets a variable
via ``:context:``.


.. plot::
   :context:

   plt.plot(range(2))
   a = 1  

The second plot must not use ``:context:``. It doesn't necessarily have to
produce an image. The important thing is that it must close the figure from the
previous plot, so that the third plot doesn't actually produce an image (if
figures aren't closed, then the third plot will reuse the same image from the
first plot).

.. plot::

   plt.plot(range(3))

The third plot must try to use a variable previously saved in `:context:`` and
must not produce an image.


.. plot::
   :context:

   assert a == 1

Lastly we include another file.

.. include:: b.rst

b.rst

File B
======

This can be anything.

Steps to reproduce

1. Put the four files above in a directory and cd into it.
2. Build the docs the first time using sphinx-build -b html . ./_build/html.
3. Modify b.rst in any way.
4. Build the docs again.

Actual outcome

The third plot in a.rst throws an error.

/Users/yongrenjie/test/rst/a.rst:21: WARNING: Exception occurred in plotting a-3
 from /Users/yongrenjie/test/rst/a.rst:
Traceback (most recent call last):
  File "/Users/yongrenjie/progs/matplotlib/lib/matplotlib/sphinxext/plot_directive.py", line 497, in _run_code
    exec(code, ns)
  File "<string>", line 1, in <module>
NameError: name 'a' is not defined

The reason for this, as suggested above, is because of the out_of_date() function. When sphinx-build is invoked again, Sphinx decides that both a.rst and b.rst must be recompiled. Now:

• Plot 1 is not considered out of date, because the image file already exists and a.rst was not modified. So the code is never run and a is never saved to the context.
• Plot 2 is there to ensure that figures are closed prior to Plot 3, so that Plot 3 never generates an image file.
• Plot 3 is considered out of date, because there is no image file that corresponds to it. Thus it is run again, and doesn't see a in the context, hence the warning.

Expected outcome

There shouldn't be any errors.

One easy way to accomplish this is to make sure that Sphinx re-runs all code snippets which are context-dependent, whenever a file is recompiled. That is, if a plot directive has :context: on, then the code should always be considered out of date regardless of the file modification times.

This will lead to some excessive regeneration of plots whenever included files are modified. For example, in the above code, whenever b.rst is modified, Plots 1 and 3 will always be re-created, even if a.rst is untouched. But IMO this is more sensible behaviour than the current bug. It would also be in line with what happens if any part of a.rst is modified, including the text outside the plot directives: all the plots in a.rst would be re-created.

This doesn't change the case where neither a.rst nor b.rst are modified, because in that case Sphinx will never attempt to recompile either file and plot_directive will never be called.