Interactive matplotlib backend for notebook by andreabedini · Pull Request #4077 · ipython/ipython

andreabedini · 2013-08-21T01:50:34Z

Hi,

this is a matplotlib web backend tailored to work with ipython's notebook. The code is mostly taken from matplotlib's backend_webagg but it's been reworked, simplified and adapted to the notebook workflow.

How it works

Essentially it works this way:

at instantiation it creates a httpserver hooked up to the same event loop the kernel uses
this serves a page which displays the rendered canvas communicating with the backend through a websocket. this logic is identical to backend_webagg
the figure manager has been slightly adapted to work like backend_inline. It keeps a list of figures the user has been working on and on show() it displays them.
The embedding of a figure in the notebook is done with an iframe to the page served by the backend.
The iframe automatically resizes itself sending a message to the parent frame.
I tested it with few matplotlib examples for the gallery: Zoom/Pan/3D rotation/Animation/Interactivity/Widgets all work fine.

to load the interactive backend just do

import matplotlib
matplotlib.use('module://IPython.kernel.zmq.pylab.backend_interactive')
import matplotlib.pyplot as plt

Then you can %load something from the gallery at http://matplotlib.org/examples/index.html

At this time the code works well, at least for local use.

Limits:

The configuration is not integrated with IPython's because I still understand very little about how it works.
A bunch of machinery IPython already implements to deal with matplotlib is ignored but it can be integrated back if useful.
At the moment the backend binds to a random port on the local interface, because I couldn't think of anything else. The backend runs on the kernel, but, ideally, it should be able to serve pages on the same interface the notebook does (although on a different port).
It is not entirely clear to me which kind of interactivity is most appropriate. At this time, the user can display the same figure more than once (and this works just fine), but changes will be propagated to all visualizations which might not be what the user expects. Perhaps it would be better to have a way of duplicating a figure.
Of course when the notebook is saved, the iframe is saved too but the backend state is not. Maybe we can save a screenshot. I don't know.

Let me know what you think of this.

jasongrout · 2013-08-21T02:32:14Z

We've implemented communication patterns like this (sage interacts) by registering a handler for a new custom message type. If you went that route, you wouldn't have to start a new server on a new port. Instead, you could pass messages back and forth on the existing iopub channel, right?

Presumably the IPython json<->backend message spec they are working on will make this even easier.

jasongrout · 2013-08-21T02:34:31Z

By the way, I think this is really cool and I've been looking forward to this since before Michael posted his demo. Thanks for working on it! We hope to integrate something like it into Sage as well.

andreabedini · 2013-08-21T02:47:45Z

Thanks for the feedback. I am aware this might be an under-engineered solution but it was too easy not to have a try. Using the existing iopub channel would make much more sense indeed (think of multi-user sessions).

The advantage of this solution is that it requires IPython to do nothing it already does. This so true that this PR is actually silly: everything can be implemented as an external module or extension and there's no need to merge.

Can you give me a link to the custom message type implementation?

andreabedini · 2013-08-21T03:00:17Z

@mdboom had implemented the same thing in matplotlib/matplotlib#2054

jasongrout · 2013-08-21T04:21:37Z

We use a bit of customization on kernel managers and the like to implement custom messages. We've had discussions with IPython devs, but never had a consensus about a plan for user-defined messages. Our registering function is at https://github.com/sagemath/sagecell/blob/master/receiver.py#L251, and our interact registration is at https://github.com/sagemath/sagecell/blob/master/interact_sagecell.py#L248. I don't know how much help it will be---we've slowly diverged with a lot of customizations from the IPython notebook kernel code.

minrk · 2013-08-21T07:54:39Z

@andreabedini if this is implemented in matplotlib, why should it be duplicated in IPython?

andreabedini · 2013-08-21T09:28:53Z

I didn't know it was. Nevertheless I don't see why it should be implemented in matplotlib. It is IPython-specific behaviour and should be implemented there, just like backend_inline. Unless matplotlib decides to provide an API for an embeddable html canvas widget.

mdboom · 2013-08-21T14:16:31Z

Thanks for doing this.

We plan to provide an embeddable html canvas widget in matplotlib for a number of reasons -- it's just incomplete work at this point. There are other webapps already that want to embed it, so I think it makes sense for it to be in matplotlib and not in IPython (so that it can support things were IPython is not part of the equation).

[I'm on vacation, so can't look at this in detail now, but it would be useful to compare notes between this and matplotlib/matplotlib#2054]

ellisonbg · 2013-08-21T17:17:42Z

I think this should not go in until we work out the new JavaScript message
spec stuff.

On Wed, Aug 21, 2013 at 7:16 AM, Michael Droettboom <
notifications@github.com> wrote:

Thanks for doing this.

We plan to provide an embeddable html canvas widget in matplotlib for a
number of reasons -- it's just incomplete work at this point. There are
other webapps already that want to embed it, so I think it makes sense for
it to be in matplotlib and not in IPython (so that it can support things
were IPython is not part of the equation).

[I'm on vacation, so can't look at this in detail now, but it would be
useful to compare notes between this and matplotlib/matplotlib#2054https://github.com/matplotlib/matplotlib/issues/2054
]

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/4077#issuecomment-23020052
.

Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger@calpoly.edu and ellisonbg@gmail.com

ellisonbg · 2013-08-21T17:18:12Z

Also, I think as much of this should be in matplotlib as possible.

On Wed, Aug 21, 2013 at 10:17 AM, Brian Granger ellisonbg@gmail.com wrote:

I think this should not go in until we work out the new JavaScript message
spec stuff.

On Wed, Aug 21, 2013 at 7:16 AM, Michael Droettboom <
notifications@github.com> wrote:

Thanks for doing this.

We plan to provide an embeddable html canvas widget in matplotlib for a
number of reasons -- it's just incomplete work at this point. There are
other webapps already that want to embed it, so I think it makes sense for
it to be in matplotlib and not in IPython (so that it can support things
were IPython is not part of the equation).

[I'm on vacation, so can't look at this in detail now, but it would be
useful to compare notes between this and matplotlib/matplotlib#2054https://github.com/matplotlib/matplotlib/issues/2054
]

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/4077#issuecomment-23020052
.

Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger@calpoly.edu and ellisonbg@gmail.com

Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger@calpoly.edu and ellisonbg@gmail.com

andreabedini · 2013-08-21T22:26:41Z

@ellisonbg is the new javascript message spec the one developed in IPEP 8 (and links, I guess)? It's hard to understand what's going on where.

@mdboom I started from your code and simplified it, so I can provide a detailed comparison, enjoy your vacation in the meantime!

ellisonbg · 2013-08-21T22:29:42Z

No, it is different. We need to catch up on writing things down - but this
stuff is still being discussed. Here's our current notes:

https://hackpad.com/IPython-Interactive-JavaScript-Widget-Design-jRD6dHFt5I8

On Wed, Aug 21, 2013 at 3:26 PM, Andrea Bedini notifications@github.comwrote:

@ellisonbg https://github.com/ellisonbg is the new javascript message
spec the one developed in IPEP 8 (and links, I guess)? It's hard to
understand what's going on where.

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/4077#issuecomment-23055408
.

Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger@calpoly.edu and ellisonbg@gmail.com

mdboom · 2013-08-26T20:30:05Z

@andreabedini: However this ends up at the end of the day, this work is very helpful, and I especially appreciate the notes you left over at matplotlib. Just so I'm clear, is this work based off of matplotlib master, or my ipython-webagg-integration branch?

On first glance, while I think some of the IPython-specific stuff may ultimately live in IPython, there is too much code here that is copied and pasted from matplotlib that should instead be reused. (get_diff_image for example). The matplotlib version is more complex, of course, because it needs to work independently of IPython as well. I think the real thing to do is to refactor what's in matplotlib now so it can be used independently of Tornado (something we need to do internally anyway to include matplotlib plots as part of an in-house Django application), and once that's done, maybe we have IPython hooks into that, but I would think those would end up quite small at the end of the day.

andreabedini · 2013-08-26T22:10:36Z

@mdboom Cheers. It's based on the master branch. You are completely right about the duplicate code: it was needed simply because mpl lacks a suitable interface. If an embeddable widget is what we are after I think it's quite possible to work on it straight away, I'll comment on matplotlib/matplotlib#2054.

ellisonbg · 2013-09-20T18:32:16Z

We have a PR open that implements the start of our widget message spec: #4195

Can we close this PR though? I really do think all matplotlib stuff should be in matplotlib - even the notebook integration.

mdboom · 2013-09-20T18:34:50Z

I agree. It belongs in matplotlib.

ellisonbg · 2013-09-20T19:50:52Z

Future work on this will be in matplotlib, closing.

Interactive matplotlib backend for notebook

ee29f7e

andreabedini mentioned this pull request Aug 21, 2013

Ipython/Webagg integration matplotlib/matplotlib#2054

Merged

ellisonbg closed this Sep 20, 2013

andreabedini deleted the backend_interactive branch May 22, 2014 01:08

Uh oh!

Conversation

andreabedini commented Aug 21, 2013

How it works

Limits:

Uh oh!

jasongrout commented Aug 21, 2013

Uh oh!

jasongrout commented Aug 21, 2013

Uh oh!

andreabedini commented Aug 21, 2013

Uh oh!

andreabedini commented Aug 21, 2013

Uh oh!

jasongrout commented Aug 21, 2013

Uh oh!

minrk commented Aug 21, 2013

Uh oh!

andreabedini commented Aug 21, 2013

Uh oh!

mdboom commented Aug 21, 2013

Uh oh!

ellisonbg commented Aug 21, 2013

Uh oh!

ellisonbg commented Aug 21, 2013

Uh oh!

andreabedini commented Aug 21, 2013

Uh oh!

ellisonbg commented Aug 21, 2013

Uh oh!

mdboom commented Aug 26, 2013

Uh oh!

andreabedini commented Aug 26, 2013

Uh oh!

ellisonbg commented Sep 20, 2013

Uh oh!

mdboom commented Sep 20, 2013

Uh oh!

ellisonbg commented Sep 20, 2013

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants