Rich Output

Rich Output#

In Python, objects can declare their textual representation using the __repr__ method. IPython expands on this idea and allows objects to declare other, rich representations including:

HTML
JSON
PNG
JPEG
SVG
LaTeX

A single object can declare some or all of these representations; all are handled by IPython’s display system. This Notebook shows how you can use this display system to incorporate a broad range of content into your Notebooks.

Basic display imports#

The display function is a general purpose tool for displaying different representations of objects. Think of it as print for these rich representations.

from IPython.display import display

A few points:

Calling display on an object will send all possible representations to the Notebook.
These representations are stored in the Notebook document.
In general the Notebook will use the richest available representation.

If you want to display a particular representation, there are specific functions for that:

from IPython.display import (
    display_pretty, display_html, display_jpeg,
    display_png, display_json, display_latex, display_svg
)

Images#

To work with images (JPEG, PNG) use the Image class.

from IPython.display import Image

i = Image(filename='images/ipython-logo.png')

Returning an Image object from an expression will automatically display it:

../../_images/468b9eed71a12cc7c5fd9209539f54308fa6136ad9d2b90f8781c9783bbfea22.png

Or you can pass an object with a rich representation to display:

display(i)

An image can also be displayed from raw data or a URL.

Image(url='http://python.org/images/python-logo.gif')

SVG images are also supported out of the box.

from IPython.display import SVG
SVG(filename='images/python_logo.svg')

../../_images/c3ba28319cf812f329c2b0a3ea8daed14cf0a07ad7c807acda8f276483258034.svg

Embedded vs non-embedded Images#

By default, image data is embedded in the notebook document so that the images can be viewed offline. However it is also possible to tell the Image class to only store a link to the image. Let’s see how this works using a webcam at Berkeley.

from IPython.display import Image
img_url = 'http://www.lawrencehallofscience.org/static/scienceview/scienceview.berkeley.edu/html/view/view_assets/images/newview.jpg'

# by default Image data are embedded
Embed      = Image(img_url)

# if kwarg `url` is given, the embedding is assumed to be false
SoftLinked = Image(url=img_url)

# In each case, embed can be specified explicitly with the `embed` kwarg
# ForceEmbed = Image(url=img_url, embed=True)

---------------------------------------------------------------------------
SSLCertVerificationError                  Traceback (most recent call last)
File /srv/conda/envs/notebook/lib/python3.9/urllib/request.py:1346, in AbstractHTTPHandler.do_open(self, http_class, req, **http_conn_args)
   1345 try:
-> 1346     h.request(req.get_method(), req.selector, req.data, headers,
   1347               encode_chunked=req.has_header('Transfer-encoding'))
   1348 except OSError as err: # timeout error

File /srv/conda/envs/notebook/lib/python3.9/http/client.py:1285, in HTTPConnection.request(self, method, url, body, headers, encode_chunked)
   1284 """Send a complete request to the server."""
-> 1285 self._send_request(method, url, body, headers, encode_chunked)

File /srv/conda/envs/notebook/lib/python3.9/http/client.py:1331, in HTTPConnection._send_request(self, method, url, body, headers, encode_chunked)
   1330     body = _encode(body, 'body')
-> 1331 self.endheaders(body, encode_chunked=encode_chunked)

File /srv/conda/envs/notebook/lib/python3.9/http/client.py:1280, in HTTPConnection.endheaders(self, message_body, encode_chunked)
   1279     raise CannotSendHeader()
-> 1280 self._send_output(message_body, encode_chunked=encode_chunked)

File /srv/conda/envs/notebook/lib/python3.9/http/client.py:1040, in HTTPConnection._send_output(self, message_body, encode_chunked)
   1039 del self._buffer[:]
-> 1040 self.send(msg)
   1042 if message_body is not None:
   1043 
   1044     # create a consistent interface to message_body

File /srv/conda/envs/notebook/lib/python3.9/http/client.py:980, in HTTPConnection.send(self, data)
    979 if self.auto_open:
--> 980     self.connect()
    981 else:

File /srv/conda/envs/notebook/lib/python3.9/http/client.py:1454, in HTTPSConnection.connect(self)
   1452     server_hostname = self.host
-> 1454 self.sock = self._context.wrap_socket(self.sock,
   1455                                       server_hostname=server_hostname)

File /srv/conda/envs/notebook/lib/python3.9/ssl.py:500, in SSLContext.wrap_socket(self, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname, session)
    494 def wrap_socket(self, sock, server_side=False,
    495                 do_handshake_on_connect=True,
    496                 suppress_ragged_eofs=True,
    497                 server_hostname=None, session=None):
    498     # SSLSocket class handles server_hostname encoding before it calls
    499     # ctx._wrap_socket()
--> 500     return self.sslsocket_class._create(
    501         sock=sock,
    502         server_side=server_side,
    503         do_handshake_on_connect=do_handshake_on_connect,
    504         suppress_ragged_eofs=suppress_ragged_eofs,
    505         server_hostname=server_hostname,
    506         context=self,
    507         session=session
    508     )

File /srv/conda/envs/notebook/lib/python3.9/ssl.py:1040, in SSLSocket._create(cls, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname, context, session)
   1039             raise ValueError("do_handshake_on_connect should not be specified for non-blocking sockets")
-> 1040         self.do_handshake()
   1041 except (OSError, ValueError):

File /srv/conda/envs/notebook/lib/python3.9/ssl.py:1309, in SSLSocket.do_handshake(self, block)
   1308         self.settimeout(None)
-> 1309     self._sslobj.do_handshake()
   1310 finally:

SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)

During handling of the above exception, another exception occurred:

URLError                                  Traceback (most recent call last)
Input In [14], in <module>
      2 img_url = 'http://www.lawrencehallofscience.org/static/scienceview/scienceview.berkeley.edu/html/view/view_assets/images/newview.jpg'
      4 # by default Image data are embedded
----> 5 Embed      = Image(img_url)
      7 # if kwarg `url` is given, the embedding is assumed to be false
      8 SoftLinked = Image(url=img_url)

File /srv/conda/envs/notebook/lib/python3.9/site-packages/IPython/core/display.py:952, in Image.__init__(self, data, url, filename, format, embed, width, height, retina, unconfined, metadata, alt)
    950 self.unconfined = unconfined
    951 self.alt = alt
--> 952 super(Image, self).__init__(data=data, url=url, filename=filename,
    953         metadata=metadata)
    955 if self.width is None and self.metadata.get('width', {}):
    956     self.width = metadata['width']

File /srv/conda/envs/notebook/lib/python3.9/site-packages/IPython/core/display.py:327, in DisplayObject.__init__(self, data, url, filename, metadata)
    324 elif self.metadata is None:
    325     self.metadata = {}
--> 327 self.reload()
    328 self._check_data()

File /srv/conda/envs/notebook/lib/python3.9/site-packages/IPython/core/display.py:987, in Image.reload(self)
    985 """Reload the raw data from file or URL."""
    986 if self.embed:
--> 987     super(Image,self).reload()
    988     if self.retina:
    989         self._retina_shape()

File /srv/conda/envs/notebook/lib/python3.9/site-packages/IPython/core/display.py:357, in DisplayObject.reload(self)
    354 elif self.url is not None:
    355     # Deferred import
    356     from urllib.request import urlopen
--> 357     response = urlopen(self.url)
    358     data = response.read()
    359     # extract encoding from header, if there is one:

File /srv/conda/envs/notebook/lib/python3.9/urllib/request.py:214, in urlopen(url, data, timeout, cafile, capath, cadefault, context)
    212 else:
    213     opener = _opener
--> 214 return opener.open(url, data, timeout)

File /srv/conda/envs/notebook/lib/python3.9/urllib/request.py:523, in OpenerDirector.open(self, fullurl, data, timeout)
    521 for processor in self.process_response.get(protocol, []):
    522     meth = getattr(processor, meth_name)
--> 523     response = meth(req, response)
    525 return response

File /srv/conda/envs/notebook/lib/python3.9/urllib/request.py:632, in HTTPErrorProcessor.http_response(self, request, response)
    629 # According to RFC 2616, "2xx" code indicates that the client's
    630 # request was successfully received, understood, and accepted.
    631 if not (200 <= code < 300):
--> 632     response = self.parent.error(
    633         'http', request, response, code, msg, hdrs)
    635 return response

File /srv/conda/envs/notebook/lib/python3.9/urllib/request.py:555, in OpenerDirector.error(self, proto, *args)
    553     http_err = 0
    554 args = (dict, proto, meth_name) + args
--> 555 result = self._call_chain(*args)
    556 if result:
    557     return result

File /srv/conda/envs/notebook/lib/python3.9/urllib/request.py:494, in OpenerDirector._call_chain(self, chain, kind, meth_name, *args)
    492 for handler in handlers:
    493     func = getattr(handler, meth_name)
--> 494     result = func(*args)
    495     if result is not None:
    496         return result

File /srv/conda/envs/notebook/lib/python3.9/urllib/request.py:747, in HTTPRedirectHandler.http_error_302(self, req, fp, code, msg, headers)
    744 fp.read()
    745 fp.close()
--> 747 return self.parent.open(new, timeout=req.timeout)

File /srv/conda/envs/notebook/lib/python3.9/urllib/request.py:523, in OpenerDirector.open(self, fullurl, data, timeout)
    521 for processor in self.process_response.get(protocol, []):
    522     meth = getattr(processor, meth_name)
--> 523     response = meth(req, response)
    525 return response

File /srv/conda/envs/notebook/lib/python3.9/urllib/request.py:632, in HTTPErrorProcessor.http_response(self, request, response)
    629 # According to RFC 2616, "2xx" code indicates that the client's
    630 # request was successfully received, understood, and accepted.
    631 if not (200 <= code < 300):
--> 632     response = self.parent.error(
    633         'http', request, response, code, msg, hdrs)
    635 return response

File /srv/conda/envs/notebook/lib/python3.9/urllib/request.py:555, in OpenerDirector.error(self, proto, *args)
    553     http_err = 0
    554 args = (dict, proto, meth_name) + args
--> 555 result = self._call_chain(*args)
    556 if result:
    557     return result

File /srv/conda/envs/notebook/lib/python3.9/urllib/request.py:494, in OpenerDirector._call_chain(self, chain, kind, meth_name, *args)
    492 for handler in handlers:
    493     func = getattr(handler, meth_name)
--> 494     result = func(*args)
    495     if result is not None:
    496         return result

File /srv/conda/envs/notebook/lib/python3.9/urllib/request.py:747, in HTTPRedirectHandler.http_error_302(self, req, fp, code, msg, headers)
    744 fp.read()
    745 fp.close()
--> 747 return self.parent.open(new, timeout=req.timeout)

File /srv/conda/envs/notebook/lib/python3.9/urllib/request.py:517, in OpenerDirector.open(self, fullurl, data, timeout)
    514     req = meth(req)
    516 sys.audit('urllib.Request', req.full_url, req.data, req.headers, req.get_method())
--> 517 response = self._open(req, data)
    519 # post-process response
    520 meth_name = protocol+"_response"

File /srv/conda/envs/notebook/lib/python3.9/urllib/request.py:534, in OpenerDirector._open(self, req, data)
    531     return result
    533 protocol = req.type
--> 534 result = self._call_chain(self.handle_open, protocol, protocol +
    535                           '_open', req)
    536 if result:
    537     return result

File /srv/conda/envs/notebook/lib/python3.9/urllib/request.py:494, in OpenerDirector._call_chain(self, chain, kind, meth_name, *args)
    492 for handler in handlers:
    493     func = getattr(handler, meth_name)
--> 494     result = func(*args)
    495     if result is not None:
    496         return result

File /srv/conda/envs/notebook/lib/python3.9/urllib/request.py:1389, in HTTPSHandler.https_open(self, req)
   1388 def https_open(self, req):
-> 1389     return self.do_open(http.client.HTTPSConnection, req,
   1390         context=self._context, check_hostname=self._check_hostname)

File /srv/conda/envs/notebook/lib/python3.9/urllib/request.py:1349, in AbstractHTTPHandler.do_open(self, http_class, req, **http_conn_args)
   1346         h.request(req.get_method(), req.selector, req.data, headers,
   1347                   encode_chunked=req.has_header('Transfer-encoding'))
   1348     except OSError as err: # timeout error
-> 1349         raise URLError(err)
   1350     r = h.getresponse()
   1351 except:

URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)>

Here is the embedded version. Note that this image was pulled from the webcam when this code cell was originally run and stored in the Notebook. Unless we rerun this cell, this is not todays image.

Embed

../../_images/86ce2b55d3b58dc7487eeeddf97680fd92b66284044adb8f447bbbfb3380ead0.jpg

Here is today’s image from same webcam at Berkeley, (refreshed every minutes, if you reload the notebook), visible only with an active internet connection, that should be different from the previous one. Notebooks saved with this kind of image will be smaller and always reflect the current version of the source, but the image won’t display offline.

SoftLinked

Of course, if you re-run this Notebook, the two images will be the same again.

HTML#

Python objects can declare HTML representations that will be displayed in the Notebook. If you have some HTML you want to display, simply use the HTML class.

from IPython.display import HTML

s = """<table>
<tr>
<th>Header 1</th>
<th>Header 2</th>
</tr>
<tr>
<td>row 1, cell 1</td>
<td>row 1, cell 2</td>
</tr>
<tr>
<td>row 2, cell 1</td>
<td>row 2, cell 2</td>
</tr>
</table>"""

h = HTML(s)

display(h)

Header 1	Header 2
row 1, cell 1	row 1, cell 2
row 2, cell 1	row 2, cell 2

You can also use the %%html cell magic to accomplish the same thing.

%%html
<table>
<tr>
<th>Header 1</th>
<th>Header 2</th>
</tr>
<tr>
<td>row 1, cell 1</td>
<td>row 1, cell 2</td>
</tr>
<tr>
<td>row 2, cell 1</td>
<td>row 2, cell 2</td>
</tr>
</table>

Header 1	Header 2
row 1, cell 1	row 1, cell 2
row 2, cell 1	row 2, cell 2

JavaScript#

The Notebook also enables objects to declare a JavaScript representation. At first, this may seem odd as output is inherently visual and JavaScript is a programming language. However, this opens the door for rich output that leverages the full power of JavaScript and associated libraries such as d3.js for output.

from IPython.display import Javascript

Pass a string of JavaScript source code to the JavaScript object and then display it.

js = Javascript('alert("hi")');

display(js)

The same thing can be accomplished using the %%javascript cell magic:

%%javascript

alert("hi");

LaTeX#

The IPython display system also has builtin support for the display of mathematical expressions typeset in LaTeX, which is rendered in the browser using MathJax.

You can pass raw LaTeX test as a string to the Math object:

from IPython.display import Math
Math(r'F(k) = \int_{-\infty}^{\infty} f(x) e^{2\pi i k} dx')

\[\displaystyle F(k) = \int_{-\infty}^{\infty} f(x) e^{2\pi i k} dx\]

With the Latex class, you have to include the delimiters yourself. This allows you to use other LaTeX modes such as eqnarray:

from IPython.display import Latex
Latex(r"""\begin{eqnarray}
\nabla \times \vec{\mathbf{B}} -\, \frac1c\, \frac{\partial\vec{\mathbf{E}}}{\partial t} & = \frac{4\pi}{c}\vec{\mathbf{j}} \\
\nabla \cdot \vec{\mathbf{E}} & = 4 \pi \rho \\
\nabla \times \vec{\mathbf{E}}\, +\, \frac1c\, \frac{\partial\vec{\mathbf{B}}}{\partial t} & = \vec{\mathbf{0}} \\
\nabla \cdot \vec{\mathbf{B}} & = 0 
\end{eqnarray}""")

\[\begin{split}\begin{eqnarray} \nabla \times \vec{\mathbf{B}} -\, \frac1c\, \frac{\partial\vec{\mathbf{E}}}{\partial t} & = \frac{4\pi}{c}\vec{\mathbf{j}} \\ \nabla \cdot \vec{\mathbf{E}} & = 4 \pi \rho \\ \nabla \times \vec{\mathbf{E}}\, +\, \frac1c\, \frac{\partial\vec{\mathbf{B}}}{\partial t} & = \vec{\mathbf{0}} \\ \nabla \cdot \vec{\mathbf{B}} & = 0 \end{eqnarray}\end{split}\]

Or you can enter LaTeX directly with the %%latex cell magic:

%%latex
\begin{align}
\nabla \times \vec{\mathbf{B}} -\, \frac1c\, \frac{\partial\vec{\mathbf{E}}}{\partial t} & = \frac{4\pi}{c}\vec{\mathbf{j}} \\
\nabla \cdot \vec{\mathbf{E}} & = 4 \pi \rho \\
\nabla \times \vec{\mathbf{E}}\, +\, \frac1c\, \frac{\partial\vec{\mathbf{B}}}{\partial t} & = \vec{\mathbf{0}} \\
\nabla \cdot \vec{\mathbf{B}} & = 0
\end{align}

\[\begin{split}\begin{align} \nabla \times \vec{\mathbf{B}} -\, \frac1c\, \frac{\partial\vec{\mathbf{E}}}{\partial t} & = \frac{4\pi}{c}\vec{\mathbf{j}} \\ \nabla \cdot \vec{\mathbf{E}} & = 4 \pi \rho \\ \nabla \times \vec{\mathbf{E}}\, +\, \frac1c\, \frac{\partial\vec{\mathbf{B}}}{\partial t} & = \vec{\mathbf{0}} \\ \nabla \cdot \vec{\mathbf{B}} & = 0 \end{align}\end{split}\]

Audio#

IPython makes it easy to work with sounds interactively. The Audio display class allows you to create an audio control that is embedded in the Notebook. The interface is analogous to the interface of the Image display class. All audio formats supported by the browser can be used. Note that no single format is presently supported in all browsers.

from IPython.display import Audio
Audio(url="http://www.nch.com.au/acm/8k16bitpcm.wav")

A NumPy array can be auralized automatically. The Audio class normalizes and encodes the data and embeds the resulting audio in the Notebook.

For instance, when two sine waves with almost the same frequency are superimposed a phenomena known as beats occur. This can be auralised as follows:

import numpy as np
max_time = 3
f1 = 220.0
f2 = 224.0
rate = 8000.0
L = 3
times = np.linspace(0, L, int(rate*L))
signal = np.sin(2*np.pi*f1*times) + np.sin(2*np.pi*f2*times)

Audio(data=signal, rate=rate)

Video#

More exotic objects can also be displayed, as long as their representation supports the IPython display protocol. For example, videos hosted externally on YouTube are easy to load:

from IPython.display import YouTubeVideo
YouTubeVideo('sjfsUzECqK0')

Using the nascent video capabilities of modern browsers, you may also be able to display local videos. At the moment this doesn’t work very well in all browsers, so it may or may not work for you; we will continue testing this and looking for ways to make it more robust.

The following cell loads a local file called animation.m4v, encodes the raw video as base64 for http transport, and uses the HTML5 video tag to load it. On Chrome 15 it works correctly, displaying a control bar at the bottom with a play/pause button and a location slider.

from IPython.display import HTML
from base64 import b64encode
video = open("images/animation.m4v", "rb").read()
video_encoded = b64encode(video).decode('ascii')
video_tag = '<video controls alt="test" src="data:video/x-m4v;base64,{0}">'.format(video_encoded)
HTML(data=video_tag)

External sites#

You can even embed an entire page from another site in an iframe; for example this is today’s Wikipedia page for mobile users:

from IPython.display import IFrame
IFrame('https://jupyter.org', width='100%', height=350)

Links to local files#

IPython provides builtin display classes for generating links to local files. Create a link to a single file using the FileLink object:

from IPython.display import FileLink, FileLinks
FileLink('01-Intro-Jupyter.ipynb')

01-Intro-Jupyter.ipynb

Alternatively, to generate links to all of the files in a directory, use the FileLinks object, passing '.' to indicate that we want links generated for the current working directory. Note that if there were other directories under the current directory, FileLinks would work in a recursive manner creating links to files in all sub-directories as well.

FileLinks('.')

Rich output and security#

The IPython Notebook allows arbitrary code execution in both the IPython kernel and in the browser, though HTML and JavaScript output. More importantly, because IPython has a JavaScript API for running code in the browser, HTML and JavaScript output can actually trigger code to be run in the kernel. This poses a significant security risk as it would allow IPython Notebooks to execute arbitrary code on your computers.

To protect against these risks, the IPython Notebook has a security model that specifies how dangerous output is handled. Here is a short summary:

When you run code in the Notebook, all rich output is displayed.
When you open a notebook, rich output is only displayed if it doesn’t contain security vulberabilities, …
… or if you have trusted a notebook, all rich output will run upon opening it.

A full description of the IPython security model can be found on this page.

Rich output and nbviewer#

Much of the power of the Notebook is that it enables users to share notebooks with each other using http://nbviewer.ipython.org, without installing IPython locally. As of IPython 2.0, notebooks rendered on nbviewer will display all output, including HTML and JavaScript.