Packaging#

In our previous lecture about modularization, we already discussed the concept of modules and how to import them. A package is a collection of modules that we can now import together and document separately. For this lecture, we are going to follow the following minimal Python package, mytoy. You can clone this repository in you local machine:

git clone https://github.com/fperez/mytoy.git

This repository contains a minimal, toy Python package with a few files as illustration for students of how to lay out their code to meet minimal Python packaging requirements.

It has a single source directory (mytoy) with an __init__.py file and one “implementation” file (toys.py) as well as a few tests in mytoy/tests.

In addition to this README.md it includes some basic infrastructure: LICENSE, requirements.txt, setup.py and .gitignore files.

The only docs included are this README.md file - a larger package would have a proper docs directory and associated Sphinx/JupyterBook build.

This is more or less the absolute minimum for a “real” python package that can be installed from source, tested and experimented with on Binder. For a more “official” version of this same idea, see the PyPA sample project repo, documented in detail in the Packaging Tutorial. The Python Packaging User Guide contains comprehensive documentation on this topic.

Let’s now disentangled some of the contents in mytoy. Let’s begin with some simple example.

1. Import a package#

Let’s begging by creating a simple analysis notebook Analysis.ipynb. Inside the notebook we make an import of a new package called mytoy with the syntax:

import mytoy

where in the same folder we have a folder called my toy with the following contents. You folder should like something like this:

The actual Python code is contained in the .py files, where we include some basic functions, classes and other objects we want to import. In order to the import to work, we need to create the index file __init__.py with the following contents:

"""
My Toy package
"""

__version__ = "0.0.1"

from .toys import *

The first part of the script is the docstring we already discussed in the lecture about documentation. The __version__ variable follows the convention of specifying the version number with a sequence of three digits, labeled as major, minor and patch (more information about the convention of these three digits in the Semantic Version documentation).

Note

In the case of being constantly making changes to your package code, you can include the autoreload command so the changes in the package are updated in the notebook without the need of restarting the kernel

%load_ext autoreload
%autoreload 2

2. Making a package installable#

Now, in order to import mytoy we need to be in the folder where mytoy lives. However, we want mytoy to be available everywhere where Python is available, in the same way we can import numpy and matplotlib from every notebook and Python session.

In order to make a package installable, we need to include these three files inside mytoy

  • pyproj.toml

  • setup.py

  • setup.cfg

For the first two files, you can use the same ones that are available inside mytoy without the need of making any change on them. The file setup.cfg needs to be customizable based on the specifics of the package. This is an example of how the setup.cfg file in mytoy looks like:

# Declarative configuration for setup.py

# For more details on this format, see the official docs here:
# https://setuptools.pypa.io/en/latest/userguide/declarative_config.html
# And a useful annotated template can be found here:
# https://gist.github.com/althonos/6914b896789d3f2078d1e6237642c35c

[metadata]
name             = mytoy
version          = attr: mytoy.__version__
author           = My Name
author_email     = me@myemail.com
description      = A Python library to make toys
long_description = file: README.md, LICENSE
long_description_content_type = text/markdown
keywords         = tools, toys
license          = BSD 3-Clause License
classifiers      =
	Programming Language :: Python :: 3
    License :: OSI Approved :: BSD License
    Operating System :: OS Independent

[options]
include_package_data = True
packages = find:
# These should be consistent with what is specified in the environment.yml
python_requires  = >= 3.6,
install_requires =
	tqdm

[options.packages.find]
exclude =
    examples*
    docs*

Notice that to your package you can also include a LICENCE file.

Note

The license is what specifies the condition under which you code can or cannot be used by someone else. Just by putting code in GitHub doesn’t imply that code is open source. That extra step requires you to specify the type of license. Notice that when you create a new repository in GitHub, one of the available options is to create the repository with a given license. Some examples of popular licenses are

  • BSD (Berkeley Software Distribution) / MIT.

  • GPL (GNU Public License) or copy-left licenses.

  • LGPL (GNU Lesser General Public License)

Here you can find the recommendation of UCB regarding which license to use.

Once these files are present in your package folder, you can install the Python package using pip. From the folder where mytoy lives and you have the configuration files, you can run from a terminal

pip install .

and this will install mytoy. After you complete the installation, you can import mytoy from any Python session (for example, now you can open python from a terminal and make a import mytoy from there).

Note

You can also install in development mode with the editable flag

pip install -e .

This is useful when we are making changes in the package as we make progress. If we don’t do this, every time we make a change in the library we need to uninstall and install again in order to those changes to be reflected.

3. Running tests#

As we mentioned in previous lectures, a very important element of writing code is having easy way of testing it. Just as we did using pytest, we can automatically run the tests included inside a Python package. After you had installed mytoy, you can run all the tests located inside the folder tests by running

pytest mytoy

Notice that you can run this command from any directory in your system.

4. Publishing our package#

You can eventually publish your code in the Python Package Index. We are not going to do that here, but from the structure of mytoy it’s easy to make a package available for other users to install using some package management system.

An example of this is the small jupitee project, that you can directly install using pip install jupytee (see the associated PyPI documentation documentation for extra information).