Developers for PySAL and its submodules are expected to follow Github Standard Operating Procedures.
In order for packages to be included into PySAL as a submodule, it must follow the Submodule Contract below:
- must support python 3.6 and 3.7.
- must be organized in a hierarchical fashion. More specifically:
- tests should be inside of the package folder, rather than outside.
- data within tests is only referred to by tests, and is used as comparison output for testing.
- notebooks is outside of the package folder.
Documentation & Testing
- must have unittesting on user-facing classes (those exposed by the API)
- must have docstrings for all user-facing functions
- must run these tests nightly (using either nightli.es or CRON jobs on travis)
- must have three notebooks in
- must host documentation (see the next section for guidelines of building a doc website)
- must refer to data in
- must only have module-level imports that the package supports in its
- must write
from submodule import functioninstead of
import submodule.functionif it wants to use a function of a PySAL submodule
- all function-level imports (those made at the top of a function)
must be decorated with the requires
or raises an explicit exception (i.e.
raise ImportError('this function requires "PACKAGE", which is not installed.)) if its requirements are not met.
- must be pep8 compliant. We recommend using autopep8 to automatically convert non-compliant code, and using flake8 thereafter.
The submodules are expected to release on The Python Package Index (PyPI), github, and conda-forge. Detailed instructions of releasing a submodule can be found in the wiki page of pysal/submodule_template.
The repo pysal/submodule_template provides a useful template for submodules covering:
how to configure travis-CI dual tests: .travis.yml - assuming that
libpysalis a dependency of the submodule, an example of dual travis-CI testing against
libpysalis provided. - basically, the dual testing configures travis-CI to test against two versions of
libpysal: pypi released stable version and the github development version.
how to describes the metadata about the submodule: setup.py
how to structure the package:
PACKAGE_NAME/ LICENSE.txt README.txt setup.py .travis.yml PACKAGE_NAME/ __init__.py notebooks/ requirements.txt requirements_tests.txt requirements_docs.txt readthedocs.yml doc/ Makefile conf.py index.rst installation.rst api.rst references.rst _static/ references.bib pysal-styles.css images/ pysal_favicon.ico _build/
Building an online documentation for the submodule mimicking giddy_docs
We recommend building a documentation website for your package with:
- sphinx, the Python Documentation Generator, which semi-automatically creates beautiful documentation.
- readthedocs, which pull the code from the github repository, build documentation and host it for free. It is capable of hosting multiple versions of documentation.
- An alternative host of the documentation website to readthedocs is Github Page. The instruction of publishing the docs website with Github Page will also be given.
The workflow is briefly introduced as follows:
Set up a
sphinx_bootstrap_themeas well as several
sphinxextensions which are very useful in generating an elegant and user-friendly online documentation from
pip install sphinx sphinx_bootstrap_theme sphinxcontrib-bibtex numpydoc
Instead of initializing a new
sphinxproject using the tool, sphinx-quickstart, developers wishing to mimic the giddy_docs can use the templates provided by the pysal/submodule_template by copying its directory doc/ (together with all the files inside) to your submodule.
- Open the copy of
in your submodule, and change all the
PACKAGE\_NAMEto your submodule name so that the Sphinx input and output behavior of your submodule are configured.
- Open the copy of
in your submodule, and change the
PACKAGE\_NAMEto your submodule name.
- Open the copy of doc/conf.py in your submodule, and change all the
Now the online docs should be rendered in the same fashion as giddy_docs. The next steps will be to fill in the content.
Creating documentation for Python docstrings
The pysal submodule is expected to contain two distinct forms of
documentation: inline and non-inline. Inline docs are contained in the
source code itself, in what are known as docstrings. Non-inline docs
are written in reStructuredText and their sources (such as
are in the
The non-inline docs should be written in
their sources live in the
sphinx will create HTML (and LaTeX (for printable PDF
versions), ePub, etc) files from the .rst sources. Currently, four .rst
files are living in
each of which will be read and built into four HTML files by `sphinx`:
- doc/index.rst: landing page for the online documentation. Should contain a brief introduction of the submodule.
- doc/installation.rst: installation page.
- doc/api.rst: API reference page providing links to inline documentation with the help of sphinx extensions numpydoc, sphinx.ext.autodoc and sphinx.ext.autosummary.
- doc/reference.rst: bibliography page capable of cross-referencing with the help of the sphinx extension sphinxcontrib-bibtex.
To add all of them to the navigation bar menu of the online docs, the navbar_links property in doc/conf.py needs to be properly configured. Follow the same procedure to add other webpages to the online documentation of your submodule.
The following Sphinx extensions are used for parsing inline docstrings and semi-automatically pulling in documentation from docstrings:
To utilize the sphinx extension
which could import the module, and pull in documentation from docstrings
in a semi-automatic way, you need to list all the classes/functions you
wish to expose to users. As shown in
we recommend organizing classes/functions and listing them in
More specifically, open the copy of
in your submodule, and change
PACKAGE\_NAME right after
currentmodule to your submodule. Then, replace the
giddy.markov.Spatial_Markov with the
classes and functions of the submodule.
Interactive Python examples
It should be noted that the interactive Python examples in the docstring
are not required, but if given, they are required to follow the
doctest format and
pass all doctests. They can be checked together with unit testing using
nosetests --with-doctest submodule/
matplotlib is imported in the interactive example, the
should be used for including the matplotlib plot (a .png file) in the
html docs. An example is given in the
API of the giddy docs.
Bibliography and cross-reference
We leverage the functionality provided by the Sphinx extension for BibTeX style citations - sphinxcontrib-bibtex to quickly and conveniently build a bibliography list in the online docs. The bibliography list is customized to only show those cited in the online docs in doc/reference.rst. To properly create cross-reference and generate the bibliography list, the followings are needed:
- update doc/_static/references.bib with the relevant references of your submodule .
- change the format of citations in docstrings and no-inline
documentation to follow
Press2007is the key of a bibtex entry in doc/_static/references.bib.
Generate and test html locally
Now that we have configured the behaviour of input and output of
sphinx and added the content, we can run the build locally
to test whether there is any issue and to see how the created html docs
looks like. Since we have
we can achieve that very easily by running
\_build folder will be created where all the created
html files live. You can double click to open the html files in a
default browser to check the content and layout.
There are two options for publishing the documentation:
- github pages
Building, versioning, and hosting the docs with readthedocs
readthedocs is free and able to host and build multiple versions.
After signing in with github to readthedocs, you can click Import a Project to import the submodule and start to build the online docs. Up to now, it is possible that the docs building will fail because of some configuration issues:
- Programming Language: change the Programming Language value to
- Python interpreter: since the submodule supports 3.X only, you need
to change the Python interpreter value from the default
CPython 3.xin Admin-Advanced Settings.
- Docs building dependencies: since we are relying on the python
to build an elegant bootstrap-Yeti
website, we need to configure
readthedocsto install the package when building the docs. A successful example of configuration is given with the help of a readthedocs.yml and proper configuration of tests and docs dependencies for setup.py.
Publishing the docs with Github Page
Publishing the documentation website wih Github Page can avoid the the additional efforts of configuring and debugging readthedocs as the locally built webpages will be published. Follow the procedure below to set up:
- Assuming you already have a sphinx project for your submodule following the instructions above, and
assuming the name of the folder for this project is
doc, what you need to do is (1) rename
docsrc(2) replace the old Makefile with a new one from pysal/submodule_template (3) copy the folder
docsfrom pysal/submodule_template to your package
- Generating the documentation html files locally by running
make htmlfrom the command line (make sure you are currently in the
docsrcdirectory). Then run
make syncto move the generated html files to the
- Update settings of your github repo as seen below:
Having setting up the github page for your project documentation website, you need to follow procedure 2 (
make html and
to update the website to reflect any changes you have made to the package.