Developers for PySAL and its submodules are expected to follow Github Standard Operating Procedures.

In order for packages to be included into PySAL as a submodule, it must follow the Submodule Contract below:

Submodule Contract

Python Version

• must support python 3.6 and 3.7.

Structure

• must be organized in a hierarchical fashion. More specifically:
  • tests should be inside of the package folder, rather than outside.
  • data within tests is only referred to by tests, and is used as comparison output for testing.
  • notebooks is outside of the package folder.

Documentation & Testing

• must have unittesting on user-facing classes (those exposed by the API)
• must have docstrings for all user-facing functions
• must run these tests nightly (using either nightli.es or CRON jobs on travis)
• must have three notebooks in notebooks
• must host documentation (see the next section for guidelines of building a doc website)

Code Standards

• must refer to data in libpysal.examples
• must only have module-level imports that the package supports in its requirements.txt
• must write from submodule import function instead of import submodule.function if it wants to use a function of a PySAL submodule
• all function-level imports (those made at the top of a function) must be decorated with the requires decorator or raises an explicit exception (i.e. raise ImportError('this function requires "PACKAGE", which is not installed.)) if its requirements are not met.
• must be pep8 compliant. We recommend using autopep8 to automatically convert non-compliant code, and using flake8 thereafter.

Release Instructions

The submodules are expected to release on The Python Package Index (PyPI), github, and conda-forge. Detailed instructions of releasing a submodule can be found in the wiki page of pysal/submodule_template.

Submodule Template

The repo pysal/submodule_template provides a useful template for submodules covering:

• how to configure travis-CI dual tests: .travis.yml - assuming that libpysal is a dependency of the submodule, an example of dual travis-CI testing against libpysal is provided. - basically, the dual testing configures travis-CI to test against two versions of libpysal: pypi released stable version and the github development version.

• how to describes the metadata about the submodule: setup.py

• how to customize Sphinx input and output behavior for mimicking giddy's online documentation: doc/conf.py

• how to structure the package:

  PACKAGE_NAME/
      LICENSE.txt
      README.txt
      setup.py
      .travis.yml
      PACKAGE_NAME/
          __init__.py
      notebooks/
      requirements.txt
      requirements_tests.txt
      requirements_docs.txt
      readthedocs.yml
      doc/
          Makefile
          conf.py
          index.rst
          installation.rst
          api.rst
          references.rst
          _static/
              references.bib
              pysal-styles.css
              images/
                  pysal_favicon.ico
          _build/
  

All the files in the doc/ directory together with requirements_docs.txt and readthedocs.yml are essential for building the online docs for the submodule. More details are given below.

Building an online documentation for the submodule mimicking giddy_docs

We recommend building a documentation website for your package with:

• sphinx, the Python Documentation Generator, which semi-automatically creates beautiful documentation.
• sphinx_bootstrap_theme, a python package providing Sphinx themes which integrates the Bootstrap CSS/JavaScript framework with various layout options.
• readthedocs, which pull the code from the github repository, build documentation and host it for free. It is capable of hosting multiple versions of documentation.
• An alternative host of the documentation website to readthedocs is Github Page. The instruction of publishing the docs website with Github Page will also be given.

The workflow is briefly introduced as follows:

Set up a sphinx project

• Install sphinx, sphinx_bootstrap_theme as well as several sphinx extensions which are very useful in generating an elegant and user-friendly online documentation from pip:

  pip install sphinx sphinx_bootstrap_theme sphinxcontrib-bibtex numpydoc
  

• Instead of initializing a new sphinx project using the tool, sphinx-quickstart, developers wishing to mimic the giddy_docs can use the templates provided by the pysal/submodule_template by copying its directory doc/ (together with all the files inside) to your submodule.

  • Open the copy of doc/conf.py in your submodule, and change all the PACKAGE\_NAME to your submodule name so that the Sphinx input and output behavior of your submodule are configured.
  • Open the copy of doc/Makefile and doc/make.bat in your submodule, and change the PACKAGE\_NAME to your submodule name.

• Now the online docs should be rendered in the same fashion as giddy_docs. The next steps will be to fill in the content.

Creating documentation for Python docstrings

The pysal submodule is expected to contain two distinct forms of documentation: inline and non-inline. Inline docs are contained in the source code itself, in what are known as docstrings. Non-inline docs are written in reStructuredText and their sources (such as doc/installation.rst) are in the doc/ directory.

Non-inline Documentation

The non-inline docs should be written in reStructuredText(.rst) and their sources live in the doc/ directory. sphinx will create HTML (and LaTeX (for printable PDF versions), ePub, etc) files from the .rst sources. Currently, four .rst files are living in doc/ of pysal/submodule_template, each of which will be read and built into four HTML files by `sphinx`:

• doc/index.rst: landing page for the online documentation. Should contain a brief introduction of the submodule.
• doc/installation.rst: installation page.
• doc/api.rst: API reference page providing links to inline documentation with the help of sphinx extensions numpydoc, sphinx.ext.autodoc and sphinx.ext.autosummary.
• doc/reference.rst: bibliography page capable of cross-referencing with the help of the sphinx extension sphinxcontrib-bibtex.

To add all of them to the navigation bar menu of the online docs, the navbar_links property in doc/conf.py needs to be properly configured. Follow the same procedure to add other webpages to the online documentation of your submodule.

Inline Documentation

The following Sphinx extensions are used for parsing inline docstrings and semi-automatically pulling in documentation from docstrings:

• numpydoc
• sphinx.ext.autodoc
• sphinx.ext.autosummary

Since the extension numpydoc is used, it is recommended that the python docstrings in the submodule follow the Numpy Docstring Standard.

To utilize the sphinx extension sphinx.ext.autodoc which could import the module, and pull in documentation from docstrings in a semi-automatic way, you need to list all the classes/functions you wish to expose to users. As shown in pysal/submodule_template, we recommend organizing classes/functions and listing them in doc/api.rst. More specifically, open the copy of doc/api.rst in your submodule, and change PACKAGE\_NAME right after the directive currentmodule to your submodule. Then, replace the classes giddy.markov.Markov and giddy.markov.Spatial_Markov with the classes and functions of the submodule.

Interactive Python examples

It should be noted that the interactive Python examples in the docstring are not required, but if given, they are required to follow the doctest format and pass all doctests. They can be checked together with unit testing using `nose`: nosetests --with-doctest submodule/

If matplotlib is imported in the interactive example, the plot directive should be used for including the matplotlib plot (a .png file) in the html docs. An example is given in the Rose API of the giddy docs.

Bibliography and cross-reference

We leverage the functionality provided by the Sphinx extension for BibTeX style citations - sphinxcontrib-bibtex to quickly and conveniently build a bibliography list in the online docs. The bibliography list is customized to only show those cited in the online docs in doc/reference.rst. To properly create cross-reference and generate the bibliography list, the followings are needed:

• update doc/_static/references.bib with the relevant references of your submodule .
• change the format of citations in docstrings and no-inline documentation to follow :cite:`Press2007 in which Press2007 is the key of a bibtex entry in doc/_static/references.bib.

Generate and test html locally

Now that we have configured the behaviour of input and output of sphinx and added the content, we can run the build locally to test whether there is any issue and to see how the created html docs looks like. Since we have doc/Makefile and doc/make.bat, we can achieve that very easily by running make html:

make html

A \_build folder will be created where all the created html files live. You can double click to open the html files in a default browser to check the content and layout.

Hosting

There are two options for publishing the documentation:

• readthedocs
• github pages

Building, versioning, and hosting the docs with readthedocs

readthedocs is free and able to host and build multiple versions.

After signing in with github to readthedocs, you can click Import a Project to import the submodule and start to build the online docs. Up to now, it is possible that the docs building will fail because of some configuration issues:

• Programming Language: change the Programming Language value to python.
• Python interpreter: since the submodule supports 3.X only, you need to change the Python interpreter value from the default CPython 2.x to CPython 3.x in Admin-Advanced Settings.
• Docs building dependencies: since we are relying on the python package sphinx_bootstrap_theme to build an elegant bootstrap-Yeti website, we need to configure readthedocs to install the package when building the docs. A successful example of configuration is given with the help of a readthedocs.yml and proper configuration of tests and docs dependencies for setup.py.

Publishing the docs with Github Page

Publishing the documentation website wih Github Page can avoid the the additional efforts of configuring and debugging readthedocs as the locally built webpages will be published. Follow the procedure below to set up:

1. Assuming you already have a sphinx project for your submodule following the instructions above, and assuming the name of the folder for this project is doc, what you need to do is (1) rename doc to docsrc (2) replace the old Makefile with a new one from pysal/submodule_template (3) copy the folder docs from pysal/submodule_template to your package
2. Generating the documentation html files locally by running make html from the command line (make sure you are currently in the docsrc directory). Then run make sync to move the generated html files to the docs directory.
3. Update settings of your github repo as seen below:

Having setting up the github page for your project documentation website, you need to follow procedure 2 (make html and make sync) to update the website to reflect any changes you have made to the package.