About this project

This page contains information to help you learn a bit more about the cc-pydata project and how it is configured.

  • If your goal is to learn more about the Cookiecutter tool used as the basis for the cc-pydata project, your best source of information will be the official Cookiecutter documentation.

  • If you just want to know what the cc-pydata project is, please see the README documentation for this project.

  • If you want to (a) learn more about the how to generate or use the resulting cc-pydata template or (b) better understand the features and functionality embedded in the resulting cc-pydata template, please see the Tutorial for this project.

On this page

Development philosophy

As is discussed in the Design decisions section of the cc-pydata documentation, I have sought to balance Python best practices and standards with my own needs across a wide range of projects. What’s more, I’ve also sought to embed structural characteristics of other great Cookiecutter templates (see Inspiration and sources).

Starting from those initial principals, I hope to have built a Cookiecutter template that is flexible and robust enough for others to find useful for their own Python data science projects.

I have taken the time to carefully test and document the cc-pydata project, both for my own sake, as well as for the sake of others who might find this project and wish to use it themselves.

I hope that the Tutorial for this project not only enlightens the user, but provides (a) the blueprint needed for those who may not be familiar with all of the tools and methods employed in the template, and (b) enough of a foundation for those who wish to fork and modify the project to better suit their own needs.

License

The cc-pydata Cookiecutter template is offered for use under the MIT open source license.

The content of this license is shown below and can also be found in the LICENSE file contained in the project repository on GitHub:

The MIT License (MIT)

Copyright (c) 2020 Michael Sedelmeyer

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Contributing

As is mentioned above in the Development philosophy section of this page, I have sought to balance best practices and features with my own particular needs. Therefore, I am not actively seeking contributions to the cc-pydata project from others. My primary reason being that, as soon as others begin adding features to this project, the more likely it will be that the template no longer fits my specific needs.

If however, you do see opportunities to improve this project and its resulting template, I am still interested in hearing from you.

If you’d like to suggest changes or to get in touch regarding the cc-pydata project, please feel free to open an issue and we can discuss.

I am not quite sure how firm I am on my position of “no contributions.” Therefore, if you have great ideas on how to improve this project…there’s a very good chance that I can be convinced!

Project testing and test API

It has been my experience that testing a Cookiecutter template presents its own set of unique challenges.

Not only do you need to:

  1. Test that the template renders, but you also need to…

  2. Test that the template renders appropriately according to whatever sorts of build options you provide when invoking the template, and you also need to…

  3. Test that the default functionality baked into template after it’s been rendered functions appropriately as well (i.e. you need to perform “tests within tests”).

Thus far, I have been unable to find any clear documentation online on how to do this correctly.

Therefore, I am providing more detail than might otherwise be warranted on how I have configured my cc-pydata project tests to deal with (what I can only imagine are) common challenges encountered by anyone who has ever sought to thoroughly test their own custom Cookiecutter template.

Using pipenv to install the required development dependencies

Prior to running any of the tests outlined below, you will need to first install the Pipenv virtual environment required for development of the cc-pydata Cookiecutter template.

For details regarding Pipenv and its usage, please see the packaging section section of the Tutorial.

Assuming you already have Pipenv installed on your development machine, you should be able to install the required dependencies and virtual environment by running the following commands from the cc-pydata repository’s base directory:

pipenv install --dev
pipenv shell

Once you are working from your active pipenv shell, you will be able to run all other testing commands outlined in the sections below.

Testing tools employed for this project

I have sought to develop a thorough set of tests to ensure that the cc-pydata project and resulting template function as expected. Programmatic tests for this project can be found in the tests directory, where I seek to test each unique feature of this template.

Tests for this project occur in several ways.

  1. There are the programmatic tests using Python’s standard unittest library to author test cases for each template feature I wish to test.

    • My governing principal here, is that I should not add any additional functionality into the template, or the means by which the overarching cc-pydata project renders the template, without also developing a test (or set of tests) to test that functionality.

    • There are additionally unit tests built into the rendered template produced by this project. Those in-template tests ensure that the baseline package provided in that rendered template functions correctly (presenting the opportunity for tests within tests).

  2. There are a set of automated tests configured using tox to ensure that the cc-pydata project functions correctly on several different versions of Python (those versions are python 3.6, python 3.7, and python 3.8 as of cc-pydata==v0.3.0 at the time of my writing this).

    • This tox configuration also runs a documentation test build to ensure that the cc-pydata Sphinx-based documentation renders successfully, and it runs a linter to ensure that the project code meets PEP 8 standards.

    • Also, dependent on whether the tox option is enabled for the rendered cc-pydata template, the rendered template itself will also contain a tox-automated test-configuration. That in-template tox configuration will perform similar tasks for the resulting rendered template (and here is our true layer of tests within tests).

  3. Lastly, continuous integration is implemented using Travis-CI (and soon to also be implemented using GitHub Actions) to run and test automated builds each time committed revisions to the cc-pydata project are pushed to the GitHub-hosted remote develop or master branches for the project.

tox test matrix and automation

If you are not familiar with Python’s test automation tool Tox, learning to use it is well worth the investment in time.

If you review the tox.ini configuration file contained in the cc-pydata project repository, you will see that tox automation for this project is configured to:

  1. Run tests on several different versions of Python to ensure compatibility with each of those versions;

  2. Run a test build of the cc-pydata project’s Sphinx documentation to ensure docs build successfully;

  3. Run a flake8 linting test to ensure all of the Python syntax in this project meets PEP 8 standards; and…

  4. Run a full build of the cc-pydata default template and then run that rendered template’s own automated tox tests (see details concerning the tests.toxtest test module outlined in this section)

To run these automated tox tests, you simply run the tox command from within your active pipenv development environment.

Additionally, you can run individual tox environments (instead of all at once) by explcitly specifying the environment you wish to run, such as:

tox -e docs

Continuous integration test builds with Travis-CI

Continuous integration (CI) build tests are set to run via Travis-CI every time a change is pushed to either the master or develop branches.

These CI tests ensure that the cc-pydata tox-automated test matrix runs successfully on a Linux system.

Please see the cc-pydata project’s .travis.yml configuration file for more detail.

Note

There are currently plans to migrate this CI automation from Travis-CI over to GitHub’s native GitHub Actions service.

  • The primary reason for this planned change is that GitHub Actions offers Windows OS images for CI testing, while Travis-CI does not.

  • During this planned CI service migration, MacOS builds will also be added to the CI build matrix.

  • I occasionally use all three of these operating systems for my development work and would appreciate the added assurance that my project and rendered templates run successfully on all three platforms.

Custom tests module using unittest and the pytest test-runner

To run unit tests for the cc-pydata project, enter your development environment by running pipenv shell and invoke the pytest test-runner by running the following command from your top-level project directory:

pytest

You should see output similar to this:

============================ test session starts =============================
platform linux -- Python 3.7.5, pytest-5.4.3, py-1.8.2, pluggy-0.13.1
rootdir: /home/Code/cookiecutter-pydata, inifile: setup.cfg, testpaths: tests
plugins: cov-2.10.0
collected 28 items

tests/test_defaults.py ..........                                       [ 35%]
tests/test_options.py .........                                         [ 67%]
tests/test_testutils.py .........                                       [100%]

============================= 28 passed in 5.55s =============================

Testing rendered Cookiecutter templates

Before getting into the details of the tests test module, it is probably worth acknowledging that:

  1. There exists a pytest plugin for testing Cookiecutter templates. That plug-in is named pytest-cookies, and it provides a boilerplate-free experience for building and testing Cookiecutter templates.

  2. However, I DO NOT use that plug-in in any way for testing this project.

If you’d like to learn more about the pytest-cookies plugin for your own use, please see that project’s documentation.

While I use pytest as the test-runner for this project, I do not use the pytest framework for writing my tests. I have attempted to keep my tests written entirely using unittest from the Python standard library. This approach requires a bit more boilerplate in my test code, but it also helps to ensure that I am not locked into pytest as a testing requirement. Besides, I have also found a simple-enough approach to building and testing my Cookiecutter template using just unittest test cases. As a result, I haven’t felt a need to use pytest or the pytest-cookies plug-in.

If you examine the TestCase classes in the tests.test_default and tests.test_options test modules, you will see that each TestCase contains a setUp() method that:

  1. Uses the contextlib.ExitStack() context manager to generate a tempfile.TemporaryDirectory() temporary directory in which I can build each of my cc-pydata rendered templates for testing; and…

  2. Contains a call to self.addCleanup(stack.pop_all().close) to ensure the temporary directory gets cleaned up after each test, regardless of whether the test setup fails in any way.

Each test case additionally uses cookiecutter.main.cookiecutter(), the main entry point to the cookiecutter command, which allows you to easily initiate a template build directly from your Python code.

By putting these pieces together, cc-pydata template builds become predictable and easy to manage for testing.

The structure of the tests module

As you will see in the tests module API documentation below, cc-pydata tests are split among several submodules.

  1. tests: At the highest level are a set of utility functions that make testing easier and reduce boilerplate code in each test case. These utility functions themselves are tested in tests.test_testutils.

  2. tests.test_defaults: This sub-module contains a set of tests focused on the default cc-pydata template build.

  3. tests.test_options: This sub-module focuses on testing versions of the cc-pydata template produced when using the optional arguments available when rendering the template.

  4. tests.toxtest: Finally, this is a sub-module that only runs when explicitly called using the command pytest -s tests/test_toxtest.py.

    • tests.toxtest contains a costly test to run because, it not only renders the cc-pydata template, but it also invokes that template’s own tox.ini to run all of its own tox environments.

    • Think of it as tests-within-tests-within-tests, all running in their own embedded temporary environments.

    • This sub-module can take several minutes to run, so considered yourself warned.

    • See tests.toxtest for more detail.

Located below is the full API documentation for each of these modules.

API documentation for the tests module

tests

This module contains global variables and utility functions required for testing the cc-pydata cookiecutter template.

Unit tests for these functions can be found in tests.test_testutils module

Module Variables:

CCDIR

Define absolute path to cc-pydata cookiecutter project directory

CCJSON

Define path to cookiecutter.json default choice variables file

JINJA_REGEX

Define regex string required to identify all jinja-related brackets

Module Functions:

bake_cookiecutter_template(output_dir[, …])

Generate the cookiecutter template defined in this project repository

find_jinja_brackets(string[, regex])

Find all instances of input string that contains jinja brackets

get_default_template_args([filepath])

Load cookiecutter.json to a dictionary object with rendered jinja

working_directory(directory)

Change working directory temporarily with context manager

tests.CCDIR = '/home/sedelmeyer/Code/cc-pydata'

Define absolute path to cc-pydata cookiecutter project directory

tests.CCJSON = '/home/sedelmeyer/Code/cc-pydata/cookiecutter.json'

Define path to cookiecutter.json default choice variables file

tests.JINJA_REGEX = '(\\{{|\\}}|\\{%|\\%}|\\{#|\\#})'

Define regex string required to identify all jinja-related brackets

tests.bake_cookiecutter_template(output_dir, template='/home/sedelmeyer/Code/cc-pydata', extra_context=None)

Generate the cookiecutter template defined in this project repository

Parameters

  • output_dir (str) – directory path in which to render the template

  • template (str) – name of cookiecutter template, defaults to CCDIR

  • extra_context (dict) – dictionary of non-default arguments for cookiecutter template build, defaults to None

Returns

path to rendered cookiecutter template directory

Return type

str

tests.find_jinja_brackets(string, regex='(\\{{|\\}}|\\{%|\\%}|\\{#|\\#})')

Find all instances of input string that contains jinja brackets

Parameters

  • string (str) – text within which to search for jinja brackets

  • regex (str) – regular expression pattern, defaults to JINJA_REGEX

Returns

regex search result, True if pattern found, None if not

Return type

bool, None

tests.get_default_template_args(filepath='/home/sedelmeyer/Code/cc-pydata/cookiecutter.json')

Load cookiecutter.json to a dictionary object with rendered jinja

Parameters

filepath (str) – filepath to cookiecutter.json file, defaults to CCJSON

Returns

dictionary of cookiecutter default arguments

Return type

dict

tests.read_template_file(builtdir, filename)

Read the contents of a file contained in the baked cookiecutter template

Parameters

  • builtdir (str) – file path to the rendered cookiecutter template

  • filename (str) – name of the target file within the cookiecutter template

Returns

text content contained within the target file

Return type

str

tests.working_directory(directory)

Change working directory temporarily with context manager

Parameters

directory (str) – path to desired working directory

tests.test_testutils

This module contains unit tests for utility functions in the tests module

class tests.test_testutils.TestTestsUtilityFunctions(methodName='runTest')

Test utility functions contained in the tests/__init__.py file

Ensure the recipe renders a cookiecutter template as expected

test_find_jinja_brackets()

Ensure find_jinja_brackets finds all jinja bracket types

test_find_jinja_brackets_ignores_json()

Ensure find_jinja_brackets ignores json and dict brackets

test_fix_cookiecutter_jinja_var_nonstring()

Ensure function returns input value when not of type str

test_fix_cookiecutter_jinja_var_string()

Ensure function removes ‘cookiecutter.’ substring only

test_get_default_template_args()

Ensure get_default_template_args reads json to a dictionary

test_read_template_file()

Ensure read_template_file returns file content

test_render_json_dict_jinja()

Ensure only double curley-bracket jinja values are rendered

test_working_directory()

Ensure working_directory context manager works as expected

tests.test_defaults

This module contains tests for the default cc-pydata template build

class tests.test_defaults.TestBuildDefaultTemplate(methodName='runTest')

Test default cc-pydata cookiecutter template build

setUp()

Render default cc-pydata template in temporary directory

test_cli_argparse_works()

Ensure template’s CLI default argparse function works

test_default_docs_build()

Ensure default Sphinx docs build in rendered template

test_default_docs_make_html()

Ensure default Sphinx docs build in rendered template

test_default_tests_pass()

Ensure all default unit-tests pass in rendered template

test_files_exist()

Ensure specified top-level files exist

test_jinja_rendered_dirs()

Ensire no jinja brackets exist after rendering directory names

test_jinja_rendered_files()

Ensure no jinja brackets exist after rendering template files

test_project_exists()

Ensure rendered template directory exists in temporary directory

test_setup_py()

Ensure rendered template package setup.py returns version number

test_subdirs_exist()

Ensure all expected sub-directories exist

tests.test_defaults.command_line_interface_bin_name = 'nameless-cc-pydata'

Define command_line_interface_bin_name for default template

tests.test_defaults.json_dict = {'_extensions': ['jinja2_time.TimeExtension'], 'command_line_interface': ['argparse'], 'command_line_interface_bin_name': 'nameless-cc-pydata', 'distribution_name': 'nameless-cc-pydata', 'email': 'Enter your email', 'full_name': 'Enter your full name', 'github_username': 'Enter your GitHub username', 'license': ['MIT license', 'BSD 2-Clause License', 'BSD 3-Clause License', 'ISC license', 'Apache Software License 2.0', 'Not open source'], 'linter': ['flake8'], 'package_name': 'nameless_cc_pydata', 'project_name': 'nameless cc-pydata', 'project_short_description': 'Enter a short description of this project.', 'release_date': 'today', 'repo_name': 'nameless-cc-pydata', 'scm_versioning': ['yes'], 'test_runner': ['pytest'], 'tox': ['yes', 'no'], 'travis': ['yes', 'no'], 'version': '0.0.0', 'website': 'Enter your website', 'year_from': ['2020', '2019', '2018', '2017'], 'year_to': "{% now 'utc', '%Y' %}"}

Define project_name for default template

tests.test_defaults.package_name = 'nameless_cc_pydata'

Define package_name for default template

tests.test_defaults.template_directories = ['data', 'data/raw', 'data/interim', 'data/processed', 'docs', 'docs/_static/figures', 'docs/_templates', 'models', 'notebooks', 'references', 'references/third-party', 'reports', 'reports/figures', 'src', 'src/nameless_cc_pydata/data', 'src/nameless_cc_pydata/features', 'src/nameless_cc_pydata/logger', 'src/nameless_cc_pydata/models', 'src/nameless_cc_pydata/visualizations', 'tests', 'tests/data', 'tests/features', 'tests/logger', 'tests/models', 'tests/visualizations']

Define list of sub-directories expected in default template

tests.test_defaults.template_files = ['.editorconfig', '.env', '.gitignore', '.travis.yml', 'CHANGELOG.rst', 'LICENSE', 'logging.json', 'Pipfile', 'README.rst', 'setup.cfg', 'setup.py']

Define list of top-level files expected in default template

tests.test_defaults.template_submodules = ['data', 'features', 'logger', 'models', 'visualizations']

Define list of src submodule directories expected in default template

tests.test_options

This module contains tests for optional cc-pydata template build options

class tests.test_options.TestBuildTemplateOption(methodName='runTest')

Test optional cookiecutter template build arguments

setUp()

Generate temporary directory in which to render templates

test_build_pre_hook_invalid_package_name_fails()

Ensure template build fails with invalid package_name

test_license_not_open_source()

Ensure ‘Not open source’ license option builds correctly

test_license_open_source_options()

Ensure open source license options build correctly

test_tox_no_ini()

Ensure tox ‘no’ option removes tox.ini

test_tox_no_pipfile()

Ensure tox ‘no’ option removes tox install from Pipfile

test_tox_no_travis_yes_yaml()

Ensure tox ‘no’ option builds with correct .travis.yml

test_tox_yes_ini()

Ensure tox ‘yes’ option builds with tox.ini file

test_tox_yes_pipfile()

Ensure tox ‘yes’ option adds tox install to Pipfile

test_tox_yes_travis_yes_yaml()

Ensure tox and travis ‘yes’ option builds correct .travis.yml

test_travis_no_badge()

Ensure travis ‘no’ option does not include badge in docs

test_travis_no_yaml()

Ensure travis ‘no’ option removes .travis.yml

test_travis_yes_badge()

Ensure travis ‘yes’ option includes badge in docs

test_travis_yes_yaml()

Ensure travis ‘yes’ option builds with travis.yml file

tests.test_options.json_dict = {'_extensions': ['jinja2_time.TimeExtension'], 'command_line_interface': ['argparse'], 'command_line_interface_bin_name': 'nameless-cc-pydata', 'distribution_name': 'nameless-cc-pydata', 'email': 'Enter your email', 'full_name': 'Enter your full name', 'github_username': 'Enter your GitHub username', 'license': ['MIT license', 'BSD 2-Clause License', 'BSD 3-Clause License', 'ISC license', 'Apache Software License 2.0', 'Not open source'], 'linter': ['flake8'], 'package_name': 'nameless_cc_pydata', 'project_name': 'nameless cc-pydata', 'project_short_description': 'Enter a short description of this project.', 'release_date': 'today', 'repo_name': 'nameless-cc-pydata', 'scm_versioning': ['yes'], 'test_runner': ['pytest'], 'tox': ['yes', 'no'], 'travis': ['yes', 'no'], 'version': '0.0.0', 'website': 'Enter your website', 'year_from': ['2020', '2019', '2018', '2017'], 'year_to': "{% now 'utc', '%Y' %}"}

Load cookicutter.json to dictionary

tests.test_options.license_list = ['MIT license', 'BSD 2-Clause License', 'BSD 3-Clause License', 'ISC license', 'Apache Software License 2.0', 'Not open source']

Define 'license' choice variable argument options

tests.toxtest

This module contains tests to ensure the cc-pydata template’s tox tests run without error.

This is accomplished by rendering the cc-pydata template to a temporary directory, then running that template’s default tox configuration to ensure all tox environment tests pass without error.

Warning

Tests contained in this module can take considerable time to run and will, by default, be ignored when the pytest test-runner command is invoked.

  • This “ignore” behavior is the result of the naming convention used for this module.

In order to run the tests contained in this module, this module will need to be explicitly specified while invoking the pytest command.

For example, use this command to run tests contained in this module:

pytest -s tests/toxtest.py

The -s option will disable all stdout capturing by pytest, ensuring that the underlying tox processes are made visible in stdout while the template’s tox script runs.

class tests.toxtest.TestTemplateToxConfig(methodName='runTest')

Test that the rendered template’s default tox configuration runs

setUp()

Render the cc-pydata template in a temporary directory

test_template_tox_runs()

Ensure tox runs without error for the rendered template