Continuous Integration with GitHub Actions using Poetry, Pylint and Pytest with coverage check
Summary
This is a follow-up to the previous post. In this article, I will document how I
- Add a built and test Github action customized for Poetry
- Add the action as part of the status check requirement for branch protection rules
- Added/Edit lint check and test coverage requirements
- Added test cases for the CLI functions
Add a GitHub action customized for Poetry
We will start with the official documentation page, jumping to the guide for using a workflow template we can reference. We are interested in the template for a Python project, so the python-app.yml file will be a good starting reference. However, our project will use Poetry instead of Pip for package management, hence we need to modify the YML file. One good reference that I found online is here. Let's review the details section by section to understand what each part does.
Note GitHub action flow should be created in the directory .github/workflows.
We started with the action definition above. The name at the beginning (Python application in line 5) defines the name of this action flow. We should able to rename it as required.
The on keyword specified when the action will be triggered. In this case, the action will triggered whenever we push or create a pull request to the main branch.
Permissions specify the action to the repository that can be taken by this action flow. Read permission should be sufficient in most cases.
Jobs specify the continuous integration and continuous deployment (CI/CD) stages that will be performed by this action flow. In this example, we started with a build stage. Note you can also give a name to the stage, and this is important for the required status check later when adding the branch protection rule.
Line 19 runs-on specifies the docker image to run the subsequent commands for this stage. In this example we use ubuntu-latest.
The rest of the codes are explained in the comments.
Note we are using some of the action scripts written by others. You can find all available actions in the GitHub marketplace. In this example, anything that starts with the keyword actions (like checkout, cache) is developed by GitHub. The only action script from third parties that we used is the snok/install-poetry@v1.
Once you commit and push the yml file to the main branch, you should see all actions being triggered under the Actions tab as shown below, along with the pass and fail status.
Add Status Check Requirement for Branch Protection Rules
The official guide detailed how to add Branch Protection Rules, with instruction step 7 to add the required status check, but it's murky about the last step. To save you some time from figuring out how to add the checks required:
When you click the Add Checks button to add the Status checks required, if you don't enter anything into the search text box, there will be no checks available to select from. You have to know the exact stage name (not the action flow name) to be able to search and add to the list of status checks required.
In this example, we name our build stage as “poetry build, lint and test”, thus if we type it in the search text box, we should be able to add it to the status check required. Note if you don't give any name to the build stage, it will use the “build” as a name by default.
Now once we add the status check, our pull request will require this check to pass to merge. Failed checks will prevent us from merging the pull request.
Add Lint Check and Test Coverage Requirements
For a Python project, a lint check can be done using pylint, which checks for coding styles as well as static code analysis. With pylint installed, the syntax to run is as follows
# pylint <directory> --fail-under=<pass_score>
# Note pylint return a score of x / 10. Example
pylint ./src --fail-under=10
# will check every .py file under ./src directory, and fail if the score
# is under 10Note sometimes we need to disable certain pylint warnings, refer to here on how to do it.
The next thing to do is to add a coverage check for the tests. One good tool with good integration with Pytest is Pytest-cov. After installation, we can add the following configurations to our pyproject.toml file.
[tool.pytest.ini_options]
# This add three options to the coverage tool
# --cov=src to check for coverage under src directory only
# --cov-report html will generate html report under htmlcov directory
# --cov-fail
-under=50 specifies a passing score of 50%, de
fault is 100%
addopts = "--cov=src --cov-report html --cov-fail-under=50"
Now if we run pytest, it will fail if the target coverage ratio is not reached. We can go to the index.html page to view the statements that we are missing. The sample below shows the majority of the coverage missing is on the CLI commands module.
Add CLI Test Cases
We can start with the official guide and modify it from there. Lets say we have the following content as per the previous post.
Now to test the power cli function,
# In test_commands.py file
# Import CliRunner from click.testing module
from click.testing import CliRunner
# Import the commands module from calc package
from calc import commands
def test_power():
""" Test the power cli
"""
runner = CliRunner()
# Here we invoke the power function of commands module
# with the supplied arguments, notice all args are strings
# and in order of what you will type in the command line
result = runner.invoke(commands.power, ['--base', '2', '--powers', '3'])
# 0 means successfully exit
assert result.exit_code == 0
# result.output will be the stdout from entering the command in terminal
# usually end with \n like '8\n', thats why we need to strip it
assert result.output.rstrip() == '8'Reference and Additional
You can check the full repository here.
For some advanced Pytest using mock objects, you can check this previous post.
