Notes on Poetry Package Manager for Python


I've recenly used Poetry for a Python 3 project; this post includes some notes on the experience plus some bits of configuration I used for integrating it to my workflow.


Poetry is a new package manager plus development utility for Python development. It does not enjoy nearly the popularity of Pipenv, but it is a fresh project, and has the advantage of utilising a new Python standard defined in PEP 517 for everything, eliminating the need for maintaining the clumsy requirements.txt and setup.py manually. It can maintain virtualenvs behind the scenes for you and the poetry run command allows running processes within the virtualenv without activating the virtualenv in your shell. This has allowed me to configure Emacs to use this feature in order to start a Python process without the need to change the environment Emacs is in or using new Elisp programs for it: I just use a script that can determine if Poetry is used and run poetry run python3 to start an interpreter from within the virtualenv for me.

Overall, I was satisfied with the experience. I’ve found a comparison of the thing with Pipenv, and IMO Poetry comes out as superior from it. Also, the author of Pipenv is definitely not the nicest person out there, which is a factor too. I’ve read various negative stuff about Pipenv, mainly regarding speed and awkward UI, but I don’t recall the arguments right now and don’t have many links either, but here is a few of them. But generally, Poetry does not have drama around it or its author, and the CLI is nice to work with, so the decision was easy.

I needed to write a script and set a variable in Emacs to get it to work seamlessly. Also, having a Makefile is useful if you don’t want to type python run ... commands over and over manually. The first script is run-python.sh:

#!/bin/sh
# run-python.sh --- run python, with poetry if applicable

start="$PWD"

while true; do
    if [ -e pyproject.toml ]; then
        exec poetry run python $@
    elif [ / = "$PWD" ]; then
        cd $start
        exec ${RUN_PYTHON:-python3} $@
    else
        cd ..
    fi
done

This one is aliased to py in my aliases file, and what it does is, it tries to determine if current $PWD belongs to a project that uses Poetry, and if so, it runs poetry run python $, which starts an interactive Python shell within the relevant virtualenv, which Poetry keeps in a separate well known place. If this is not a Poetry project, then it runs the program @$RUN_PYTHON denotes, which defaults to python3.

In Emacs I use the following settings to use this script with run-python:

(setf python-shell-interpreter "run-python.sh")

Finally, below is a makefile that uses Poetry to run various Python code quality tools, Mypy, Sphinx and pytest on the project. Should be fairly self-evident, so I won’t comment it.

PKGDIR=scissors
STUBSDIR=etc/stubs
export MYPYPATH=$(etc/stubs)

PYLINT_DISABLED=W0511,R0903,C0305
FLAKE8_IGNORE=W391

PRETEST = lint flake8 typecheck

# Whether to measure coverage after tests:
COV=no

# See: (info "(make) Syntax of Functions"), "Commas and unmatched
# parentheses or braces cannot appear in the text of an argument as
# written".
empty :=
comma := ,
space := $(empty) $(empty)
rest  := $(subst $(space),$(comma)$(space),$(PRETEST))

all: help

help:
	@echo "Targets:"
	@echo "	test		run tests and $(rest);"
	@echo "			with COV=yes, measure coverage at the end"
	@echo "	quicktest	run tests and nothing else"
	@echo "	typecheck	check types with mypy"
	@echo "	apidoc		generate api docs"
	@echo "	docs		generate docs output"
	@echo "	stubs		generate typing stubs"
	@echo "	lint		lint source code"
	@echo "	flake8		lint source code"
	@echo
	@echo "Variables:"
	@echo "	COV=$(COV), PRETEST=$(PRETEST)"

quicktest:
	@$(MAKE) PRETEST= COV=no $(MAKEFLAGS) _test

test: $(PRETEST) _test

_test:
ifeq ($(COV), yes)
	@echo \> Run tests and measure coverage \(pytest --cov\)...
	@poetry run pytest --cov=$(PKGDIR)
else
	@echo \> Run tests \(pytest\)...
	@poetry run pytest
endif
	@echo \> Complete\!

typecheck:
	@echo \> Check types \(mypy\)...
	@poetry run mypy $(PKGDIR)/

apidoc:
	poetry run sphinx-apidoc -f --separate --module-first -o docs/api $(PKGDIR)

# Mainly intended for readthedocs.org
docs-deps:
	poetry run pip freeze | grep -E '(Sphinx|numpydoc)' > docs/requirements.txt

# TODO(2019-04-28): add info too
docs: docs-deps apidoc
	$(MAKE) -C docs $(MAKEFLAGS) singlehtml html coverage && \
		cat docs/_build/coverage/python.txt

stubs:
	poetry run stubgen -p mako -o $(STUBSDIR)

lint: pylint
pylint:
	@echo \> Lint Python code \(pylint\)...
	@poetry run pylint -rn -sn --jobs=0 --disable=$(PYLINT_DISABLED)\
		$(PKGDIR)

flake8:
	@echo \> Python style checks \(flake8\)...
	@poetry run flake8 --ignore $(FLAKE8_IGNORE) $(PKGDIR)

.PHONY: all help test typecheck apidoc docs docs-deps stubs lint pylint flake8