Effective Repository Templates with Copier
Copier is immensely useful as a vehicle for building and distributing developer tooling that remains maintainable at scale
The critical feature it provides beyond tools like cookiecutter is the ability to update the code you have generated from it to a newer version of the template, while respecting any changes you’ve made to it
In this post I’ll detail a number of sensible defaults, techniques, design patterns and best practices that enhance the utility of copier templates
I’ve since packaged most of these into A Production-Ready Meta-Template for Copier if you’d rather start from a template that already has them applied than apply them by hand
Set _min_copier_version
Unless you’re using CI to test your template with multiple versions of copier I suggest setting this to the latest version available.
With tools like uv or pipx for isolated installs of python CLI tools, in the vast majority of cases there is no need to support older versions and so you can nudge them to update.
This ensures your users are using the most up to date software with all the fixes to the copier cli tool
and that as the template author, you can use all features described in the docs
Set _subdirectory
As a sensible default, I suggest setting
# copier.yml
_subdirectory: template
which means your template will be the contents of the template/ directory rather than the root of the template repository
This is quite useful for separating the template contents from files for the maintenance of the template itself such as; tests, docs, CI/CD config, and developer tooling
A more complex pattern, that can be useful if you’re supporting multiple variants of template that require a fundamentally different structure, is to template this value with the answer to a question
# copier.yml
_subdirectory: '{{template}}'
template:
choices:
- Tech Stack A: template_dir_for_tech_stack_a
- Tech Stack B: template_dir_for_tech_stack_b
However, if the templates are really so different that they require their own variant like this, I would recommend that you consider if they should be entirely separate templates (i.e. in separate repos) so they can be maintained and versioned separately
If only part of your template is fundamentally different you can achieve similar results with conditional directories: e.g.
template/{% if your_variable == 'feature_A' %}feature_A_dir{% endif %}/
Hardcoded template values
Sometimes it’s useful to include variables that are used within the template as a helper for the template maintainers, but don’t need to be asked to users. You can achieve this with answers that are never asked
template_name:
type: str
default: "your-template-name"
when: false # hidden question: reusable variable with default value
Note: These values do not appear in the users answers file
The template_name variable specifically is one I recommend including in all templates
as it’s useful for namespacing content, to avoid clobbering existing content
📁 scripts/
└── 📁 {{ template_name }}/
└── 📄 my-script.sh
Error loudly on undefined variables
Avoid unexpected behaviour from variables (question answers) not being defined by setting this jinja option:
_envops:
undefined: jinja2.StrictUndefined
This setting pairs very well with testing your templates (discussed later in this post) allowing you to catch bugs with your question asking more easily
Highlight merge conflicts
The way copier provides the ability to update code from an older version of the template is by using git to apply the differences between the old template and the new template to your code. Copier raises merge conflicts for you to resolve if you’ve modified the code differently in your instance of the files.
By including a .pre-commit-config.yaml file with check-merge-conflict for use with
pre-commit in the template, you can help
ensure users more easily find conflicts if they occur when they update the template
# .pre-commit-config.yaml
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v6.0.0
hooks:
- id: check-merge-conflict
args: [--assume-in-merge]
Ideally pre-commit validation should be checked in CI with a job that runs pre-commit run --all-files in case a developer has forgotten to install it
Reduce merge conflicts
It’s fairly infrequent that this issue comes up, but annoying for users as it’s quite unintuitive!
The git 3-way merge algorithm that copier uses when doing copier updates
will catch stable differences from the template in user code adjacent
to matching updated lines
e.g. if a template has a pyproject.toml like this say:
[project]
name = "example"
requires-python = "==3.13.*"
And a user updated their instance to look like this:
[project]
name = "my-package"
requires-python = "==3.14.*"
Then updating the requires-python to 3.14 in the template:
[project]
name = "example"
requires-python = "==3.14.*"
This will result in users seeing merge conflicts for things that are stable long term differences to the template and feel like they should be respected by copier update:
[project]
<<<<<<< before updating
name = "my-package"
=======
name = "example"
>>>>>>> after updating
requires-python = "==3.14.*"
To stop this happening, put stable (e.g. comment) or blank lines between lines you expect to change in the template
[project]
name = "example"
requires-python = "==3.13.*"
This issue can come up often if e.g. renovate is updating a user’s template and the source template similarly
Handling templates inside your template
If files you’re including in your template are also themselves templates and use the same template delimiters as jinja defaults ({{ }}) used by copier,
then things can get a bit awkward.
For instance if you include task’s Taskfile.yaml or just’s justfile or you use jinja
in the output project, and you want to template values inside those files, you can end up with the delimiters conflicting.
Then you either have to use lots of {% raw %}{% endraw %} blocks to escape the templating manually.
Alternatively, there are a number of settings to modify the (jinja) templating within _envops which are helpful to change the delimiters used by copier in the template
_envops:
block_end_string: "%]"
block_start_string: "[%"
comment_end_string: "#]"
comment_start_string: "[#"
variable_end_string: "]]"
variable_start_string: "[["
You might also consider customizing the suffix of files that get templated (from the default of .jinja) if it clashes with your desired output too
_templates_suffix: .tmpl
Whitespace control
Familiarity with the whitespace-control functionality offered by jinja templating is very helpful for maintaining functional and tidy output files
This includes the use of - or + to control whitespace around the jinja blocks
{%- -%}
and/or setting global defaults you can customize within _envops
_envops:
trim_blocks: true
keep_trailing_newline: true
lstrip_blocks: true
Loop helpers
Jinja templating provides you with some helpers when you are using for loops via the loop variable which has various helper
attributes and methods attached
For instance to construct a valid json array, you need comma’s for all the lines except the last:
[
{% for item in items %}
"{{ item }}"{% if not loop.last %},{% endif %}
{% endfor %}
]
See the jinja docs for details of other attributes and method
Validate your inputs
Empty strings are valid inputs to stings but might not be what you want, so make sure to validate your inputs where appropriate
question:
type: str
validator: >-
{% if question == "" %}
Answer must not be an empty string
{% endif %}
You can utilize the guard clause pattern to create more complex validation.
The built-in regex_search filter can be valuable for somewhat complex checks.
question:
type: str
validator: |-
{% if question == "" %}
Answer must not be an empty string
{% endif %}
{% if len(question) > 20 %}
Answer must be less than (or equal to) 20 characters
{% endif %}
{% if not (question | regex_search('^[A-Za-z][A-Za-z0-9-]*$')) %}
Can only contain letters, numbers, and hyphens and must start with a letter.
{% endif %}
If describing the validation via jinja templating is getting unmanageable you can use jinja extensions to do more complex validation. You can add jinja “filters” (a python function that receives the value piped in) that will parse the input and produce the validation sting.
question:
type: str
validator: |-
{{ question | your_validation_filter }}
Customisable pre-made options include:
- https://pypi.org/project/copier-pydantic/ (Includes multiline error message support)
- https://github.com/copier-org/jinja2-jsonschema
Or if you need something more custom, the copier_templates_extensions
extension enables integrated extensions as part of the template.
_jinja_extensions:
- copier_templates_extensions.TemplateExtensionLoader
There are two main downsides of this approach. Firstly, it requires the user to use the --trust flag (as it can run arbitrary code)
which potentially will cause them some concern or require them to investigate what it’s doing before being happy using it.
However, if your template already requires the use of the trust flag for other reasons (e.g. migrations) then this might be less of an issue.
Secondly, it also complicates the copier installation process, requiring installing the extension into the virtual environment
uv tool install copier --with <the-extension-python-package>
# or
pipx install copier
pipx inject copier <the-extension-python-package>
Creating files/directories with for loops
Copier recently (in version 9.5) introduced support for looping over iterables to construct files and directories
With this change, it increases the value in asking questions that have iterable answers, to construct data structures to loop over
There are a number of different ways to support this with copier
For lists of values known statically ahead of time you can use choices with multiselect:
environments:
multiselect: true
choices:
- dev
- test
- staging
- production
📁 dotenv
└── 📄 {% yield env from environments %}{{ env }}{% endyield %}.env.jinja
# {% yield env from environments %}{{ env }}{% endyield %}.env.jinja
ENVIRONMENT={{ env }}
Using yaml (or json) inputs which expect nested maps allows you to construct more useful files, but can require more complex validation
# helper list
required_config:
type: yaml
default: |-
db_host: string
db_port: number
when: false
environments_config:
type: yaml
multiline: true
default: |-
dev:
db_host: localhost
db_port: '5432'
validator: |-
{% if environments_config is not mapping %}
Environment config must be a yaml map with keys as the environment name
{% endif %}
{% for env, config in environments_config.items() %}
{% if config is not mapping %}
Environment '{{ env }}' value must be a yaml map with keys: {{ required_config | join(',') }}
{% endif %}
{% for var in required_config %}
{% if var not in config %}
{{ env }} must contain {{ var }} setting
{% endif %}
{% if config[var] is not string %}
{{ var }} must be a string setting
{% endif %}
{% endfor %}
{% endfor %}
Then your file can be more advanced like this
# {% yield env from environments_config %}{{ env }}{% endyield %}.env.jinja
ENVIRONMENT={{ env }}
DB_HOST={{ environments_config[env].db_host }}
DB_PORT={{ environments_config[env].db_port }}
Sane validation
As you can see above, the validation for more complex data structure starts to get quite complicated in pure copier/jinja validators, and you’re more likely to miss a case, or need complex testing.
I highly recommend that you check out these extensions, which you can use to simplify the validation specification, and improve the user experience of the errors:
- https://pypi.org/project/copier-pydantic/ (Includes multiline error message support)
- https://github.com/copier-org/jinja2-jsonschema
Modular templates
A collection of approaches supporting composition of multiple copier templates
Use Separate Directory structure
Put your files in separate directories using locations like
.gitlab-ci/{{ template_name }}/.gitlab-ci.yml
scripts/{{ template_name }}/your_templates_script.sh
.task/{{ template_name }}/Taskfile.yaml
{{ user_specified_name }}/your_files.py
Where needed these file can be imported/included into top level files with tools that support this
Set a unique _answers_file name
Not only is this the only way to set a non-default location, but it’s also the only way
to move the answer file outside the target location root! (Previously, putting
{{_copier_conf.answers_file}}.jinja in a non root location was supported, but that’s deprecated now)
_answers_file: .copier.{{ template_name }}.yaml
# or
_answers_file: .copier-answers.{{ template_name }}.yaml
# or
_answers_file: .copier/{{ template_name }}.yaml
Note: this will require your users to update the template using:
copier update -a path/to/copier-answer-file.yaml
So it’s a good idea to include this in a task runner task you ship with the template! You can even look up the answer file location, in case they edit it
copier update -a {{_copier_conf.answers_file}}
Answering questions from existing yaml files
If you construct sets of related modular copier templates, then you might be able to reuse answers from another copier answers file, or construct a special interface file in a parent template for use in a child template to reduce the number of questions users need to answer.
The most basic way to do this is with
copier --data-file answers-in-file.yaml
For more advanced cases _external_data can be used with jinja templating e.g. to convert names
# path/to/file/relative/to/copier/target/directory.yaml
different_name: value
_external_data:
existing_file: "path/to/file/relative/to/copier/target/directory.yaml"
question:
default: "{{ _external_data.existing_file.different_name }}"
Discoverability
Add the github topic copier-template to help others find your template
Testing your template
Copier does not have a well-trodden path for testing in the community, and patterns are immature, but it’s hugely important for a template’s long-term maintainability (and developer’s mental health).
It’s worth separating two different questions and testing them separately:
- Does the template itself behave correctly i.e. does
copier copydo what I expect, maybe given specific answers? - Does the generated output actually work? Does the code it produces run? Do the files lint correctly? Etc
The first is a thin wrapper over copier.run_copy / copier.run_update with pytest:
def test_copy_default(template_dir, tmp_path):
run_copy(
str(template_dir),
tmp_path,
vcs_ref='HEAD',
defaults=True,
)
assert (tmp_path / 'some_file').exists()
The second is where things start to get more complex and so I recommend checking out my copier template for copier projects that helps set up the testing harness for this. See A Production-Ready Meta-Template for Copier for an introduction.
Include a task running tool
I recommend you include a task runner like go-task or just, to
give users of the generated project a consistent set of commands (task test, task lint) regardless of what’s
underneath them, and to give yourself a scriptable interface for the template’s own CI and local development.
See A Production-Ready Meta-Template for Copier
for a template with this already wired up.