Posts tagged: copier

Copier has a few quirks with vcs that I just discovered by trying to test out some changes. I may have some config that I have long forgotten about somewhere deep in my dotfiles, I don’t think so, but id love to be wrong and corrected, please reach out.

What Doesn’t Work #

I tried throwing everything at this template to make it work. I tried a bunch of flags that did not work. I tried making commits to the local repo to get rid of the dirty warning. I really wanted to test new changes locally without committing and pushing untested and potentially broken changes.

uvx copier copy ../markata-blog-starter .
uvx copier copy gh:waylonwalker/markata-blog-starter@develop .
uvx copier copy ../markata-blog-starter . -wlg --trust

What Works - –vcs-ref #

Finally after trying everything to get the local copy to work, and my guess of @branch not working I found this to work. It does require me to go to the repo on my develop branch.

uvx copier copy gh:waylonwalker/markata-blog-starter --vcs-ref develop .

git">What Works - delete .git #

Really this might be my best option to make quick changes and test them locally without going through a version control system. It is not ideal, but makes it easy to quickly iterate on. I might be renaming .git, or copying to /tmp for quick iteration.

rm -rf .git
uvx copier copy ../markata-blog-starter .

Copier I love #

Copier is a great templating tool. I really love it. I use it every single day to create posts on this blog using Tmux hotkey for copier templates. This is the first time this quirk has got me and it had me puzzled for 45 minutes as I did not expect this behavior whatsoever.

I’ve long used copier to create all of my posts for my blog, and it works really well for my workflow. I think of a title, call a template, and give it a title. out of the box obsidian did not seem to work this way. It seems like it wants me to right click a file tree and make a new file using the tree, this is not my jam.

Here is what I came up with to replace my til template.

---
date: <% tp.file.creation_date() %>
templateKey: til
title: <%*
  const originalFileName = await tp.system.prompt("Enter file name");
  const toTitleCase = str => str.replace(
    /\w\S*/g,
    txt => txt.charAt(0).toUpperCase() + txt.substr(1).toLowerCase()
  );
  const title = toTitleCase(originalFileName);
  tR += title + '\n'; // Add the title to the template result
-%>
published: true
tags:
  -
---
<%*
const fileName = originalFileName.toLowerCase().replace(/\s+/g, '-');
const newFilePath = `pages/til/${fileName}`;
await tp.file.move(newFilePath);
-%>

<% tp.file.cursor() %>

• tR is a return value, and it gets placed directly into the place it is in the file
• to.file.cursor() creates a tab-index point so I can tab into the content

I recently had to update my copier-gallery command to trust my own templates because some of them have shell scripts that run afterwards. Be warned that this could be a dangerous feature to run on random templates you get off the internet, but these are all mine, so if I wreck it its my own fault.

copier copy --trust <template> <destination>

All the the copier copy api can be found with help.

❯ copier copy --help
copier copy 8.3.0

Copy from a template source to a destination.

Usage:
    copier copy [SWITCHES] template_src destination_path

Hidden-switches:
    -h, --help                         Prints this help message and quits
    --help-all                         Prints help messages of all sub-commands and quits
    -v, --version                      Prints the program's version and quits

Switches:
    -C, --no-cleanup                   On error, do not delete destination if it was
                                       created by Copier.
    --UNSAFE, --trust                  Allow templates with unsafe features (Jinja
                                       extensions, migrations, tasks)
    -a, --answers-file VALUE:str       Update using this path (relative to
                                       `destination_path`) to find the answers file
    -d, --data VARIABLE=VALUE:str      Make VARIABLE available as VALUE when rendering the
                                       template; may be given multiple times
    -f, --force                        Same as `--defaults --overwrite`.
    -g, --prereleases                  Use prereleases to compare template VCS tags.
    -l, --defaults                     Use default answers to questions, which might be
                                       null if not specified.
    -n, --pretend                      Run but do not make any changes
    -q, --quiet                        Suppress status output
    -r, --vcs-ref VALUE:str            Git reference to checkout in `template_src`. If you
                                       do not specify it, it will try to checkout the
                                       latest git tag, as sorted using the PEP 440
                                       algorithm. If you want to checkout always the
                                       latest version, use `--vcs-ref=HEAD`.
    -s, --skip VALUE:str               Skip specified files if they exist already; may be
                                       given multiple times
    -w, --overwrite                    Overwrite files that already exist, without asking.
    -x, --exclude VALUE:str            A name or shell-style pattern matching files or
                                       folders that must not be copied; may be given
                                       multiple times

It’s no secret that I love automation, and lately my templating framework of choice has been copier. One hiccup I recently ran into was having spaces in my templated directory names. This makes it harder to run commands against as you need to escape them, and if they end up in a url you end up with ugly %20 all over.

Cookiecutter has the solution #

Yes the solution comes from a competing templating framework.

I install copier with pipx, so I need to inject cookiecutter in to my copier environment to use the slugify filter.

pipx inject copier cookiecutter

If you are using a normal virtual environment you can just pip install it.

pip install copier cookiecutter

add the extension to your template #

copier.yml

Now to enable the extension you need to declare it in your copier.yml file in your template.

_jinja_extensions:
    - cookiecutter.extensions.SlugifyExtension

Use it | slugify #

use-it

Now to use it, anywhere that you want to slugify a variable, you just pipe it into slugify.

❯ tree .
.
├── copier.yml
├── README.md
└── {{ site_name|slugify }}
    └── markata.toml.jinja

1 directory, 3 files

Here is a slimmed down version of what the copier.yml looks like.

site_name:
  type: str
  help: What is the name of your site, this shows in seo description and the site title.
  default: Din Djarin

_jinja_extensions:
    - cookiecutter.extensions.SlugifyExtension

Results #

Running the template looks a bit like this.

straight from their docs #

The next section is straight from the cookiecutter docs

Slugify extension #

The cookiecutter.extensions.SlugifyExtension extension provides a slugify filter in templates that converts string into its dashed (“slugified”) version:

{% "It's a random version" | slugify %}

Would output:

it-s-a-random-version

It is different from a mere replace of spaces since it also treats some special characters differently such as ' in the example above. The function accepts all arguments that can be passed to the slugify function of python-slugify_. For example to change the output from it-s-a-random-version to it_s_a_random_version, the separator parameter would be passed: slugify(separator='_').

I have added a hotkey to my copier template setup to quickly access all my templates at any time from tmux. At any point I can hit <c-b><c-b>, thats holding control and hitting bb, and I will get a popup list of all of my templates directory names. Its an fzf list, which means that I can fuzzy search through it for the template I want, or arrow key to the one I want if I am feeling insane. I even setup it up so that the preview is a list of the files that come with the template in tree view.

bind-key c-b popup -E -w 80% -d '#{pane_current_path}' "\
    pipx run copier copy ~/.copier-templates/`ls ~/.copier-templates |\
    fzf --header $(pwd) --preview='tree ~/.copier-templates/{} |\
    lolcat'` . \
    "

I’ve had this on my systems for a few weeks now and I am constantly using it for my tils, blogs, and my .envrc file that goes into all of my projects to make sure that I have a virtual environment installed and running any time I open it.

Copier allows you to run post render tasks, just like cookiecutter. These are defined as a list of tasks in your copier.yml. They are simply shell commands to run.

The example I have below runs an update-gratitude bash script after the copier template has been rendered.

# copier.yml
num: 128
_answers_file: .gratitude-copier-answers.yml
_tasks:
  - "update-gratitude"

I have put the script in ~/.local/bin so that I know it’s always on my $PATH. It will reach back into the copier.yml and update the default number.

#!/bin/bash
# ~/.local/bin/update-gratitude
current=`awk '{print $2}' ~/.copier-templates/gratitude/copier.yml | head -n 1`
new=`expr $current + 1`
echo $current
echo $new
sed -i "s/$current/$new/g" ~/.copier-templates/gratitude/copier.yml

The copier answers file is a key component to making your templates re-runnable. Let’s look at the example for my setup.py.

❯ tree ~/.copier-templates/setup.py
/home/walkers/.copier-templates/setup.py
├── [[ _copier_conf.answers_file ]].tmpl
├── copier.yml
├── setup.cfg
└── setup.py.tmpl

0 directories, 4 files

Inside of my [[ _copier_conf.answers_file ]].tmpl file is this, a message not to muck around with it, and the ansers in yaml form. The first line is just a helper for the blog post.

# ~/.copier-templates/setup.py/\[\[\ _copier_conf.answers_file\ \]\].tmpl
# Changes here will be overwritten by Copier; NEVER EDIT MANUALLY
[[_copier_answers|to_nice_yaml]]

Inside my copier.yml I have setup my _answers_file to point to a special file. This is because this is not a whole projet template, but one just for a single file.

# copier.yml
# ...
_answers_file: .setup-py-copier-answers.yml

Once I change the _answers_file I was incredibly stuck

Run it #

I’m making a library of personal copier templates in my ~/.copier-templates directory and I am going to run it from there.

copier copy ~/.copier-templates/setup.py

Results #

After rendering the template we have the following content in our .setup.setup-py-copier-answers.yml file. This will allow us to update quick if we ever change our template.

# .setup-py-copier-answers.yml
# Changes here will be overwritten by Copier; NEVER EDIT MANUALLY
_src_path: /home/walkers/.copier-templates/setup.py
author_github: waylonwalker
author_name: Waylon Walker
description: awesomeness
framework: null
keywords: null
package_name: my-package

Update it #

This is where I was most stuck, primarily becuase -a <answers_file> must come exactly after the base command copier. This felt a bit odd to and not where I expected it so it.

copier -a .setup-py-copier-answers.yml update

Stop asking all these damn questions #

So the defaults are now changed to our previous results, but it keeps asking for them. To stop asking we can simply add a -f flag.

copier -fa .setup-py-copier-answers.yml update

I was completely stuck for awhile. copier was not replacing my template variables. I found out that adding all these _endops fixed it. Now It will support all of these types of variable wrappers

# copier.yml
_templates_suffix: .jinja
_envops:
  block_end_string: "%}"
  block_start_string: "{%"
  comment_end_string: "#}"
  comment_start_string: "{#"
  keep_trailing_newline: true
  variable_end_string: "}}"
  variable_start_string: "{{"

!RTFM: Later I read the docs and realized that copier defaults to using [[ and ]] for its templates unlike other tools like cookiecutter.

I’ve been looking for a templating tool for awhile that works well with single files. My go to templating tool cookiecutter does not work for single files, it needs to put files into a directory underneath of it.

template variables #

By default copier uses double square brackets for its variables. variables in files, directory_names, or file_names will be substituted for their value once you render them.

# hello-py/hello.py.tmpl
print('hello-[[name]]')

note! by default copier will not inject variables into your template-strings unless you use a .tmpl suffix.

Before running copier we need to tell copier what variables to ask for, we do this with a copier.yml file.

# copier.yml
name:
  default: my_name
  type: str
  help: What is your name

installing copier #

I prefer to install cli tools that I need globally with pipx, this always gives me access to the tool without worrying about dependency conflicts, bloating my system site-packages, or managing a separate virtual environment for it myself.

pipx install copier

running copier #

When running copier copy we pass in the directory of the template, and the directory that we want to render the template into.

copier copy hello-py .

note! the directory ‘.’ is often referred to in cli programs to represent the current working directory that we are calling the command from.

results #

The resulting files will have your variables injected into them if you have setup your template and copier.yml up correctly.

print('hello-you')