Posts tagged: actions

dash e your github docker actions

2022-12-07T13:09:37Z

Give github actions the -e flag in the shebang #! so they fail on any one command failure. Otherwise each line will set the exit status, but only the last one will be passed to ci.

#!/bin/bash -e

What is -e #

The -e flag to the bash command allows your script to exit immediately if any command within the script returns a non-zero exit status. This can be useful for ensuring that your script exits with an error if any of the commands it runs fail, which can help you identify and debug issues in your script. For example, if you have a script that runs several commands and one of those commands fails, the script will continue running without the -e flag, but will exit immediately if the -e flag is present. This can make it easier to troubleshoot your script and ensure that it runs correctly.

Solution for Windows #

In windows the solution is not quite as simple. You can define a function in a Windows batch script that wraps an if statement to check the exit status of a command and handle any errors that may have occurred. Here is an example of how you might define a function called “check_error” that does this:

:check_error
if errorlevel 1 (
  echo An error occurred!
  exit /b 1
)

To use this function in your script, you would simply call it after running a command, like this:

some_command
call :check_error

This would run the “some_command” and then call the “check_error” function to check the exit status and handle any errors that may have occurred. This approach allows you to reuse the error-checking logic in your script, which can make it easier to write and maintain.

Site Down During Build

2021-03-22T00:00:00Z

Recently I noticed a new netlify site of mine was down while I was checking to see if new content was live. Later found out this was consistent after each and every push the site would go gown as soon as I hit push, and would not come back until the build finished.

Is this normal? #

Do other Netlify sites go down during build???

Short Answer NO. All of my google fu lead me to believe I was alone and none of my other sites do this.

Digging into my build #

My deploy script ends with the following. After resetting keys and watching it build half a dozen times I determined that everything was working as normal here.

- name: Deploy to Netlify
uses: nwtgck/actions-netlify@v1.1.12
with:
    publish-dir: "./markout"
    production-branch: markout
    production-deploy: true
    deploy-message: "Deploy markout from GitHub Actions"
env:
    NETLIFY_AUTH_TOKEN: ${{ secrets.NETLIFY_AUTH_TOKEN }}
    NETLIFY_SITE_ID: ${{ secrets.NETLIFY_SITE_ID }}

Opening the Nelify Console #

After poking at the netlify console for hours I realized that the issue was that netlify was still auto-deploying from a no longer existing directory and would cause 404’s for every page. During build, then my build from GitHub Actions would deploy with the netlify cli.

Netlify really likes to put a lot of warnings up when you are not deploying from them. I tured off automatic deploys, swore to the netlify gods this is what I wanted. Pushed a new deploy and 🎉 THE SITE DID NOT GO DOWN.

jdiv class=‘center-img’>

Review of the git-auto-commit-action

2020-08-03T05:00:00Z

It’s a really cool GitHub action that will automatically commit files changed during the action. I was using this to render a new readme based on a template.

Check out the repo for git-auto-commit-action.

It’s a really cool GitHub action that will automatically commit files changed during the action. I was using this to render a new readme based on a template.

This has been by far the easiest way to commit back to a repo that I have seen. Other patterns often require fully setting up the git user and everything. While it’s not all that hard, this action already has all of that covered.

You must give it a commit message and thats it. Optionally you can configure a number of things. Its possible to configure the commit_user_name, commit_user_email, and commit_author. I often also scope the file_pattern to a certain subset of files.

If you’re new to actions check out this article on using actions.

You do need to be careful to checkout the repo just a bit different than normal.

Limitations & Gotchas #

directly from the repo

Checkout the correct branch #

You must use action/checkout@v2 or later versions to checkout the repository. In non-push events, such as pull_request, make sure to specify the ref to checkout:

- uses: actions/checkout@v2
  with:
    ref: ${{ github.head_ref }}

You have to do this do avoid that the checkout-Action clones your repository in a detached state.

Usage #

from the repo

- uses: stefanzweifel/git-auto-commit-action@v4
  with:
    # Required
    commit_message: Apply automatic changes

    # Optional branch to push to, defaults to the current branch
    branch: feature-123

    # Optional options appended to `git-commit`
    # See https://git-scm.com/docs/git-commit for a list of available options
    commit_options: '--no-verify --signoff'

    # Optional glob pattern of files which should be added to the commit
    # See the `pathspec`-documentation for git
    # - https://git-scm.com/docs/git-add#Documentation/git-add.txt-ltpathspecgt82308203
    # - https://git-scm.com/docs/gitglossary#Documentation/gitglossary.txt-aiddefpathspecapathspec
    file_pattern: src/*.js tests/*.js

    # Optional local file path to the repository
    repository: .

    # Optional commit user and author settings
    commit_user_name: My GitHub Actions Bot
    commit_user_email: my-github-actions-bot@example.org
    commit_author: Author <actions@github.com>

    # Optional tag message
    # Action will create and push a new tag to the remote repository and the defined branch
    tagging_message: 'v1.0.0'

    # Optional options appended to `git-push`
    push_options: '--force'

    # Optional: Disable dirty check and always try to create a commit and push
    skip_dirty_check: true

More Actions #

If you’re new to actions check out these articles on using actions.

Four github actions for your website

2020-05-18T13:02:00Z

GitHub’s actions are a new GitHub feature that will trigger GitHub to spin up a virtual machine and run some tasks with some special access to your repo. It can interact with comments/issues, it can clone your repo, You can explicitly pass in secrets so that it can commit back to the repo or deploy to another service. The environment may be a Linux, windows, or even a mac machine. I believe this is wildly incredible for the open-source community, putting these tools in the same place that we are already collaborating is so convenient.

What can they do for my personal website? 🤔 #

GitHub actions can give you confidence that your site is up and running, with the latest JavaScript packages, does not have broken links, and can even take screenshots of what your website looks like on different screen sizes and operating systems.

periodically check that the website is up
update npm
url checker
screenshot website

srt32/uptime #

srt32/uptime is an action that you can run on any public website. I run this one several times every day and it gives me confidence that my various sites are still up and running. It ensures that my build didn’t break something, nothing is wrong with my hosting provider, or my DNS.

name: check if site is up
on:
  schedule:
    - cron: '0 0 * * *'

jobs:
  ping_site:
    runs-on: ubuntu-latest
    name: Ping the site
    steps:
    - name: Check the site
      id: hello
      uses: srt32/uptime@master
      with:
        url-to-hit: "https://waylonwalker.com/"
        expected-statuses: "200,301"

taichi/actions-package-update #

Here is another one to make sure that your package.json does not get too far out of date, or have any vulnerabilities. taichi/actions-package-update will submit a PR back to your repo with any updated dependencies. Since it submits it as a pr, your tests triggered by PRs should also run. giving you confidence that you are ready to update.

Example to update package.json every Wednesday night at midnight.

on:
  schedule:
  - cron: 0 0 * * 3
name: Update
jobs:
  package-update:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@master
    - name: set remote url
      run: git remote set-url --push origin https://$GITHUB_ACTOR:${{ secrets.GITHUB_TOKEN }}@github.com/$GITHUB_REPOSITORY
    - name: package-update
      uses: taichi/actions-package-update@master
      env:
        AUTHOR_EMAIL: john@example.com
        AUTHOR_NAME: john
        EXECUTE: "true"
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        LOG_LEVEL: debug
      with:
        args: -u --packageFile package.json --loglevel verbose

urlstechie/urlchecker-action #

urlstechie/urlchecker-action is an action to collect and check URLs in a project and report on broken links. This is another one to give yourself some confidence that you are not linking out to a broken site, and can give you a heads up before you have frustrated users.

name: Check URLs

on: [push]

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v2
    - name: urls-checker
      uses: urlstechie/urlchecker-action@0.2.1
      with:
        # A subfolder or path to navigate to in the present or cloned repository
        subfolder: docs

        # A comma-separated list of file types to cover in the URL checks
        file_types: .md,.py,.rst

        # Choose whether to include file with no URLs in the prints.
        print_all: false

        # The timeout seconds to provide to requests, defaults to 5 seconds
        timeout: 5

        # How many times to retry a failed request (each is logged, defaults to 1)
        retry_count: 3

        # A comma separated links to exclude during URL checks
        white_listed_urls: https://github.com/SuperKogito/URLs-checker/issues/1,https://github.com/SuperKogito/URLs-checker/issues/2

        # A comma separated patterns to exclude during URL checks
        white_listed_patterns: https://github.com/SuperKogito/Voice-based-gender-recognition/issues

        # choose if the force pass or not
        force_pass : true

swinton/screenshot-website #

swinton/screenshot-website will take a screenshot of your website. It can even run a matrix of sizes and os’s to check how your site looks on various systems.

name: screenshot-website
on:
  schedule:
    - cron: '0 0 0 * *'

jobs:
  screenshot:
    name: Screenshot
    strategy:
      matrix:
        os: [ubuntu-latest, macos-latest, windows-latest]
        width: [1200, 992, 768, 600]
    runs-on: ${{ matrix.os }}

    steps:

    - name: Screenshot Website
      uses: swinton/screenshot-website@v1.x
      with:
        source: https://waylonwalker.com/
        destination: screenshot-${{ matrix.os }}-${{ matrix.width }}.png
        width: ${{ matrix.width }}

Four Github Actions for Python

2020-04-13T06:23:00Z

If you are developing python packages and using GitHub here are four actions that you can use today to automate your release workflow. Since python tools generally have such a simple cli I have opted to use the cli for most of these, that way I know exactly what is happening and have more control over it if I need.

Lint
Test
Package
Upload to PyPi

Lint With flake8 #

flake8 is pythons quintessential linting tool to ensure that your code is up to the standards that you have set for the project, and to help prevent hidden bugs. I am a heavy user of black and isort as well, but for ci flake8 is typically considered the gold standard. black and isort will help you automate many fixes suggested by flake8.

    - name: Lint with flake8
      run: |
        pip install flake8 isort black
        # stop the build if there are Python syntax errors or undefined names
        flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
        # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
        flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics

Testing with pytest #

pytest is such an amazing project, definitely one to check out and start using if you are not already doing so.

    - name: Test with pytest
      run: |
         pip install pytest
         pytest

Building with setuptools #

I am still using the older, less hipster, setuptools to build my projects. Primarily because I am used to to, partly because things such as editable installs are not possible with the newer build tools, and I am a HEAVY user of editable installs.

    - name: build
      run: |
        pip install wheel
        python setup.py sdist bdist_wheel

Publishing to pypi #

Here I am going to use an amazing action from the GitHub marketplace by @webKnjaZ. It is super simple. First you need to log into your pypi.org account, go to account settings, enable 2FA, and add a Token, then paste that toke into a secret inside your repos settings. Next just drop the name of that secret into the password field of the action and you are off.

note: I did put a check in to make sure that push event comes from main.

    - name: pypi-publish
      if: github.ref == 'refs/heads/main'
      uses: pypa/gh-action-pypi-publish@v1.1.0
      with:
        # PyPI user
        # Password for your PyPI user or an access token
        password: ${{ secrets.pypi_password }}
        # The repository URL to use
        # repository_url: # optional
        # The target directory for distribution
        # packages_dir: # optional, default is dist

That’s my four top python actions #

These are the easiest and most basic four actions that every python project on GitHub should have. Now that actions are available for free on any public repo there is no reason not to use GitHub Actions for any new project.

Send Emails with GitHub Actions

2020-03-31T03:39:00Z

Here is one useful thing that you can do with GitHub actions no matter what language you use, send email. You might want to know right away when your ci passes. You might want to give your team a nice pat on the back when a new release is deployed. There might be subscribers wanting to see the latest release notes in their inbox as soon as the latest version is deployed. Whatever it is, its pretty easy to do with an action right out of the actions marketplace.

Mail on Star #

Here is a silly example that sends an email to yourself anytime someone stars your repo.

name: Mail on Star

on:
  watch:
    types: [ started ]

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  # This workflow contains a single job called "email"
  email:
    # The type of runner that the job will run on
    runs-on: ubuntu-latest

    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:
      - name: ✨ Send email, you star
        uses: dawidd6/action-send-mail@v1.3.0
        with:
          server_address: smtp.gmail.com
          server_port: 465
          username: quadmx08
          password: ${{ secrets.GMAIL_PASS }}
          subject: Your a star ✨
          body: ${{ github.actor }} just starred your mail-on-star repo!!! ${{ github.repository }}
          to: ${{ secrets.GMAIL_ADDRESS }}
          from: ${{ secrets.GMAIL_ADDRESS }}

What Are GitHub Actions

2020-03-16T05:00:00Z

GitHub actions are an amazing tool that allows us to run code based on triggers inside of our repo. Their is a large and growing community of actions inside the marketplace to use with very little effort. Best of all they are free for public repositories, and private repos have a very generous free tier.

I have been diving deep into Github actions for about a month now and they are wicked good! They allow you to run any sort of arbitrary code based on events in your repo, webhooks, or schedules. They are very reasonably priced. The interface that GitHub hs developed for them is top-notch! It’s so good I have done 90% of my editing of them right from github.com.

TLDR #

some interaction to your repository triggers code to run.

#

The online editor for actions is pretty amazing. When creating a new workflow it automatically sets up a new blank workflow or a workflow from the marketplace for you in your .github/workflows directory. This is all it takes to get an action running, a yaml or yml file in the .github/workflows directory.

The editor does a great job of detecting syntax errors, misplaced keys. It also does a great job at autocompletion. As you type it will suggest keys that are accepted by the workflow syntax. There is an embedded side pannel with docs and the marketplace to the right.

#

see this article from GitHub for a full set of details: https://help.github.com/en/actions/reference/events-that-trigger-workflows

You can trigger actions to run based on about any interaction with the repo that you can imagine, push, PR, webhooks, follows, create a branch, delete a branch, deployment, fork, wiki, issues, comments, labels, milestones, just check out the GitHub article for the full list.

#

The most common and default trigger you will come across is the on push. This means that on every push/pull_request the given action will run. This is typically at the start of the file and will trigger the workflow for the whole file.

# Trigger the workflow on push or pull request
on: [push, pull_request]

You can also filter to only run on specific branches. You probably only want to run your release workflow on the main branch, but want linting and testing on all branches.

push:
  branches:
   - main
pull_request:
  branches:
    - main

#

It is also possible to set up your workflows to run on a schedule. I have set a few of these up myself to do things such as updating/auditing npm dependencies and checking if the site is up.

on:
  schedule:
    # * is a special character in YAML so you have to quote this string
    - cron:  '*/15 * * * *'

#

One issue that I have with GitHub actions is that there really isn’t a good way to manually run workflows. A workaround I found is that you can run a workflow when the repo is starred.

on:
  watch:
    types: [ started ]

If you have a public repo with some traction, you might want to avoid this hack, but if you did want to use it on a repo that may potentially get some stars randomly make sure that you filter to only your stars.

on:
  watch:
    types: [ started ]

jobs:
  run-on-star:
    runs-on: ubuntu-latest
    steps:
      - name: ✨ you starred your own repo
        if: github.actor == 'WaylonWalker'

#

GitHub offers quite a generous free tier to get you started.

I think that GitHub’s pricing just shows its commitment to the open-source. Any public repo has unlimited build minutes! I believe this goes for not only Linux actions, but the more expensive windows and mac actions as well.

#

You will find that a lot of actions need things such as a GitHub personal access token. You may even be hitting a third party API such as twitter or Gmail that require an API key. These are things that need to be kept secret DO NOT put these as raw text inside your action. The first tutorial I followed to deploy to GitHub pages did this 🤦‍♂️ and I followed.

GitHub offers a wonderful secrets manager. From your repository go to settings > secrets. You can just add settings/secrets to the URL of your repo to get there as well. From there add a new secret. Now your secret is accessible by secret key using ${{ secrets.<your-key> }} from anywhere in your workflows yml file.

GitHub has done an amazing job at hiding these secrets. Anywhere that I have seen try to echo these secrets out into the console or anywhere just shows ***. I am not sure if you can 100% rely on this, but they appear to have done a good job with it.

#

One great feature of actions is the live logs. As you are developing them it is likely that you are anxiously watching them with anticipation. Watching those logs go, and turn green is a great experience.

#

As with all things open source, much of the power of actions comes through the community and in actions case the marketplace. Reusable actions can be deployed to the github marketplace. Here they can be found from search, starred, and example workflows can be copied in one click.

I find that many times while I can write all of the code necessary in a shell script to do most of what I need, there is already an action in the marketplace that takes care of everything for me. In fact there are usually several to choose from.

#Discuss #

What Actions are you excited about?
Are you using actions today?
What struggles have you encountered with actions?
Do you like these silly image headers I used? Do they kill A11y? I attempted to use good alt text to counter.

Getting Started with GitHub Actions

2020-03-16T05:00:00Z

Github actions are written in configuration files using the YAML syntax. YAML is a superset of JSON. Most YAML can be expressed inline with JSON syntax. Similar to python YAML is whitespace driven by whitespace rather than brackets tags. The argument for using YAML for configuration files such as actions is that it is more human-readable and editable. It’s much easier to see the whitespace layout than it is to get closing brackets correct. For actions, I believe this is mostly true. I don’t see any use case to get past 3-5 indents, which is completely manageable.

Can I just say that I learned more than I realized about YAML by writing this article

Arrays and Objects #

In YAML or JSON, the most basic containers for data are arrays, a 1D list of things, and objects, for key-value pairs.

Arrays #

The start of an array container is signified with a leading -. This is probably one of the big things I didn’t understand about YAML before writing this post, but hats off to the GitHub actions editor as it took care of a lot of my misunderstanding for me.

- one
- two
- three

['one', 'two', 'three']

Objects #

Objects are just {key: "value"} pairs without any - before them. I find that objects make 💯 sense to me. Unlike arrays, they feel very intuitive.

name: one
who: me

{'name': 'one', 'who': 'me'}

Example Combining arrays and objects #

Let’s start writing something that looks a bit more like a GitHub action. GitHub actions are built from an object containing name, on, jobs. Where jobs is a list of jobs, that contain a list of steps. Simple actions will only need a single job, but commonly a list of steps.

Shortlist of GitHub action keys #

These are the keys, with their parents, that I found most useful. You can find a complete list on workflow-syntax-for-github-actions.

name
on
- push
- pull_request
- schedule
- watch
env
jobs
- name
- needs
- env
- if
- steps

Combining arrays and objects #

Here is a very small example that contains an object with one key, jobs. That jobs object contains one job called build that contains a list of steps. Notice the - before each step, and how each step repeats the same object keys.

jobs:
    build:
        runs-on: ubuntu-latest
        steps:
        - name: step-one
          uses: checkout
        - name: step-two
          uses: test
        - name: step-three
          uses: package

{
  "jobs": {
    "build": {
      "runs-on": "ubuntu-latest",
      "steps": [
        {
          "name": "step-one",
          "uses": "checkout"
        },
        {
          "name": "step-two",
          "uses": "test"
        },
        {
          "name": "step-three",
          "uses": "package"
        }
      ]
    }
  }
}

Multiline Strings #

Multiline strings are super important in GitHub actions. You will likely use the | to preserve newlines for shell scripts most commonly but may also have some raw text fields that need to be concatenated without a newline character using the > operator.

| preserves newlines
folds newlines

preserved: |
    cd my-dir
    ls
    mv public ../
folded: >
    This is some long text
    that I do not want on
    one line, but it is
    really a one-liner

{
    "preserved": "cd my-dir\nls\nmv public ../\n",
    "folded": "This is some long text that I do not want on one line, but it is  really
   a one-liner"
}

🔥 I always use | for steps that are scripts

Integration testing with Python, TestProject.io, and GitHub Actions

In 👆 This article you can see how I use the | character to run some docker compose commands in a GitHub action to run some integration tests wtih pytet, testproject.io, a headless browser, and selenium.

Anchors are not supported #

YAML has this amazing feature for reducing repetative content called anchors. You can save part of your configuration as a reusable variable in other sections. I see this being really cool if you had separate jobs that all needed similar steps. Look for this improvement in the future, for now just be aware that it is part of the YAML syntax.

See support ticket 👉 Support-for-YAML-anchors

secrets: &secrets
    github-pat: ${{ gh-pat }}
    gmail-pass: ${{ gmail-pass }}

jobs:
    build:
        - name: step-one
          uses: checkout
          <<: *secrets
        - name: step-two
          uses: test
          <<: *secrets
        - name: step-three
          uses: package
          <<: *secrets

Notice how the nice clean YAML syntax gets exploded with much more data in the JSON format.

{
    "secrets": {
        "github-pat": "${{ gh-pat }}",
        "gmail-pass": "${{ gmail-pass }}"
    },
    "jobs": {
        "build": [
            {
                "github-pat": "${{ gh-pat }}",
                "gmail-pass": "${{ gmail-pass }}",
                "name": "step-one",
                "uses": "checkout"
            },
            {
                "github-pat": "${{ gh-pat }}",
                "gmail-pass": "${{ gmail-pass }}",
                "name": "step-two",
                "uses": "test"
            },
            {
                "github-pat": "${{ gh-pat }}",
                "gmail-pass": "${{ gmail-pass }}",
                "name": "step-three",
                "uses": "package"
            }
        ]
    }
}

Writing an Action #

With a basic understanding of YAML you can probably go to your repo and click actions > new workflow > setup my own workflow right from the ui, and create your own. Or read through the official syntax docs for deeper information workflow-syntax-for-github-actions. Let’s finsh off with a really simple action, the default one from GitHub.

Important first step #

It is important to know that when running an action you will likely need access to your code in order to lint, test, build, package, whatever you want to do with it. Your first step for any action requiring code from your repo is to checkout your repo.

 steps:
 # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
 - uses: actions/checkout@v

Default example #

This example runs a workflow called CI on ubuntu on every push or PR to the main branch. Within the build job it does a checkout of the repo, then runs two shell steps.

# This is a basic workflow to help you get started with Actions

name: CI

# Controls when the action will run. Triggers the workflow on push or pull request
# events but only for the main branch
on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  # This workflow contains a single job called "build"
  build:
    # The type of runner that the job will run on
    runs-on: ubuntu-latest

    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:
    # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
    - uses: actions/checkout@v2

    # Runs a single command using the runners shell
    - name: Run a one-line script
      run: echo Hello, world!

    # Runs a set of commands using the runners shell
    - name: Run a multi-line script
      run: |
        echo Add other actions to build,
        echo test, and deploy your project.

Four github actions for your website

Python #

Four Github Actions for Python

If you are a python deveoper like me you will likely want to use GitHub actions to Lint, Test, Package, and deploy your python projects. 👆 This article covers just that.