Posts tagged: cli

Authentication from cli tools can be a bit of a bear, and I have to look it up every time. This is my reference guide for future me to remember how to easily do it.

I set up a fastapi server running on port 8000, it uses a basic auth with waylonwalker as the username and asdf as the password. The server follows along with what comes out of the docs. I have it setup to take basic auth, form username and password, or a bearer token for authentication.

curl #

The og of command line url tools.

# basic auth
curl -u 'waylonwalker:asdf' -X POST localhost:8000/token
# basic auth with password prompt
curl -u 'waylonwalker' -X POST localhost:8000/token
# token
curl -H 'Authorization: bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJ3YXlsb253YWxrZXIiLCJleHAiOjE3MDI5NTI2MDJ9.GeYNt7DNal6LTiPoavJnqypaMt4vYeriXdq5lqu1ILg' -X POST localhost:8000/token

wget #

My go to if I want the result to go into a file.

# basic auth
wget -q -O - --auth-no-challenge --http-user=waylonwalker --http-password=asdf --post-data '' localhost:8000/token

# token
wget -q -O - --header="Authorization: bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJ3YXlsb253YWxrZXIiLCJleHAiOjE3MDI5NTI2MDJ9.GeYNt7DNal6LTiPoavJnqypaMt4vYeriXdq5lqu1ILg" -O - --post-data '' localhost:8000/token

httpx #

An http client written in python, primarilty used with the python api, but has a nice cli.

# install
python3 -m pip install httpx

# basic auth
httpx -m POST --auth waylonwalker asdf http://localhost:8000/token

# basic auth with password prompt
httpx -m POST --auth waylonwalker - http://localhost:8000/token

# token
httpx -m POST --headers="Authorization" "bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJ3YXlsb253YWxrZXIiLCJleHAiOjE3MDI5NTI2MDJ9.GeYNt7DNal6LTiPoavJnqypaMt4vYeriXdq5lqu1ILg" http://localhost:8000/token

httpie #

A modern http client written in python.

# install
python3 -m pip install httpie

# basic auth
http POST localhost:8000/token -a waylonwalker:asdf

# basic auth with password prompt
http POST localhost:8000/token -a waylonwalker

# token
http POST localhost:8000/token -A bearer -a eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJ3YXlsb253YWxrZXIiLCJleHAiOjE3MDI5NTI2MDJ9.GeYNt7DNal6LTiPoavJnqypaMt4vYeriXdq5lqu1ILg

httpie with plugin #

# install
python3 -m pip install httpie-credential-store
# usage
http POST localhost:8000/token -A creds

httpie prompt #

http-prompt comes from the httpie org, and has an interactive cli interface into apis. You can even specify a spec file to autocomplete on api methods.

http-prompt localhost:8000 --auth waylonwalker:asdf --spec openapi.json

jpillora/installer is the install script generator I have been looking for. It downloads binaries for your machine from GitHub releases and unzips them for you. It grabs the latest release, so you can easily update them. I have tried scripting these installs in the past and struggled to consistently get the latest version for every package and unpack it correctly.

Also these pre-compiled binaries install rediculously fast compared to building them from source.

Check out some example links.

opening in a browser will show metadata

https://i.jpillora.com/serve

If you pass in script=true it will instead return the install script as it would by default through curl.

https://i.jpillora.com/serve?script=true

Use it to install neovim #

All you need to do to generate an install script is to pass in the GitHub repo slug with the org.

curl https://i.jpillora.com/neovim/neovim | bash

The shell script that it generates for neovim looks like this.

#!/bin/bash
if [ "$DEBUG" == "1" ]; then
    set -x
fi
TMP_DIR=$(mktemp -d -t jpillora-installer-XXXXXXXXXX)
function cleanup {
    rm -rf $TMP_DIR > /dev/null
}
function fail {
    cleanup
    msg=$1
    echo "============"
    echo "Error: $msg" 1>&2
    exit 1
}
function install {
    #settings
    USER="neovim"
    PROG="neovim"
    ASPROG=""
    MOVE="false"
    RELEASE="stable"
    INSECURE="false"
    OUT_DIR="$(pwd)"
    GH="https://github.com"
    #bash check
    [ ! "$BASH_VERSION" ] && fail "Please use bash instead"
    [ ! -d $OUT_DIR ] && fail "output directory missing: $OUT_DIR"
    #dependency check, assume we are a standard POISX machine
    which find > /dev/null || fail "find not installed"
    which xargs > /dev/null || fail "xargs not installed"
    which sort > /dev/null || fail "sort not installed"
    which tail > /dev/null || fail "tail not installed"
    which cut > /dev/null || fail "cut not installed"
    which du > /dev/null || fail "du not installed"
    #choose an HTTP client
    GET=""
    if which curl > /dev/null; then
        GET="curl"
        if [[ $INSECURE = "true" ]]; then GET="$GET --insecure"; fi
        GET="$GET --fail -# -L"
    elif which wget > /dev/null; then
        GET="wget"
        if [[ $INSECURE = "true" ]]; then GET="$GET --no-check-certificate"; fi
        GET="$GET -qO-"
    else
        fail "neither wget/curl are installed"
    fi
    #debug HTTP
    if [ "$DEBUG" == "1" ]; then
        GET="$GET -v"
    fi
    #optional auth to install from private repos
    #NOTE: this also needs to be set on your instance of installer
    AUTH="${GITHUB_TOKEN}"
    if [ ! -z "$AUTH" ]; then
        GET="$GET -H 'Authorization: $AUTH'"
    fi
    #find OS #TODO BSDs and other posixs
    case `uname -s` in
    Darwin) OS="darwin";;
    Linux) OS="linux";;
    *) fail "unknown os: $(uname -s)";;
    esac
    #find ARCH
    if uname -m | grep -E '(arm|arch)64' > /dev/null; then
        ARCH="arm64"

        # no m1 assets. if on mac arm64, rosetta allows fallback to amd64
        if [[ $OS = "darwin" ]]; then
            ARCH="amd64"
        fi

    elif uname -m | grep 64 > /dev/null; then
        ARCH="amd64"
    elif uname -m | grep arm > /dev/null; then
        ARCH="arm" #TODO armv6/v7
    elif uname -m | grep 386 > /dev/null; then
        ARCH="386"
    else
        fail "unknown arch: $(uname -m)"
    fi
    #choose from asset list
    URL=""
    FTYPE=""
    case "${OS}_${ARCH}" in
    "linux_amd64")
        URL="https://github.com/neovim/neovim/releases/download/stable/nvim-linux64.tar.gz"
        FTYPE=".tar.gz"
        ;;
    "darwin_amd64")
        URL="https://github.com/neovim/neovim/releases/download/stable/nvim-macos.tar.gz"
        FTYPE=".tar.gz"
        ;;
    *) fail "No asset for platform ${OS}-${ARCH}";;
    esac
    #got URL! download it...
    echo -n "Downloading"
    echo -n " $USER/$PROG"
    if [ ! -z "$RELEASE" ]; then
        echo -n " $RELEASE"
    fi
    if [ ! -z "$ASPROG" ]; then
        echo -n " as $ASPROG"
    fi
    echo -n " (${OS}/${ARCH})"

    echo "....."

    #enter tempdir
    mkdir -p $TMP_DIR
    cd $TMP_DIR
    if [[ $FTYPE = ".gz" ]]; then
        which gzip > /dev/null || fail "gzip is not installed"
        bash -c "$GET $URL" | gzip -d - > $PROG || fail "download failed"
    elif [[ $FTYPE = ".tar.bz" ]] || [[ $FTYPE = ".tar.bz2" ]]; then
        which tar > /dev/null || fail "tar is not installed"
        which bzip2 > /dev/null || fail "bzip2 is not installed"
        bash -c "$GET $URL" | tar jxf - || fail "download failed"
    elif [[ $FTYPE = ".tar.gz" ]] || [[ $FTYPE = ".tgz" ]]; then
        which tar > /dev/null || fail "tar is not installed"
        which gzip > /dev/null || fail "gzip is not installed"
        bash -c "$GET $URL" | tar zxf - || fail "download failed"
    elif [[ $FTYPE = ".zip" ]]; then
        which unzip > /dev/null || fail "unzip is not installed"
        bash -c "$GET $URL" > tmp.zip || fail "download failed"
        unzip -o -qq tmp.zip || fail "unzip failed"
        rm tmp.zip || fail "cleanup failed"
    elif [[ $FTYPE = ".bin" ]]; then
        bash -c "$GET $URL" > "neovim_${OS}_${ARCH}" || fail "download failed"
    else
        fail "unknown file type: $FTYPE"
    fi
    #search subtree largest file (bin)
    TMP_BIN=$(find . -type f | xargs du | sort -n | tail -n 1 | cut -f 2)
    if [ ! -f "$TMP_BIN" ]; then
        fail "could not find find binary (largest file)"
    fi
    #ensure its larger than 1MB
    #TODO linux=elf/darwin=macho file detection?
    if [[ $(du -m $TMP_BIN | cut -f1) -lt 1 ]]; then
        fail "no binary found ($TMP_BIN is not larger than 1MB)"
    fi
    #move into PATH or cwd
    chmod +x $TMP_BIN || fail "chmod +x failed"
    DEST="$OUT_DIR/$PROG"
    if [ ! -z "$ASPROG" ]; then
        DEST="$OUT_DIR/$ASPROG"
    fi
    #move without sudo
    OUT=$(mv $TMP_BIN $DEST 2>&1)
    STATUS=$?
    # failed and string contains "Permission denied"
    if [ $STATUS -ne 0 ]; then
        if [[ $OUT =~ "Permission denied" ]]; then
            echo "mv with sudo..."
            sudo mv $TMP_BIN $DEST || fail "sudo mv failed"
        else
            fail "mv failed ($OUT)"
        fi
    fi
    echo "Downloaded to $DEST"
    #done
    cleanup
}
install

Self Host Your Own #

I’d reccomend self hosting your own. This way you know that it’s consistent and unlikely to change in a way that breaks your use.

curl -s https://i.jpillora.com/installer | bash

Repos I am using installer for #

Here are the repos I am using installer for.

atuinsh/atuin
benbjohnson/litestream
bootandy/dust
BurntSushi/ripgrep
chmln/sd
cjbassi/ytop
dalance/procs
dbrgn/tealdeer
ducaale/xh
go-task/task
imsnif/bandwhich
imsnif/diskonaut
kovidgoyal/kitty
mgdm/htmlq
neovim/neovim
ogham/dog
ogham/exa
pemistahl/grex
sharkdp/bat
sharkdp/fd
sharkdp/pastel
sirwart/ripsecrets
starship/starship
topgrade-rs/topgrade
zellij-org/zellij

Quickly and easily create new versions of your Python package with the gh release command. Get the version number, changelog, and

Releasing a new version of your Python package can be a daunting task. You need to make sure that all the necessary files are included, and that the version number is correct. But now, with the help of the gh release command, you can make the process much smoother.

The gh release command allows you to quickly and easily create a new version of your Python package. All you need to do is provide the version number, the changelog, and the distribution files. For example, if you wanted to create a new version of your package with the version number v1.2.3, you could use the following command:

gh release create v1.2.3 -F CHANGELOG.md dist/*.whl dist/*.tar.gz

This command will create a new version of your package with the specified version number, and include the changelog and the distribution files. It’s a great way to make sure that all the necessary files are included in the release, and that the version number is correct.

The gh release command is a great tool for quickly and easily creating new versions of your Python package. With just a few simple commands, you can make sure that all the necessary files are included, and that the version number is correct. So if you’re looking for an easy way to release a new version of your Python package, give the gh release command a try.

Give github actions the -e flag in the shebang #! so they fail on any one command failure. Otherwise each line will set the exit status, but only the last one will be passed to ci.

#!/bin/bash -e

What is -e #

The -e flag to the bash command allows your script to exit immediately if any command within the script returns a non-zero exit status. This can be useful for ensuring that your script exits with an error if any of the commands it runs fail, which can help you identify and debug issues in your script. For example, if you have a script that runs several commands and one of those commands fails, the script will continue running without the -e flag, but will exit immediately if the -e flag is present. This can make it easier to troubleshoot your script and ensure that it runs correctly.

Solution for Windows #

In windows the solution is not quite as simple. You can define a function in a Windows batch script that wraps an if statement to check the exit status of a command and handle any errors that may have occurred. Here is an example of how you might define a function called “check_error” that does this:

:check_error
if errorlevel 1 (
  echo An error occurred!
  exit /b 1
)

To use this function in your script, you would simply call it after running a command, like this:

some_command
call :check_error

This would run the “some_command” and then call the “check_error” function to check the exit status and handle any errors that may have occurred. This approach allows you to reuse the error-checking logic in your script, which can make it easier to write and maintain.