Why I Built Litestream - Litestream
Despite an exponential increase in computing power, our applications require more machines than ever because of architectural decisions made 25 years ago. You can eliminate much of your complexity ...
litestream.io [1]
As applications scale to the edge, to put compute as close to the user as possible, database queries back to the master node get slower and slower. Enter sqlite replication, put the database wtih the application code and replicate from master.
References:
[1]: https://litestream.io/blog/why-i-built-litestream/
Publishing rhythm
I'm All-In on Server-Side SQLite
Ben Johnson has joined Fly.io
Fly · fly.io [1]
SQLite is the next big database trend. with more horizontal scaling, close to user read heavy applications, having your database in the same application stack makes a lot of sense. Tools like litestream are going to enable global distribution in an impressive way.
References:
[1]: https://fly.io/blog/all-in-on-sqlite-litestream/
LiteFS Cloud: Distributed SQLite with Managed Backups
Documentation and guides from the team at Fly.io.
Fly · fly.io [1]
Fly.io’s solution to sqlite managed backups.I definitely want to look into this a bit, but moreso the tech under the hook litestream.
References:
[1]: https://fly.io/blog/litefs-cloud/
If you’re into interesting projects, don’t miss out on litestream [1], created by benbjohnson [2].
Streaming replication for SQLite.
References:
[1]: https://github.com/benbjohnson/litestream
[2]: https://github.com/benbjohnson
I’m impressed by flameshow [1] from laixintao [2].
A terminal Flamegraph viewer.
References:
[1]: https://github.com/laixintao/flameshow
[2]: https://github.com/laixintao
Looking for inspiration? installer [1] by jpillora [2].
One-liner for installing binaries from Github releases
References:
[1]: https://github.com/jpillora/installer
[2]: https://github.com/jpillora
GitHub - jpillora/installer: One-liner for installing binaries from Github releases
One-liner for installing binaries from Github releases - jpillora/installer
GitHub · github.com [1]
This is a sick looking bash script generator for installing binaries off of github releases. it reccomends curl into bash, but you could curl into install.sh and toss that in your dotfiles repo or wherever.
Install installer with installer
curl -s https://i.jpillora.com/installer | bash
References:
[1]: https://github.com/jpillora/installer
How to run pods as systemd services with Podman
Podman is well known for its seamless integration into modern Linux systems, and supporting systemd is a cornerstone in these efforts. Linux commonly uses th...
redhat.com [1]
podman comes with a nice command for generating systemd service files (units).
$ podman pod create --name=my-pod
635bcc5bb5aa0a45af4c2f5a508ebd6a02b93e69324197a06d02a12873b6d1f7
$ podman create --pod=my-pod --name=container-a -t centos top
c04be9c4ac1c93473499571f3c2ad74deb3e0c14f4f00e89c7be3643368daf0e
$ podman create --pod=my-pod --name=container-b -t centos top
b42314b2deff99f5877e76058ac315b97cfb8dc40ed02f9b1b87f21a0cf2fbff
$ cd $HOME/.config/systemd/user
$ podman generate systemd --new --files --name my-pod
/home/vrothberg/.config/systemd/user/pod-my-pod.service
/home/vrothberg/.config/systemd/user/container-container-b.service
/home/vrothberg/.config/systemd/user/container-container-a.service
References:
[1]: https://www.redhat.com/en/blog/podman-run-pods-systemd-services
I like MordechaiHadad’s [1] project bob [2].
A version manager for neovim
References:
[1]: https://github.com/MordechaiHadad
[2]: https://github.com/MordechaiHadad/bob
makeplane [1] has done a fantastic job with plane [2]. Highly recommend taking a look.
🔥 🔥 🔥 Open Source JIRA, Linear, Monday, and Asana Alternative. Plane helps you track your issues, epics, and product roadmaps in the simplest way possible.
References:
[1]: https://github.com/makeplane
[2]: https://github.com/makeplane/plane
Pagefind
Pagefind is a fully static search library that aims to perform well on large sites, while using as little of your users’ bandwidth as possible, and without hosting any infrastructure.
Pagefind · pagefind.app [1]
Pagefind is absolutely insane. I’ve tried a number of static site searches, and found them all hard to get get going, clunky and not the best experience as a user or developer.
I setup pagefind in about 2 minutes on my site where it found and indexed 833 pages in 2 minutes.
The only downside I see so far is that it is a lot of bandwidth to the user. On simulated slow 3G you can definitly feel it, but not terrible. Anything slower and its going to start feeling frustrating.
edit: I have actually fully deployed it on waylonwalker.com, and its fast!
create the index
npx -y pagefind --site public --serve
Then I put this on a page, it looks really nice on a white background, but would need some work to drop into a dark theme.
<link href="/pagefind/pagefind-ui.css" rel="stylesheet">
<script src="/pagefind/pagefind-ui.js"></script>
<div id="search"></div>
<script>
window.addEventListener('DOMContentLoaded', (event) => {
new PagefindUI({ element: "#search", s...
If you’re into interesting projects, don’t miss out on project.nvim [1], created by ahmedkhalf [2].
The superior project management solution for neovim.
References:
[1]: https://github.com/ahmedkhalf/project.nvim
[2]: https://github.com/ahmedkhalf
I’ve recently given tailwindcss a second chance and am really liking it. Here is how I set it up for my python based projects.
https://waylonwalker.com/a-case-for-tailwindcss
Installation #
npm is used to install the cli that you will need to configure and compile tailwindcss.
npm install -g tailwindcss-cli
Setup #
You will need to create a tailwind.config.js file, to get this you can use the cli.
npx tailwindcss init
Using tailwind with jinja templates #
To set up tailwind to work with jinja templates you will need to point the tailwind config content to your jinja templates directory.
module.exports = {
content: ["templates/**/*.html"],
};
Setting up the base styles #
I like to use the @tailwind base;, to do this I set up an input.css file.
@tailwind base;
@tailwind components;
@tailwind utilities;
Compiling #
Now that it’s all setup you can run the tailwindcss command. You will get an output.css with base tailwind plus any of the classes that you used.
tailwindcss -i ./input.css -o ./output.css --watch
-
Dang Mariah, killing it with continuous learning perspective.
A Case For Tailwindcss
I was watching @theprimeagen recently and I think he sold me on using
tailwindcss. The thing about tailwind is that it is not a big component
library, it’s a set of css classes mapped to a few (usually one) style.
All css classes are shitty, so you might as well use someone else’s shitty
css classes on all your projects rather than thinking you’re being smart with a
new set of classes that you will hate in 6 months when you come back to the
project. roughly quoted from memory of @theprimeagen
It’s tiny # [1]
So unlike big component libraries like tailwind, it comes with a cli that that
it uses to create the final css file. It is able to treeshake out all the
tailwind classes that you are not using and only ship the ones that you are
using.
It’s hard to clash # [2]
Since the classes are so small and single purpose it’s hard to end up with
something like .card in two places that mean different things causing you to
duplicate most of that css anyways so that the whole design doesn...
Simon Willison (@simonw) on X
Anyone got a lead on a good embedding model that can embed both images and text into the same space, so you can search for "dog" and get back images most likely to contain a dog?
It looks like Vis…
X (formerly Twitter) · twitter.com [1]
Kinda mindblown that this is even possible. This is so far outside of my current thinking that i didn’t even think of an elegant way to implement semantic search accross images and text at the same time. I know it happens at Google, but I envision that as still text search accross tags and meta data about the image.
Based on the number of responses CLIP is the thing that does this.
References:
[1]: https://twitter.com/simonw/status/1700528222382027039
I came across textual-web [1] from Textualize [2], and it’s packed with great features and ideas.
Run TUIs and terminals in your browser
References:
[1]: https://github.com/Textualize/textual-web
[2]: https://github.com/Textualize
GitHub - aca/emmet-ls: Emmet support based on LSP.
Emmet support based on LSP. Contribute to aca/emmet-ls development by creating an account on GitHub.
GitHub · github.com [1]
This is the greatest nvim emmet plugin I have tried. In the past I had tried the vim plugin a few times and just could not get a good flow with the keybindings and found it confusing for my occasional use. emmet-ls just uses lsp-completion, so its the same flow as other completions.
You can try it out by installing with :Mason
config # [2]
local lspconfig = require('lspconfig')
local configs = require('lspconfig/configs')
local capabilities = vim.lsp.protocol.make_client_capabilities()
capabilities.textDocument.completion.completionItem.snippetSupport = true
lspconfig.emmet_ls.setup({
-- on_attach = on_attach,
capabilities = capabilities,
filetypes = { "css", "eruby", "html", "javascript", "javascriptreact", "less", "sass", "scss", "svelte", "pug", "typescriptreact", "vue" },
init_options = {
html = {
options = {
-- For possible options, see: https://github.com/emmetio/emmet/blob/master/src/config.ts#L79-L267
["bem.enabled"] = true,
},
},
}
})
References:
[1]: https://github.com/aca/emme...
LLM now provides tools for working with embeddings
LLM is my Python library and command-line tool for working with language models. I just released LLM 0.9 with a new set of features that extend LLM to provide tools …
Simon Willison’s Weblog · simonwillison.net [1]
Simon’s llm cli is getting quite interesting. I really want to run some clustering on my website content.
References:
[1]: https://simonwillison.net/2023/Sep/4/llm-embeddings/
Formatter
How to use the Biome formatter.
Biome · biomejs.dev [1]
Tried out biome today and it worked better than prettier on jinja templates, I might adopt this over prettier.
References:
[1]: https://biomejs.dev/formatter/
Make the easy things easy
It’s so easy to get out of rhythm, get busy, and drop the ball on some things
that you really want to do or should do. This blog is a good example. I took
some time off for some family reasons, but have taken a long time to get back
to it simply because I am out of rhythm. As I am trying to get back into the
rhythm there is some tooling that I have set up for it that I completely forgot
about that feel good to use again.
Repetitive Tasks # [1]
Simple Repetitive Tasks that I have to do often can just feel soul crushing,
and one main thing that got me interested in programming.
AI tools are becoming more and more useful at solving these problems. For
instance code generation tools like co-pilot or codeium are really good at
boilerplate and pattern repetition. Things that used to be a few vim macros is
now just banging on tab.
I often look for setting up templates or some sort of snippet to replace a big
chunk of boilerplate that I know I will need over and over.
timebox # [2]
Do...
Check out aboutfeeds [1] by genmon [2]. It’s a well-crafted project with great potential.
Web feeds/RSS “getting started” guide for new users.
References:
[1]: https://github.com/genmon/aboutfeeds
[2]: https://github.com/genmon
htmx ~ The disable-element Extension
htmx gives you access to AJAX, CSS Transitions, WebSockets and Server Sent Events directly in HTML, using attributes, so you can build modern user interfaces with the simplicity and power of hypert...
v1.htmx.org [1]
An extension to disable elements during flight of an htmx [2] request, Looks super useful for things like a create or delete button where the server would end up with an error if you double delete or double create. This eliminates an error path that the user might see under normal use of the ui.
References:
[1]: https://v1.htmx.org/extensions/disable-element/
[2]: /htmx/
htmx ~ hx-indicator Attribute
The hx-indicator attribute in htmx allows you to specify the element that will have the `htmx-request` class added to it for the duration of the request. This can be used to show spinners or progre...
htmx.org [1]
The htmx-request class is added to htmx-target elements. You can target this css selector to create loading state throbbers.
By default the target element will the self, but you can use the typical htmx [2] css selector to select which element will recieve the htmx-request class while the request is running.
The only way to override the name of the class is through config.
References:
[1]: https://htmx.org/attributes/hx-indicator/
[2]: /htmx/
-
Prime concisely made sense of why htmx is so awesome compared to what has become modern reactive web dev in 2 minutes. I had never thought of it this way and it’s incredible.
One thing I have comepletely missed out on with my use of htmx is setting the disabled state while the server is working, what a genius move!
References:
[1]: /htmx/
htmx ~ Examples ~ Updating Other Content
htmx gives you access to AJAX, CSS Transitions, WebSockets and Server Sent Events directly in HTML, using attributes, so you can build modern user interfaces with the simplicity and power of hypert...
htmx.org [1]
Three ways to support updating other content. Fantastic article walking through the different ways to update other parts of the screen using htmx [2].
In htmx there is no 2 way data binding, the dom is your state, and if you have elements derived from the same data on the screen in different places you need to think about how to keep them in sync.
References:
[1]: https://htmx.org/examples/update-other-content/
[2]: /htmx/
Bigger Applications - Multiple Files - FastAPI
FastAPI framework, high performance, easy to learn, fast to code, ready for production
fastapi.tiangolo.com [1]
Fastapi [2] lets you tag your APIRouter’s so that the swagger docs are grouped according to the router.
router = APIRouter(tags=['router'])
Now all routes in router will appear in the router group in the swagger docs.
References:
[1]: https://fastapi.tiangolo.com/tutorial/bigger-applications/#another-module-with-apirouter
[2]: /fastapi/
Custom pages and templates - Datasette documentation
docs.datasette.io [1]
Datasette has its own static server that can host assets such as style sheets.
datasette -m metadata.json --static assets:static-files/
References:
[1]: https://docs.datasette.io/en/stable/custom_templates.html#serving-static-files
Check out hedgedoc [1] by hedgedoc [2]. It’s a well-crafted project with great potential.
HedgeDoc - Ideas grow better together
References:
[1]: https://github.com/hedgedoc/hedgedoc
[2]: https://github.com/hedgedoc
Just starred htmx-lsp [1] by ThePrimeagen [2]. It’s an exciting project with a lot to offer.
its so over
References:
[1]: https://github.com/ThePrimeagen/htmx-lsp
[2]: https://github.com/ThePrimeagen
[1]
Tailwind css component library. There are many examples with copy and pastabily with the tailwind classes already setup.
References:
[1]: /static/https://preline.co/docs/index.html
Tailwind CSS Cheat Sheet
Cheat sheet to learn Tailwind CSS quickly. Browse and search all Tailwind utility classes or CSS properties on one page.
nerdcave.com [1]
A nice searchable cheatsheet for tailwindcss classes.
References:
[1]: https://nerdcave.com/tailwind-cheat-sheet
cURL Command Without Using Cache | Baeldung on Linux
A quick and practical guide to using curl without cache.
Baeldung on Linux · baeldung.com [1]
Busting cache with curl. I’m not sure how much gets cached by curl, but I have ran into several cases where I am looking for new content and I want to ensure the content is new and no chance of being cached.
This article suggests 3 different techniques.
curl -H 'Cache-Control: no-cache, no-store' http://www.example.com
curl -H 'Pragma: no-cache' http://www.example.com
curl http://www.example.com/?xyzzyspoon
References:
[1]: https://www.baeldung.com/linux/curl-without-cache#adding-the-pragma-http-header
[1]
sqlite has 3 different tokenizers, porter, ascii, trigram.
These can be used with sqlite-utils.
sqlite-utils enable-fts --tokenize porter database.db post title message tags
And with the python api.
db = Database('database.db')
db["post"].enable_fts(
["title", "message", "tags"], create_triggers=True, tokenize="trigram"
)
posts = list(db["post"].search(search))
References:
[1]: /static/https://www.sqlite.org/fts5.html
GitHub - sharkdp/bat: A cat(1) clone with wings.
A cat(1) clone with wings. Contribute to sharkdp/bat development by creating an account on GitHub.
GitHub · github.com [1]
Bat is my favorite pager, its the one for me that seems to just work more than the rest. colors, syntax highlighting, line numbers search, it just feels the most natural.
References:
[1]: https://github.com/sharkdp/bat
Check out server-hot-reload [1] by mikeckennedy [2]. It’s a well-crafted project with great potential.
Include in your web projects for dev-time auto reloading of web browser when any change is detected in content.
References:
[1]: https://github.com/mikeckennedy/server-hot-reload
[2]: https://github.com/mikeckennedy
sqlite_utils Python library - sqlite-utils
sqlite-utils.datasette.io [1]
sqlite-utils is primarily a cli tool for sqlite operations such as enabling full text search, and executing searches, but it also has a nice python api that is exposed and pretty straightforward to use.
from sqlite_utils import Database
db = Database("database.db")
db["post"].enable_fts(["title", "message", "tags])
db["post"].search("water")
This returns a generator object that you can iterate over the row objects with.
References:
[1]: https://sqlite-utils.datasette.io/en/stable/python-api.html#full-text-search
External Link
levelup.gitconnected.com [1]
Use prettier to format all files in a directory. By default prettier does not write, it just echos out the format that it would do. Give it the --write and it will write the changes to the files.
prettier --write .
I just used this on my thoughts repo.
prettier --write templates
References:
[1]: https://levelup.gitconnected.com/how-to-format-all-files-in-a-directory-with-prettier-5f0ff5f4ffb2
GitHub - simonw/datasette-render-markdown: Datasette plugin for rendering Markdown
Datasette plugin for rendering Markdown. Contribute to simonw/datasette-render-markdown development by creating an account on GitHub.
GitHub · github.com [1]
datasette really does everything doesn’t it!
References:
[1]: https://github.com/simonw/datasette-render-markdown
GitHub - simonw/shot-scraper: A CLI utility for taking screenshots of websites, recording video demos and scraping sites using JavaScript
A CLI utility for taking screenshots of websites, recording video demos and scraping sites using JavaScript - simonw/shot-scraper
GitHub · github.com [1]
> A command-line utility for taking automated screenshots of websites
Daaaang, this is such an elegantly simple way to get web screenshots with a cli. I was literally up and running with two commands on my arch linux machine (which it warned was unsupported by playwright).
pip install shot-scraper
# Now install the browser it needs:
shot-scraper install
shot-scraper waylonwalker.com
shot-scraper https://datasette.io/
shot-scraper https://datasette.io/ -h 1280 -w 1920
shot-scraper https://datasette.io/ -h 480 -w 720
shot-scraper shot --selector '#posts' https://thoughts.waylonwalker.com/post/89
Note shot-scraper https://datasette.io/ is a full length screenshot of the entire page.
Oh and its pretty dang fast, let alone the setup time, this crushes on startup time in my attempts to use a headless browser in the past.
References:
[1]: https://github.com/simonw/shot-scraper
shot-scraper: automated screenshots for documentation, built on Playwright
shot-scraper is a new tool that I’ve built to help automate the process of keeping screenshots up-to-date in my documentation. It also doubles as a scraping tool—hence the name—which I …
Simon Willison’s Weblog · simonwillison.net [1]
An interesting way to build automatically annotaatd docs with arrows pointing to elements on a webpage.
References:
[1]: https://simonwillison.net/2022/Mar/10/shot-scraper/#a-complex-example
External Link
youtube.com [1]
I’d never given this much thought, but there are so many guides that are complete guides for beginner workflows, but once you get beyond beginner there is likely no manual for what you are trying to do in programming. There is no guide that will tell you the best way to get your companies salesforce data, alongside of the ERP data and present it to the users who need to know in a way that compels them to make the right decisions. You are going to have to build this out for yourself by piecing together knowledge about each subject.
References:
[1]: https://www.youtube.com/shorts/wsEuPYFpDgk
HTML Over The Wire | Hotwire
Hotwire is an alternative approach to building modern web applications without using much JavaScript by sending HTML instead of JSON over the wire.
hotwired.dev [1]
An alternative approach to building modern web withhout heavy js and json, but instead html [2] over the wire, keeping the logic in the backend of rails.
References:
[1]: https://hotwired.dev/
[2]: /html/
How do I post form data using Curl?
ReqBin is the most popular online API testing tool for REST, SOAP and HTTP APIs.
ReqBin · reqbin.com [1]
How to pass form data with curl, give it the d.
curl -X POST https://reqbin.com/echo/post/form
-H "Content-Type: application/x-www-form-urlencoded"
-d "param1=value1¶m2=value2"
References:
[1]: https://reqbin.com/req/c-sma2qrvp/curl-post-form-example
I’m really excited about SHARK-Studio [1], an amazing project by nod-ai [2]. It’s worth exploring!
SHARK Studio – Web UI for SHARK+IREE High Performance Machine Learning Distribution
References:
[1]: https://github.com/nod-ai/SHARK-Studio
[2]: https://github.com/nod-ai
The work on AMD-SHARK-Studio [1] by nod-ai [2].
AMD-SHARK Studio – Web UI for SHARK+IREE High Performance Machine Learning Distribution
References:
[1]: https://github.com/nod-ai/AMD-SHARK-Studio
[2]: https://github.com/nod-ai
Vue.js
Vue.js - The Progressive JavaScript Framework
vuejs.org [1]
A super handy reference to the vuejs lifecycle.
[2]
References:
[1]: https://vuejs.org/guide/essentials/lifecycle.html#lifecycle-diagram
[2]: https://vuejs.org/assets/lifecycle.16e4c08e.png
How to Use HTML to Open a Link in a New Tab
Tabs are great, aren't they? They allow the multitasker in all of us to juggle a bunch of online tasks at the same time. Tabs are so common now that, when you click on a link, it's likely it'll ope...
freeCodeCamp.org · freecodecamp.org [1]
Most of the time when creating links in html [2] you want to maintain the default behavior, as this is what users are going to expect, but sometimes your site behaves such that it does not fit, and it does something unexpected anyways. in this case you might want to make the default behavior to open the link in a new tab rather than relying on users to control click.
Use this with restraint as this can make your site feel janky and do things that do not feel natural to the web.
<p>Check out <a href="https://www.freecodecamp.org/" target="_blank" rel="noopener noreferrer">freeCodeCamp</a>.</p>
References:
[1]: https://www.freecodecamp.org/news/how-to-use-html-to-open-link-in-new-tab/
[2]: /html/
Create Models with a Many-to-Many Link - SQLModel
SQLModel, SQL databases in Python, designed for simplicity, compatibility, and robustness.
sqlmodel.tiangolo.com [1]
Creating many to many relationships with sqlmodel requires a LinkTable Model. The link model will keep track of the linked id’s between each of the models.
[2]
from typing import List, Optional
from sqlmodel import Field, Relationship, Session, SQLModel, create_engine
class HeroTeamLink(SQLModel, table=True):
team_id: Optional[int] = Field(
default=None, foreign_key="team.id", primary_key=True
)
hero_id: Optional[int] = Field(
default=None, foreign_key="hero.id", primary_key=True
)
class Team(SQLModel, table=True):
id: Optional[int] = Field(default=None, primary_key=True)
name: str = Field(index=True)
headquarters: str
heroes: List["Hero"] = Relationship(back_populates="teams", link_model=HeroTeamLink)
class Hero(SQLModel, table=True):
id: Optional[int] = Field(default=None, primary_key=True)
name: str = Field(index=True)
secret_name: str
age: Optional[int] = Field(default=None, index=True)
teams: List[Team] = Relationship(back_populates="heroes", link_model=HeroTeamLink)
References:
[1]...
External Link
stackoverflow.com [1]
I went down the route of leveraging the json-enc extention in htmx [2], but later realized that this completely breaks browsers/users who do not wish to use javascript. While most of the web would feel quite broken with javascript disabled, I don’t want to contribute to that without good reason.
Taking a second look into this issue, rather than using json-enc, and using as_form to get form data into a model keeps the nice DX fo everything being a pydantic model, but the site still works without js. with js htmx kicks in, you get a spa like experience by loading partials onto the page, and without, you just get a full page reload.
the implementation # [3]
copied from https://stackoverflow.com/questions/60127234/how-to-use-a-pydantic-model-with-form-data-in-fastapi
import inspect
from typing import Type
from fastapi import Form
from pydantic import BaseModel
from pydantic.fields import ModelField
def as_form(cls: Type[BaseModel]):
new_parameters = []
for field_name, model_field in cls.__fields__.items():
model_field: ModelField # type: ignore
new_parameters.append(
inspect.Parameter(
model_field.alias,
inspect.Parameter.POSITION...