BeautifulSoup is a DOM like library for python. It’s quite useful to
manipulate html [1]. Here is an example to find_all html headings. I stole
the regex from stack overflow, but who doesn’t.
Make an example # [2]
sample.html
Lets make a sample.html file with the following contents. It mainly has
some headings, <h1> and <h2> tags that I want to be able to find.
<!DOCTYPE html>
<html lang="en">
<body>
<h1>hello</h1>
<p>this is a paragraph</p>
<h2>second heading</h2>
<p>this is also a paragraph</p>
<h2>third heading</h2>
<p>this is the last paragraph</p>
</body>
</html>
Get the headings with BeautifulSoup # [3]
Lets import our packages, read in our sample.html using pathlib and find all
headings using BeautifulSoup.
from bs4 import BeautifulSoup
from pathlib import Path
soup = BeautifulSoup(Path('sample.html').read_text(), features="lxml")
headings = soup.find_all(re.compile("^h[1-6]$"))
And what we get is a list of bs4.element.Tag’s.
>> print(headings)
[<h1>hello</h1>, <h2>second heading</h2>, <h2>third heading</h2>]
I recently added a heading_link plugin to markata, you might notice the
🔗’s next to each heading on this page, that is powered by this exact
techniq...
Posts tagged: webdev
All posts with the tag "webdev"
210 posts
latest post 2026-05-01
Publishing rhythm
Today I discovered a sweet new cli for compressing images.
squoosh cli [1]
is a wasm powered cli that supports a bunch of formats that I would want to
convert my website images to.
from the future
> Unfortunately, due to a few people leaving the team, and staffing issues
resulting from the current economic climate (ugh), I’m deprecating the
CLI and libsquoosh parts of Squoosh. The web app will continue to be
supported and improved. I know that sucks, but there simply isn’t the
time & people to work on this. If anyone from the community wants to fork
it, you have my blessing.
https://github.com/GoogleChromeLabs/squoosh/pull/1321
Web App # [2]
First the main feature of squoosh is a web app [3] that
makes your images smaller right in the browser, using the same wasm. It’s
sweet! There is a really cool swiper to compare the output image with the
original, and graphical dials to change your settings.
CLI # [4]
What is even cooler is that once you have settings you are happy with and are
really cutting down those kb’s on your images, there is a copy cli command
button! If you have npx (which you should if you have nodejs and npm) already
installed it just works without instal...
Building Rich a Dev Server
Draft Post
I’ve really been digging [1]@willmcgugan [2]’s
rich [3] library for creating TUI like
interfaces in python. I’ve only recently started to take full advantage of it.
Dev Server # [4]
I am working on a project in which I want to have a dev
server running continuously in the background. I really
like dev servers theat automatically chooose an unused
port and list out the running pid so that I can kill it if
I need to.
- automatic port number
- auto-restart
- display ( port, pid, uptime )
finding the port # [5]
I am very novice at best when it comes to sockets, the following function came
from searching StackOverflow for how to tell if a port is in use. I
recursively check if a port is being used, if it is I increment by one until I
find an unused port to return.
def find_port(port=8000):
"""Find a port not in ues starting at given port"""
import socket
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
if s.connect_ex(("localhost", port)) == 0:
retur...
Site Down During Build
Recently I noticed a new netlify site of mine was down while I was checking to
see if new content was live. Later found out this was consistent after each
and every push the site would go gown as soon as I hit push, and would not come
back until the build finished.
Is this normal? # [1]
Do other Netlify sites go down during build???
Short Answer NO. All of my google fu lead me to believe I was alone and none of my other sites do this.
Digging into my build # [2]
My deploy script ends with the following. After resetting keys and watching it build half a dozen
times I determined that everything was working as normal here.
- name: Deploy to Netlify
uses: nwtgck/[email protected]
with:
publish-dir: "./markout"
production-branch: markout
production-deploy: true
deploy-message: "Deploy markout from GitHub Actions"
env:
NETLIFY_AUTH_TOKEN: ${{ secrets.NETLIFY_AUTH_TOKEN }}
NETLIFY_SITE_ID: ${{ secrets.NETLIFY_SITE_ID }}
Opening the Nelify Console # [3]
After poking at t...
Adding Audio to my blog posts
This is episode 1 of the Waylon Walker Audio experience, posts from
waylonwalker.com [1]{.hoverlink} in audio form.
So I have had this idea for awhile to add audio to my blog posts. The idea
partly comes from the aws blog, if you have ever been on their blog you will
have noticed that they have a voiced by amazon polly section.
What to Expect # [2]
Honestly I don’t know this is all new to me and I dont have much to go off of.
For now its a test that may or may not work out.
I will say that the time that I have available for clean audio is a bit limited
so expect these to come out in batches as I get time to go back and record.
What Not to Expect # [3]
One thing that makes the aws blog really hard to listen to is the robotic
voice, I definitely don’t want that. This will be voiced by a real human, Me.
At the same time written text doesn’t translate directly to audio well so don’t
necessarily expect the audio to be
word for word.
Code blocks # [4]
There are a lot of code block...
gatsby-remark-embedder
Inspired by discourse’s link expansion I am rolling out expansions for one line
links on the blog waylonwalker [1]. I was able to find
a gatsby plugin
gatsby-remark-embedder [2]
that expands one line links for social cards for popular platforms like twitter
and YouTube through a repose from Kyle Mathews to my tweet.
https://twitter.com/kylemathews/status/1329817928666005504
Use Cases # [3]
This covers a couple of use cases I have with very little effort.
- Twitter
- YouTube
install # [4]
npm i gatsby-remark-embedder gatsby-plugin-twitter
This was super quick and simple to setup, the only thing that was extra was to
install the gatsby-plugin-twitter plugin as well as the
gatsby-remark-embedder.
enable # [5]
// In your gatsby-config.js
module.exports = {
// Find the 'plugins' array
plugins: [
`gatsby-plugin-twitter`,
{
resolve: `gatsby-transformer-remark`,
options: {
plugins: [
{
resolve: `gatsby-remark-embedder`,
options: {
customTransformers: [
// Your custom t...
Expand One Line Links
I wanted a super simple way to cross-link blog posts that require as little effort as possible, yet still looks good in vanilla markdown in GitHub. I have been using a snippet that puts HTML [1] into the markdown. While this works, it’s more manual/difficult for me does not look the best, and does not read well as
Goals for new card # [2]
The new card should be fully automated to expand with title, description, and
cover image. Bonus if I am able to attach a comment behind it.
- fully automated
- card expansion
- Title
- description
- cover image
Old Card # [3]
If you can call it a card 🤣. This card was just an image wrapped in an anchor tag and a paragraph tag. I found this was the most consistent way to get an image narrower and centered in both GitHub and dev.to.
<p style='text-align: center'>
<a href='https://waylonwalker.com/notes/eight-years-cat/'>
<img
style='width:500px; max-width:80%; margin: auto;'
src="https://images.waylonwalker.com/eight-years-cat.png"
al...
Why use a cms
When first learning to code its very common to hard code everything right into
the code. This happens with most folks in just about any language. Whether its
HTML [1] or markdown for front end content, or even hardcoding parameters in our
backend languages like python, or node.js.
🤷♀️ What’s wrong with hard coding everything? # [2]
Hard coding everything right into your code makes it really hard for
non-technical collaborators to join. It makes it nearly impossible to hand
websites off to clients without needing to come back for routine updates.
The cms generally come with a rich content editor that feels more like
something most folks are used to. There are buttons for changing the font,
font-size, adding images, bold, italics, etc.
Sometimes I don’t feel technical # [3]
Even when you are developing for a technical audience there is a layer of
polish that comes from giving them a nice interface to edit their content in.
YouTube doesn’t have you manually inserting records into...
How I Built My GitHub Profile
I ran a discussion on dev that collected quite a list of examples in the comment section. So many great calls to action, animations, memes, and weird tricks.
[1]
My current profile # [2]
[3]
social icons # [4]
Upload all of your icons to the repo in a directory such as icons or assets, then link them with a height attribute like below. I used html [5] for mine, not sure if you can set the height in markdown.
<a href="https://dev.to/waylonwalker"><img height="30" src="https://raw.githubusercontent.com/WaylonWalker/WaylonWalker/main/icon/dev.png"></a>
note I did add a bit of (non-breaking-whitespace) between my icons. Without adding css this seemed like the simplest way to do it.
Center # [6]
Aligning things in the center of the readme is super simple. I used this trick to align my social icons in the middle.
<p align='center'>
...html
</p>
right # [7]
For my latest post [8] I floated it to the right with a little bit of align='right' action.
<p>
<a ...
Refactoring your blog urls
I just did a quick refactoring of my JAMStack blog urls. Some didn’t fit with
my style, some had _ that I wanted to switch to -, and others were
ridiculously long. I’ve been using forestry as my CMS, I write many of my
posts there, and sometimes it picks some crazy file names (based on my titles).
It was time to refactor.
Large Refactor At The Command Line [1]
When refactorings similar to this get really big I often need to do some
project wide find an replace, I usually do this right from the command line.
🖊 Rename posts change the filename # [2]
My post urls are based on the file name of my markdown file, so I can simply go
through my filesystem and rename anything I want. From here its probably best
to only commit the addition of the new file name, until the redirects clear,
but these are all low traffic posts for me so I just commited both at once.
Safely redirect without breaking links
_redirects ⤴ /redirects # [3]
I am hosted on netlify, which automatically puts very ⚡ ...
Gatsby Scripts with onload
This might be useful
https://github.com/nfl/react-helmet/issues/146
https://github.com/gatsbyjs/gatsby/issues/13136
How to get Dev Comments from an article Url
I want to incorporate some of the wonderful comments, \U0001F495, \U0001F984,
and \U0001F516’s that I have been getting on dev.to on my website. I have
dabbled once or twice with no avail this time I am taking notes on my journey,
so follow along and let’s get there together. By the end of this post, I will
have a way to get comments from posts on the client-side thanks to the
wonderfully open dev.to API.
I want to incorporate some of the wonderful comments, 💕, 🦄, and 🔖’s that I have been getting on dev.to on my website. I have dabbled once or twice with no avail this time I am taking notes on my journey, so follow along and let’s get there together. By the end of this post, I will have a way to get comments from posts on the client-side thanks to the wonderfully open dev.to API.
The API # [1]
dev.to has an open API that allows us to easily get comments as HTML [2]. They have their API hosted at https://docs.forem.com/api/#tag/comments, let’s take a look at it.
[3]
Here we can...
Adding google fonts to a gatsbyjs site
stack overflow link [1]
References:
[1]: https://stackoverflow.com/questions/47488440/how-do-i-add-google-fonts-to-a-gatsby-site
Explicit vs Implicit Returns in Javascript
Often when reading through javascript examples you will find some arrow
functions use parentheses () while others use braces {}. This key
difference is that parentheses will implicitly return the last statement while
braces require an explicit return statement. It is important to understand the
difference between them because it is likely that you will find code examples
of both and trying to edit code written differently than you’re used to may
have unintended consequences.
[1] # [2]
Arrow functions are one-liner functions in javascript that have two main syntactical ways to create the code block. with parentheses and braces. Let’s take a look at both ways of creating arrow functions so that when we come accross them in the wild it will all make sense.
[3] # [4]
Here is an example of an arrow function that will implicitly return the last
statement without the return keyword. I believe that these are a bit more restricted
in that you cannot set variables inside them. They are ...
Do You Hoist
I am working through Wes Bos’s
beginnerjavascript.com/ [1] I just hit module 18
on hoisting. It’s something that I always knew was there, Its not something I
typically see used or use myself.
Do you Hoist? # [2]
Do you have any use cases that you use hoising? Why? It seems like a really cool feature in any language that uses it, but I dont really notice it in use.
What is Hoising # [3]
There are many articles that cover this in far more depth, but its the idea that variable declarations and functions are defined before they are executed. This means that it doesnt matter if you call a function before or after it is defined.
Hoisting # [4]
console.log(`Hello ${getUser()}`)
function getUser() {
return 'Waylon'
}
Running this code will log out “Waylon”
What about variable hoisting # [5]
I am most familiar with python which does not variable hoist so this one kinda confused me at first. It only hoists the variable declaration not the value of the variable. It defines whether th...
Custom Scrollbar Design
Getting a custom scrollbar on your site makes it stand out a bit compared to
the very plain stock one that are on most sites. This is how I set mine up on
my gatsby site.
Inspired by Wes Bos’s new uses.tech [1] I wanted a custom
scrollbar on my personal site. I had tried to do it in the past, but gave up
after it was not working.
Looking at the Source # [2]
Since uses.tech [1] is open source I jumped on github, searched for scroll and found this layout.js [3].
Copy it to my own component # [4]
My first step was to take his css and copy it into a styled component for my entire layout, but it failed. I do not fully understand why. None of the custom style came through at all. If you know please leave me a comment.
[5]
I suspect for some reason it has to do with attatching to the html [6] element inside of a styled-component. I think wes was able to get around this by using createGlobalStyle. But I was still using much of the default gatsby template, so I did not have a createG...
2020 waylonwalker.com rebrand
Moving into 2020 I have been really leaning on using purple as my theme color
everywhere more and more. Its time for an update to my personal site, not
just because it feels plain, not just because the cover art I am using for
dev.to doesn’t fit my current card layout, but because I feel inspired and
I want to.
Starting point # [1]
This is what we are working with. It has been my card design for at least a
year now. Its not bad but, its a bit play, doesnt fit my new cover art style,
and that date is not working over top of the cover art text.
- plain
- cover art does not fit
- I am not digging the date on cover art that also has text
Colors # [2]
I have been really into using a deep purple lately. It is a neutral color that
does not get enough respect, i.e. it’s not used as frequently and kinda stands
out when used.
How I pick colors # [3]
I am really bad at picking colors t...
I just added react-headroom to my site
It was so easy to get a professional looking navbar with just 3 lines of code.
This package seriously is so usable on mobile it is ridiculous. I found this
package from
day-4 [1]
of the 100 days of gatsby challenge. It is by the wonderful man who brought us
gatsbyjs Kyle Mathews, so you know its gotta be good.
install react-headroom # [2]
installation is easy
npm i react-headroom
Import Headroom # [3]
There was no instructions for es6 style imports that are common with gatsbyjs
sites like mine, but it was intuitive to figure out.
import Headroom from 'react-headroom'
Using Headroom # [4]
Simply wrap your existing content, Nav in my case, with the <Headroom />
component and your off to the races. The content will pop back into view when
you scroll past then back up.
<Headroom>
<-- Your content goes here -->
</Headroom>
See it in action # [5]
I think this simple package completely changes the ux of your site on mobile.
You can get that sticky nav out of the way, but its still...
Serverless things to investigate
Social Automation # [1]
Zappier # [2]
- Free
- only one step is free
- can take rss to social
Automate.io # [3]
-
Form submission # [4]
https://getform.io
Free to get started
https://formkeep.com/?#pricing [5]
Free 7 day trial $4.99 cheapest after
References:
[1]: #social-automation
[2]: #zappier
[3]: #automateio
[4]: #form-submission
[5]: https://formkeep.com/
I finally fixed my Styled-Components in gatsby.js
I finally fixed my Styled-Components in gatsby.js. I am starting a redesign of
my website. I have started cross posting to dev.to [1] more
regularly. With that I have been making more detailed cover images at the
recommended 1000x420. These images get cut off on my own site, which is a
bit ridiculous to have my own content not look right on my own site. But
before we start a heavy redesign I have a small issue that has plagued the site
for at least a year!
I have a small issue that has plagued the site for at least a year!
I have been using styled components in my gatsby.js site for about a year now. And it has been plagued by styled-components not being in the ssr causing some jank in the styles being loaded. You can see it in the lighthouse performance report below. It generally loads super quick, and is only caused on first load. Anything using the gatsby <Link/> component typically is fine and unaffected by the issue.
The fix # [2]
The fix was so simple it was only 2 lines t...