ryan.himmelwright.net

2026 Yearly Theme

2026-05-08T01:55:01Z

Durham, NC

Better late than never, right?

Deciding the Theme

This year I was a bit different in deciding my yearly theme, mostly in how and when I named it. When the year started, I had no major drive to figure out my theme immediately. I opted to mull over it instead. During those first few months, I quickly recognized a pattern in my thinking for my theme, the phrase “everything is up for reconsideration”. However, I couldn’t determine a nice Cortex-style name for it.

Fast-forward to April, and I decided that I really needed to clean up and formalize this vague idea now that it was the second quarter of the year. After some brainstorming and dabbling, I stumbled on The Year of Sovereign Inquiry.

Year of Sovereign Inquiry

Weird name, right? But it works. To expand a bit, the breakdown is something like this:

Sovereign → self-governing, independent, in ultimate control of your life and choices

Inquiry → active questioning, investigation, testing assumptions

Why it works: It frames the year as a process of questioning everything under my own authority. This includes habits, philosophies, systems, and tech.

The name goes back to that feeling that everything in my life, no matter how long I’ve held onto it, is really up for evaluation this year. I’m slowly going through various areas of my life and looking at everything and wondering “does it have to be this way?”. I then test out the alternatives.

I had been working with that mindset throughout 2026 already, and I finally had a name for it.

So Far

As a yearly theme, it’s been working quite well. By default, I am already someone that questions everything. Well, almost everything. This theme has encouraged me to push that extra bit whenever I come up against something I would normally consider untouchable. Instead of blindly moving on, I think “Go ahead. Anything is up for reconsideration, it’s the ‘Year of Sovereign Inquiry’…”.

Since the start of the year, there have been a number of ‘inquiries’ I’ve dug into. Many of these might appear as habit changes I would likely investigate regardless, but now, I am doing so with much less hesitation. I am also willing to test out alternatives no matter how long-standing or important the current system is. Here are a few examples:

Reliance on Apple

I’ve been using NixOS more heavily on the GPD Pocket as my secondary daily driver.
This has pushed me to slowly move away from services and apps that are locked within Apple.
I got an Android device (original Google Pixel) to test out GrapheneOS.
I’m researching how to use a stack like Flutter+ClojureDart for app development instead of straight Swift+SwiftUI. It would be cross-platform and allow for the majority of the development (still need to build/debug iOS/macOS apps on Mac) to also be done on Linux if I choose.

Shifting Local-first and Home-labbing again

I’ve been moving some services and systems to self-hosted options when it makes sense. Sometimes it’s to enable the service outside of Apple, sometimes it’s to get my data out of the cloud. Other times, the self-hosted option is simply better. Here are a few I set up:
- Immich - Photo backup outside of Apple Photos formatting. Available to GrapheneOS and Linux devices now.
- Pinchflat - continuing to test out systems like this to view YouTube content like my text RSS feeds.
- Nextcloud - my sync service, notes on mobile, and honestly a playground for other stuff (calDAV, etc).
- Switched to Unifi gear - Better network control. I can disable WAN on my client devices (for better focus), and still use my services on the homelab (which is still connected to WAN to do its thing).

Notes/Journal

Switched from org mode to simply using markdown files, except when org makes more sense (when tables, TODO lists, and other features are important).
Configuring various clients to properly use links in my notes was a pain. So I stopped using links. (Do I need links? Do I actually use them often, if at all? --> No)

Time Tracking

I switched to using time-warrior instead of dedicated apps. I have it configured on my server and make entries via ssh using a shell alias or Apple Shortcut. It works surprisingly well. It started as a short experiment, but I have been using it now for months.

Task Management

I tested out Super Productivity for dev and hobby tasks. I wanted something more open since I was writing dev notes in my project tasks.
I tested Vikunja for self-hosting options.
I am trying to use a simpler tag-based workflow (ex: using day, week, season and other tags instead of dates for management).

Limiting Mental Inputs

Video

I’ve dramatically pulled back on video content I consume. I don’t watch videos at lunch, and block YouTube for most of the day (and only sometimes work around it XD )
I’m trying to use systems like Pinchflat for better intentional viewing

Podcasts

I’m testing out blocking my podcast player in the morning (!). Podcasts are great and I love listening to them during the day, but given the topics I listen to, I often want to ‘look something up’, or spec out a configuration for tech I won’t buy (which is a big time-suck of mine). So, I’m at least delaying that during the day. It’s going well so far.

Internet

Like podcasts, I’ve started to dabble with periods of working offline.
With the Unifi and home server changes, I can have my client devices blocked from the WAN, but still able to connect to the server. So, I can still sync my notes, or even push my code (I added git repo mirrors on my server, so I can push to that, and the server will push the changes to source hut for me).

Reading

After a reading lull at the start of the year, I’m reading more. This is partially due to replacing a bunch of my browsing and video habits with books.
I also listen to audiobooks, often instead of podcasts when they’re blocked.
In light of the yearly theme, I’m less worried about whether something will “count” for my reading. Those old thoughts of “Does an audiobook count? What about a zine?” are out the window, and I’m learning more because of it.

Exercise

Back to simple, quick workouts, and overall activity. Going well.
Also trying out sandbag exercises.
Not worried about hitting all my Apple Watch goals, or ensuring I use the various workout services we have (Apple Fitness+, Tempo). Do what works, not what I have sunk-cost guilt over.

Target Second Language

I always go back to German whenever I am interested in learning a second language. Last time I did this, I realized it made more sense for me to learn Spanish, or even French, instead (although I stuck with German).
Prompted by our Montreal trip and encouraged by the yearly theme to reconsider - I think I’ll finally switch my target language to ~~French~~ Spanish (changed my mind while editing this post XD ).

Going Forward

Lastly… I might move my theme’s start to Q2 each year (April 1st)? It’s when I finally defined the name for my theme this year, and it also lines up with when I left Red Hat last year, so it actually works well. I don’t need to conflate it with the changing of the year, which is sort of the point of the themes in general. We’ll see.

In summary though, I’ve enjoyed this theme thus far and hope to continue experimenting as the year moves on.

Tech Got Expensive - Revisiting My Order History

2026-04-05T13:00:00Z

My MeLE fanless mini PC, with an external SSD on top of it.

I’ve known the last few months that largely due to AI companies, RAM prices are skyrocketing. I also knew that this would trickle down: SSD prices would also be affected, which would eventually cause HDDs to also rise. Still, I was shocked (but not surprised :-( ) realizing just how bad it’s gotten even for devices like little mini PCs.

The Data

On a whim, I decided to comb through the last few years of my Amazon order history, looking at tech purchases and comparing the purchase price with the current listed price. Given that some of these purchases go back to 2021, I even adjusted the prices to today’s largely inflated dollars to make it appear like I know what I am doing (Inflation calculations done using the US Inflation Calculator).

That said, there is one big issue with this data to be aware of. My purchase prices are directly from my order history, which included tax and (sometimes) shipping. The current prices, however, are just what I found when clicking on the current listing, which does not include shipping or tax. So in practice, this table actually understates the change.

Name	Purchase Date	Purchase Price	Inflation adj.	Current Price	Time (days)	Price Delta (adj)	Percent (adj)	Notes
Samsung 860 EVO 2TB SSD	2021-01-11	$217.17	$261.90	$291.30 (used)	1904	$74.13 ($29.40)	+34% (11%)	link
Seagate IronWolf 4TB 3.5" NAS HDD	2021-07-07	$112.86	$136.11	$215.00	1727	$102.14 ($78.89)	+90% (58%)	link
Micron 8GB DDR4 SDRAM Memory	2021-07-21	$58.32	$70.33	$78.81	1713	$20.49 ($8.48)	+35% (12%)	link
Samsung EVO Plus 2TB NVMe	2022-11-29	$168.76	$188.34	$468.88	1217	$300.12 ($280.54)	+178% (149%)	link
Silicon Power 2TB NVMe M.2	2024-11-12	$97.81	$101.89	$279.97	503	$182.16 ($178.08)	+186% (175%)	link
MeLE Fanless Mini PC 4C, N100 16GB RAM 512GB	2025-03-27	$197.42	$200.49	$469.99	368	$272.57 ($269.50)	+138% (135%)	link
SanDisk 128GB microSDXC Memory Card	2025-11-20	$14.95	$15.17	$29.99	130	$15.04 ($14.82)	+100% (97%)	link

(Current prices collected on 2026-03-31)

The item that caught my attention (and prompted this post in the first place) was the MeLE Fanless Mini PC. I was so excited to get a device like that for under $200 roughly a year ago, and now, it’s $500 with tax.

There were a few items that I looked at but didn’t include in the chart. For example, the 2x12TB WD NAS HDD I bought last year didn’t really have a listing anymore, but similar models I found appeared to be double the price. I also didn’t include tech that doesn’t seem to be affected. Portable monitors, for instance, got cheaper.

Conclusion

So given all this data, what’s my big conclusion? I dunno. Mostly just that it sucks. I also find it unfortunate that at a time when so many people are feeling subscription fatigue, burned out by AI, and even returning to pirating content, these prices make solutions like self-hosting even further out of reach for many. (This all feels conspiracy-worthy…)

While writing this draft, there have been a few reports that RAM prices have started to fall, but even so it will take quite some time before things truly stabilize again, if ever, across a very shaky global supply chain. Let’s hope for the best. Good luck everybody.

BTW, My Chuwi Minibook X Died...

2026-03-05T16:25:34Z

The internals of the Minibook X from when I tried to fix it.

Intro

As a quick update, shortly after I wrote about my Chuwi Minibook X… it died.

After publishing that post, I did indeed purchase a GPD Pocket 4, with the idea of it replacing both the minibook and my old Minisforum HX99g. When it arrived, I took a week or so to set it up and ensure it would fit my needs before selling the previous two devices. When I went to wipe the minibook… it wouldn’t turn on.

As I started searching the internet for solutions, it quickly seemed like this is a common issue with the Chuwi Minibook, unfortunately.

Fixing Attempts

Periodically over the span of three months, I tried a number of the suggested fixes. At first, I tried the various use charger X, hold power button for Y seconds, then charge for Z hours solutions. They didn’t get me anywhere. Finally, one night I decided to crack it open and try some of the more invasive debugging methods. This included (but was not limited to) the following:

Checked for visible damage (burn marks, loose components, disconnected cables)
Identified the battery connector and disconnected the internal battery from the motherboard
Performed a full embedded controller discharge (held power 45–60 seconds, waited several minutes)
Reseated battery connector carefully and ensured full alignment
Attempted proper charger-first boot sequence (plug in → wait → single press power)
Attempted AC-only boot with battery disconnected
Tested battery output using a multimeter (battery was fine and charged)

During these tests, there were no LEDs, fan twitches, or screen activity at any point. So, I concluded, with my limited knowledge, that it was likely something related to the connector or board logic.

Conclusion

Even if I missed something simple, or the fix was easy, at that point I was done wasting time on it. Instead of selling it, I pulled out the NVMe, and donated it to the local e-cycling location. Hopefully they’re able to figure it out and give it a new life…

Now Update - 2026-02-24

2026-02-24T22:43:32Z

The inside of the dead Chuwi Minibook X I tried to save.

I can’t believe that it’s almost two months into 2026 already! Anyway, enough has changed that I should probably do a Now update…

Indie Dev

For coding projects, I hinted that I had started a new project in the last now post, and I’ve continued chipping away at it since. It’s tentatively called ParenLists and is a simple, template-based, list app. It’s still rough around the edges, but the core functionality is getting there. I most recently wrote a simple sync server and added multi-user support, so my wife and I should be able to dogfood it now…

I also made a minor update to Perspective Pixels to support changing the scaling percentages. I have the build in TestFlight, but haven’t completed the full process to push out an official release in the App Store.

Hardware Experiments

I continue to love using the GPD Pocket 4 running NixOS. I have been working on it most days, including to build out a homelab again on my tiny n100 NixOS server. However, I’ve started to use an even smaller, truly pocketable, setup on the go.

I grabbed a refurbished Pixel Fold (first gen) for roughly $350 USD. I wanted something to keep up with the current state of ‘Android’ (or Android-alternatives) and try a foldable device for the first time. I have successfully installed GrapheneOS on it, set up the Android Terminal app, and overwrote the default Debian VM to use NixOS. I have it fully working with my flake configuration and everything! (I actually wrote the rough draft for this on it).

Home

It was nice to travel a bit over the holidays to see family, but it’s good to be back home for a short travel break. We’ve been busy most weekends with kids’ birthday parties, but those tend to be short, so our weekends aren’t too crowded yet.

I loved getting to watch the Olympics with my family again, but am honestly glad they’ve wrapped up as it was quite the time suck.

Lastly, we finally got around to swapping out the legs on my home office desk to ones that support floor sitting all the way up to standing. Floor sitting at my main desk in the mornings and standing in the afternoons has been great for my flexibility.

Books

I haven’t finished a book yet this year, but that’s likely because I’m reading several in parallel and all are essentially textbook-style non-fiction. This week, though, my library hold on Apple in China by Patrick McGee finally became available. So hopefully the time limit and change of pace will help me finish it.

Clojure Script for Project Git Logs

2026-02-16T21:20:07Z

As mentioned in my last week notes post, I wrote a simple Clojure script to help me review what I’ve completed across projects during a given time period.

As of now, here’s my weekly-commits.clj, which I execute using babashka:

#!/usr/bin/env bb

(require '[babashka.process :refer [shell]])
(require '[babashka.fs :as fs])
(require '[clojure.string :as str])

(def since (if (seq *command-line-args*)
             (str/join " " *command-line-args*)
             "last Monday"))

(def home (System/getProperty "user.home"))

(def projects
  [(str home "/code/home-manager")
   (str home "/code/ryan.himmelwright.net")
   (str home "/code/ParenProjects")
   (str home "/code/PerspectivePixels")])

(defn git-log [git-dir]
  (:out (shell {:dir git-dir :continue true :out :string}
               "git" "--no-pager" "log" (str "--since=" since)
               "--pretty=format:%ad  %s"
               "--date=format:%a %Y-%m-%d %H:%M")))

(defn dirname [dir-path]
  (last (str/split dir-path #"/")))

(defn project-header [dir]
  (str "=== " (dirname dir) " ===\n"))

(defn git-log-string [git-dir]
  (let [git-log (git-log git-dir)]
    (if (str/blank? git-log)
      ;; If blank log
      (str (project-header git-dir) "No commits in time range.\n")
      ;; Else: return normal message
      (str (project-header git-dir) git-log "\n"))))

(doseq [dir projects]
  (if (fs/directory? dir)
    (println (git-log-string dir))
    (println (str (project-header dir) "Project dir (" dir ") not on this computer.\n"))))

In the script, I list out project directories (the projects var) that contain git repos. By default, the script will print out all git log results for each project since last Monday. The logs are printed in a specific format that displays a timestamp to easily see what was completed and when.

Additionally, I added the ability to change the --since flag by providing different values as arguments to the script. For example, bb weekly-commits.clj last Friday would show logs since last Friday (instead of the last Monday default).

For good measure, I added some conditional handling so that if there are no commits, or a project simply isn’t located on a system, it handles the issue gracefully with a message.

For example:

$> bb bin/weekly-commits.clj
=== home-manager ===
Fri 2026-02-13 10:26  Moved PerspectivePixels project to ~/code/
Fri 2026-02-13 09:41  split out ghostty configs, added some elfeed updates
Thu 2026-02-12 17:37  adding immich
Wed 2026-02-11 21:59  Rewrote weekly-commits.clj myself to fix some edge cases
Wed 2026-02-11 21:58  Added zeal to porygon
Wed 2026-02-11 12:21  added weekly-commits script
Wed 2026-02-11 09:27  added vm support

=== ryan.himmelwright.net ===
Sun 2026-02-15 16:04  changed button style
Sat 2026-02-14 15:52  finished editing, publishing
Fri 2026-02-13 15:47  Merge branch 'main' into week-post-1797
Fri 2026-02-13 14:10  added to archive
Fri 2026-02-13 14:10  Updated uses page
Fri 2026-02-13 13:05  Working through edit
Fri 2026-02-13 10:54  Started rough draft to links
Tue 2026-02-10 15:07  Template for week post

=== ParenProjects ===
Thu 2026-02-12 12:27  fixed client issues
Thu 2026-02-12 12:21  added sync support for cli
Thu 2026-02-12 12:02  Added skew automated testing
Thu 2026-02-12 11:49  Added conflict sync testing, and fixed issue with lists editing
Wed 2026-02-11 11:47  fixed bug so that pull sync doesn't run when sync isn't enabled
Wed 2026-02-11 11:41  Further cleanup, sync notifications
Wed 2026-02-11 10:46  refactor cleanup to consolidate app state
Tue 2026-02-10 11:20  Giant refactor to breakdown the clojure bridge in swift

=== PerspectivePixels ===
No commits in time range.

The script is nothing fancy, and I don’t have much else to say about it. It’s just something I threw together one night, and I thought I’d share it. I love the ability to script simple solutions to little problems like this one.

Week Post #1797

2026-02-14T20:55:29Z

Durham, NC

Welp, the weekly note officially isn’t a one-off now. Although, I purposefully refrained from posting one last week as I didn’t want to appear over-eager and set the bar for recurring posts too high out the gate.

First, some quick home-life updates since my previous week post:

Most of the snow and ice have melted
I’ve loved watching the 2026 Olympics
My wife and I finally paired up and re-applied debounce grease to our washer. So far, so good.

This weekend, we have several kids’ birthday parties and my parents are visiting. It should be busy, but fun.

Projects

A few days ago, I did write a quick Clojure script to help with the week-note endeavor, though I might write about that in its own post.

Next, I made an update to PerspectivePixels for the first time since the 1.1.0 release. I added the ability for users to customize the list of percentages used for scaling (ex: 200%, 250%). This has been a feature I’ve wanted for quite a while, and recently I’ve found myself curious about scaled resolutions above the previous max of 200%… so it finally got added. I’m realizing now the update is only in TestFlight still… so maybe I’ll push that over the weekend…

For ParenLists, I continued work on adding the MVP server and sync support. It’s going well and I have it mostly working and tested. I think the next phase will be building out user support and collaboration so that my wife and I can start using the app in our day-to-day life to really work out the issues. That’s quite the step up, but again… the MVP doesn’t have to be perfect. It just has to work for the two of us right now.

Lastly, I’ve continued to enjoy working in my Linux setup, and have been curious about testing out other systems too. So, I’ve actually ordered a refurbished Pixel 6 that should arrive today. I’m not planning on using it for my main phone, but to test out several different things I have in mind:

A physical Android test device for when I start to dabble more with ClojureDart/Flutter builds.
Progress of Linux on Android with things like Termux and the official Android 16 Terminal
GrapheneOS/LineageOS, etc.
Building fun handheld devices, as seen on r/writerdeck and r/cyberdeck

Oh yeah, and I also set up Nextcloud on our home server 😅…

Music

In addition to a ton of the Les Misérables Soundtrack, I’ve been enjoying the new album, Singin’ to an Empty Chair, by ratboys that was released last week.

Links

I’ve logged some of the posts I’ve read over the last 2 weeks. Here are a few of them:

I rediscovered that Curtis McHale has a blog, and went on a bit of a reading spree last night.

That’s it! Enjoy the weekend!

I Asked Gemini Why My Linux Computer Feels Peaceful

2026-02-06T16:12:16Z

My Linux setup working on this quick post.

Curious about what it would say, I asked Google Gemini “Why does it feel so much more quiet/peaceful when I’m using my Linux computer compared to if I use one like my Mac or even when I’ve used Windows in the past?”

I know I shouldn’t be, but I was surprised by its answer:

That “quiet” feeling isn’t just in your head—it is a byproduct of fundamentally different design philosophies. Windows and macOS are designed as services that want to engage you, whereas most Linux desktop environments are designed as tools that wait for you.

Gemini then broke down a few, specific, reasoned examples. It covered common ones like building your own environment on Linux vs. having it built for you, and the increase of ‘Dark Patterns’ due to monetization on other OSes. These were fine, but I think my favorite statement was simply the concluding summary:

That “peace” you feel is the absence of a billion-dollar company trying to “optimize your engagement” while you’re just trying to write code or browse the web.

Probably the smartest thing I’ve ever seen Gemini say.

Don’t worry though. It immediately followed that gem with the following question:

Would you like to know how to further “silence” your COSMIC environment, such as disabling workspace animations or fine-tuning notification “Do Not Disturb” settings?

Eager for engagement, and apparently oblivious to its own role in the ‘Dark Patterns’ it had just thoroughly described above.

Week Post #1795

2026-02-01T03:45:00Z

Some ice in our yard this week -- photo credit to my wife.

This year, as a way to stay motivated with all the various projects I’m working on, I want to be better about reflecting on my weeks. One way to do that while also complying with my goal to write and share more is to publish it in a “week post”. I’m not committing to writing something every week, but I do want an outlet for when I do.

So, I took the numbering idea Robb Knight uses (who I think got from someone else). Basically, instead of worrying about continuously incrementing a weekly note counter, I simply number the post with the week number of my life. I was born on a Friday, so it works out well enough.

To make things easier, I coded a new option in my webcli.clj script to calculate the week number and generate the post file for me:

󱄅  [website] ⚡  scripts/webcli.clj new week

Select a week:
0. Last week: Week #1795 (2026-01-23 to 2026-01-29)
1. This week: Week #1796 (2026-01-30 to 2026-02-05)
2. Next week: Week #1797 (2026-02-06 to 2026-02-12)

Enter Choice (0-2): 0

Generating weekly post for 1795 [2026-01-23 to 2026-01-29]
Created week post: ~/code/ryan.himmelwright.net/src/post/2026/week-post-1795/index.md
You can now edit the content there.

I’m testing it out for the first time with this post.

Projects

I accomplished a good bit of project work, despite the shorter week (an ice storm last weekend kept daycare closed the first few days).

Before that storm, it was my turn to DM the session with my D&D group. I’m glad that 1) we managed to squeeze it in before the storm started, 2) the session went well, and 3) I don’t have to worry about planning it anymore XD.

Next, I wrote and published a short post, along with some other backend work on the website.

Regarding coding, I’ve been working on a list app and spent this week adding archiving functionality and performing some initial performance testing. I also planned out the preliminary design for building a sync server that I’ll start… next week.

Music

I listen to quite a bit of online radio during the day. One of the stations I frequently play, WJRH, is actually the college station for my alma mater, Lafayette College. Over the past few months, they’ve been playing a Joyce Manor single that gets stuck in my head every time I hear it - All my friends are so depressed. This week, the full album (I used to go to this bar by Joyce Manor) was released, so I bought it. Love it.

Also, if you haven’t listened to Streets of Minneapolis by Bruce Springsteen yet, give it a listen.

Links

Lastly, just a few videos/links to share from this week:

Enjoy the rest of the weekend! If it’s snowing again where you live like it is here, stay safe and warm!

Re-enabled Contrast Accessibility Settings on my Apple Devices

2026-01-30T03:25:51Z

Settings I toggled (left) and one before -> after compare (center, right).

Two weeks ago, I switched on some of the display accessibility settings. Specifically, I enabled the ones that focus on increasing contrast and readability:

Bold Text
Larger Text
Show Borders
On/Off Labels
Reduce Transparency
Increase Contrast
Differentiate Without Color

Background

This isn’t my first time using these settings. I previously enabled them, probably around the time I was dabbling with e-ink devices and was used to seeing blunt contrast in app UIs. However, I disabled the settings last year while coding the front-end of PerspectivePixels, and thought I was experiencing bugs only on my devices as a result of the settings (my phone looked different from the simulator and my wife’s phone XD ).

Afterward, I kept default settings during *OS 26 beta and release cycles which contained the initial liquid glass implementation, so that I could follow the progress.

Why Switch Back

So why switch back now? Basically, I’ve given liquid glass a good amount of time, but think I’ve had enough of it for now. To be clear, I’m not one of those people who think it’s ‘the absolute worst thing Apple has ever done!’. Beyond the visibility issues, it does look… cool?.. pretty?

However, it represents a design progression that is moving in the complete opposite direction from where I am. I’ve always loved simplicity and contrast, and recently have started to double down on those aspects by dabbling with Neobrutalism design for my website and my in-progress apps. Over time, my appreciation for readability, high contrast, and simplicity in applications has only increased, even at the expense of ‘beauty’.

The Switch

On a whim one day, I toggled on the accessibility settings on my MacBook. However, not long after that I had them enabled on all my Apple devices. Once it was set up, I immediately wanted it enabled whenever I switched to a device without the settings. It’s not perfect, and by all accounts ‘uglier’ than the default settings, but I much prefer it for my UIs.

To try it out, go to Settings -> Accessibility -> Display & Text Size and play around with the different options. Over time, I ended up using most of them.

Conclusion

I just wanted to share a small but very satisfying change I’ve gone back to recently. It might not be for everyone… but I enjoy it. Try it out if you never have, especially if liquid glass isn’t your thing.

Timewarrior Trial - My iOS Shortcut

2026-01-17T03:00:00Z

I made an iOS shortcut that, while overly simplistic, actually works quite well to connect with Timewarrior!

Over the last year I’ve been using apps like timery and timelines to dabble with time tracking. Both of these apps have been great, and I highly recommend both of them. However…

Equally, I’ve been lured back into experimenting with text-based apps this year, after I fell once again to the infamous siren song of Emacs. To cut to the chase, I haven’t been able to get Timewarrior out of my head.

Desiring Timewarrior

Timewarrior, based on the great CLI task management app Taskwarrior, is also a simple CLI app, but focused on time tracking. I’ve been eyeing it for some time but have never fully tried it. On occasion I’d clock projects while at my computer, but I also wanted to track house tasks or projects… times when I usually only have my phone on me. I know there are different Timewarrior servers and potential clients I could dive into, but that was more of a lift than I wanted to dedicate time to before I knew if the system even worked for me.

The Setup

Earlier this week I decided to use Timewarrior to track my computer projects, if only for a day. To simplify logging my time regardless of whether I was working on my MacBook or NixOS computer, I decided to actually use Timewarrior on diglet, my tiny Intel N100-based computer we use as a home server.

Terminal Computers

I started out by just using an open terminal window with a persistent ssh/mosh connection which I could enter timew commands into. It worked fine, but I quickly got annoyed with having to switch windows (often on a different workspace) to enter the commands, when I was already working in a perfectly capable terminal. Eventually, I switched to passing my queries over ssh:

ssh diglet.local timew summary :today

With that method working, I decided to abstract some of the repetitive parts, and added this alias to my zsh.nix config file:

twdl = "ssh diglet.local timew"; # Local only

With that alias saved and applied to all my computers’ zsh configurations, I could now type twdl <timewarrior commands>, and the command would execute on diglet via ssh, from my device just as if I was using Timewarrior locally.

Note: the shorthand twdl stands for “Timewarrior diglet local” 😉

My Phone

The shortcut was easy to configure, at least for a basic setup.

When I finished up my computing tasks for the day, I wanted to continue logging with Timewarrior, but away from my computers. Passing commands over ssh worked so seamlessly that I was sure I could set up something similar for my iPhone using Apple Shortcuts.

There was only one “problem”… my ssh setup relied on being connected to my home network. However, when I want to track time from my phone, it is often when I am out and about. Luckily, this wasn’t a real issue, as I already have Tailscale configured and running on both my iPhone and diglet.

I fired up the Shortcuts app and got to work. I started with getting the ‘Run script over SSH’ step working, as it was the most complicated and critical one. I set up the host using my diglet login and its Tailscale address. For the script part, I started with a hard-coded timew summary until I confirmed the ssh commands were working.

With the ssh section working, I moved on to adding a few extra steps to make a basic but flexible shortcut. First, I added a starting prompt that asks the user what timew commands to execute. I saved that input to a variable, which I then passed to the ssh step: e.g., timew $VAR.

Next, I knew from running timew commands over ssh on the computer that the output is still returned. At first, I just displayed it in a text window, but it was too hard to read. I debated formatting the output, but it seemed like more effort than I wanted, and likely less flexible across commands. After a quick search, I learned that shell output displays more nicely when using ‘Quick Look’. Good enough.

For ease of use, I bound the shortcut to the action button on my iPhone 17. Now all I have to do is hold the button for a second, and a window pops up to prompt me for Timewarrior commands (I don’t have to write the timew part). I enter it, and then see the result on screen. That’s it. Simple, easy.

Conclusion

I’ve been using this for my time tracking throughout the week and honestly… it works really well. At this point, the biggest issue I face is that I’m not familiar with the more advanced Timewarrior commands… yet. There are more advanced ways to manage what I’m doing, and I could configure a proper server/client, but this works great. The one thing I might do is add additional options to the shortcut prompt to single-tap my most frequent commands, but still leave the custom input at the bottom.

For now, I’m happy as is. Sometimes the best solution isn’t the fanciest one, but one that is simple and works

2025 Recap

2025-12-30T02:30:00Z

I wanted to share and catalog what I did in 2025 (ex: books read and shows attended), but didn’t want to go crazy writing a full-blown post like my 2023 review picks (notice I didn’t do it in 2024 XD). So, I’m dumping a bunch of lists into a post. Enjoy!

Home & Family

We added a bunch of pollinator plants this year, and saw a huge increase in pollinators as a result :D

We’ve done quite a bit of traveling, and continued to transform and set up gardens outside our house. Some highlights:

Several snow events in Durham, NC
First year turning side yard into pollinator garden
Added several garden beds
Did a massive chip drop in the late spring
Dog (Maggie) had double knee surgery
Family vacations to Virginia Beach VA, Outer Banks NC
Travel to see friends & family in Seattle/Everett WA, Columbia SC, PA
First ‘Friends’ Birthday Party for son’s 3rd birthday
Continued swim lessons and first time doing Soccer Shots with son
Brother’s Wedding (Co-Best Man)
Got my wife to finally play Stardew Valley

Career

Big changes this year for my career. That about sums it up XD.

Left job @ Red Hat
Built and released my first-ever app (Perspective Pixels)

Website Changes/Improvements

I forgot I was still using hugo for my website at the start of the year. I’ve done so much since that switch!

Hugo -> Eleventy
Nginx -> Caddy
Added /uses archive
Added Now/Then Pages
Added pages under /explore
Added /mumbles, /POSSE, /colophon, /projects
Added /blogroll, /reading page generators
Added /chipotle, /defaults, /guy, and /save
A smattering of /now updates and blog posts
Theme tweaks

Notable Tech Hardware Changes

By the end of the year, I've switched from a two-Mac to a Mac+Linux laptop setup.

While I feel like I did a bunch of hardware swapping as per usual, by the end of the year the net changes essentially boiled down to a few things:

16" M3 Pro MBP + 13" M3 MBA -> 14" M4 Pro (14-Core) MBP
Minisforum HX99g -> GPD Pocket 4
iPhone 15 -> iPhone 17
Added MeLE Mini PC Quieter 4C as a home server

Notable Tech Software Changes

I’ve continued to jump around with software quite a bit. Here are the most notable ones I can think of.

Obsidian -> Emacs
TickTick -> Todoist -> Reminders/GoodTasks -> TickTick
Readwise Reader -> newsboat -> newsraft -> Elfeed (emacs) for rss
Fastmail webUI -> neomutt -> mu4e (emacs) for email
Apple Music -> Mostly local music and online radio
Daily driving NixOS in addition to macOS

Concerts/Shows

The Meymandi Concert Hall, Raleigh, NC

A year or two ago, my wife and I made it a priority to attend concerts and shows when possible. This year we were fortunate enough to see:

2025-02-01 | Harry Potter and the Chamber of Secrets Orchestra – DPAC
2025-02-15 | Symphony - Beethoven’s Piano Concerto #4 – NC Symphony
2025-03-22 | Alice in Wonderland Ballet – NC Ballet
2025-05-31 | Carnival of the Animals – NC Symphony
2025-07-19 | Les Misérables – DPAC
2025-09-03 | Blink-182 w/ Alkaline Trio & Beauty School Dropout - Coastal Credit Union Music Park at Walnut Creek (Raleigh, NC)
2025-09-11 | Beauty & The Beast – DPAC
2025-10-17 | Stardew Valley: Symphony of Seasons – DPAC
2025-10-28 | Lenny Pearce – The Ritz (Raleigh, NC)
2025-10-30 | Six - The Musical – DPAC
2025-11-13 | Neil deGrasse Tyson: An Astrophysicist Goes to the Movies (III) – DPAC
2025-11-23 | Nutcracker Ballet – DPAC
2025-12-12 | The Front Bottoms – The Ritz (Raleigh, NC)
2025-12-19 | The Nutcracker Ballet & Symphony – NC Ballet/Symphony

(I think that’s at least all the big shows)

Books Read/Listened to

While it wanes at times, I think overall I’ve been able to maintain a good reading habit throughout most of the year.

Still in Progress

Elements of Clojure by Zachery Tellman
The Joy of Clojure (2nd edition) by Michael Fogus and Chris Houser
Shadow Dark (TTRPG)

Completed

Wicked by Gregory Maguire (book)
Code Name: Pale Horse by Scott Payne (audiobook)
Shadow Dark Quickstart Guide (TTRPG)
iOS and iPadOS 26 Review by Federico Viticci (ebook)
macos 26 Tahoe by John Voorhees (ebook)
Plato and a Platypus Walk Into a Bar - Understanding Philosophy Through Jokes (book)
Beardy Moving away from Apple (Blog Collection)
The 8 Minute Organizer by Regina Leeds (book)
Fahrenheit-182 by Mark Hoppus (audiobook)
Company of One - Paul Jarvis (ebook)
Grow Wild - Katy Bowman (book)
Life-changing Homes by Nicolás Boullosa, Kirsten Dirksen (eBook)
Leonardo Da Vinci by Walter Isaacson (audiobook)
The Almanack of Naval Ravikant - Eric Jorgenson, Jack Butcher, and Tim Ferriss (book)
Only Say Good Things - Crystal Hefner (audiobook)
Slow Productivity - Cal Newport (book)
Everything I Know - Paul Jarvis (book)
A System for Writing - Bob Doto (book)
The Pathless Path - Paul Millerd (book)

Podcasts

[2026-01-01] Update: I didn’t include my podcast stats when I first published this post because they were still accumulating. But now, I have my completed Overcast stats for 2025! Here’s the podcasts I logged at least an hour listening to this year (in Overcast).

#	Podcast	Listening Time
1	Accidental Tech Podcast: Unedited Live Stream	113:22:33
2	The WAN Show	91:03:13
3	Connected Pro (Private)	60:01:12
4	The Rest Is History	54:22:46
5	Upgrade	49:59:14
6	Mac Power Users	27:55:01
7	LINUX Unplugged	25:27:18
8	Hemispheric Views	18:59:46
9	Analog(ue)	12:49:37
10	Moretex (Private)	12:18:55
11	The Rest Is Politics: US	11:02:27
12	Under the Radar	8:36:48
13	Ruminate	8:06:05
14	AppStories	6:30:06
15	The Glass Cannon Podcast	5:26:22
16	Focused	3:23:49
17	Crossover (Private)	3:20:12
18	Comfort Zone	3:07:05
19	The Rest Is Classified	3:06:42
20	MacStories Unwind	3:05:32
21	Reconcilable Differences	2:33:18
22	The Rest Is Politics	2:27:36
23	Move Your DNA with Katy Bowman	2:07:06
24	Conduit	1:38:58
25	Syntax – Tasty Web Development Treats	1:10:30
26	defn	1:04:09

Conclusion

My wife Rebecca and I at the Museum of Natural Science, Durham, NC

I’m sure I’ve missed a ton of stuff I should have probably listed. It’s been a busy year. However, at least I recorded something. All in all, it’s been a great year, and I’m excited to keep things flowing into the next.

Now Update - 2025-12-20

2025-12-21T03:41:19Z

Museum of Life & Science, Durham NC

… It’s been a few months since the last update. I’ve meant to do one several times, but kept tweaking various parts of the website first (which I wrote about in my latest post). Even prior to starting this draft I went in and tweaked my config to get the /now page functioning as a redirect like I wanted (so hopefully this won’t bust people’s RSS feeds… fingers crossed). Anyway, let’s get to some updates.

Learning & Projects

Since wrapping up features for Perspective Pixels, I’ve moved on to some other coding projects. I’ve started work on a simple app, but I’m taking my time to truly learn and understand both the code and philosophies behind what I’m doing. I’m making the core (and CLI tooling) in Clojure, and I have loved getting back into functional programming with a LISP language. I’m finally getting to the rewarding point of my code actually building something … tangible.

On the tech front, I’ve doubled down on nixOS by experimenting with using it on portable hardware to essentially make a better iPad+Keyboard setup for my needs. It’s gone very well and I’ve actually sold my old MinisForum HX99g to help purchase a GPD Pocket 4. It’s been great and not only functions as a tiny convertible computer, but is powerful enough to plug into my dock and use as a legitimate Linux device. So now I’m at least partially back on desktop Linux, and loving it.

Home

Let’s start with the garden updates: Yes, it all died off by now XD. Not much else to report.

As I noted in the last post, the end of the year has been busy, but fun. We’ve attended a smattering of shows, concerts (including a toddler rave), and even my brother’s wedding. Each week was filled with swimming + soccer lessons, and a slew of other activities held together by intermittent colds and allergies.

We’re preparing for a few days in PA to see family for the holidays, which should be nice. Assuming we don’t bring back any form of ‘cold’, that is.

Books

One of the website updates I completed was adding a /reading page to the site. I’ve been keeping my reading list in an org note (of course), and have a script to generate the website page from the current state of that note. So I’m thinking I might retire this section of my /now updates.

But as an update from last time anyway, I basically started and spent that entire time reading Wicked by Gregory Maguire, along with an audio book or two. I’m currently working through the Shadowdark RPG Book, as well as some Clojure books (The Joy of Clojure, 2nd edition and Elements of Clojure).

Added New Pages to Website

2025-12-01T02:45:00Z

Oh yeah, I also updated my favicon...

A large motivation when I switched my website to eleventy at the beginning of the year was to make modifying and tweaking much easier. So far, I can gladly say that’s been the case. This past month, I finally got around to adding a bunch of pages I’ve had on my todo list for quite some time. With 11ty, the process was super smooth. Here’s a quick rundown of what I’ve added.

Explore Page

First up was figuring out how to manage navigation with this influx of pages on the website. I solved it by implementing a page I had created months ago, but never enabled: a site index page. This is simply a page where I can link to everything else on my website. While I could get fancy with drop-down navigation menus and other methods (and might one day), why over-complicate things just yet?

The hardest problem I had was deciding what I wanted to name the page. At first it was simply named more… but I didn’t love it. Then while browsing robb knight’s website, I noticed his site referred to this as the Explore page. I liked it, and took the idea for my own website.

Mumbles

Another feature (inspired by Robb’s website, among others) that I’ve long wanted to implement is making a page where I can write short-form snippets that are then forwarded as posts to my mastodon account. Setting it up in 11ty wasn’t too difficult: I just have an index page that takes all the dated entries and displays them in order. I also scripted the process of writing a new entry so I didn’t have to fuss with timestamps and other hindrances.

Once again… naming was my biggest hurdle. Pages like this are often referred to as notes on the indy-web. However… I didn’t want to use that name. To me, notes sound more like a page for a living document. For example, I could have a note page about a project, or programming language. While I don’t define ‘notes’ for anything on my website currently, I wanted to keep the option open.

Same as before, my solution surfaced while browsing the indy-web. I was reading a post on thatalexguy.dev and realized he referred to these short online quips as ‘mumbles’. Again, I loved it and used it myself.

/POSSE

Once I had my ‘mumbles’ working, I set up a feed and linked it with echofeed to forward the mumbles as my Mastodon posts. Now that I was starting up a proper POSSE, I added a /posse slash-page for my strategy to be published if/as it expands.

Script-Generated Pages

My new_mumble.clj wasn’t the only script I wrote during this process. For the next pair of pages, I wrote a Clojure script that generates the page from .org files I already maintain in my Emacs setup. If I update either of those files, I simply run the script with babashka and the markdown is generated for me to push to the site.

Reading List

First up is a reading list/log. While writing my /now updates over the last few months, I usually included a small log at the bottom that listed what books I was currently reading, and recently finished. While writing it, I would often check my reading log note for the information and realized one day… this was stupid. If I have the log, why not just automate that into an update. So I did.

Instead of needing to include my reading progress in /now updates, I just run a script to have the relevant sections from my reading list post to /reading. Easy.

Various Other Slash Pages

The last few pages I added are various other slash pages that I’ve wanted to add to my site for one reason or another.

/defaults

As a Hemispheric Views Podcast listener, I feel obligated to have a /defaults page. I still don’t understand the scoring (who does?), and might consider tweaking the categories over time to better fit what software I need and use. But for now, I don’t mention any software selections on my /uses page, so this is a good home for that.

/guy

Another slash-page from Hemispheric Views. I really enjoy this page as a sort of quick and to the point /about supplement. It can be very similar to /interests/, but I enjoy this format.

/save

Many services I use have referral programs. Not only do I never take advantage of them when telling others about a service… I often forget which services even have them. At the very least I hope maintaining /save will be a good reference for me 😂.

/chipotle

Last, but not least, we have probably the most important slash-page that exists: /chipotle. While I’m mostly joking, being a burrito lover that has a go-to Chipotle order, it’s not as much of a joke as you may think…

Conclusion

So that’s the rundown of all the pages I’ve added in the past few weeks. I’ve also added a new favicon image instead of just my initials (image included at top of the post). I’m happy with the results, and hope to continue updating and tweaking the pages over time. It’s the little things like this that make having a presence on the indy web so much fun. Enjoy!

Chuwi Minibook X - Initial Thoughts

2025-11-14T16:50:00Z

As I’ve continued to dig deeper into Emacs and TUI life again, I naturally cultivated a new interest in cyberdecks. After researching cyberdeck options, I was drawn to the more realistic mini laptops. These devices are tiny, portable, and similar to the netbooks that I used to supplement my main desktop.

Additionally, I am still fascinated by the concept of modular computers. This is a device that can function as a tablet, laptop or desktop all in one. For some, this can be an iPad Pro. Unfortunately, I have concluded that at least for me, I don’t see the iPad fully fitting that role any time soon. I need a shell with full file system access, a good (ideally, declarative) package manager, and of course… Emacs.

The Minibook X is a great Emacs machine...

Luckily for me, the same technology that has enabled the modern mini-PCs I love… has also brought forth a renaissance in mini laptops that can be far more powerful than the ‘netbooks’ of my college days.

After a bit of research, I settled on the Chuwi Minibook X, which was ‘marked down’ to somewhere in the mid-$300 USD range at the time of purchase (about the base price of Apple’s cheapest iPad). I got the newer N150 model, and bumped the RAM from 12GB to 16GB for an extra $20 or so.

The good

Playing Stardew Valley - with a controller...

After using this device for several weeks now, I have some initial thoughts, most of which are positive. Here are a few:

Build/value

At roughly $300-360 USD, it’s a solid machine. The build quality isn’t as good as my MacBook, or even the Thinkpads I used to own, but it is much better than any other ‘netbook-like’ device I’ve used in the past.

Performance

In addition to being the best built laptop I’ve had at this size, it’s performed quite well and has even surprised me in some areas. Specifically, I was delightfully surprised that when I connected it to my Thunderbolt 4 Dock, with the Apple Studio Display attached… everything including the display just worked… at the full 5k resolution (2x scaled)!

Similarly, while I know it’s likely possible to run Stardew Valley on a potato, I was still happy to fire it up without any issue and get playing.

To be clear, it’s not a beast when it comes to performance… after all it only has an Intel N150 in it. It definitely chugs a bit compared to my minisforum HX99g, especially if I’m managing windows on the 5k display. For general tasks however, like browsing, watching simple videos, or writing/coding around in Emacs, everything has largely just worked (and on nixOS too)!

Size

Comparing size with an 11" iPad Air (M2) + Keyboard, and my iPad Mini (7th Gen).

Lastly, the size has been great. It’s noticeably smaller than my MacBook, yet large enough that it still feels like a laptop, and I don’t feel too cramped on the keyboard. In fact, when I predominantly use it, I don’t feel like it’s small - I think everything else is large. Switching back to my 14" MBP it feels like I have a 16" again 😂.

It doesn’t fit in my pocket, but I’m able to easily slide it into a small sling bag I already have with me when I’m out with my kid (my ‘dad bag’). This makes it easy to pull out the Minibook to work on some writing or research when waiting at swim lessons, for example. This was my primary motivation for trying out one of the machines, and so far, so good. It’s all the benefits of an 11" iPad… but with more useful software available to me (Emacs, nixOS, and a terminal).

What’s… fine (but understandable at this price)

Not everything has been perfect, and there are a few things that are just fine. However, I do want to re-state that at this price point… they still far surpass expectations 😆.

Screen and audio

The screen, while okay, clearly isn't as vibrant as even my iPad Mini.

The screen is actually quite good for text and reading with its 215 PPI. However, when watching videos, I do notice that the colors are a bit flat. Again, it’s fine but is noticable enough that I sometimes switch to my iPad Mini for watching videos instead.

The other problem that usually forces me off of it while watching video is the audio. It doesn’t have much punch, but even worse… it just isn’t loud enough. Often I have to hook up headphones to it in order to hear something as simple as Critical Role while working on house tasks.

Trackpad

Another area of slight disappointment is the trackpad. It feels mushy and I don’t think taps always register. This was apparent when I was trying to fish in Stardew Valley. It was such a problem that I’ve started to use a gaming controller whenever playing on this laptop, as it’s simply a better experience (even with a low-quality controller). I think even just adding dedicated clicking buttons like on a Trackpad or GPD device would mitigate the worst issues, but the Minibook unfortunately only has the mediocre trackpad.

Annoying

There are just a few things that are flat out annoying about the Minibook, despite the price point.

Tablet support setup…

While tablet mode is great for reading all my e-books, messing with the exposed trackpad + keyboard on the back makes it difficult to use in practice.

This isn’t really the fault of the Minibook, but is an annoyance I’ve had to deal with so I’m writing about it here. Essentially, the ‘tablet-mode’ has been a pain to get working. While it’s possible that this works great out of the box on Windows, I installed nixOS immediately. It seems like others have worked out these issues, but I can’t seem to get everything happily working together. When I manage to have the keyboard disable in tablet mode, the screen orientation doesn’t work (and the system seems less stable in general). When I have the rotation and stability all working, the keyboard is still active.

So, I’ve settled somewhere in the middle. I use my config that leaves the system more stable, with the automatic screen orientation working, as well as binding a trackpad toggle to a key command. The trackpad is what I most often accidentally hit, and it is nice to still hit the physical keyboard buttons in tablet mode (volume, etc) compared to fiddling with a virtual keyboard setting, so the solution is fine.

Flaky WiFi

By far the most annoying and detrimental to my use with the device is that the WiFi is very flaky. It can often take minutes until it fully connects and I can ping a host. I’m not sure if it’s specific to my network (doesn’t seem to be as big an issue on the go, but I’ve only taken it to one or two places so far), but happens all the time at home. Super frustrating.

Conclusion

I like being able to use the laptop in 'tent mode' with my split keyboard.

That’s about it. So far other than the WiFi flakiness, most of the problems I have are understandable, or I have figured out a workaround for them.

My biggest takeaway is that I love this kind of device setup. It’s small and easy to throw in a sling, I can flip back the screen to read, watch a video, or use with my split keyboard. When I’m home, I can connect to my desk setup and forget how small it is. Looking into more expensive alternatives in this category, it seems these devices don’t even need to fully sacrifice power.

Which brings me to my ultimate conclusion: I love this type of device and it has me actually using nixOS more consistently again, something that my hx99g alone struggled to do. I don’t really want to own multiple linux computers though, especially since I also have a Mac. If I could somehow combine the best parts of these two Linux boxes into one … (cough… GPD Pocket… or cough GPD WIN mini/max…), that would be wonderful.

For now though, the Minibook has been a great experiment into this form of computing, especially at its price. And I’m thrilled to be back.

PerspectivePixels -- Version 1.1.0 Released

2025-09-29T16:00:00Z

Last week, I wrapped up some of the lingering features I wanted to add before the end of the month. Over the weekend, Apple approved the updates and they’re now live in the App Store.

What’s New in PerspectivePixels 1.1.0

New features include import/exporting saved lists (left), saved list sorting (center), and delete all saved items (right).

This update introduces some key features I wanted to add once the MVP was out, as well as some tweaks for the new iOS 26 release.

Release Notes:

New Features:

Import/Export Your Monitor List: You can now export your saved monitor list to a JSON file for backup or sharing. Use the new import feature to restore your list from a file. These options can be found in the new dropdown menu on the Saved Items tab.

Bulk Delete: A new option to delete all items at once has also been added to that dropdown menu.

Saved Items List Sorting: You can now sort your saved list by name, screen size, PPI, or PPD in either ascending or descending order.

Improvements:

Updated App Icon: The app icon has been updated to support liquid glass transparency in the latest iOS version.

Back-end Refinements: I’ve made some under-the-hood changes to improve performance and compatibility with the latest iOS updates.

Bug Fixes:

Fixed a visual bug in the info pop-ups.

That’s all for this update. I just wanted to share the release notes here ;).

Now Update - 2025-09-22

2025-09-22T15:30:00Z

Nags Head, NC

Life has remained busy since I last wrote. Between family vacations, I turned 34 at the beginning of the month and released the MVP of my first app. It’s felt good though, as I’m finally making continued progress on my goals again.

Learning & Projects

After the last update, I released Perspective Pixels to the App Store. Since releasing it, I’ve mostly been working on wrapping up some new features I still want to add. I have a shortlist of items I’m hoping to wrap up soon, so I can start working on another project next month. Outside of the vacations, I’ve become a bit more consistant with coding most days, even if it’s for a short time. Generally, I feel happier and more productive during the rest of the day when I am able to sit down and focus on some code first.

I’ve continued digging into my doom emacs configuration, although luckily the tweaking has slowed down a bit. I have also been playing around with nixOS again, expanding my home-manager flake configuration to additionally manage a few nixOS systems. I must say, the combination of diving back into emacs and nixOS at the same time really has me eyeing up Guix 😅.

Home

For home life, we’ve had a few vacations, added soccer to the list of weekly activities for my 3 year old, and my wife and I attended a few shows/concerts including a blink-182 concert during my birthday week. Lots of fun.

On the gardening front, we’ve mostly been letting it do its thing without much intervention. Most of it is starting to die off, as we’re nearing the end of the season, but it was a great experimental year.

Looking at the family calendar, the rest of the year looks just as busy, but it’s all good. I’m enjoying the activity.

Books

I returned to my reading habit over the last month or so, and finished off several partially-read books. Currently, I’m working my way through the MacStories.net OS reviews before starting my next book.

Currently Reading

iOS and iPadOS 26: The MacStories Review by Federico Viticci (ebook version)
The Courage to Be Disliked: How to Free Yourself, Change Your Life, and Achieve Real Happiness by Fumitake Koga and Ichiro Kishimi (Audiobook)

Recently Finished

macOS 26 Tahoe: The MacStories Review by John Voorhees (ebook version)
Plato and a Platypus Walk Into a Bar: Understanding Philosophy Through Jokes by Thomas Cathcart and Daniel Klein
Beardy Guy Blog: Apple to Linux Journal Series (I exported all series posts into a giant PDF ebook)
The 8 Minute Organizer: Easy Solutions to Simplify Your Life in Your Spare Time by Regina Leeds
Fahrenheit-182: A Memoir by Mark Hoppus (Audiobook)
Company Of One: Why Staying Small Is the Next Big Thing for Business by Paul Jarvis

See more of my reading history or connect with me via my story graph profile.

Released My First iOS App - PerspectivePixels

2025-09-02T16:00:00Z

A week and a half ago, I released my first-ever iOS app… actually, my first real app in general. Before anyone gets too excited, I should mention that this is the first iteration of a basic app, and it probably shows 😅. Still, I’m happy to have passed that first milestone.

About

The app is called PerspectivePixels, and it’s available on the app store. It’s a simple iOS application that calculates several display metrics. For example, it calculates PPI, PPD, FOV, including their scaled values on HIDPI displays. You can also save display specs to tweak or compare later.

That’s the general summary so far. I have ideas for future features, but this had everything I needed, so it seemed like a good cutoff point for the MVP release.

Why

Why did I make this app in the first place?

First: I wanted it. Whenever I see the specs for a laptop or display, I inevitably have questions that require deeper calculations to answer:

“Will this monitor have a high enough PPI for proper 2x scaling on macOS?”

With PerspectivePixels, I can enter the stats and immediately see the PPI. Additionally, I can even see what the ‘scaled’ resolution and PPI are if I would use it at a 2x scale

“If I downsize from a 32” to 27" monitor, but move it 6 inches closer, will it effectively appear to be the same size to me?"

Here, I enter the specs of the two displays in question, including the distance from my eyes and compare. I can look to see if the field of view (FOV) is roughly the same, and even toggle the distances to see how the values change. I can also look at the ppd values to see how sharp the device remains while moving it. It doesn’t matter if a moved monitor covers the same field of view if each pixel suddenly looks like a LEGO brick.

“Will this device’s screen have a ‘usable’ resolution?”

Sometimes when I’m looking at unusual devices (for example mini-laptop/netbooks) it can be hard to tell what type of working space I will have. In this case, I can enter the specs and play around with the scaling values. For example I might see an 8" 1440x900 device. At 200 PPI, I’ll probably want to use 2x scaling, but with a scaled working resolution of 720x450, will it be worth it?

Before building PerspectivePixels, I relied on a Python module I cobbled together years ago. Having an app that I can quickly open up and use, with saved items, is much better.

Left: The Calculations Tab, Center: The Save Items List (with details), Right: Settings Pane.

Second: I wanted to build my first app, and this was a great way to start. I had most of the back-end logic from my python code and it was easy enough to port to Swift. All the remaining tasks: designing + coding the UI, creating graphical assets (ex: app icons, screenshots), figuring out App Store Connect, TestFlight, and App Store Review process… were all things I had never done before.

Being a simple, low-risk, app meant it was perfect as a learning project. At the end of the day, even if no one likes it, it’s something simple I wanted… and now have.

Conclusion

So, feel free to check it out! I admit it’s not the best app out there, but I’ve already found it useful. I have a list of improvements I hope to make now that the MVP out, but I’m also always open to feedback.

Enjoy, and thanks if you give it a try!

Now Update - 2025-08-04

2025-08-05T02:00:00Z

My first ever sunflower bloom!

With summer here, the last few weeks have been a bit different than the ones from when I previously wrote. It’s been hot outside, and my wife’s work schedule has been a bit crazy in preparation for upcoming vacations we’re taking. I’ve spent more of my time working on one-off tasks and projects, including helping our dog recover from surgery in May (fully recovered now… but having some other issues 😅). All in all, things have been great, but it does feel a bit ‘off-track’ from what I imagined. I find summers typically feel that way though, so maybe it’s just normal.

Learning & Projects

I’m at the point to where I’m ready to release the MVP of my simple iOS app… so I’ve started to drag my feet. I’ve worked on cleaning it up, fixing some bugs, and setting up basic testing. Progress has slowed down though for now.

Part of the slowdown has also been my full dive into using tui-apps, and from there getting back into emacs after a several year hiatus. The switch has been wonderful, but setting up a new config does take some up-front time. However, I’m starting to settle in now, only needing to make minor tweaks here and there. And I’m loving it.

Home

Home life has been a bit busy since my last update. My son turned 3, and we traveled up to Pennsylvania to celebrate with extended family, and then also had his first ‘kid/friend’ party when we returned.

Project-wise, we’ve done a bunch of work on the yard/garden this summer. We added some new raised beds, ~~are still working through~~ finished (today) a massive chip-drop, and enjoyed the first year bloom of my wildflower garden!

Books

I fell off my reading habit, letting my WIP list balloon and not finishing anything. However, over the last week I’ve gotten back on track. Here’s what I’m reading:

Currently Reading

Company Of One: Why Staying Small Is the Next Big Thing for Business by Paul Jarvis
Daggerheart Core Set by Spenser Starke, Rowan Hall, Matthew Mercer (Darrington Press)

Recently Finished

Grow Wild: The Whole-Child, Whole-Family, Nature-Rich Guide to Moving More by Katy Bowman
Life-Changing Homes: Eco-Friendly Designs that Promote Well-Being by Nicolás Boullosa and Kirsten Dirksen
Leonardo da Vinci by Walter Isaacson (audiobook)

See more of my reading history or connect with me via my story graph profile.

Rethinking my RSS use

2025-08-01T02:47:00Z

Elfeed RSS reader

Background

Over the past month, I’ve been cleaning up and organizing my personal digital music library to bring it back to life after years of streaming era neglect. Oddly enough, that task has pushed me back into searching for and using TUI applications, as I now use mpd and rmpc to listen to my local library. I set up neomutt for email and continue looking for more TUI opportunities.

When it comes to read-it-later and RSS clients, I have been using Readwise Reader for almost a year now. It’s an expensive, but amazing system, and one I’ve configured to automatically import all my highlights into obsidian.

However… I haven’t been reading much with it lately. On top of that, I’ve been reconsidering how I take notes on what I read. I rarely use or refer back to all my blind highlighting, and the self-imposed pressure to import everything to my system has become a deterrent. It’s discouraged me from reading physical books, or even digital ones with limited Readwise import support.

Without the need to be using Readwise Reader, and the desire to have some TUI options, I started experimenting.

Trialing Apps

I loved nom's ease and simple UI, but it didn't support categorical lists like I wanted.

I started with Newsboat, a classic TUI feed reader, but I didn’t love its default configuration. So, I tried Newsraft and really enjoyed it. But it didn’t support back-ends for feeds, which I needed to sync across my devices.

Combining what I learned, I decided to set up a FreshRSS server and attempted to configure Newsboat to work more like Newsraft. I got the server up and running, but was unable to configure Newsboat to my liking.

I tried more TUI applications, including nom. Nom’s defaults and simplicity were wonderful, but it didn’t list the feed source next to each article, which made it a bit more chaotic.

I apparently couldn’t get a system that matched what I wanted… But what did I actually want?

Rethinking my needs

Reframing the problem, I thought “What am I trying to get out of RSS… Do I even NEED to worry about sync?”

After some consideration I realized I mostly want RSS to be somewhere I can go to read and have articles come to me. I don’t care about having highlights for every little thing I read anymore. If I want to note something, I’ll spin up a note for it, but I don’t need them by default. Similarly, I don’t intend to read everything in my feed, as that just adds unneeded stress to my life.

Given my new awareness, I reasoned that I didn’t actually need my feeds to sync. Having consistent read history across devices, while nice, was no longer a necessity. I now see RSS feeds as a place to get curated articles to read, from my favorite sources.

The solution

Newsraft worked perfect out of the box, but didn't support feed syncing

With the requirements redefined, I was able to come to a new solution:

I use Newsraft when on my MacBook.
As for iPhone/iPad Mini, I started using Reeder (classic) again

I used the switch as an opportunity to clean out my feed list, and break it down into different categories for both applications. So far, things have been going well. I’ve found myself actually reading my feeds again, and am much less pressured while doing so. I am very happy with this setup for what it is.

~Update~

I can cleanly maintain my RSS feed list in an `org` file, and export it as an opml file to import on my mobile devices.

It’s taken me so long to finish editing and publishing this draft, that I’ve moved beyond playing with terminal apps… and have been happily consumed by Emacs once again after a multi-year hiatus. Consequently, I’m now using Elfeed for RSS on the mac, and continue to use Reeder Classic on my mobile devices.

Elfeed was a little weird to get used to at first, but I love it now. I am able to define my feed list in an org file, which has built-in support to then export that file as an opml one which I use to easily import an updated feed list into Reeder. Amazing.

Been using the Aerospace Tiling WM on macOS

2025-06-23T14:12:30Z

My Aerospace-tiled window setup while outlining this post.

Back when I used Linux as my main (and only) desktop OS, I often enjoyed the plethora of great tiling window managers. I most commonly turned to i3/gaps (and later sway), but dabbled with others like dwm, awesomewm, and even stumpwm (I had to try the common lisp one 😂).

Since switching to the Mac as my primary desktop computer, I hadn’t found a comparable tiling alternative that worked for me. I tried yabi and Amethyst, but they didn’t stick. Eventually, I settled on using the window management commands in Raycast to manually ‘tile’ my windows when needed.

Enter Aerospace

Several months ago (I can’t believe it’s been that long already) I re-discovered Aerospace WM and gave it a shot.

I disabled jankyboarders for the Xcode simulator and iPhone-Mirroring.

Being an ‘i3-like tiling window manager’, I was able to easily drop in the default config and already have 90% of what I wanted . During the next few days/weeks, I slowly tweaked the rest to my needs.

Some of these tweaks included:

Adding the janky-borders config inside the aerospace one… and switching the colors to match.
Defining a few applications to default to floating windows, and not have borders (i.e. iPhone-Mirroring app)
Set secondary monitors to default to workspace 9 when connected.

I even got most of the configuration integrated into my nix home-manager configuration, which is awesome.

Bar config

When I couldn't get sktychybar to work with my nix config, I themed bartender to make the menu bar look good enough.

I tried setting up sketchybar, but was never able to convince it to play nice with my nix setup. While fighting with the config, I learned that bartender can actually modify its appearance to one that better fits what I wanted…so I’ve just been using that instead. It works.

What I Like

Quick to get up and going

The default config works well and the keybindings are largely what I want.

Easily customizable for things that need tweaking

Modifying the configuration for the most part is rather straightforward, especially coming from a history using i3. Being able to add it to my nix setup was a nice touch too.

It just flows nicely

I like how easy it is to move around the workspaces and windows. Even the multi-monitor approach, while simple, is straight-forward. I just hit opt-shift-tab from a space that I want to move to a new monitor, and it jumps over. Easy.

What could be better

Moving Around Window Layouts

One thing I think i3 (and other tiling managers) handled a bit better is moving around window layout configurations in a single workspace. It’s hard to dial in a good layout hierarchy as you open new windows. In i3, I was able to hit a key to tell the system that my next window should be either a horizontal or vertical split, which enabled me to build the structure as I went. In aerospace, the philosophy seems to be after-the-fact adjustments. That’s fair, but I still find it hard to organize the windows in complex nested trees. Likely, I haven’t fully made the mental adjustment on how to do this yet though.

Single (Mac) Workspace Quirks

Windows hidden *mostly* off screen. Sometimes they get stuck there.

I understand that Aerospace works better for me than alternatives I’ve tried because it forgoes trying to use Apple’s built-in spaces mechanism, and just manages everything on its own in a single workspace.

While this is fine, I do miss being able to swipe between workspaces. This might be something that I can configure a workaround for, but I haven’t gotten there yet.

Additionally, because all the windows are managed in a single workspace, sometimes things get a bit off and you can see them hanging out waiting in the corner of the screen. Or, they become stuck off-screen and the only way I can get the application to show is to close it out and re-open it.

Occasional Freeze/glitch

Every now and then, Aerospace will get clogged up while switching around and the WM will freeze for a bit. This isn’t too common and I seem to notice it less often now.

Conclusion

A 'r/unixporn/' style screenshot.

So far, Aerospace has been great! I’ve finally hit that sweet spot where I can’t remember the last time I’ve had to tweak the config. Best of all, most days I forget about Aerospace as a program… it’s just how I use my computer now. The ultimate goal for a window manager 🙂.

Now Update - 2025-05-05

2025-05-06T14:30:00Z

Just got back for a wonderful trip to Seattle/Everett WA to visit friends (photo: Richmond Beach, Shoreline WA USA)

My days still generally involve a few hours of learning or project work over coffee in the morning, followed by gardening and house tasks in the afternoon. I might flip this order as the percentage of my outdoor tasks continues to steadily rise alongside the average afternoon temperature…

Learning & Projects

I’ve been slowly working through a simple iOS app I started months ago. It’s still ugly, but slowly getting… better. I’ve also been making small tweaks and updates to the website here and there.

Home

General house tasks and maintenance still consume the majority of this time as I continue to learn how to organize and optimize the work. The main projects right now involve setting up the gardening for this year, and converting parts of our yard to expand our growing space. I’m thinking about wood chipping the rest of our property once we get the garden established…

We just returned a few days ago from an amazing trip to Seattle to visit friends.

Books

I’ve been building a better reading habit over the past year. Here’s what I’ve been working through recently:

Currently Reading

Grow Wild: The Whole-Child, Whole-Family, Nature-Rich Guide to Moving More by Katy Bowma
Company Of One: Why Staying Small Is the Next Big Thing for Business by Paul Jarvis
Leonardo da Vinci by Walter Isaacson (audiobook)

Recently Finished

The Almanack of Naval Ravikant: A Guide to Wealth and Happiness by Eric Jorgenson
Only Say Good Things: Surviving Playboy and Finding Myself by Crystal Hefner (audiobook)

See more of my reading history or connect with me via my story graph profile.

Easily Ejecting macOS Disks

2025-04-29T16:15:00Z

I can easily ejects my disk by clicking a single dock icon.

I particularly enjoy my laptop setup, which includes connecting it to a dock that supports multiple SSDs. This arrangement streamlines my backup process and provides additional storage space for infrequently used large files like games, local large language models (LLMs), and virtual machine disk images.

However, I hate how macOS yells at me when I abruptly unplug the dock. It has the right to nag… I wouldn’t want to yank the cord in the middle of a Time Machine backup. Nevertheless, I’m still too lazy to manually eject all the drives each time I disconnect.

Solution

Editing the shortcut. It just runs the AppleScript.

So, I devised a solution to streamline this process. It’s not original, and in fact, I adapted elements from various online resources to create it. Unfortunately, too much time has passed, and I’ve lost track of the specific sources I used.

In short, I configured an Apple Shortcut to disconnect known disks from my Mac using Apple Script:

tell application "Finder"
 -- Get a list of all ejectable disks
 set ejectableDisks to every disk whose ejectable is true
 
 -- Loop through each disk and check its name
 repeat with aDisk in ejectableDisks
  if name of aDisk is "MasterBall" or name of aDisk is "PokeBall" then
   eject aDisk
  end if
 end repeat
end tell

This script:

Gets a list of ‘ejectable disks’ from Finder
Iterates through the disk list
Disconnects disks with names matching those I’ve specified (In my case, MasterBall and PokeBall)

This approach ensures the script doesn’t accidentally eject disk images or drives I might want to keep connected once undocked.

Lastly, I added the shortcut to my dock. Now, I just tap an icon and those drives eject. Easy.

Conclusion

So far, it has performed excellent. Occasionally, I have to wait a few seconds for it to complete, but that’s likely due to the system waiting for the disks to eject, and isn’t really avoidable.

I have thought about adding a physical smart button to trigger the script, but the required complexity to accomplish that hasn’t been worth the effort yet 😅 . Right now, this is perfect.

Now Update - 2025-04-24

2025-04-24T14:30:00Z

Lots of photos like this in my photo feed currently as I capture images to identify and log various plants.

I’m still settling into a new routine after leaving my full-time job. My days generally involve a few hours of learning or project work over coffee in the morning, followed by gardening and house tasks in the afternoon. When my wife is home (her schedule has been quite busy lately!), we’ve been able to spend some quality time together again.

Learning & Projects

A few weeks ago, I started to learn some design principles, but quickly realized the best way for me to learn is by doing. So, I pulled an iOS app I started months ago back of the shelf to tackle. It evolved from bunch of python scripts I’ve used for years, but would prefer in proper app-form for my phone. It’s still ugly, but it’s slowly looking… better.

Home

At home, much of our focus is still on adjusting to my career transition. The main projects right now involve setting up the gardening for this year, and converting parts of our yard to expand our growing space.

We are also looking forward to a trip to see friends in Seattle WA next week!

Books

I’ve been building a better reading habit over the past year. Here’s what I’ve been working through:

Currently Reading

Company Of One: Why Staying Small Is the Next Big Thing for Business by Paul Jarvis
Leonardo da Vinci by Walter Isaacson (audiobook)
- I had to renew it at the library. It’s a long book…

Recently Finished

The Almanack of Naval Ravikant: A Guide to Wealth and Happiness by Eric Jorgenson
Only Say Good Things: Surviving Playboy and Finding Myself by Crystal Hefner (audiobook)

Added 'now' and 'then' Pages

2025-04-08T15:30:00Z

An old (filtered) photo of my computer setup in college. I thought it might fit the 'now and then' topic...

I didn’t plan on having back-to-back “I added Y type of page to the site” announcement posts… but here we are. There’s not much to say for this one, but I’m about to publish the changes and figured I might as well include another short announcement post, similar to when I added the uses archive last month.

I’ve wanted to create a now page for a very long time. However, I wasn’t really working on anything I wanted to share, and I wanted to implement the uses archive first. With that done, I’ve finally added now pages.

The implementation is basically the same as the uses pages. I reused the same code from the ‘uses archive’ and created new collection to grab and list the ‘then’ pages:


  // Then Archive Collection
  eleventyConfig.addCollection("thenArchivePages", function(collectionApi) {
    return collectionApi.getFilteredByGlob("src/pages/now/then/*/*.md")
          .sort((a, b) => new Date(b.data.date) - new Date(a.data.date));
  });

(I know this probably still isn’t the best way to do it… but hey, it works)

After that, it was just some minor skeleton work and actually writing the content. But that was it.

You can find the now page here, and eventually – once I have more than one entry – the then pages will be listed here. Enjoy!

Now Update - 2025-04-08

2025-04-08T15:00:00Z

We're in the peak of pollen season here in Durham NC, right now.

About a week ago, I left my job at Red Hat to open up more flexible time for family life and to grow my skills in various areas. So… right now I’m mostly figuring that all out 😅.

Learning/Projects

I’m starting out by dedicating some time to learn more about design - both general and specifically UI/UX design. I want to work on some projects, but they all tend to become blocked by design since I have zero experience in it. So I’m working to fix that.

I’ve also been editing my website to support now/then pages. I have put it off for some time, until I could safely share that much of what I’ve been working on was related to leaving my job.

Home

On the home and family front, I’m setting up an organized calendar/task management system so we can all be (roughly) on the same page. Beyond that, the big timely project I’ve been dragging my feet on is prepping and setting up our garden for this year. We tarped our side yard over the winter, and I also plan to try spreading some wildflowers.

My family is taking a trip to see friends in Seattle WA at the end of the month, so that’s my next big countdown item.

Books

I’ve been building a better reading habit over the past year. Here’s what I’ve been working through:

Currently Reading

The Almanack of Naval Ravikant: A Guide to Wealth and Happiness Eric Jorgenson
Leonardo da Vinci by Walter Isaacson (audiobook)

Recently Finished

Slow Productivity: The Lost Art of Accomplishment Without Burnout by Cal Newport
Only Say Good Things: Surviving Playboy and Finding Myself by Crystal Hefner (audiobook)

Added a 'uses' Page Archive

2025-03-16T20:37:00Z

A stack of my current and precious MacBooks.

I’ve long wanted to dig through my website’s git history to unearth some of my old homelab and /uses pages to resurrect them on an archive page… someday. But with discussions about transient slash pages popping up on other blogs lately, I finally decided last week that it was time.

One of my motivations for switching to Eleventy was to make website customization like this easier. So far, I’m happy to say the switch was worth it, as implementing the archive wasn’t too difficult. I essentially added a subdirectory to store the archived /uses pages, and defined new collection to grab the pages and populate the the list of a new index page:

  // Uses Page Archive Collection
  eleventyConfig.addCollection("usesArchivePages", function(collectionApi) {
    return collectionApi.getFilteredByGlob("src/pages/uses/archive/*/*.md")
          .sort((a, b) => new Date(b.data.date) - new Date(a.data.date));
  });

(I know this probably isn’t the best way to do it… I’m still finding my way around eleventy – but it seems to work 😉)

I combed through the history and pulled out a snapshot page from every few months - usually when my setup had changed enough to justify a new entry. Overall, the selection provides a good enough historical timeline of my setup.

I plan to follow a similar approach going forward. I won’t spin off every little update into its’ own archive page. However, once enough changes have accumulated - usually when there’s a new photo, or I’ve swapped out a main device, I’ll make a new entry.

There’s still more I want to refine, like cleaning up the page formatting, and optimizing duplicate images, but for now it’s good enough.

Feel free to take a look if interested!

Embracing Notification Center Widgets

2025-03-08T13:40:00Z

When desktop widgets arrived in macOS Sonoma, reviewers were generally enthusiastic about the update. But for me, the excitement quickly faded, as I found that always-visible widgets made my desktop feel cluttered and overwhelming.

Desktop Widgets

Previously using a few widgets on the desktop.

As a result, I didn’t use widgets much, except for a few on my work computer. These widgets provided simple, but useful information: the weather, battery life on my devices, and a small calendar displaying my upcoming meetings. However, my personal computer desktop remained widget-free.

With widgets permanently visible on the left side of my desktop, I noticed a shift in my window management habits. Similar to the behavior changes I observed when my display was too large, I subconsciously positioned non-fullscreen windows to the right side of the desktop, or even sized them smaller than I normally would so that the widgets fit nicely next to it. I felt like I always had to see them, even if they weren’t important to the task at hand.

What I want[ed]

At first, I wished I could define desktop widgets to show on just one of my many virtual desktops.

Initially, I thought the solution was simply a missing feature: Optionally adding widgets to specific virtual desktop(s). This would let me keep them in a space I already reserve for Activity Monitor and other status apps. Accessible when I needed it, while keeping the other workspaces clean and uncluttered.

But alas, that’s not how desktop widgets were implemented. So, I carried on, accepting that I simply wouldn’t be satisfied.

Sigh.

If only macOS had a central place for both my notifications and widgets… one that slid out of sight when I needed to focus (😂).

Notification Center

One day… I (finally) realized the solution had been sitting there this whole time, hiding just off the edge of my screen.

While working, I absentmindedly opened Notification Center to clear out message alerts that had piled up over time. After dismissing them, I glanced down to the random assortment of widgets below… and it hit me: “Hey! Widgets…here…”.

I’d always known about the widgets in the Notification Center, and even clean up and re-organize the ones on my iPhone and iPad. But on the Mac? I had mostly just ignored them and hadn’t intentionally thought about curating them, or that they could perfectly replace my desktop widgets.

Embracing Notification Center Widgets

My cleaned up and organized widgets in Notification Center.

I opened up the edit panel and got to work refining my setup. I removed widgets I didn’t need, added ones that I found useful, and positioning them in a way I found pleasing. With macOS able to use widgets from an iPhone, the pool of useful options was better than ever.

For the finishing touch, I added a custom keyboard shortcut (CMD + \) to quickly toggle Notification Center.

Conclusion

It’s been several weeks since I made this switch and overall, it’s been great. I’ve tweaked my setup here and there as I experiment with different apps, and while I use it more on some days than others, it’s proven to be a fantastic solution. The keyboard shortcut makes all the difference. Being able to press two keys, glance at the info, and then dismiss it has been amazing.

I’m glad that macOS allows desktop widgets for those that enjoy them. But for my workflow, this is a better fit.

FYI - Switched Site to Eleventy

2025-02-21T00:00:00Z

American Tobacco Campus, Durham NC

Just a quick “heads up” post… Last night, I switched my website to a new version that is generated by Eleventy instead of Hugo. So, if things look a little “off” compared to before (fonts, spacing, navigation), or if I inadvertently broke the RSS feeds…that’s why. (Sorry!)

Additionally, while working on the conversion, I decided to switch my web-server from trusty old Nginx to Caddy.

I won’t go into the detailed reasoning behind the switches, but the short version is I’ve been wanting to experiment and play around more with my website - but I often found myself fighting with Hugo. By contrast, Eleventy felt more like a “build from the ground up” system, which better aligns with what I’m aiming for right now.

As far as the Caddy switch, I really don’t need the power of Nginx for this minimal site. I explored alternatives and thought Caddy seemed like a good match. I like the simplicity managing everything with a single Caddyfile for configuration. So, I’m trying it out.

I might publish more detailed posts about the switch in the future, but I wanted to at least get this one out quick in case I did break anything, or people noticed that things look… ~~worse~~ different.

Touch ID on the iPad Mini

2025-02-03T00:00:00Z

Durham, NC

Last year, I sold my 11" M1 iPad Pro after experimenting with some e-ink devices. After some time and a bit of family device shuffling, I’ve found myself rocking an iPad mini for over a month now.

I often have issues using fingerprint readers over time (my hands peel due to allergies), so FaceID has been a key selling point and motivator for me to get the Pro line of iPads. However, now that I’ve been using the mini for a while, I’ve realized that in fact, Touch ID may be ideal on a tablet, and the iPad mini in particular.

Here are just a few benefits I’ve noticed so far:

Touch ID works more reliably than I expected. Nearly every time I touch it, it works. FaceID would often get upset if I didn’t ‘look at it’ correctly.
It functions perfectly in low or even no light. This is especially useful when reading in bed at night.
By scanning multiple fingers (and possibly your palm), TouchID works no matter how the device is orientated or how it’s being held. By comparison, FaceID doesn’t work when the camera is accidentally covered or oriented poorly.

While I do think FaceID is a fantastic technology, there are still situations where TouchID has its advantages. In a perfect world, we’d have devices with both methods, but I doubt that will happen due to cost. It would be nice though.

Disabling Safari on iOS and iPadOS

2024-12-30T00:00:00Z

Dowdy Park, Nags Head NC

Why

As an easily distracted person, I sometimes set up limits and blockers on my tech to stay focused on what I’m trying to accomplish. This most commonly takes the form of restricting sites like YouTube and Reddit, or configuring downtime settings when I repeatedly stay up too late.

Earlier this year, I read Low Tech Life by Jose Briones, which led me to reflect on my smartphone use. I concluded that a true “dumb phone” wouldn’t be a great solution for me personally. The biggest reason being that I do want to carry a camera with me, which would mean I’d always have 2 devices instead of one… and currently, my only camera is my iPhone anyway.

So, I switched to thinking about where it made sense to ‘dumb down’ my smart phone and identify the areas I wanted to limit. I deleted my video apps (YouTube, Plex, Nebula, etc.), and moved on to cleaning up the other apps I don’t use on my phone.

I don’t use much social media, which is what people typically remove when going through an exercise like this. If not social media, what is the next activity I most often waste time on? Web browsing. It’s the “let me look up this random fact”, or “I want to verify something”, or “I need to search for and re-read the same forum posts about this tech product I’m never going to buy anyway” browsing that gets me.

But removing a browser can be tricky. And a little scary.

How

Disabling Safari in the Screen Time settings.

Disabling Safari on the iPhone and iPad can be done through Screen Time settings. This means that Screen Time must first be enabled and setup. Once it is:

Open Settings and navigate to the Screen Time Section.
Tap Content & Privacy Restrictions under the Restrictions section.
Ensure the Content & Privacy Restrictions toggle is enabled.
Tap Allowed Apps & Features below the toggle. This will open a list of built-in iOS/iPadOS apps, including Safari.
To disable Safari, switch the toggle off.

Once disabled, Safari should no longer appear as an app to open on the device.

(Disclaimer: These steps were accurate when this post was written in December 2024, but may differ in the future.)

Concerns

I was still able to view useful information, like restaurant web pages through in-app browsers. For example, in Google Maps.

After disabling Safari, there were a few concerns I had as I entered this experiment. Most of them revolved around not being able to look up information for “legitimate” uses. For example:

Viewing store or restaurant hours while on the go
Skimming linked documents in the show notes of a podcast
Looking up the weather or news while out and about

Fortunately, it turned out that even with the Safari app disabled, all of these tasks were still possible – through apps. While Safari is disabled, the built-in browser APIs that apps use still worked, most of the time. To maintain required functionality, use specific apps in place of general Safari searching.

Issues

I said the built-in browsers work most of the time because there were a few instances where I hit issues. Sometimes, a link wouldn’t even try to open when I tapped it in an app. I think this happens when an app immediately attempts to handoff a link to Safari, but nothing happens. Here’s two specific examples of this occurring:

While trying to open links in an Apple Note. The preview card loads, but the link itself doesn’t open.
A more concerning case was when connecting to Wi-Fi, where the connection normally directs a user to a captcha page or requires verifying certs that are expected to complete through Safari. I encountered this while attempting to connect to my work network after renewing my access credentials. I had to re-enable Safari to complete the process. I imaging this could be a significant headache in public locations.

Conclusion

I don't know if I'll continue this method or try something like ScreenZen. But as a built in method, it works.

Still, issues aside, for the most part, this is exactly what I wanted when I decided to test it. I still had access to most of my non-distracting uses of a smartphone, without the ability to pull it out of my pocket and start browsing the web whenever I wanted. By intentionally installing apps for the uses I wanted ahead of time (and optionally using Screen Time to disable installing new ones), I was able to dial in exactly how “dumb” my iPhone should be.

I think I’ll keep Safari disabled by default, with maybe a few exceptions for trips or vacations. I’m also tempted to re-enable Safari on my iPad Mini… but I’m pretty sure I would just waste time on it if I did. I might be able to find a middle ground, using something like ScreenZen, but I have yet to fully test it.

Anyway, I highly recommend giving this a try if you also find yourself living in a browser all the time. There’s nothing to lose; Safari can always be added back if it doesn’t work out.

Switched to Todoist for Personal and Home Tasks

2024-11-30T00:00:00Z

P!NK Concert - Lenovo Center, Raleigh NC

It’s been just shy of a year since I setup the obsidian-tasks plugin to manage tracking my events and tasks. However, I recently switched to using Todoist for my personal and home task management.

Why the switch?

The better support for sub-tasks is what initially prompted the switch from ticktick to todoist.

In that obsidian tasks post, I hinted at trying to use a ‘family’ obsidian vault for notes/planning/tasks. It didn’t work out. In the end, I was the only one that used it.

Defeated, I moved our family tasks and projects back into ticktick. As my wife and I tried to more consistently use ticktick, I worked to restructure our shared list to support not only general tasks, but organized projects as well. Unfortunately, ticktick just didn’t seem to support a structure that made sense for what I wanted. In particular, it had weird limitations with nested tasks.

Researching work-arounds, I started to see evidence that todoist might actually be a better fit for our use. To test the idea, I spun up a proof-of-concept project… and it worked. So, only a day or two after moving all of our family tasks from Obsidian -> TickTick… I did it all again, this time from TickTick -> Todoist 😅.

After moving the family tasks, I went ahead and converted my personal ones as well. I had already been debating splitting out my actionable tasks items from obsidian into TickTick for a few reasons anyway. First, I used TickTick to track my general one-off and home tasks, and Obsidian was used for project items, but some other general tasks. There wasn’t a clear line of when to put a task in one verses the other. This made it a pain to keep everything in order, and even caused duplication.

Beyond that, with so many tasks built up over the past year, Obsidian was becoming quite sluggish. It was frustrating to work in, and multiplied the time required when planning in my notes. I knew if I could phase out the majority of the tasks items it was calculating, it should help with performance.

My Todoist Setup

As stated above, I started with moving over the family stuff first, and then almost immediately converted my personal tasks as well.

Home Workspace

Our HomeTasks Project, and the sections we group tasks into.

I created a shared workspace for our home, with two projects in it: HomeTasks and HomeProjects (I also later added a Home_Inbox to dump rough task ideas).

The HomeTasks project is a list view that contains general family tasks. For example, taking out the trash, daycare drop off, bathing the dog, etc. It is divided into sections representing different family areas (house, pets, kid, etc). This organizes the tasks into their domain, for easy viewing

HomeProjects is a board view containing different ‘Project’ task cards, organized in columns designating project status (ex: idea, planning, in progress, on hold, done). Each card represents a larger family project that contains many sub (or even sub-sub…) tasks for it.

Personal Workspace

My todoist habits project contains my common recurring tasks.

My personal setup is similar to the family one. I have a Tasks project list that contains my general, non-project tasks that I dump into the system.

I also have my own RyanProjects board, containing ‘project tasks’ (this post, for example) that move across the various board states. The project parent task contains all the relevant sub-tasks related to it.

Compared to the Home workspace, I have an extra todoist project called Habits. It started as a replacement for the excellent Habits Tracker in TickTick, but didn’t live up to my expectations for that role. So, I moved the tracking logs back to the goal notes I keep in Obsidian. The Todoist project remained, and now houses my daily/weekly recurring tasks, broken into sections, roughly grouping when each one occurs.

Filters

I have a handful of filters, 3 of which I have favorited.

So far, I have just a few filters I rely on:

Remaining Habits: Un-completed Habit tasks that remain for the day,
Overdue: Un-completed tasks that are past their due date/time.
Today’s assigned tasks: Tasks for the given day that have an assignee. This lets me check if there is anything assigned to my wife (which is usually filtered out in my other views) that I might need to re-assign to myself.

I expect that as I become more familiar Todoist’s capabilities, I will continue to create more filters as needed. I also haven’t setup any labels, yet.

Conclusion

While I enjoy todoist, I will miss the habit tracker in TickTick.

So far, I’m happy with this new setup. I still use Obsidian with the obsidian-tasks plugin for my task manager at work, but it makes more sense there as my work-vault is isolated. For my very integrated personal and home life however, I think this switch has introduced some appreciated clarity. My current organizational system:

Notes/Journal/Logs:
- Personal: Obsidian
- Family: Shared Apple Notes Folder
Task Management:
- Both: Todoist (With my personal workspace and a shared family workspace)
Events
- Both: Calendars (My wife and I each have one we share with each other, and I subscribe to some additional ones for extra information)

In summary, I love Todoist’s flexibility, integrations, and efficiency. There are a few features I miss from TickTick, mostly the habit tracking. However, I’m glad to have some separation in my systems to re-enforce clear definitions of what I consider a task vs. event. vs. note. We’ll see how long it lasts this time 😆…

Thoughts on 'Portability'

2024-10-31T00:00:00Z

Nags Head, NC

I find it interesting that reviews for tech products with two size options (ex: 14" vs 16" MacBook Pros), will often take a somewhat lazy approach when comparing the two variants. Usually, the argument is boiled down to “the smaller device is ‘more portable’ but the bigger one, ‘has a larger screen, which is great for multi-tasking’. I guess both statements are technically true, but they assume “portability” and "multi-tasking” are universal, constant, traits that match every person and use case. Worse, I think this quick passing remark prevents consumers from digging a little deeper into how they will actually use a device once they obtain it.

Thoughts

My 14-inch M1 Pro setup on Vacation in 2023. Despite being the 14-inch, it doesn't seem very portable...

After several years using a mix of 14" and 16" MacBooks Pros, two 13" MacBook Airs, sampling my wife’s 15" MacBook Air, and a smattering of all sizes of laptops and tablets before that, I’ve developed a couple of opinions about device sizes, that might seem counter-intuitive.

‘Multi Tasking’ and Screen Size

While doable for quick fixes, Xcode feels cramped on smaller laptop screens.

The first lesson, is that the screen size requirement for “multi tasking” is heavily dependent on the workflow and tooling of a given task. Specifically, screen size only really matters when considering what needs to be displayed on the screen concurrently This may be all the panels and menus of a complex application UI, or the simultaneous use of many simple ones.

For instance, I prefer a smaller screen when I am writing a post, as it helps keep my focus on the task at hand. However, I wouldn’t consider that process ‘mono-tasking’. I’m often flipping between windows writing the draft, editing photos, grabbing source links, checking words in the dictionary, etc. However, none of those tasks require an application with an extensive UI, and I don’t need to see them all at the same time.

By contrast, when I’m coding using a heavier IDE like Xcode, the UI of that single application contains many sections I like to have open (the code editor, simulator preview pane, debugger pane, file tree, etc). This makes my 16" MacBook Pro much more suited for the job. It should be noted that I didn’t simply state ‘the 16" is better at coding’'. If I’m coding some Clojure in Neovim, there’s not much of a difference between sizes.

‘Portability’ Preferences

Portability can refer to a lot of things. Does it imply the ability to pull a device out of a bag with one hand to check a few emails at the airport or a coffee shop? Or does it represent a computer that travels in a suitcase, and is then setup on a desk to complete a week’s worth of full days while working from a remote office on a business trip? Both cases require a “portable” machine, but my recommendations would differ greatly for each of these roles.

The best example I’ve heard discussing this difference is from Marco Arment on the ATP podcast, when he describes living both the “laptop-laptop” and “desktop-laptop” lifestyle (a setup I also have…).

The “laptop-laptop” is a small computer to be used as a typical laptop, for “laptop-things”. This can be around the house for small tasks, when frequently moving from meeting room to meeting room at the office, or thrown in a bag for quick trips. It’s the grab and go computer.

The “desktop-laptop” by contrast, lives most of its life indistinguishable from a desktop computer. It remains at a desk, often tethered to an external monitor, mouse (trackball), keyboard, audio equipment, external hard drives, and other peripheral devices. However, it also functions as an all-in-one portable workstation, containing everything required (or close enough) to work when relocated away from that desk.

Example

My setup on vacation in 2024. While the larger 16inch, I didn't feel as though I *needed* all the other stuff given the larger screen.

For example, let’s compare my computer setup during two back-to-back years of my family October vacation. On both, I planned to (and did) work on some projects. Last year, I worked on the massive Website Redo (2023), and this year, a few programming projects, as well as starting this post.

Last year, I still had my 14" M1 Pro MacBook Pro. However, given its smaller size, I felt I would need more periphery items for the setup to work comfortably. While I didn’t need to go as far as bringing the mouse and ferris sweep, once you pack a portable monitor and stand for the additional screen space, the other devices snowball quickly.

By comparison, after switching to the 16" M3 Pro MacBook Pro as my main device this year, I didn’t feel I needed to bring everything else to get work done. The 16" MacBook Pro works great as a full portable (there’s that word again) workstation. I still have the option to bring the extra screen, keyboard, etc (and I might have considered it if I was working full days, or for more than a week), but I was fine without it, whereas the 14" felt cramped. Despite being the larger laptop, in the end the 16" setup took up much less volume in my bag.

Conclusion

Hopefully, this didn’t sound too much like a rant. With endless advice following the “portable vs good for multi-tasking” template, I wanted to put something out there to counter it. I’ve formed these thoughts through much trial and error, and they still continue to evolve over time.

macOS Sequoia iPhone Mirroring

2024-09-24T00:00:00Z

Portland, ME

With macOS 18 - Sequoia being released to the general public, I decided to finally update my MacBooks to it. As reviews indicate, with the focus largely on Apple Intelligence this year (still to be released), there are few major items in the… un-intelligent?… aspects of Sequoia. However, there is one feature I was interested in when announced at WWDC, and have enjoyed even more now that I have my hands on it: iPhone Mirroring.

What is iPhone Mirroring?

iPhone mirroring passes my phone's audio through my MacBook speakers. This enables me to listen to iOS only applications on my Mac. For example, the Apple Classical Music App.

As part of the “continuity” features in the Apple Ecosystem, iPhone mirroring allows the user to access and control their iPhone via a macOS application. This is comparable to how the “Screen Sharing” app enables remotely controlling other Macs on the network. While connected, the phone remains locked, but can be fully interacted with through the app.

What I Like

Similar to the screen sharing updates released last year, iPhone mirroring has been implemented quite well. In particular, there are a few details that I didn’t expect, but am delighted to see.

Audio Passthrough

First, iPhone audio is passed through to the MacBook. While this is pretty obvious, the results are wonderful. As an example, this allows using iOS audio apps, like Overcast or Apple Music Classical, “on the Mac” through its much better speakers. It also nullifies common issues I experience with sync. I can listen to a podcast through my computer speakers, and then simply continue on the phone when I get up and walk away. It’s the same device controlling the audio, so syncing is not required.

Keyboard Shortcuts

A second nicety are the keyboard shortcuts. They cover common actions that are typically completed using swipe gestures on iOS:

CMD + 1: Navigates to the Home Screen
CMD + 2: Opens the app switcher
CMD + 3: Launches a spotlight search

Additionally, there’s CMD++/CMD+- to increase/decrease the window size, as well as the usual edit/window shortcuts you know and love (cut, copy, paste, etc).

Performance

iPhone Mirroring is useful for debugging 'on-device' in Xcode.

Lastly, in my experience the performance has been great. Beyond improving user experience, the responsiveness encourrages other useful workflows. For example, this makes on-device testing in Xcode much nicer. You can build a project on the device, but test and debug it in the iPhone Mirror, essentially treating it as a better version of the iOS simulator.

Conclusion

While it’s still early days, I have enjoyed iPhone mirroring in macOS Sequoia. Like the screen sharing app, I imagine it will become a staple in my lazy toolbox, powering the use of all my devices from a single computer.

Obsidian Tasks Plugin - Events

2024-08-24T00:00:00Z

Bar Harbor, ME

Earlier this year, I wrote about how I started using the obsidian tasks plugin to manage my tasks, in both my work and personal obsidian vaults. One topic I didn’t discuss in that post, is how I also track “events” as tasks in my work vault.

Why

I like to organize my meeting events in a log for that note.

After I switched to having each ‘task’ living in its related obsidian note, I wanted to also have scheduled/timed items (events) housed there too. For example, I could make a note for each of my recurring meetings, and have a recurring task setup in it. This would allow me to see meetings listed in my daily/weekly notes, and remind me of them while planning out my work. Additionally, as I completed the recurring task over time, it would serve as a log of when I actually had each meeting, in the note for that meeting.

This desire increased when I switched into a manager role, and the majority of my days became littered with meetings and 1x1s.

How

After initially exploring the possibility of defining multiple global task filters, I opted for an equally simple solution instead. Short answer: regex.

Long answer: I defined a convention that if tasks started with (HH:MM), I would consider them to be events. Additionally, when I use a time of (00:00), I consider it an “all day” event. With the format decided, I was able to use regex expressions in the filters of my tasks boards. Specifically, I used the following line to either include or exclude tasks that match this format (match in this example):

description regex matches /\(\d{2}\:\d{2}\)/

Next, I modified the callout task boards in my daily and weekly note templates to split out “event” tasks from my normal “todo” ones. This makes it easier to see all my scheduled events without them getting buried in my normal task list. Additionally, since the naming convention starts with the time (in 24 hour format), I can use sort by description to ensure the events are in chronological order:

>[!hint]+ Meetings & Events
>```tasks
>(scheduled on {{date:YYYY-MM-DD}}) OR (due on {{date:YYYY-MM-DD}})
>description regex matches /\(\d{2}\:\d{2}\)/
>short mode
>hide tags
>hide task count
>sort by description
>```

I was also able to apply this filter in the “Todo” and “Completed” tasks lists to remove the event tasks, since they are displayed in their own list:

>[!todo]+
>```tasks
>not done
>(scheduled on {{date:YYYY-MM-DD}}) OR (due on {{date:YYYY-MM-DD}})
>description regex does not match /\(\d{2}\:\d{2}\)/
>short mode
>hide tags
>hide task count
>group by filename
>sort by priority
>```

That’s it.

Conclusion

I wanted to have timed events show in my daily/weekly notes, separate from my todo list.

I’ve been using this setup at work for probably 6 months now, and it’s worked extremely well. I have a note for each recurring meeting, and I added ‘Meeting Log’ sections to my people notes to track individual 1x1s with them. When I have a working group or one-off meeting related to a particular project, I just make the event in that note. This continues the benefit of having project notes tracking all their stuff, so it can be easily found in one place.

In my personal vault, I use the same naming conventions for events, but haven’t updated my note filters to split them out, as I don’t use as many timed event tasks in that vault currently.

Still, if you have a need to track or manage ‘events’ in the tasks plugin, this method works surprisingly well.

Boox Page Initial Thoughts

2024-07-24T00:00:00Z

Durham, NC

Over the last few years, I have periodically wanted to replace my old kindle paperwhite. To mix things up (and for the good library support at the time), I planned to switch to a kobo device. But, I never got one. During that same period, I also became increasingly interested in e-ink technology, and the android e-ink tablet market. After several months researching devices… I bought one.

Why the Page?

The larger e-ink tablet use cases overlapped too much with the 11" iPad Pro I already have.

Boox Tab Ultra C

The Boox Tab Ultra C with its optional keyboard case, is what initially caught my attention. It was a small, e-ink tablet computer that seemed perfect to me at first. However, once I started seriously considering purchasing a device, I admitted I didn’t need to jump right to the high end. Being my first e-ink tablet device, this was especially true. What if I hated the experience? On top of that, much of the ideal use cases of the Tab Ultra C overlapped with the 2021 11" iPad Pro and iPad magic keyboard that I already have. So I moved on.

Boox Note Air 3 ©

Next, I looked at the Boox Note Air 3, which seems to be considered the best all around device for most people. While researching the Air models, I realized I would likely prefer a black and white device instead of color. I wanted the clearest text possible, and to not need a backlight most of the time. However, the current color eink screens are apparently dimmer and not as sharp compared to black and white.

During this research period, I started using my very old kindle paperwhite, just to confirm my desire for e-ink and to verify I would use it day to day. As interested as I was in larger tablets, the more I used the smaller kindle, the more I realized what I actually needed was an e-reader. Something lightweight that I can grab and read on the go. So the practical choice would be to get a reader that I could also use to experiment with android on e-ink first. If I liked it, I could look into larger devices in the future if it made sense. (Also, my wife has been interested in how this goes for me, as she’s picked up reading again and would gladly accept a hand-me-down e-reader eventually 😂).

Boox Page

So, I started looking at 7” options. This was right when Boox announced the “go” series devices and while the go color 7 looked nice… I really didn’t want color (the go 10.3 however… I might still have my eye on it…). I also looked at the Mini C which was a good writing + reading option. Once again though… I wanted B+W e-ink, and the slightly smaller size of the page made it more likely to fit that handheld size I wanted. Apart from being an older device and running a slightly older version of Android, the Page had everything else I wanted: a Black and White 300ppi display, a front light in case I needed it, handheld size, and even page turn buttons. Buttons weren’t a requirement for me, but having never used them, it was something I was interested in trying.

After a quick scramble researching all the competitor devices, I ordered a Boox Page from BestBuy.

Initial Thoughts

I like to use the Boox Page as a photo frame when it's in standby.

When the Boox Page arrived, I set it up and started playing around. It took some time to figure out the configuration and my preferences for everything, but I eventually got it all dialed in. During the process, I periodically jotted down some notes about experience:

The First Day:

The screen is (physically) smoother than I thought it would be. I’m sort of sad by this as I’ve loved the matte feel of the kindle the last few days.

I’m going to want a case… I don’t want the screen breaking when I throw it in my bag or walk around with it outside of the house.

Refreshing the screen takes some getting used to, but going okay so far…

Typing in Obsidian on it with the ZSA Voyager Keyboard was actually nice. I thought the refresh lag would be unbearable, but I find it manageable.

(The most annoying thing while typing is not being able to use the MacOS/iOS (emacs) keyboard navigation I love and am used to 😅).

The Second Day:

I ordered a case. Not sure when it will come.

I managed to change how the page turn button taps and long holds work, which makes a huge difference in usability. I have a long hold on the top one set to open the multi-tasking view, and the bottom one to trigger a full screen refresh (to clear ghosting).

I added some of my images for the screensavers, and it’s amazing. I like placing the Boox Page on a stand under the light on my desk when not in use. It’s basically a black and white photo frame. Delightful.

I configured a few other apps, including setting up f-droid and getting termux light mode setup.

Several days later:

Things are going better with it. I am experimenting with different ways to be able to read on it, but have my progress sync with my other devices.

I played with typing on it more using several keyboards.

Battery life overall seems good.

Honestly, I love e-ink…

Summary thoughts/conclusion

The Boox Page is a great e-reader, but also a way for me to experiment with e-ink.

Overall, I’ve been very happy with the purchase. The size is ideal to use as an e-reader, but yet large enough for me to experiment with using e-ink in different scenarios. And I love e-ink. It’s such a cool technology - a blend of our digital tech, but more adjacent to the analog world, with it’s magical ink pages that don’t require a backlight (or power once set) to read. It’s really cool.

Anyway, I’m glad I went with the Page, as I think it was a perfect entry device into this world for me, but also filled a gap I had in my tech setup. I love reading on it, and hope to continue experimenting.

Started Browsing Geminispace

2024-06-29T00:00:00Z

Over the past year, I’ve been intrigued by the “small web” and have been trying to spend more of my online time there. I touched on this a little bit in my website redo 2023 post [1].

While browsing some blogs in a web-ring recently, I noticed a few of them referenced and linked to ‘gemlogs’. I decided to dig a little deeper, and watched a few videos about gemini [3] [4]. One of them also talked about gopher, which I had heard about but never used.

After these videos, I still wasn’t too interested in gopher… However, I DID want to look into Gemini. So, I downloaded the Lagrange gemini client, and started browsing.

So what is gemini?

The Lagrange Gemini Client

Gemini is a newer internet protocol, that serves as an alternative to the prevalent http(s). The project was started in June 2019, and really took off during COVID (I’m a little late to the party). It embraces the text-focused aspects Gopher, while improving the security and ease of use to more modern standards. Gemini approaches the internet with a “keep it simple” and “less is enough” guiding philosophy. To learn more, the project has a wonderful FAQ [5].

[5] Gemini FAQ

How does it differ from the web?

The obvious main difference between gemini and the standard http(s) web is that gemini is a simplified protocol, by design. So what does this mean in actual use?

To start, nothing is embedded in-line. That means no text formatting (i.e. bold, italic, strikethrough), in-line code snippets, or links within the text. Everything takes place on its own line, and as result, the purpose of each line can be determined from the first 3 characters. In total, there are 6 types of lines:

Text
Link
Heading
List
Quote
Preformat toggle (code blocks)

“Gemtext” syntax [6] is similar to markdown, but again, much more stripped down. For example:

Headers use ‘#’, but have a limit of H1 -> H3.
Lists are supported and indicated by a leading ‘*’, but they are only allowed to be one level deep (no nested lists).
Quote lines also start with ‘>’
‘Preformat toggle lines’ work similarly, using ‘```’ (three back ticks) to start and end a block.
Links get their own line, using the syntax ‘=> LINK OPTIONAL-LABEL’. Links of any protocol are supported.
Again, no in-line text formatting

That’s it. While this might seem extreme and overly limiting, it really does feel clean and focused. There is less to worry about while writing. It also has the added benefit of making parsers MUCH easier to code.

In addition to gemtext, there are some simplifications in gemini on the server side. For example, the server should only send one response per request. This means that videos, scripts, fonts, style sheets, or images shouldn’t autoload, as just the page source is returned for the initial request of a page. Images and other files are still supported, but as links. Many clients will load images in the page after clicking them or optionally when scrolling. There are also different status codes and requirements found in the spec [7].

Why does this post look different?

If you haven’t caught on by now, this post looks different from my others. I’ve formatted it to somewhat represent what browsing a ‘gem’ (page) looks like. There’s no inline formatting, no list nesting, and (probably most noticeably) links are on their own line, usually at the end of a paragraph or section.

Please note, this isn’t a perfect recreation, as I have left some items in, like the images and captions, as just linking wouldn’t render them like I experience them in Lagrange anyway. I also didn’t include a header image for this post. It felt superfluous for a gemini ;) .

How to get started

Get a client

The Amphora Client

Getting started with gemini isn’t too difficult. You just need a browser that supports the protocol. Actually, you don’t even need that, as you can view gemini content through a proxy, like the SmolNet Portal [8].

That being said, a client really does provide the best experience. I use Lagrange [9] as my GUI client, and was even able to install the mobile beta via TestFlight on my iPad and iPhone.

For the cli, I adore amfora’s [10] user interface and design. However, it is in “maintenance mode”, so I’ve also tried telescope [11], but I am having trouble getting it to look as glamorous as amfora. Telescope does seem highly configurable though, so it might be possible.

For other operating systems, or for alternative options, take a look at the known software page [12].

Browse

One of my favorite pages has been 'chilly weather' (despite the misleading name for the current weather...)

Once you have a client, start browsing! In addition to the ‘awesome gemini’ page [13], there are a few other landing pages that can be helpful. I mostly used the ‘Getting Started’ page bookmarked by default in the Lagrange client [14], but something like geminiquickst.art [15] could also work.

Another great place to find content is by looking at one of the many aggregators. I occasionally take a peek at antenna[16] or cosmos [17] to see if any newly posted logs catch my interest.

Lastly, you can always search gemini for something! I’ve used TLGS.one [18] for search, but there might be others out there.

Basically, have fun! There’s all sorts of cool gems that people have made, despite the minimal protocol. For example, I’m quite fond of Chilly Weather [19].

[13] (gemini) Awesome Gemini
[14] (gemini) Lagrange Getting Started Page
[15] (gemini) geminiquickst.art
[16] (gemini) Antenna
[17] (gemini) Cosmos
[18] (gemini) TLGS.one
[19] (gemini) Chilly Weather

Conclusion

So, what have I thought browsing geminispace the last few months? Honestly, it’s been great. It feels different than reading on the normal web. Quieter, I guess. And the quiet isn’t just because there’s less people (and content), but due to the design limitations. There’s less crap screaming for my attention, because it simply isn’t allowed. With gemini, the author is mostly concerned with the content, and not so much the style of it. That is up to the clients to implement and control.

This (client-side styling) can be a mixed trade off. While I do miss the creativity you can often find on http(s) blogs, it is nice having things all look concise and consistent in my gemini clients. Additionally, browsing gemipace is SO much better than the web when using alternative clients, like terminal based browsers, or if you have a poor internet connection.

Overall, I love it, and hope to continue using it as an escape from the big web.

New ZSA Voyager Keyboard

2024-05-15T00:00:00Z

The Shipped ZSA Voyager Keyboard

In February, I decided to finally order the ZSA Voyager keyboard. By mid-March, it had arrived. Here are some of my thoughts after using it for over 2 months now.

The Ferris Sweep’s few Downsides

While I still enjoy my ferris sweep keyboards, I did start running into a few problems with them. My biggest issue was the long term reliability. Over time, some keys would become sensitive and misfire constantly. My original sweep build has a misfiring u key that is so bad (likely from my poor soldering skills) that I can’t really use it anymore. This is particularly bad given that I use vim-bindings in many of my apps, where u is “undo” 😬.

Second, while I still think for most uses, a 34-key layout is perfect for my needs, I admit that it would be nice to have a little more wiggle room designing layouts for my less common activities (i.e. gaming). Additionally, the sweep is a little too compact for someone with my sized hands.

Deciding to get the Voyager

The main body of the Voyager isn't too much larger than the sweep.

When the Voyager was announced, I was instantly interested. It looked like a well-built, well-designed, slightly larger version of the ferris sweep that I cherish so much. The main ‘con’ people seemed to have with it was that the thumb clusters only had two keys. Coming from the Sweep, this wasn’t a problem for me.

My only issue with it was that, in contrast to what others were saying online, I thought it had too many keys 😆. I didn’t want to sacrifice portability to have more than 34 keys. However, comparing with the size printout, the Voyager wasn’t much larger than my Sweep.

I decided that when the stability of my ‘good’ sweep got worse, the voyager would eventually be my next keyboard. With time, my main sweep did get worse, so I ordered the voyager mid February. After some debate, I finalized my configuration:

“Cosmic Nomad Black” Color
“Blank” keycaps
Kailh Choc Pro Red switches (35gf)

The Pro Red switches became available shortly before ordering my Voyager. Without my beloved Silvers (40gf) as an option, I decided to try the slightly lighter Pro Reds (35gf) instead of the heavier Reds (50gf). Note: these are all liner switches.

With that, the order was submitted and I eagerly waited for it to be built and shipped.

Setup (Flashing my Layout)

Setting up my layout in the Oryx configurator.

About a month later, the Voyager arrived. After unboxing and checking it out, I immediately worked to flash my layout on it. This is something I was able to start before the Voyager even arrived, using ZSA’s Oryx configuration tool.

Despite the Voyager having 50% more keys… I still love my 34-key layout on the sweep. So I mostly stuck to that when making this layout. I just added a number row to the base layer’s additional top row, and some lighting controls to that row on another layer. I figured I can always adjust it over time as I get used to the extra keys and have ideas for them. Additionally, it makes switching back to the sweep easier when I want to use it.

With the firmware compiled, I easily flashed the board using the ZSA flashing tools, and pressing the physical reset button on the board. No paper clip/tweezers required!

Initial Thoughts

The Voyager makes for a great desk *and* portable setup.

I started using the Voyager as my daily driver immediately once it was flashed. I adjusted rather quickly, alternating between the Voyager and my laptop keyboards. I took a few notes about the initial experience:

While typing, it feels good so far. Honestly, the weirdest thing has been the additional outer pinky column of keys, and the additional top row. It seems I’ve gotten used to using the edges when placing my fingers on the sweep.

I did a “short burst” monketype test and got over 100 WPM, with only 1 error. It was a test without punctuation or caps, but using my normal 34 key layout is working well.

My basic typing feels really great after some (more) time. I do need to add the number layer still though. While I can use that top row on the base layer… I’m used to (and apparently love) slamming down on those two thumb keys and just using the numbers above my home row fingers. I think it’s easier for my brain to know where a number is that way.

I’ve also noticed a few tweaks I need to fix. For example, I might need to edit the timings for the hold keys, as I’m noticing some SHIFT issues… so far though… it still feels excellent.

Things I like:

Magnetic. I love plopping this on my MacBook’s palm rest

The tenting legs

The choc keys, but with MX spacing. I didn’t know if I’d like it, but it feels great and by comparison my Ferris Sweep feels so tight whenever I use it now.

Longer Term Thoughts

I love how the metal case allows the Voyager to attach to the magnets in the MacBook's palm rests.

Over a longer period, I continued to make a few layout changes, mostly minor tweaks here and there. I also added a layer to experiment with mouse controls one day when I forgot to bring my trackball to the office. After some extended use there are many things I appreciate. Here’s just a few:

Sturdy Construction: The Voyager feels like a premium board. There is no flex, and everything is designed and built so well.
Magnetic: I underestimated how much I would enjoy this. Having a magnetic case enables so many more tenting possibilities. Additionally, due to the magnets in my MacBook (that keep the lid closed), I can attach the keyboard halves to my laptop’s palm rests. This works so much better than trying to place it on top of the built-in keyboard!
Switches and keycaps: Both the Choc Pro Red switches and the custom keycaps for the Voyager feel delightful. (They also sound quite nice too).
MX spacing: I was leery about the MX spacing with choc switches. I was worried that it would take away from what I enjoy so much about choc, but honestly it just made it better. I get all of the low profile, portable goodness, but without my hands cramping up.
Column Stagger: The way the column stagger has fit my hands so perfectly on the sweep was largely what kept me from more mild ones on boards like the corne. The layout and stagger of the Voyager seems to be spaced perfectly for my hands.
Portability: The carrying case with integrated tenting feet really assist the portability of the keyboard. I just fold it up, and throw it in my bag, like a smaller pair of headphones.

With all that said, there are a few things that could be improved (seriously, I had to think hard to even come up with these).

TRRS Cable Position: Personally, I would prefer the TRRS ports to be closer to the center of the board. It connects the two halves so I usually like it closer there.
- Thinking about it now though, I think the ports on the outside is better for tenting the board. Fine… I’ll revoke this one 😆.
Tenting Feet: While the feet are great, they can be annoying to attached and remove when frequently packing up and traveling the board like I do. This isn’t really a fault of the design, and just of a pain point with how I use it. Additionally, a portable design that provides ability to adjust the level of tenting would be awesome.

Conclusion

Using two magnetic stands for tenting.

Moving forward, I’m planning to construct tenting setups for both my work and home office desks. I hope this will help increase the ease of commuting the Voayager even more, as I should be able to just pull it out of it’s case and plop it on magnets instead of messing with the feet each time. I’ve. tested this out in the office using two cheap magnetic phone stands, and so far it’s worked well.

To conclude, I adore the Voyager. It is essentially my ideal keyboard. It’s a mix my previous Ergodox EZ and my Ferris Sweep. Comfortable, solid, portable… perfect.

Minisforum um560 Died

2024-04-24T00:00:00Z

Durham NC

Last week while trying to do a fresh install of Fedora, my Minisforum um560 mini computer died. I plugged it in, heard a faint “pop”, and haven’t been able to power it on since. Unfortunately, after a history of a questionable power supply issues, this wasn’t much of a shock…

History

I purchased the um560 in 2022, after researching several mini-computer models. I wanted a lower powered computer (compared to my desktop), that I could leave running as my “linux dev” machine, and remotely connect to from my MacBook.

I was interested in the the um560 specifically, because it was Minisfourm’s first model to feature ‘alt-mode charging’. This meant that it could (in theory) be both powered and connected from the same cable via a thunderbolt dock. In other words, the ”one cable lifestyle”, just like I have with my MacBook. Even though I wanted the um560 to mostly run headless, I figured this feature would be convienient when I occasionally work on it with a display. Shutdown, plug in, and boot it up.

Previous Issues

In practice, this never truly worked. The um650 would shut off trying to load the OS after the boot screen. It didn’t seem to be getting enough power, despite using connections that had more than enough PD. On top of that, the general power delivery was very unstable and the computer would randomly shutdown while doing tasks like running updates (even when using the included adapter + cable).

Eventually, I tweaked the bios settings to use a lower TDP threshold, hoping it would help. It made the system a bit more stable when running as a headless server, but I never really got the alt mode working consistently.

I ran it for a year or so as a headless server, mostly serving Plex and building my website. It remained stable for longer periods of time, but was always a gamble of when it would crash, not if. Not a great feature for a server. I eventually spun it down to only use from time to time as a Linux test machine. That is until it broke.

Conclusion

That’s all. I wanted to log the issues I had with the um560, just in case others experience anything similar. Hopefully, it’s just an issue with my unit or even just this (now discontinued) model. I haven’t had the same issues with my Minisforum hx99g, although it doesn’t have TB power delivery, and admittedly I don’t have it powered on 24/7 or for very long periods of time.

As for the um560, I’ve removed the SSD and 64 GB (!) of RAM, and will send the unit to e-waste recycling. Bummer. I might hold onto the RAM and keep an eye open for a cheap small computer to use as a new test server, if I need to eventually. For now, I can just setup the Linux SSD in my hx99g finally…

Started Using the Obsidian Tasks Plugin

2024-03-23T00:00:00Z

Durham NC

Jumping back into work after the holidays this year, I realized how cumbersome my notes and task management system had become. I was copying lists of TODO checkboxes across weekly notes. Each day, I moved anticipated work items to my daily note, only to pop the unfinished items back onto the weekly note. It was stupid, inefficient, and not how I usually handle task management. So, I finally tried the Obsidian Tasks plugin.

I started with my ‘work’ vault because I desperately needed an improved task management system there. The current arrangement wasn’t working for me anymore. I also had a completely different setup for personal tasks already in use. However, after a few weeks with the Tasks Plugin at my day job… I had to switch over my ‘personal’ vault too. At least for personal tasks and projects that is. Family tasks remain in ticktick, but I’m testing a family sub-vault for project notes too. Let’s take a deeper look into this system.

Plugin Setup & Settings

Tasks plugin settings. The global task filter is a must for my vaults.

The first step was to install the Obsidian Tasks plugin. Like any Obsidian community plugin, ensure “Restricted Mode” is disabled, and then search for and install the “Tasks” plugin. After enabling it, I realized there were 2 settings changes I needed to make for my system to work properly:

Set and use the “global task filter”. This option defines a string that is used to differentiate between a normal checkbox and an actual task. This was key, especially when converting an older vault that contains all sorts of historical opened/closed checkboxes littered throughout it. I used the tag #task for my filter.
I also turned on the option: “Next recurrence appears on the line below”. This toggles the behavior so that when a recurring task is completed, the new recurring item appears below the previous one, instead of the default setting of above it. This is just a personal preference for me, although I can see why the default might make more sense, particularly with longer lists of completed recurring tasks.

I made a few other tweaks, but those are the two big items. I saw there’s a way to include global settings to auto append to every task block, which I might do in the future as I nearly always include short mode, hide tags, and hide task count.

Vault Conversion

The Tasks section of my weekly note.

Templates

With the plugin configured, I started the process of converting my vault(s). My first step was to add task views to my current daily and weekly notes. Once I had what I initially wanted, I applied the formatting to the note templates. This was an iterative process, and I ended up going back and tweaking the templates over the following weeks.

After some time to soak, my template modifications slowed down, and I was left with the following (for my ‘personal’ vault):

Daily Note

>[!danger]- Overdue
>```tasks
>not done
>(scheduled before {{date:YYYY-MM-DD}}) OR (due before {{date:YYYY-MM-DD}})
>short mode
>hide tags
>hide task count
>group by filename
>sort by priority
>```

>[!todo]+
>```tasks
>not done
>(scheduled on {{date:YYYY-MM-DD}}) OR (due on {{date:YYYY-MM-DD}})
>short mode
>hide tags
>hide task count
>group by filename
>sort by priority
>```

>[!done]+
>```tasks
>(done on {{date:YYYY-MM-DD}}) OR (cancelled on {{date:YYYY-MM-DD}})
>short mode
>hide tags
>hide task count
>group by filename
>sort by priority
```

This is at the top of my daily note template, and lists tasks scheduled for the day in three callout boxes: Overdue, Todo, and Completed. Tasks in each box are also grouped by filename, which usually corresponds to the project the tasks are part of. As I complete tasks, they display in the different boxes accordingly.

Weekly Note

### Monday
- **Daily Log**: [[{{monday:YYYY-MM-DD}}]]
>[!tldr]+  Events & Tasks
>```tasks
>(scheduled on {{monday:YYYY-MM-DD}}) OR (due on {{monday:YYYY-MM-DD}}) OR (done on {{monday:YYYY-MM-DD}}) OR (cancelled on {{monday:YYYY-MM-DD}})
>short mode
>hide tags
>hide task count
>sort by priority
>sort by done
>```

After the switch to using the tasks plugin, my previous manual weekly note slimed down to just a bunch of task views 😅. I had to learn how to have a template set specific days for each day of the week, but after some research, I found out that you can use {{DAY:FORMAT}} in weekly templates, and the date for that day that week will be swapped in. I think this is handled by periodic notes, but I’m not positive. I was also able to use this trick to finally have each day’s link to its corresponding daily note be set automatically.

The weekly template mostly consists of a list of blocks for each day of the week. For each day, I have a link to the day’s daily note, and then a simplified, single callout box for the day’s tasks. This makes it easier to see tasks across several days at a glance. If I want more detail or tasks split apart, I can go to a specific day’s note.

Monthly/Quarterly Templates

```tasks
(scheduled in {{date:YYYY-MM}}) OR (due in {{date:YYYY-MM}}) OR (done in {{date:YYYY-MM}}) OR (cancelled in {{date:YYYY-MM}})
folder does not include Logs/Weekly Plans
folder does not include Logs/Daily Notes
short mode
group by filename
sort by scheduled
```

My monthly/quarterly notes each have a section that lists all the tasks for that time period, again grouped by filename. This allows me to quickly see everything that I have scheduled, or completed in a given month or quarter, which is awesome when I want to review at the end. Yes, it does get long, but that’s usually a good thing (at the end of the time period, only completed items remain).

The only real difference between the two is that the quarterly one uses {{date:YYYY-[Q]Q}} to match the quarters instead of months ({{date:YYYY-MM}}).

(Note: I don’t currently have these in callout boxes, as I thought their size wouldn’t work well for being in one. However, I’m second-guessing that now as I write this post 😉)

Task Conversion

My tasks live in the note they're related to. Like the note for this post, for example.

After setting up my periodic note templates with task views, my next step was to get my data into the system. This mostly involved finding all my current “task” items, converting them to proper obsidian tasks by adding the #task tag to them, and setting a scheduled date. Many of these checkbox items lived in my monthly or quarterly notes, so I often created new project notes for tasks to live in, if there wasn’t one already. Lastly, I added a tasks section to my project note template, which made this procedure a bit easier.

The whole process wasn’t too bad, as I only worried about converting current items, and a few from the last month or so. Moving forward, I’ve been creating tasks in the proper format, and when I bring up old items to work on, I convert them if they haven’t been for some reason. So far, it’s worked.

My Task Workflow

My workflow is much easier than it used to be. Now, I simply add tasks to project or item notes, and they appear when scheduled. If I want to know all tasks for a particular item, I just look at them in the note. I can also periodically spin up a view to see all tasks without scheduled dates, so nothing gets lost.

If needed, I can (and do) add small, one-off tasks in my daily, weekly, or “Reminders” notes, but I prefer having a specific note to log them in. Visiting the Library? Make a new “CITY library” note, and add a task section to it. Now you have a library ‘visit log’. The same applies to doctor’s appointments, restaurants, car maintenance, vet visits, etc. Adjusting my note structure and workflow for obsidian tasks has made my entire vault much more useful as a knowledge base for my life.

I mostly view tasks in my daily note, or directly in a project note as I work on it. As dates change, I either use the convenient button to push them to a future day, or use the edit window to make more complicated adjustments to the scheduling of the task. Very handy.

Conclusion

Adding event 'tasks' to notes about places has served as an easy way to maintain a visit log.

Like callouts, the tasks plugin has completely changed how I use obsidian. There are a few improvements I still want to figure out, like using two global task filters (#event in addition to my current #task), but overall everything works pretty much as I want it to. The tasks plugin has been the last piece I’ve needed to make Obsidian the perfect combined task+note system. I’ve been longing for this since moving away from Emacs org-mode several years ago. It’s quite the feat.

Reading my First Comic

2024-02-28T00:00:00Z

CRITICAL ROLE: THE MIGHTY NEIN ORIGINS--CALEB WIDOWGAST

While watching the announcements portion of a Critical Role episode one night, the cast highlighted an upcoming issue to their comic book series . Listening in, I thought to myself “I really should look into reading that series. I bet I would enjoy it.” So I decided to give it a shot.

Obtaining a Copy

After some quick research on where I could find Critical Role comic books, it turned out that they are available on Hoopla through the Durham Library. I love libraries. That night, I read some articles on how to read comic books, in order to better recognize subtleties in the art, how the panels are organized, gutter spacing, and other techniques that are often used. This article was very helpful in getting me going.

After brushing up on how to read a comic book, I read the intro/sample to the issue I intended to checkout: Critical Role: The Mighty Nein Origins - Caleb. The next morning (there’s a daily limit on electronic checkouts), I was able to checkout the full version electronically from my library.

Reading

At first, I paged through the issue, skimming and taking in the art. I didn’t have a large dedicated time in that moment to properly read it, but I was excited to give it a quick first pass. A week or two later, I sat down on a Sunday night while I was watching my son. He read his books next to me, and I cruised through the comic on my iPad. I was a lovely time.

Thoughts

Overall, I really enjoyed it. It was fun to go through the story and delightful to appreciate the art. Even more than ‘normal’ book reading, I can feel that comics are more likely to make me want to periodically go back and page through them, as there is so much visual complexity. The library issues are a great start, but for ones I really love and want to revisit, I might buy a few of my own copies. We’ll see.

Moving forward, I want to start occasionally reading comics, probably with continuing the Critical Role series. I think it could be a nice treat to sit down and read through an issue once or twice across a few nights. If you’ve never read a comic book before, I recommend it. See what your library has to offer 😉.

Obsidian Callouts

2024-02-15T00:00:00Z

Durham NC

Several months ago, I read about obsidian callouts in a post. At the time, I thought it was a neat feature, but didn’t know what I would use them for. “Why would I need an announcement box for in my personal notes?”

Recently while testing out the Obsidian Tasks Plugin, I realized I could use callout boxes to clean up and organize tasks in my notes. Since that realization, I use callouts for all of my task views, and a few other places too.

What are Callouts?

My example callouts, in Obsidian.

Callouts are a way to easily break up an obsidian note. They are colored boxes that contain content, and can be similar to a html details element. They are in the announcement notes you see after updating Obsidian (which is how I was familiar with them).

To create a callout in Obsidian, you just create a normal block quote, but have a callout tag for the first line:

>[!info]
>This is the formatting for a basic callout.
>Here is another line

The !info defines the type of callout (ex: todo, summary, error, bug…). There are many, and they determine the icon used, the color of the box, and the default title. I say default, because the title can be changed by adding it after the tag. For example, to define “I’m a Callout” as the title for a hint callout:

>[!hint] I'm a Callout
>This is the formatting for a callout.

Lastly, callouts can be made foldable by including a + or - immediately after the closing bracket. A foldable callout can be collapsed to just the title bar when clicking it. A - means that by default, the callout will be collapsed, while a + will default it to opened. An opened callout:

>[!example]+ I'm a Callout
>I am a foldable callout. By default, I am opened.

How I use Them

The 'Plans' section of my daily note, consisting of my task view callout boxes.

I use callouts for two purposes: 1) to create ‘dashboards’ of information in my notes and 2) to define a subsection of my note I can collapse.

The ‘dashboard’ use is mostly partnered with the Obsidian Tasks Plugin I started using recently. In my new setup, I define tasks under different projects notes. Next, I have task views that let me see all the scheduled tasks for a given day, week, or month in my periodic notes. These views are all defined inside callouts. This helps add some formality to task sections, and allows me to collapse the sections if it’s too overwhelming.

The less common, but increasing use for callouts, is when I want to section a large chunk of information into a collapsed box. For example, I might have a note with several long lists that I need to reference occasionally, but don’t want them visually taking up the whole note. Using callouts, I can define the lists under nice named tabs that only expand when needed.

Conclusion

Despite thinking I would never use callouts when I first learned of them, I can’t imagine obsidian without them now. They allow me to use the tasks plugin in a way which isn’t a complete mess. This has been a wonderful improvement to both my work and personal vaults.

2023 Review Picks

2024-01-23T00:00:00Z

Brightleaf Square, Durham NC

In 2022, I started keeping a yearly note in Obsidian. Partway through that year, I also began tracking my favorite ‘picks’ for different categories. For example, my favorite book or tech purchase. I liked it so much that when we rolled into 2023, I dedicated a section of the note to track picks from the beginning. Instead leaving the results sitting in my Obsidian vault, I figured I might as well share them here this year.

What are ‘My Picks of the Year’?

So what exactly are my “picks of the year”? Simply put, they are items I select that stand out for a particular category. That’s really the only rule. There are a few other guidelines I follow though:

Does the pick have to be from this year?

Not really. I tend to prefer picks that relate to the year in some way, but that doesn’t mean they have to be from that year. I might pick a song from 10 years ago because I heard it for the first time this year, or because it is meaningful now.
Does the pick have to be my favorite for its category?

Nope. It might be the most influential, even if it wasn’t my favorite.
Categories are fluid.

When I started making picks, I made each category as I thought of items I wanted to rank. Now that I’ve done this for more than a year… the categories change. There are some categories that I simply didn’t have strong nominations for (i.e. I didn’t really watch any notable movies). I also allow myself to add new ones.
This is just for fun.

At the end of the day, this is just a silly thing I started tracking in my yearly note.

Why do yearly picks?

I think I’ve enjoyed doing a yearly pick for 3 main reasons:

It is fun to look back at what I picked for different categories each year. I imagine if I continue this habit, I’ll enjoy looking back at my picks during various points of my life.
Having picks in the back of my mind brings a new level of awareness around what I consume. It forces me to really think about how I feel about different experiences.
Selecting final picks prompts me to reflect on why something was meaningful/important to me that year.

With all the background information covered, let’s dig into my 2023 picks!

2023 Picks

The Apple Studio Display and my floor desk were some of my favorite purchases in 2023.

Tech Purchase: Apple Studio Display

Honorable Mention: 16" M3 Pro MacBook Pro

This category was hard this year. Looking back, I had an arguably obnoxious amount of great tech purchases, each which could win in less crowded years. Most of the year, I thought it would be the Apple Studio Display, but then I squeezed in my 16" M3 MacBook Pro right at the end of the year, making it a tough decision.

In the end, I still choose the Studio Display. For my uses, it’s a great monitor and does everything I need it to. Having a large Retina display has been amazing, and its speakers are good enough that I’ve really been able to simplify my desk this year, allowing it to be more flexible. Despite the cost, I’ve been very happy with this purchase. Also… +1 to the Apple Refurbished store 😉!

Non-Tech Purchase: Bamboo Floor Desk

Honorable Mentions:

Vivobarefoot Primus Lite III
p a c t shorts (The Backyard Weekend Short - Storm)

This category has a weird mix of all sorts of items. I switched to using barefoot shoes this year, and my Vivo’s have been a solid all around pair for that. Additionally, I’ve experimented with using more natural fibre clothes this year, and my linen+organic cotton p a c t shorts were very comfortable this summer.

For my pick though, I think I’m going with my Bamboo Floor Desk. I use it all the time, all over the house, and can slide it under the couch if I want to store it away.

‘House’ Purchase: Schlage Encode Plus Smart Lock

Honorable Mention: Birdies Tall 8-in-1 Metal Raised Garden Bed - Mist Green

This is a category I added last minute, so I’m sure there are items that could be here I forgot.

The main reason I added this was to include our door lock as a pick. I knew it would be great to easily unlock the door when we got home, but I didn’t anticipate how nice it would be while leaving the house too. I just push the lock button and go. No more fumbling with keys. We’ve wanted to make this upgrade for quite some time, but the locks models we wanted were always out of stock. My wife and I are both very happy with this purchase.

Music Album: One More Time - Blink-182

Honorable Mention: The Land is Inhospitable and So Are We - Mitski](https://www.youtube.com/watch?v=5WW7cuAm-7Y&list=PLB0BgxYHor0sb2T8992DeaKVaknn-771p)

I was a little skeptical when I saw the new Blink album released… until I listened to it. I love it. I’m sure there are many technical reasons to criticize this album, but I don’t care. ‘One More Time’ is a meaningful album, and I’m not the only one that thinks that.

Song: I’m too small for this world - Mandelbro

Honorable Mentions:

Earlier this year I had a Max Reisinger video playing in the background while working. There’s a section of the video that uses I’m too small for this world by Mandlebro, and by the time the song’s second verse hit, I was already hooked. I continued to put it on several times a week throughout the whole year. So good.

One More Time from the Blink album was also a powerful song that I wanted to include here. Lastly, as a parent… Cat Cat Frog Frog got a bunch of playtime this year… although my son never asked for it. My wife and I both legitimately enjoy it and listen to it all the time 😆.

Game: Melvor Idle

Honorable Mentions: Baldur’s Gate 3

As someone who adores Old School Runescape, but doesn’t have enough time to truly feed that addiction… Melvor Idle is perfect. It’s based on OSRS, but an ‘idle’ game. Most of my enjoyment in OSRS is planning out when and how to level up everything, and make money doing it. That’s basically what Melvor Idle is at its core. You select at task, log off, and check in a few hours to plan and switch to something else. It’s wonderful.

I’ve had Baldur’s Gate 3 for years as a pre-release, but didn’t start to play the released version until the last few days of the year. I’m hoping to play it more in 2024.

Book: Born to Run by Christopher McDougal (Audio)

I didn’t read/listen to too many books this year, but one I did was Born to Run. I heard of it in the past, but with me switching to barefoot shoes, and starting to go on light jogs, I thought it was a good time to give it a read. I listened to the audiobook (thank you Durham Library!) and found it very engaging. It had a bunch of ‘non-fiction’ type content, packed into an entertaining story to drive everything. A perfect combination.

Exercise/Activity: ”Jogging”

Honorable Mentions: Pilates

As mentioned above, this year I occasionally went for what some would call a ‘light jog’. It sure as hell wasn’t running, but it was more than a walk. I largely blame my Earthrunners, as their springy lightness made me want to keep going after jogging through an intersection one day. A few times during the year, when I had the time and someone else watching my son, I would go out and do a few miles at different paces as needed. Usually in the rain. (I love the rain).

I also started doing more pilates classes on Apple Fitness+ and like it much more than I thought I would. I’m not sure why I’m surprised, as it combines all my favorite exercise focuses: strength, core work, flexibility, and balance.

Written Media/Website (Added): Kevin Quirk’s Blog

Honorable Mention: Low-Tech Magazine

This was a new category I added this year. I redid my rss and read-later setup in 2023, and focused on consuming more ‘small web’ content. I followed more personal blogs this year as motivation for my website redo. One blog that kept popping up everywhere was one I already had in my feed: Kev Quirk’s website. Reading a bunch of his posts (and others’ he linked to) helped push me to finally do my redesign. Moreover, I used the simplecss theme he created as a starting point for my website.

Low-Tech Magazine has influenced me to rethink how I live, and how to adapt more sustainable practices. It also influenced my website redesign, after reading about their redesign.

Hobby: Small Web/Blogging

Honorable Mention: Gardening/Learning permaculture & Ecology

As mentioned above, the largest personal project I accomplished this year was my website redo. This was prompted by the internet becoming shittier, and my desire to find better spaces on it. This lead me to the rings of the ‘small web’, personal blogging, and sustainable web design. I redid my website and want to continue exploring this space in the future.

I also started gardening for the first time this year. Just a simple raised bed, but I also read about permaculture (and ecology) this year, and hope to grow the hobby over then next few years.

Software/App: Typora

Lastly, I ended up selecting Typora for my software app this year. With the website redo, I can more easily use normal markdown editors for my posts. After trying a few, Typora has become my favorite for writing some of my drafts (including this one). Additionally, it has been helpful at work. When I want to export a markdown note I wrote in obsidian, but want it formatted nicer for an email, Typora’s formatting stays when I copy and paste to Gmail. This is a game changer.

Conclusion

I gardened for the first time in 2023 (with help from my wife)

Those are my 2023 picks. Not terribly exciting, but they do create a snapshot of my 2023, which I think is interesting. I’ve already setup my 2024 note, and have a few items to add as potential picks already. We’ll see how they fair as the entire year continues to fall into place.

Traded my 14" M1 Pro MBP for a 16" M3 Pro MBP

2023-12-19T00:00:00Z

Stacked 14" and 16" MacBook Pros

Welp, I’ve done it again. I traded in my 14" M1 Pro Macbook Pro for a 16" M3 Pro MacBook Pro, which I received a few weeks ago. Proving I could keep a Mac for over a year (2 years and 1 day! 😆), I allowed myself to do a custom build. But more on that later.

Why Change from the 14" M1 Pro?

16" M3 Pro MBP on left and 14" M1 Pro MBP on right.

Overall, the 14" M1 Pro has served me very well during the last few years, and largely still does. The CPU and general performance are wonderful and I would still recommend this model to those that want to find a nice discounted MacBook. My upgrade to the M3 Pro has definitely been more of a want than need. With that said, there were a few changes I wanted for my next computer.

First, the 512GB SSD was a little tight and annoying to always be cleaning up. Similarly, while the 16GB RAM was doable, the ‘Memory Pressure’ would climb when I would be working on several things, so I wanted a bit more head room there as well.

Mainly, I wanted a bigger display. I’ve learned that I appreciate having ample screen space when using my computer away from my desk, and I do that more often now. The 14" is good compromise for most, but I still felt like I was using a laptop with it solo on a table.

I also realized that my laptop doesn’t need to be crazy portable anymore. After getting the iPad Magic Keyboard for my 11" M1 iPad Pro, I’ve been able to grab that whenever I wanted a truly small and portable device. If there is something the iPad can’t manage, I typically want a larger screen for that task anyway. The magic keyboard was a game changer, not only for my iPad, but also for shifting my computer’s requirements.

After purchasing the magic keyboard, I figured my next personal computer upgrade would likely be a switch to the 16" MBP. Beyond fitting my work use case, a big reason I decided on the 16" at work was so that I could compare having it with the 14" I had at home.

What About the 15" Air?

When the 15" M2 Air was released, I thought “Oh wow that looks nice. I wish I could make that could work for my needs”… which led me to evaluate: could it?

A larger screen was my main motivation for wanting to switch things up from the 14".
I loved that the 15" Air had the larger screen, but maintained the weight and portability of my 14". The best of both worlds.
I wasn’t hurting for more performance. An M3 or even the M2 wouldn’t be too far off the performance of my binned 8-core M1 Pro.
The significantly lower price point compared to a 16" Pro meant it was reasonable to max out the RAM and SSD (which is something I did want), and still be below the price of a 16".

For the next few months, I was convinced that my next device would be a 15" Air. I just wanted to wait for M3 chips to be announced, and see if it was worth getting that or a discounted M2.

Deciding on the 16" Pro (again)

The 16" MBP is a solid all in one computer.

When the full M3 MacBook Pro line was announced, I started thinking about what I actually wanted. With the M3 chip also released, I had a rough idea of what to expect from an M3 15" air, likely available in a few months. I started comparing the two, and kept coming back to the 16" Pro being a better fit for all my needs and wants:

The 16 has amazing speakers, which I do very much enjoy and use often.
The screen is better. I didn’t care before, but I like having an HDR display for photos.
Support for 2 external displays.
Better IO - an extra TB port, HDMI, and SD can all be useful to have when tinkering.
Some more performance overhead
More RAM for the upgrade (36GB vs max of 24GB) … for a cost of course 😉.
I don’t actually need this computer to be the most portable. If anything, the 15 Air’s size would be a benefit for my work laptop.

Why not the M3 Max then?

If was looking to upgrade the M3 Pro to 36GB RAM and 1TB of storage, why not just consider the base model (14 Core CPU/30 Core GPU) M3 Max configuration? It starts with those specs as a “base model”. I did. Especially when I looked at the difference in shipping times 😆.

But it was still a $400 upgrade (slightly less after using a discount). For that amount, I would get 2 more supported external monitors, better CPU and GPU performance, double the memory bandwidth, a second encoder engine, and I’m sure a few other things I’m missing. Oh, and an extra 0.1 lbs in weight.

But the thing is… I don’t need most of that. I’ve already dedicated a whole section of this post talking about how performance-wise, I would be completely fine with a 15" Air. It was mostly features that are not related to the performance of the SOC that swung me back to the Pro. So, the Max isn’t worth it in my case. Maybe if I used a ton of monitors at my desk, or actually played games, things would be different.

Besides, while it was criticized in all the reviews, the M3 Pro chip has a 6 efficiency/6 performance core setup that I am very interested by. As someone that watches CPU monitors way too much and is obsessed with energy efficiency, I know those tiny efficiency cores get shit done for very little. I’m excited to have a system with 6 of them.

Why not a M1 or M2 Refurb/Discount?

The shipped 16" M3 Pro MBP.

This is something I did consider. The main reason I didn’t get an older model is that I wanted to go through Apple and trade in my old base model 14 M1 Pro (8 core). I prefer using their trade-in system over the hassle of selling devices myself, and the $905 they offered wasn’t bad. That ruled out options like the B&H, BestBuy, and Amazon Black Friday sales. Additionally, the Apple refurbished store didn’t have a configuration I wanted at the time.

…Not Space Black?

If I went with the newer M3 Pro model, why didn’t I get its ‘biggest feature’… the space black finish? From what I can tell (I haven’t seen it in person), the new finish looks lovely and I was tempted. That being said, I really hate the Space Grey of my work laptop and didn’t want to take the risk with my personal laptop.

On top of that, I am really fond of silver. It’s simple, easier to keep clean, and looks really gorgeous on this generation of MacBook Pro, with the thin black bezel and black keyboard well.

What I will miss from the 14"

I will miss how the 14" M1 Pro felt to simply pick up and walk around with while working on projects.

With my 16" M3 Pro MacBook Pro in hand for a few weeks, and my old 14" M1 Pro traded in, there are a few things I imagine I will miss about the 14:

The Feel: I’ll miss how the 14" feels when held. Walking over to a table, picking it up with one hand and walking away just feels so good. The 16 's heft does take away from that a bit.
Having my work and personal computers differ: I liked that I had the 14 for personal use and 16 for work. Switching back and forth between them meant that I was always able to appreciate their strengths by comparison. The 14" always felt so easy to grab and go, while still having just enough screen space for most tasks. Using it always made my 16" feel more expansive. Without the constant contrast of the 14", I’m afraid I’ll eventually get used to the 16" and it will feel standard or even small to me. I’ll also miss the cascading look of the 14 and 16 in my double laptop stand.

I’m sure there will be more that I miss over time as nostalgia snowballs, but initially, these are what come to mind.

Conclusion

My 16" M3 Pro MBP on my desk, starting this post.

So far using the M3 Pro 16" MacBook Pro has been superb. Its size fits perfectly for how I use my personal computer at this point in my life. On top of that, the additional RAM and storage has made a huge quality of life improvement too. I haven’t traveled with it yet, but with the Holidays approaching I’m excited to see how that goes.

Website Redo 2023

2023-12-04T00:00:00Z

Oregon Inlet, Outer Banks, NC

If you frequently visit this website (do those people exist?), you might have noticed things suddenly look a bit… different. After a long period long period of being eager to overhaul of my website, I finally did. Here’s the why and how.

Background

Ecograde report for a post on the old website

I have enjoyed my website’s theme and layout since I switched to hugo, over six years ago. However, there are a few areas that I’ve wanted to improve, such as trimming down the website’s pointless bloat. This past year, I dug more into sustainability topics, including reading about sustainable web design, ecograder, and how low-tech magazine rebuilt their website. Expectedly, this site did not score well on ecograder or gtmatrix, and I had to change that.

In addition, I have become increasingly concerned with how terrible the web is becoming. This has led me to retreat a bit and rely more on rss. I want to start consuming more of the web directly from curated personal blogs, rather than bloated websites and horrible AI-generated articles created solely to improve SEO. As I have found and added websites to my feed list, I have been inspired by the simplicity of many of these sites. Browse around the 512kb club list to see what I mean. People are focused on the content of the site, and not the ads. It is clear that many of these blogs serve as the author’s “home” on the web.

Goals

With those two areas of motivation, I wanted to finally redo my personal site. Before starting, I brain-dumped a list of goals for the project to help guide its design:

1. Lower environmental/carbon footprint

I wanted my site to be better optimized and not a resource hog. Websites from The 512kb Club are a good example, and a potential goal for this one.

2. Smaller and easier image system

I wanted to use either dithered or compressed images throughout the site, at least for initial page loads (but maybe still link to the higher quality originals).

I also wanted it to be easy to mange. I would love to use normal markdown formatting to add images instead of hard coded html, or Hugo short codes. Compatibility with ‘normal’ markdown editors and previews would be another bonus.

3. No javascript

I don’t think I have a need for JS on my site. So why bother with it?

4. New theme to support the above items

A Minimal theme that focuses on content. Simplify colors, and fonts.

5. Remove/Replace comment system

(I had already done this. I couldn’t wait) Remove for now. Maybe add a static system in the future.

6. Switching from a strictly tech to a personal blog

I want this to be my home on the internet too. As part of that, I wanted the possibly to write about interests beyond tech. Maybe gardening, life experiments, non-tech reviews, thoughts, etc. (see rss items below)

7. Improve rss feeds

I want to have rss feeds setup per category, so people can subscribe to individual topics (ex: tech only) if they wish.

I also need to maintain support for rss only posts.

8. Easy to write/maintain

I’d love to be able to draft pages in something like obsidian or other editors without having to worry about the hugo server running during writing. Being able to draft or edit on my phone and/or iPad would also be great.

9. Change ‘Homelab’ to a ‘Uses’ Pages

This is basically what I want, and it seems to be the standard amongst personal tech blogs. My ‘Homelab’ page was a cluttered mess and had way too much stuff on it. I also don’t really maintain a “homelab” anymore.

10. Fix the CI/CD/Testing/Deployment pipeline process

I want to still run tests, but I want them to work without being too picky. The setup should be easy to do on any of my machines or pipelines. (CI needs to work with redo. Tests can come later)

The Redo Process

My portable setup while working on the redo on vacation.

I began the redesign at the beginning of October (while on Vacation). I decided to start with the theme, so I created a new hugo theme from scratch, and used simple.css for the starting style sheet. I continued to tweak the style and layout a bit to have the site fit what I wanted.

After some initial tweaks, I worked on adding a render image hook. This would allow me to use standard markdown formatting (ex:![alt text](image_src_path "caption")) in my content files, and hugo would know how to convert it to a particular html formatting when rendering the site (including doing the image scaling and adding the caption). Eventually, with a bunch of trial and error and help from blog posts, I hit a point where the theme was good enough and I had the image hooks working.

Next, I moved onto converting all the content files to use the new hook. During this step, I also converted all the pages into leaf bundles. This took FOREVER. In hindsight, I should have attempted to script some of this process.

Once everything was finally converted, I worked on some remaining miscellaneous tasks, including redoing the rss feeds which involved adding new category-specific ones. I also cleaned up small issues with the theme, and added a Hugo partial to compress and convert images, which I applied to the header images in the layout code.

Lastly, I updated the site content pages, and prepared to push all the changes.

Deployment Woes

With everything ready, I merged the massive working branch into my website’s main branch to deploy… and hit a string of issues.

First, there were some build errors related to the image hooks. After investigating, I think it’s because they utilize a new feature not included in the older version of Hugo my Gitlab runners had. I tried resolving this by swapping out the build container to different images with newer hugo versions, but it didn’t help. I eventually settled on configuring my Mac mini server to be a gitlab runner I could use until supported versions are in the CI images.

The builds worked using the Mac mini runner… but during the deployment step I hit another issue:

ERROR: Uploading artifacts as "archive" to coordinator... 413 Payload Too Large id=5531626753 responseStatus=413 Payload Too Large status=413 token=64_ptorz

FATAL: too large

It turns out the limit for a Gitlab Pages build is 1 GB. The redesign had grown my site (mostly due to generating several compressed image files per image) to… just over 1GB. I could have worked to minimize the size, but that would only be kicking the can down the road. So, I decided to instead switch where I host the site.

I learned how to configure an ssl cert for my fedora Linode, and pushed the build to the nginx server running on it. This was actually the test server I used during the rebuilding process, so it was already setup to host the site.

Finally… it was deployed.

One final_final step I decided to take was moving the source into a fresh git repo and pushed to a new gitlab project. I have the old one archived for its history, but wanted to start clean with this redo. I took some time to update the CI scripting for the new deployment process, but after that, I was actually done.

Conclusion

The new website theme, in both light and dark modes.

While I was ‘done’ with the process, I still had to write this post 😆. To be honest, I’ve pushed it off for a few weeks now. This redo was such a massive endeavor that this post only scratches the surface of everything I did. That said, I am happy with the result. There are still tweaks I want to make, but the site now accomplishes all the goals I defined when I set out on this project.

The website is much leaner and scores significantly better in performance metrics. At the same time, it is easier for me to actually write the posts. In fact, I wrote much of this in Typora, and even some of the draft from my iPad (and not just ssh’d to my computer)! I’m glad this work is finally done, and I’m excited to use the new site now.

Quick Jira Card Links in Obsidian

2023-09-25T00:00:00Z

Duke Park, Durham NC

I record notes in obsidian all day. I keep daily logs to plan out my days, and sometimes write separate meetings notes. In both cases, I usually refer to Jira cards. Recently, I grew tired of writing out the obsidian and url links every time I wanted to reference a Jira card. So I found a better way.

Background

In my daily log, I have a section where I timeblock the day. I generally try to ensure what I am working on can be traced back to a Jira card. While working on a card, I also like to have a corresponding note in obsidian where I can keep a running log of my work and thoughts. When I reference a card in the daily planner, I link to the jira note for that card (even if it is just to get the benefit of auto complete 😆). For example, I might have:

*Morning*
- 08:30 Email and MR Review #general
- 09:30 [[Team Meeting]] #meeting
- 10:00 [[JIRA-9999]] #qe_dev
- 11:00 [[Project Meeting]] #meeting
- 12:00 Lunch #lunch

Here, I link to common meetings, but more importantly, a Jira card note (JIRA-9999) to track when I am working on that card (or at least plan to).

I make obsidian links for any jira card referenced in a meeting, even if I don’t create a note for the card. That way, if I ever do instantiate an obsidian note for the Jira card, obsidian will already have all the back links pointing at it. For example, during backlog refinement meetings, I write links for each card discussed. If someone has a question about the card, or I pick it up later to work on, I already have the note linked to every time I’ve referred to the card in my notes.

Over time, I learned that it’s nice to not only have the link for the obsidian note, but also a quick reference to the url for the Jira card (ex: [[JIRA-9999]]([link](https://jiraserver.com/JIRA-9999)), viewed as: Jira-9999 (url)) in the note).

The Quick-Add Plugin

After starting to include jira urls more often, I wanted to template the process. I knew it shouldn’t be too hard, as the only unique characteristic in both types of links was the Jira card number. This is especially true when considering that 90% of the cards I reference are from the same project.

I first tried the built-in template system in Obsidian. However, I wanted a pop-up to prompt me for a card number that I could quickly type in. After browsing around the community plugins, I came across the QuickAdd plugin and installed it.

Setting up Jira Links

Quick add settings to insert link at cursor of current file.

I found the QuickAdd plugin a little confusing to understand at first. It defines several types of shortcuts, which weren’t exactly clear to me how they worked. Regardless, for my needs, I settled on the “capture” type after playing around with the different options. I was able to use the following format to create a quick link for my team project jira cards:

[[PROJECT-]]([Jira link](https://JIRA-INSTANCE.com/browse/PROJECT-))

I also made a more generic one to add cards/links from other teams when I need to. It’s the same format except that I provide the full card id (including project stub) in the pop-up prompt:

[[]]([Jira link](https://JIRA-INSTANCE.com/browse/))

I also setup a shortcut in the Obsidian settings to launch quick-add. So now, I just hit CMD-CTRL-J and a pop-up window opens where I either select the project or generic template. Then, a second pop-up opens, where I can enter either the card ID for the generic template, or just the number for my project’s specific one.

# Ex: Entering "9999" in the project prompt will insert this into obsidian:

[[PROJECT-9999]]([Jira link](https://JIRA-INSTANCE.com/browse/PROJECT-9999))

After hitting enter, my fully formatted obsidian + jira url markdown link is inserted at my cursor. Wonderful!

Conclusion

So far, this setup has been great. It’s allowed me to quickly enter proper links to jira cards while taking notes. I can add full links to the notes in real time, and not need to go back and clean things up. Also, having the url whenever I refer to jira cards in obsidian has been super handy when I’m reading through my notes and want to double check something in the real card. Another simple, but very effective solution!

Setting Up RssOnly Posts in Hugo

2023-08-27T00:00:00Z

Downtown Durham NC

Awhile ago, I wanted to write posts that would show up in the website’s rss feed, but not be linked to anywhere else on the site. Unfortunately, this isn’t an option you can simply toggle on in hugo (that I could find). So, I implemented it myself in my theme layout.

Marking a post as rss only

The first thing I needed was a way to mark which pages should be published only in rss. For that, I created a marker which can be added in the header meta data of those posts:

Rssonly = "True"

I don’t think I have to define it anywhere. If it’s in the header of posts, it can be used in the templates.

Adding Home Page Filters

With a way to mark posts as rss-only, I could start adding filters so that those posts wouldn’t show elsewhere on the site. The first place was to remove them from the listing on the homepage, so I went to it’s layout file: /layouts/index.html.

There, I took the following line of code, which grabs pages to list on the home page:

{{ $section := where .Site.RegularPages "Section" "in" $mainSections }}

and replaced it with this one:

{{ $section := where (where .Site.RegularPages "Section" "in" $mainSections) "Params.rssonly" "!=" true}}

The change omits posts that have the rssonly metadata set to true.

Adding Archive Filters

Next, I wanted to remove the rss-only page from appearing on the posts archive page (or any other post lists), so I went to my ‘list’ layout at /layouts/_default/list.html.

This time, I modified the .Paginator.Pages to filter out rss-only posts:

{{ range .Paginator.Pages }}

modified to:

{{ range where .Paginator.Pages ".Params.Rssonly" "!=" true }}

Including a message at the top of `Rssonly` posts

Lastly (so I thought), I wanted to add a special case for the layout to automatically add a message at the top of rss-only pages, to declare they were rss-only. So, I added the following snippet in my /layouts/post/single.html file, right above the {{ .Content }} line:

{{ if .Params.rssonly }}
    RSSONLY MESSAGE                    
{{ end }}

At that point, I thought I was done, so I started writing this post to document and share my work.

Next/Prev post fix

However, as I was writing, I noticed an issue. Posts that were before or after an rss-only post, linked to the “rss-only” post in the next/previous links at the bottom of the page.

I looked into a fix quickly, but wasn’t able to figure out a quick solution at the time. On top of that, the more I looked into solutions, the more I second guessed my method altogether. I probably should have learned how to make a new type of page in hugo, which didn’t show in any of these lists. But that was hard to figure out. Instead, I decided to put the fix and post draft on my backlog for a month or two.

I eventually picked up the problem again and managed to figure out a solution for my prev/next page code (found in /layouts/post/single.html):

<!-- Next Post/ Previous Post Links -->
{{ $pages := where (where .Site.RegularPages "Section" .Section) ".Params.Rssonly" "!=" true }}
{{ if $pages.Next . }}
    <div style="float: right; text-decoration: underline; text-align: right;">Next Post:</div>
{{ end }}
{{ if $pages.Prev . }}
     <div style="float: left; text-decoration: underline; text-align: right;">Prev Post:</div>
{{ end }}
<br>
{{ with $pages.Next . }}
    <a href="{{ .Permalink }}" style="text-align: right; float: right; max-width: 40%;">{{ .Title }}</a>
{{ end }}
{{ with $pages.Prev . }}
    <a href="{{ .Permalink }}" style="text-align: left; float: left; max-width: 40%">{{ .Title }}</a>
{{ end }}

Here, I to added the following line to make a newly filtered list of possible pages:

{{ $pages := where (where .Site.RegularPages "Section" .Section) ".Params.Rssonly" "!=" true }}

Then, I replaced the .PrevInSection and .NextInSection lines using that $pages variable: $pages.Prev and $pages.Next, which seem to work exactaly how I want it.

With that fix, I started to work on this post again 😅.

Conclusion

As far as I am aware, I think that’s it. As I stated above, I’m sure there are many better solutions to enable rss-only posts in hugo. But this worked for me. If you want to do something similar… maybe do a quick search to make sure nothing obvious comes up first. Now I need to figure out how to add some tests to check that this continues to work…

Nearly headless VMs Using utmctl

2023-07-29T00:00:00Z

Durham NC

I’ve been using UTM since I switched to Apple silicon macs, but I recently discovered it’s cli tool, utmctl. It’s completely changed how I mange UTM and VMs on my macs.

How to install

utmctl is included when you install UTM (via the AppStore or package download available on their website). Once installed, you should have the utmctl command in a terminal:

iryan@venusaur ~ $ utmctl
OVERVIEW: CLI tool for controlling UTM virtual machines.

USAGE: utmctl 

OPTIONS:
  -h, --help              Show help information.

SUBCOMMANDS:
  list                    Enumerate all registered virtual machines.
  status                  Query the status of a virtual machine.
  start                   Start a virtual machine or resume a suspended virtual machine.
  suspend                 Suspend running a virtual machine to memory.
  stop                    Shuts down a running virtual machine.
  attach                  Redirect the serial input/output to this terminal.
  file                    Guest agent file operations.
  exec                    Execute an application on the guest.
  ip-address              List all IP addresses associated with network interfaces on the guest.
  clone                   Clone an existing virtual machine.

  See 'utmctl help ' for detailed help.

Commands

The utmctl options are all rather straightforward. I most often use the commands utmctl list, utmctl start VMNAME, and utmctl ip-address VMNAME. However, some of the other options, such as exec and attach, look interesting.

A quick example - Running (nearly) ‘headless’ VMs

The display device in the UTM settings. Right click delete.

The main improvement that utmctl has provided me is the ability to run my VMs “headless” (mostly), which I am accustomed to doing on Linux. To do this:

Install and configure a VM in UTM, and ensure that ssh is enabled so it will automatically start on boot. Also, verify that you can properly ssh into the VM.

# Enable ssh to start. Likely this command on most linux systems:
sudo systemctl enable sshd

Remove the virtual display in the VM’s settings. This will prevent the VM window from opening while it is running, as there is no GUI for the VM to run.
Use utmctl to list or check the status of the VM(s).

ryan@venusaur ~ $ utmctl list
UUID                                 Status   Name
D77A861B-61DD-483A-BA09-D647C77EB77A stopped  nixos
EADB0899-8B20-48B9-BD23-B179E3C258B3 stopped  fedora

-- or --

ryan@venusaur ~ $ utmctl status fedora
stopped

Use utmctl to start the VM:

ryan@venusaur ~ $ utmctl start fedora

ryan@venusaur ~ $ utmctl status fedora
started

Use utmctl to get the ip address from the running VM (it might take a minute until the VM is fully started to work):

ryan@venusaur ~ $ utmctl ip-address fedora
192.168.64.33
fda6:d2b:ac2e:c75a:f87:fc14:d8fc:ef8d
fe80::6b82:d89b:3f27:9c3d

ssh into the VM using the ip-address:

ryan@venusaur ~ $ ssh ryan@192.168.64.33
Last login: Fri Jul 28 05:43:17 2023
[ryan@fedora ~]$ # I'm in the VM!

Although I usually shutdown the VM from inside it (which is recommended), you can alternatively shut it down using utmctl:

ryan@venusaur ~ $ utmctl status fedora
started
ryan@venusaur ~ $ utmctl stop fedora
ryan@venusaur ~ $ utmctl status fedora
stopped

After moving the display device, running these steps may still leave the UTM application open in the Dock, but the VM window should not pop up. It’s why I consider this to be a “mostly headless” process.

Conclusion

I love a simple but useful cli tool, and utmctl has been just that. I am glad that the team added it, and that it continues to get updates. For me, it immensely improves my experience using UTM. In fact, while writing this post I finally went and bought the App Store version of UTM to help support the developers. If you also love UTM, I encourage you to do the same.

Trying Choc Sunset Switches

2023-07-01T00:00:00Z

Downtown Durham NC

A couple weeks ago, I finally ordered a set of choc sunset switches for my wireless ferris sweep. After using Kailh choc silver switches daily for over a year, I wanted to try a “good” choc tactile switch. Here are some of my notes after several weeks of use.

History

Before the ferris sweeps, my two previous keyboards (HHKB Pro and ergodox ez) each had tactile switches. Cherry MX Brown on the Ergodox and Topre (I know, but it’s close enough) on the HHKB. Before building my first Ferris Sweep Keyboard, I bought a sample pack of choc switches, and the tactile options were all disappointing. They just felt like scratchy linear switches.

At the time, a new ‘sunset’ switch was being developed and had the community hopeful for a better option, but it wasn’t quite ready yet. So I instead decided to see how lighter, linear switches feel on a low profile board, and picked choc silvers. I enjoyed the silvers enough that when I built my second ferris sweep, I ordered them again to keep the experience between the two boards similar.

Ordering the Sunsets

The package of 40 delivered choc sunset switches.

After failing to get the batteries on my wireless ferris sweep to work, I was looking around at various pre-built boards on the market (like the Corne-ish Zen, for example). While searching, realized I wasn’t sure what switches I’d want to get if I ever got a new choc board (I haven’t yet).

I love the silvers, but the now-available sunsets looked interesting. My wireless ferris sweep has hot-swap sockets, so I decided to finally splurge and get a set of choc sunsets for my board. I could try them out, and have a spare set of choc switches on hand.

Setup & Initial Impressions

My wireless ferris sweep board with choc sunsets installed.

I recorded my initial thoughts when first using the sunsets that I’ll include below. When they arrived, I pulled one out of the bag to hold:

“feels nice, slight bump, seemingly smooth…”

Next, I placed the switch in the tester (with a key cap), adjacent to my normal choc silver, to compare the two. My initial reaction surprised me:

“… oh no!.. I’m not sure if like that… it feels dirty or something, and not as smooth (duh)…”

Finally, I swapped out the silvers for the sunsets on my board, and tested typing on the full keyboard:

“Better I guess… but man, it is different. The bump is noticeable, but it’s not crazy when typing and I’m not overly thinking about it. I feel like I’m ‘popping’ the keys which I normally enjoy, (like on my 14” Macbook Pro), but idk… I don’t love it as much as I thought I would (at least initially). I can see myself enjoying it more when I am used to it though. Right now, it just feels heavier and weird from what I currently use."

“It does feel deeper or something too, more solid. That might be because the travel feels heavier, or maybe the bump gives it the illusion of being a bigger switch or something?.. I don’t know, but it is very different.”

I was surprised to learn just how much I’ve learned to love the silvers (considering how much I hated linears before using choc switches).

“My biggest take-away after using the sunsets for a few minutes though is ‘huh, I think I actually like (at least on low pro/choc switches), linear switches rather than just tolerating them…’. I think this is because I bottom out regardless, and the silvers bottom out in a way which feels good to me. However, they are a nice med-light heaviness that I can type on effortlessly and not bottom out if I want to.”

After using linears for so long, I had some trouble initially getting used to the bump:

“I can’t type as light, at least not yet on these (sunsets). Maybe when I learn where the bump is eventually it will be better. I think after the bump I just shoot to the bottom.”

“Also, I’m having trouble actuating some keys, especially things like the OSM shift and other mods. My brain might feel the bump and thinks it’s bottomed out or something on holds, idk. Going to keep playing with it and am interested to see how going back to the silvers feels…”

Despite my initial reactions, after using the sunsets a little more, specifically for some typing tasks, things got better:

I’m warming up to them a bit more, which is nice to see.

They feel fun, with the poppy-ness

They are more fatiguing. My forearm feels a bit tired and I’m not sure if it’s related or not but I think it might be 😆

Switching back and forth between this and the choc silvers is fine, but they feel very light after.

I like it’s lower sound. Some of it is the case, but even so it feels less tick-tick-tick-y…

After Some time with them

The choc silver (left) and sunset (right) switches.

After a few days using them at work, it got even better:

Much nicer today. I used them a bunch at work and tonight while writing. They’re closer to what the HHKB Pro or my 14" Macbook Pro have, with the ‘pop’. I do like it, but I also like the linears still. I think it might be a 'whatever mood I’m in will determine what I prefer at a time.

So now that I’ve used both choc silvers and sunsets on most days for several months, what do I think? Well, I like the sunsets for their ‘pop’ and different feel when typing. It feels more like using a larger mechanical keyboard to me, which is a nice change. I also favor their lower sound compared to the silvers.

On these very small, low profile split keyboards though, I think I prefer the silvers. They just pair nicely with how I use this type of board and layout. My fingers feel like they’re dancing across the keys as I tap and hold home row modifiers, and switch between layers.

Conclusion

With that said, I still love the sunsets. They are a much better tactile choc switch than any of the others I’ve tried. While using choc keyboards however, I have learned that I prefer linear switches, and so would likely choose silvers instead. I think the sunsets are a great switch and I highly recommend them to anyone that wants a tactile choc switch. I’m glad I got mine and will continue to enjoy the extra ‘pop’ in my board.

Enable Syntax Highlighting for .keymap Files in Neovim

2023-06-06T00:00:00Z

PNC Arena, Raleigh NC

I’ve made a few minor tweaks to my keyboard layout (both in qmk and zmk) in the last few weeks. Each time, I’ve been annoyed with the non-highlighted text of the .keymap files when I edit them in neovim. I learned how to fix this, and the solution is very simple.

Background

My .keymap in neovim before fixing the syntax highlighting.

Neovim has been my editor of choice for awhile now, especially when in a terminal. I converted my configuration to a .lua setup, with treesitter for syntax highlighting about a year ago. Generally, this works well. However, the system didn’t recognize to use c highlighting for my keymap files.

:setfiletype

After some digging, I was reminded about the setfiletype option in vim. This lets you change the buffer’s file type. For example, :setfiletype markdown. When I tried setting the buffer to c, everything looked much better.

Adding to my Configuration

With a potential fix, I wanted to add it to the configuration to keep it persistent. I didn’t feel like converting this to the equivalent lua, so I just used a vim.cmd in the general file of my home-manager neovim configuration:

-- Generally read .keymap files as C ones
vim.cmd[[autocmd BufNewFile,BufRead *.keymap setfiletype c]]

This line tells vim to set the file type to c for any file that ends in .keymap. Maybe one day I’ll convert it to lua, but for now … it works 😄.

Conclusion

My .keymap in neovim after fixing the syntax highlighting.

After adding that line and rebuilding my home-manager config, neovim opens .keymap files with syntax highlighting on all my machines. Quick and simple!

RSS-Only Posts

2023-05-13T00:00:00Z

I just figured out a way to enable ‘rss-only’ posts on my website. Basically, it means if you’re getting and reading my posts via rss, you see this. If you’re just browsing the website index… you won’t (however you can still access the page via direct link).

It’s a little nod and thank you to rss users out there 🙂.

I came across the idea after adding evantravers.com to my feed reader. He’s part of something called the rss-club, and I love it. There are others in the club that I also follow, but I never realized I was getting rss only posts from them 😄.

Anyway, I liked the idea and wanted to implement it on my website for a few reasons:

First, I do love RSS and want to encurrage the use of it, especially those that write and maintain personal blogs.
Second, I have been thinking about ways to semi-publish less technical and informal posts, without watering down the main content of the site… this might be a way to do that, but I’m still deciding.

Long story short, if you subscribe to this website via rss… thank you! I’ll try to add quick and fun posts to the feed on occasion.

– Ryan

My Required MacOs Apps (2023)

2023-04-30T00:00:00Z

Durham NC

Now that my work computer is a MacBook Pro, I’m spending a lot of time on a Mac. Overall, I enjoy it… after some tweaks from the default configuration. I rely on a few applications to really make macOS work for me. Here are four that make my experience in macOS much more enjoyable and productive.

Raycast

I use raycast to manage my windows.

I’ve used raycast for years now, but I just recently started heavily relying on it for a few specific commands. While this launcher is extremely versatile and can be used for basically anything, I mostly utilize three operations:

Toggle between light/dark system appearance
Window Management (ex: setting windows to half, third, or fourth screen splits)
Switching my set audio input/output devices (Great when jumping into a meeting)

Easy Move+Resize

Being able to move or resize windows quickly and easily by holding the CMD key plus right or left clicking the mouse is something I’ve always missed from Linux while in macOS. One day, I was fed up and started searching the internet for a solution because I figured I couldn’t be the only one missing it.

I eventually stumbled on a Github repo for an application called Easy Move+Resize. It’s simple little app, allowing me to select a modifier key to pair with right/left clicks to move & resize windows… exactly what I want. Now I can hold CMD and click a window to move it with my mouse, or right click and drag to easily resize it. Finally!

BetterDisplay

Using a sidecar iPad in portrait mode via BetterDisplay.

Next, BetterDisplay provides all the additional display controls I desire. It allows me to use sub monitor configurations in macOS, and also includes many nice to have features, like software brightness controls for all monitors.

Most importantly, I can configure my displays in ways normally unallowed … like disabling a connected display or using an iPad via sidecar in portrait orientation.

Again, BetterDisplay is a very powerful tool that I highly recommend.

Hand Mirror

Lastly, Hand Mirror. This might seem like a stupid app, but I find myself using it all of the time. Basically, it puts an icon in the menubar, and when you click on it (or under the notch with the pro version), a preview of your webcam pops open. I use this to check my camera and lighting before joining a meeting, or while in one (ex: I can check how everything looks before turning my back camera on).

It’s such a small feature, but very convenient.

Conclusion

Using my work macbook at the Ansible office.

That’s about it. While there are many other apps that I use daily and love, these are the core apps that help fill in the gaps and enable macOS to truely work for me and my workflows.

Work Laptop Refresh - 16\" MacBook Pro M1 Pro

2023-04-02T00:00:00Z

New and Old Work Laptops, Durham NC

Near the end of last year (2022), I was eligible for a laptop refresh at work. This time around, I chose to replace my 7th generation X1 carbon with the 16" Macbook Pro option.

Laptop Specs

Old work laptop: Gen.7 X1 Carbon Thinkpad (left). New work laptop: 16 M1 Pro Macbook Pro (right).

Before proceeding further, here are the specifications of each machine:

x1 Carbon Thinkpad (gen7)
Intel i7-8665U [1.9 Ghz (4.8 Ghz Turbo), 2 cores, 8 threads]
16 GB RAM
250 GB NVME SSD
14" 1920x1080 IPS Display

~2.4 lbs.
12.71" x 8.54" x 0.58"

2021 16" Macbook Pro
M1 Pro (10 CPU Cores [2 efficiency, 8 performance], 16 gpu cores)
16-core Neural Engine
Media engine
32 GB RAM
512 GB NVME SSD
16.2" 3456x2234 Liquid Retina XDR display

4.7 lbs.
14.01" x 9.77" x 0.66"

With that available for reference, lets continue.

Why the switch from Thinkpad to Mac?

I did enjoy the x1 carbon quite a bit, especially compared to the p50 I had previously. However, my work needs have changed over the last few years, and I wanted something more powerful. On top of that, I wanted to try out using a Mac at work. Some of my reasons for the switch to Mac include:

Great Hardware

Despite having a decent display, I often used my 4k external monitor for 200% scaling and a slightly larger text (14" vs. 15.6").

The experience I’ve had with my personal Apple hardware has been very positive. Apple Silicon has performed wonderfully, and I love how low power and silent the chips run. Beyond processors, I have become accustomed to how clear and crisp text appears on a display with 200% scaling. For this reason, I appreciate the retina displays on MacBook Pros. I don’t feel obligated to hook a portable 4k display to my MacBook to make text look great, which I do on the Thinkpad (I know Thinkpads can have 4k displays, but the models we have at work don’t, and even when they did, there usually is a noticeable hit to battery life). Lastly, the speakers in a MacBook Pro are really hard to beat in a laptop, and the remaining periphery components (keyboard, trackpad, webcam, mic) range anywhere from ‘good’ to ‘great’ for what I need in a work device.

As a bonus, Macs work well with other devices in the Apple ecosystem (although I do wish was more open). This includes the Apple Watch, iPhone, iPad, and any airplay 2 supported devices, which I’ve accumulated over the years.

Focus

I need my work machine to help me finish my work. I don’t work on a desktop team, so while playing around and learning details about desktop Linux is fun and can be useful, it doesn’t always help me stay on task with what I need to accomplish for my job. If I’m going to spend time ‘messing around’ learning something, it should probably be things like OpenShift, CICD, Coding, and I guess testing. All of those tasks I can do just fine from a Mac. I don’t even run many local VMs for work currently. If I did, I’d likely stick with Linux.

The truth is, I tend to get distracted tweaking the desktop and underlying system on Linux. While I still do this on macOS, there’s more restraint. In Linux, anything can be hacked together with enough effort (and it sometimes has to be), but in macOS, it is usually a little clearer when your rabbit hole has slammed into a wall. For me to be productive, I need that… when it comes to my desktop OS. I’m still willing to mess around with servers all I want 😆.

A Nice Change

I love Linux, but using it for my desktop OS has become a bit stale for me lately. I do find this strange, because I think the Linux desktop has had many interesting projects over the last few years. Still, it doesn’t allure me like it use to. I don’t feel the desire to distro-hop around anymore (which is good, as I currently have very limited time). While I might occasionally dabble with Silverblue, NixOS, or Elementary OS, I ultimately hop back to Fedora.

The areas I find most interesting about Linux (podman, virtualization, nix) have nothing to do with the desktop.

Opportunity to Learn and Share with Others

Lastly, the Apple Silicon transition is a great time for me to dig into macOS and learn. While core, ‘normal user’ workflows are fine, niche computing can be a different story. Things don’t always work perfectly. Workarounds have to be discovered. Hacks are often relied on. As a daily Linux user over the last 12+ years, this all sounds familiar. While many long-time Mac users might not want to bang their heads inventing solutions to edge cases, I find it to be a great opportunity jump in, investigate, and then share my knowledge with others.

A perfect example of this already in action is my m1 podman post. I have had several of my Red Hat co-workers message me that they found and used that post while searching how to setup podman their new MacBook. I want to do more of that.

Why the 16" over the 14"

My personal 14" MBP and work 16" MBP.

At first, I thought the 14" was the obvious choice. It’s just as powerful as the 16", but more portable. Coming from a 2.4 pound 14" X1 Carbon, I didn’t want the switch to be too drastic. Additionally, the computer would be docked to a monitor both when working from home and the office, so there was no real reason to carry a bigger device… right?

Then, during the weeks leading up to when I actually made the decision… my mind shifted and I decided on the 16". Here’s a few reasons why:

Still Portable

The 16" MacBook Pro is still a portable computer. It might be 4.7 lbs, but that’s still a pound lighter than the p50 work computer I had before the x1 carbon, and almost half the weight of my previous personal desktop replacement laptop, the 8.6 pound Bonobo Extreme.

As far as the added weight to my commute… as much as I’d love to walk/bike to work most days, having an infant son that attends a daycare miles from where I live or work means that at least the next few years, my commute is sadly car-based. And my car doesn’t mind the extra size of the 16".

Stand-alone Workstation

The 16" Macbook Pro on a table.

Second, I wanted a machine that functioned well as a stand-alone workstation. I’ve learned that in order to stay motivated, I need to occasionally mix up my surroundings. So, I wanted a computer that I could work on away from my desk, without any compromises. This might be in a meeting room, around the office, or just at my kitchen table.

When placed on a table in front of me, the 16" screen feels perfect. It covers enough of my field of view, and has a resolution that doesn’t feel cramped. Pair that with amazing speakers, and the 16" MBP truly does feel less like a ‘laptop’ and more like a ‘portable workstation’. This is wonderful if I’m tying to work in a meeting room, or anywhere else.

It’s different from what I have

Lastly, and this might be a silly reason, but it’s still something to consider: I already own a 14" M1 Pro Macbook Pro for my personal computer. It’s the base model with the binned cpu/gpu, and only has 16GB of RAM, but as far as the external appearance, it looks the same (except silver) as a work issued one would.

Not only would a 14" work macbook increase the likelihood of me sleepily grabbing the wrong device in the morning, but I would also miss out on a chance to experience working on ‘the other option’. I could learn what the 16" is like to use daily, and how that compares to my 14". I might hate it, but I would know that for a fact, rather than just speculation.

How’s it going?

My 16" Macbook Pro setup at the office.

After nearly 3 months with the 16" M1 Pro MacBook Pro as my new work computer, how has it been? Honestly? It’s been delightful. It took a little bit of time to figure out how to setup my work on a Mac, but it wasn’t nearly as bad as I imagined it would be. My experience using the 14" M1 Pro, and switching my dotfiles + dev environments over to nix + home-manager made the transition much smoother.

The hardware has also been perfect. It’s a brilliant machine to use, wherever I want to work. I never feel cramped when using it. In fact, because the screen is so crisp compared to the 27" 1440p office monitor, the majority of the time I’m at my desk I use the laptop display as my main one (on a stand, I still need proper ergonomics), with my desk’s monitor in portrait as a secondary.

To conclude, I am very happy with my decision to go with the 16" Macbook Pro for my work laptop. It is a incredible portable workstation, and was easily the right choice for me, especially for my work device.

Converting my 34 Key layout from QMK to ZMK

2023-03-01T00:00:00Z

Wireless Ferris Sweep Build, Durham NC

After months loving my ferris sweep, I decided to build a second one. Specifically, I wanted a wireless version so I could avoid all the wires when using it in portable setups. However, the nice!nano controllers used for bluetooth keyboard builds, uses the ZMK firmware instead of QMK, which I had previously used. So, I had to convert my ferris sweep layout to ZMK. Here’s my experience.

Background

The ferris sweep's wires can be a bit of mess, especially on the go.

A couple weeks into using my ferris sweep, I decided if I ever built a second one, that I would want it to be wireless. In September (on my birthday). I finally ordered parts for a nice!nano based ferris sweep build. Weeks after ordering, they arrived.

I finally had the parts, but there was still a road block: I was only a few months into being a new parent, with a wife still in medical training. Even when I managed to find time to build the keyboard, it was never a large enough chunk of time to also include the patience which my novice soldering skills require. Unsurprisingly, I messed up. And got frustrated. So, I decided to order a pre-built board.

A few more weeks later, it arrived. I assembling it using the switches and key caps I still had from my failed build. With the keyboard physically complete, I needed to generate and flash it with the ZMK firmware.

Setting up ZMK

The process for flashing ZMK was quite different than what I did for QMK. First, I had to install ZMK, which was easy enough to do by following the documentation. This involved running a script which prompts the user with a few questions, and then generates a repo containing the appropriate firmware templates for the keyboard model specified. After that repo is pushed to Github, Github Actions are triggered to run firmware builds.

As simple as running the script was, it was where I hit my first snag. My firmware builds kept outputting a single .bin package, which was not expected (I needed separate firmwares for each nice!nano: a left and right).

After struggling with this problem for a bit, I realized my error. During the board selection, I had been selecting Ferris 0.2 (which is the original ferris), I needed to select Cradio/Sweep instead:

Keyboard Selection:
 1) 2% Milk		             24) Helix		       47) Preonic Rev3
 2) A. Dux		             25) Hummingbird	   48) QAZ
 3) BAT43		             26) Iris			   49) Quefrency Rev. 1
 4) BDN9 Rev2		         27) Jian			   50) Redox
 5) BFO-9000		         28) Jiran		       51) REVIUNG41
 6) Boardsource 3x4 Macropad 29) Jorne		       52) REVIUNG5
 7) Boardsource 5x12	     30) Knob Goblin	   53) Romac Macropad
 8) BT60 V1 Hotswap	         31) Kyria		       54) Romac+ Macropad
 9) BT60 V1 Soldered	     32) Kyria Rev2		   55) S40NC
10) Chalice		             33) Leeloo		       56) SNAP
11) Clog		             34) Lily58		       57) Sofle
12) Contra		             35) Lotus58		   58) splitkb.com Aurora Corne
13) Corne		             36) MakerDiary m60	   59) splitkb.com Aurora Lily58
14) Corneish Zen v2	         37) Microdox		   60) splitkb.com Aurora Sweep
15) Cradio/Sweep	         38) MurphPad		   61) Splitreus62
16) CRBN Featherlight	     39) Naked60		   62) TG4x
17) eek!		             40) Nibble		       63) Tidbit Numpad
18) Elephant42		         41) nice!60		   64) Waterfowl
19) Ergodash		         42) nice!view		   65) ZMK Uno
20) Eternal Keypad	         43) nice!view adapter 66) Zodiark
21) Eternal Keypad Lefty     44) Osprette		   67) Quit
22) Ferris 0.2		         45) Pancake
23) Fourier Rev. 1	         46) Planck Rev6
Pick a keyboard:

With that switch, the system started building the firmware for the correct keyboard.

Converting the layout

Now that I was building the proper firmware for the ferris sweep, I needed to convert my QMK layout over to ZMK. I think I started by manually copying my layout over into the layout template (or maybe even tried a conversion tool).

Whatever I attempted… it didn’t work. When I tried to build the firmware, it erred on the keymap file:

devicetree error: /__w/ferris-sweep-zmk/ferris-sweep-zmk/config/cradio.keymap:38 (column 43): parse error: expected number or parenthesized expression

To isolate the problem, I commented out the layers and added them back in one at a time. With each error, I figured out where it was happening and what changes I needed to apply to make it ZMK-complaint.

While I don’t know if I recorded all my required changes, here are a few of the big ones I had to convert while going from qmk -> zmk.

Any keys that used () formatting needed to be replaced with a ZMK alternative. Ex: LCTL_T(KC_A) in QMK became &mt LEFT_CONTROL A in ZMK (A on tap, Left CTRl on hold).
The OSM keys I used in qmk are sticky keys in ZMK (doc source), so I converted those. Ex: OSM(MOD_LCTL) became &sk LCTL
Most symbols had to be converted to alternate abbreviations. Some (but not all) of the changes include:
- KC_ESC --> &kp ESCAPE
- KC_TILD --> &kp TILDE
- KC_LCBR --> &kp LBRC
- KC_RCBR --> &kp RBRC
- KC_GRV --> &kp GRAVE
- KC_SCLN --> &kp SEMICOLON
- KC_LBRC --> &kp LBKT
- KC_RBRC --> &kp RBKT
- KC_LPRN --> &kp LPAR
- KC_RPRN --> &kp RPAR
- KC_EXLM --> &kp EXCl
- KC_DLR --> &kp DLLR
- KC_PERC --> &kp PERCENT
- KC_MINS --> &kp MINUS
- KC_CIRC --> &kp CARET
- KC_AMPR --> &kp AMPS
- KC_BSLS --> &kp BACKSLASH
- KC_SLSH --> &kp SLASH
- KC_EQL --> &kp EQUAl
- KC_ASTR --> &kp ASTRK
- KC_QUES --> &kp QUESTION
- KC_MUTE --> &kp C_MUTE
- KC_VOLD --> &kp C_VOLUME_DOWN
- KC_VOLU --> &kp C_VOLUME_UP
- KC_PGUP --> &kp PAGE_UP
- KC_PGDN --> &kp PAGE_DOWN
- KC_HOME --> &kp HOME
- KC_DEL --> &kp DELETE
- KC_ENT --> &kp ENTER
- KC_CAPS --> &kp CAPSLOCK
- KC_MSTP --> &kp C_STOP
- KC_MRWD --> &kp C_REWIND
- KC_MFFD --> &kp C_FAST_FOREWARD
- KC_MPLY --> &kp C_PLAY_PAUSE
Lastly, there was at least one instance of a typo that threw me for a loop: &sl instead of &sk 😆.

Once I got all the standard keys in my layout figured out, the last thing I needed to switch was the keys to toggle between layouts. My normal thumb-key layer-switch practically stayed the same, only going from MO(NUM) to &mo NUM. However, the keys that toggle my base layer switched from a DF(NUM) to &tog NUM, which seems to work the same. I also decided to remove some of the alternate base layers I never use on my QMK sweep.

Fixing home row mods

At this point, the firmware compiled and I was able to flash my layout to the keyboard! After using it for a few seconds, it was immediately clear that the home row mods were terrible. As I continued to type, they were so bad that I thought I might have to disable them because of how often they misfired. I tweaked several timings, but nothing resolved the problem.

After some research, I learned about the flavor = "tap-preferred" setting that can be added to the keymap file:

&mt {
  flavor = "tap-preferred";
  tapping_term_ms = <285>;
 };

I made this change, and the keyboard worked exactly how I wanted it to. At this point, the layout and timing felt just like my qmk ferris sweep! Success.

Conclusion

New ZMK keyboard working great. It's been my daily driver at work.

It has been over two months now and everything has worked great, without any issues. I have been using the zmk sweep as my daily driver in the office, and seamlessly switch back and forth between it and my QMK sweep at home. I haven’t had to make many tweaks to the layout, but when I do (which I will at some point to add bluetooth controls), it should be very easy now that I have the layout converted. Hopefully this post can can help any others that need to make the QMK --> ZMK switch. Good luck!

Home-Manager Dark Mode Toggle

2023-01-30T00:00:00Z

Bodie Lighthouse, Outer Banks NC

A few months ago, I took the plunge and migrated my dotfiles system into my home-manager (nix) setup. Around the same time, I also switched to a desk in the office that is next to a window. I love all the natural light, but need to use light colored themes during different times of the day due to the brightness.

Manually switching my terminal and neovim themes every time I wanted to toggle between light and dark themes was tedious. So, I turned to my new dotfile system to see if I could figure out a more automated approach.

Implementation

Home-manager allows you to declaratively define user specific packages and dotfiles. In other words, I define everything in code (nix), and home-manager generates all the dotfiles, based on the functional nix expressions. Many applications allow you to easily set themes by declaring a variable in the configuration. For example, the following to defines settings for the terminal app, kitty:

programs.kitty = {
  enable = true;
  extraConfig = builtins.readFile ./configs/kitty.conf;
  theme = "Gruvbox Material Dark Hard";
};

The first line ‘enables’ kitty, the second one pulls in my old dotfile to be used for the majority of the configuration, and the last line sets which theme to use.

With this in mind, I wondered if I could alter my home-manager configuration to conditionally toggle between ‘light’ and ‘dark’ modes for my terminal apps.

Variables file

The first step took me a bit to figure out, but I eventually learned how to create a variables file that can be read from other config files:

{ config, pkgs, lib, ... }:

{
  options = with lib; with types; {
    darkmode = mkOption { type = bool; };
  };
  config = {
    darkmode = true;
  };
}

I used lib to add some typed options, that can be set and later accessed elsewhere in the nix/home-manager configuration. I added a boolean darkmode option, which can be used to define if ‘darkmode’ is enabled or not.

kitty theme set

The first application I wanted to toggle was the terminal. When I switched to home-manager for dotfiles, I started using kitty again, because of it’s configurability in home-manager.

For the darkmode toggle, I wanted to not only switch between a light and dark theme, but also to define which theme was used for each case (ex: maybe Gruvbox for dark, but Ayu for light). This turned out to be quite easy to do in the kitty home-manager config file:

{ config, ...}:

let
  kitty-theme = if config.darkmode then
    ## Dark Themes
     # "Ayu"
     # "Dracula"
     # "Gruvbox Material Dark Medium"
     "Gruvbox Material Dark Hard"
     # "Flatland"
     # "Kaolin Aurora"
     # "Kaolin Ocean"
     # "Monokai"
     # "Nord"
     # "Nova"
     # "Obsidian"
     # "PaperColor Dark"
     # "Obsidian"
     # "Solarized Dark"
     # "Spacemacs"
     else
    ## Light Themes
    # "Ayu Light"
    # "GitHub Light"
    "Gruvbox Material Light Hard"
    # "Material"
    # "Piatto Light"
    # "Solarized Light"
    # "Tomorrow"
    # "Spring"
; in

{
  programs.kitty = {
    enable = true;
    extraConfig = builtins.readFile ./configs/kitty.conf;
    theme = kitty-theme;
  };

}

First, I make sure to import the config module, which contains our config.darkmode variable. I then defined a local variable, kitty-theme using a conditional expression. Basically, if the config.darkmode is true, then kitty-theme is set to the name of my dark theme, otherwise it is set to the light theme name. To make it easy to change my preferred themes, I list some of my favorites, and un-comment my selection for each category (light/dark).

With the local variable defined, it is used when setting the programs.kitty configuration: theme = kitty-theme;. With this, home-manager builds the kitty configuration, and selects the theme based on what the darkmode variable set to in the variables.nix file!

neovim theme loading

Setting up the dark/light themes for my terminal isn’t very useful if neovim doesn’t also follow the dress code. To make this work for neovim, I had to get a little creative:

{ config, ...}:

let 
  colors-file = if config.darkmode then ./configs/nvim/dark-theme.lua else ./configs/nvim/light-theme.lua;
in
{
  imports = [
    ../variables.nix
  ];
  # Just use my neovim config
  home.file.".config/nvim/".source = ./configs/nvim/nvim-dotfiles;
  home.file.".config/nvim/".recursive = true;
  # Set color theme file to load
  home.file.".config/nvim/lua/user/colortheme.lua".source = colors-file;
}

Like the kitty setup, I import config and define a local variable (colors-file), based on the condition of config.darkmode. However, for the neovim toggle, I need to change multiple variables (ex: set vim.opt.background and some theme-specific ones too). So unlike kitty, the local variable defines a file path to a neovim configuration file, rather than just a theme name. Whichever file path is selected, home-manager copies it as a colortheme.lua file when building the neovim configuration.

Both colortheme.lua options set required variables, load the colorscheme, and throw an error if the theme is not found. Again, I list several themes that I have installed and un-comment the one I want:

Dark mode config (dark-theme.lua):

vim.opt.background = "dark"

-- Theme
-- local colorscheme = "ayu"
-- local colorscheme = "everforest"
local colorscheme = "gruvbox"
-- local colorscheme = "happy_hacking"
-- local colorscheme = "nord"
-- local colorscheme = "sonokai"
-- local colorscheme = "sobrio"
-- local colorscheme = "sobrio_ghost"
-- local colorscheme = "sobrio_verde"


-- Attempt to Load theme
local status_ok, _ = pcall(vim.cmd, "colorscheme " .. colorscheme)
if not status_ok then
  vim.notify("colorscheme " .. colorscheme .. " not found!")
  return
end

Light Mode config (light-theme.lua):

vim.opt.background = "light"
vim.g.ayucolor = "light"

-- Theme
-- local colorscheme = "ayu"
local colorscheme = "everforest"
-- local colorscheme = "gruvbox"
-- local colorscheme = "nord"
-- local colorscheme = "sobrio"
-- local colorscheme = "sobrio_light"
-- local colorscheme = "sobrio_verde_light"


-- Attempt to Load theme
local status_ok, _ = pcall(vim.cmd, "colorscheme " .. colorscheme)
if not status_ok then
  vim.notify("colorscheme " .. colorscheme .. " not found!")
  return
end

After setting up the separate colorscheme files, I updated my previous colorscheme.lua neovim config file:

-- Everforest specific options
vim.g.everforest_background = "hard"

-- Sonokai specific options
-- vim.g.sonokai_style = "atlantis"
-- vim.g.sonokai_style = "espresso"
vim.g.sonokai_style = "maia"

-- Try to load theme from dark/light colortheme fiUUE
local status_ok, amineeded = pcall(require, 'user.colortheme')
if not status_ok then
    local colorscheme = "gruvbox"
  vim.notify("colortheme file found. Using default (" .. colorscheme .. ") instead!")
    -- Load & Check
    local status_ok, _ = pcall(vim.cmd, "colorscheme " .. colorscheme)
    if not status_ok then
    vim.notify("colorscheme " .. colorscheme .. " not found!")
    return
    end
end

This file sets some general (non dark/light specific) variables, and then tries to load the colortheme.lua file that home-manager placed. If it cannot load it for some reason, it defaults to using gruvbox (my favorite) and notifies the user.

In summary, home-manager uses the darkmode variable to decide which colorscheme.lua file to include (dark or light) when it builds and deploys the neovim dotfiles. Then, neovim attempts to load that file for it’s colorscheme settings, falling back to a default if it encounters any issues (for example, if I’m not using home-manager on a system).

How it all works together

All of the changes made so far use dotfiles to define if an application uses a dark or light theme. Now that I use home-manager to generate my dotfiles, this means all I have to do is run home-manager, and it will apply whatever I have the darkmode variable set to:

home-manager switch

If I want to switch modes, I just change the darkmode boolean in the variables.nix file, re-run home-manager switch, and the theme changes are applied.

Script

While this system is already much simpler than manually changing all the themes, I still didn’t want to have to locate the variables.nix, edit the value, and then run home-manager every time I wanted to toggle between dark and light mode. So, I scripted it! (darkmode-toggle.sh):

VARIABLES_FILE=/home/ryan/Documents/DevOps/home-manager/variables.nix

if grep -q 'darkmode = true;' $VARIABLES_FILE; then
    echo "Currently dark-mode. Switching to light-mode."
    sed -i.bak 's/darkmode = true/darkmode = false/g' $VARIABLES_FILE
else
    echo "Currently light-mode. Switching to dark-mode."
    sed -i.bak 's/darkmode = false/darkmode = true/g' $VARIABLES_FILE
fi
echo "Running home-manager switch to apply change..."
home-manager switch
echo "Done!"

The script checks what darkmode is set to in the variables file, and switches it. It then runs home-manager switch to apply the change. There are also a few print messages to communicate what is happening.

Lastly, I added a symlink of this file to /usr/bin/darkmode-toggle, so now I just have to type darkmode-toggle (actually, dar TAB ENTER) in my terminal, and everything changes. Much better!

Wrap up/conclusion

While I have used nix/nixOS off and on for years, and switched my dotfiles over to home-manager months ago, this was really my first time playing with nix expressions. While this might not be the best approach to accomplish this, I had a ton of fun figuring it out. On top of that, this method has honestly been solid for months now.

Using gTile

2022-12-22T00:00:00Z

Downtown Durham NC

Years ago I enjoyed using minimal tiling window managers. Now, I prefer to use a ‘normal’, full desktop environment (ex: Gnome or Plasma). That said, I still prefer to arrange and manage my windows from the keyboard. On macOS, I’ve been using betterTouchTool for improved window management (rectangle is also great).

During my time using betterTouchTool, I’ve been annoyed that I didn’t have an equivalent method to set windows quite the same way on Linux. While testing out an ultra wide monitor at work, I really needed a better solution. So I searched a bit harder. It turns out there is an option that does exactly what I want in Linux (at least on Gnome): gTile.

What is gTile & How to get it

Using gTile, you can specify where on the screen you would like to place a window (red) from a list of pre-set configurations you decide (the pop up box).

Basically, gTile is a tiling/window management extension for Gnome. It allows users to define different grid systems (ex: 8x6, 4x4, 3x1) that they can use to place windows. It has many additional features and can be quite customized.

Being a Gnome extension, it is installed from the extension store: link. Note, settings for gTile are configured in the extensions app.

How I use gTile

I like how you can set the window margins (gaps).

Again, there are a ton of customization options for gTile, so the ways it can be used is nearly endless. That said, here is how I use it.

I almost exclusively use a 1x6 layout for the following reasons:
- 1x6 allows me to select halves or thirds (usually a 2/3 & 1/3 configuration) easily
- I use 6ths (instead of 3rds) because I sometimes have applications I like to use narrows windows with (ex: telegram).
- I don’t stack applications, so I only need 1 row.
There are settings to use specific keyboard shortcuts, but I quickly became accustomed to and enjoy the pop-up window. I have alt+Enter bound to open the selection window, and then use my arrow keys (and sometimes shift) to do the rest.
I enjoy having a little gap between windows. It’s not the most efficient use of space, but I enjoy the look better.
I even tend to use a gTile section to setup my single ‘full screen’ windows (by selecting all 6 sections for a window). This maximizes the window, but with a little bit of padding around it. If I want to entirely full screen the window, I can use Gnome’s default CMD+Up keyboard shortcut.

Conclusion

My windows after arranging with gTile.

gTile is exactly what I want for window management, and now defines how I use Gnome. Specifically, I enjoy having the selection pop-up, and using my keyboard to visually select how I want each window to be arranged. Figuring out the gaps options was icing on the cake. Now, I want my macOS workflow to catch up 😅. Overall I’m very happy with this solution.

Adding --baseurl Argument to My Website Pytests

2022-11-06T00:00:00Z

Sarah P. Duke Gardens, Durham NC

When I run the pytests for my website, they run against whatever hugo server I have running. While writing a post, this is usually the same machine I’m writing from. When I commit changes, the CI pipeline automatically runs the tests on my gitlab CI runner node. The IP address of the server to test is determined by a BASE_URL constant variable defined in my constants.py file.

Changing the variable in this file, and not accidentally pushing it in git is super annoying. So, I finally sat down one night and moved this variable out of the constants file, converting it to an optional pytest argument.

Steps

While this might not the best approach, it was easy enough to implement in a very short period of time, and seems to be working great so far. Here are the steps I took to convert my tests from using a constant variable to the optional argument.

Add pytest_addoption() function to the conftest.py file

def pytest_addoption(parser):
    parser.addoption(
        "--base-url", action="store", default="http://ci-runner.website.net"
    )

First, I added a pytest_addoption() function definition to my conftest.py file. This function allows me to add command line options to pytest. I added a single option for now, --base-url, with it’s action set to "store". Additionally, I set the default value to the address of my CI runner node. This way, if I don’t provide a value it defaults to using my CI runner.

Add base_url fixture to the conftest.py file

@pytest.fixture()
def base_url(request):
    return request.config.getoption("--base-url")

Next, I added a new fixture (base_url) to replace my old constant variable (BASE_URL) with. This fixture simply returns the value of the --base-url option just added.

Update other fixtures to reference the base_url fixture

## Example Before
@pytest.fixture(params=POST_NAMES)
def post_url(request):
    """Returns the post urls for testing."""
    return BASE_URL + "/post/" + request.param.lower()

## Example After
@pytest.fixture(params=POST_NAMES)
def post_url(request, base_url):
    """Returns the post urls for testing."""
    return base_url + "/post/" + request.param.lower()

Next, I had to update any other fixtures in the conftest.py file to use the base_url fixture, instead of the old constant variable. I just added base_url to the parameter list, and replaced the variable name in the fixture definitions.

Update test functions to use base_url fixture

## Example Before
@flaky
def test_md_links(post_md_link):
    """Checks that the markdown links are not broken."""
    if post_md_link in SKIP_LINKS:
        pytest.skip(f"{post_md_link} in skip list")
    elif post_md_link.startswith("http") or post_md_link.startswith("https"):
        url = post_md_link
    else:
        url = BASE_URL + post_md_link.lower()
    response = requests.get(url)
    assert response.status_code != 404, f"The link {post_md_link} is not found."
    assert response.status_code != 403, f"The link {post_md_link} is forbidden."

## Example After
@flaky
def test_md_links(base_url, post_md_link):
    """Checks that the markdown links are not broken."""
    if post_md_link in SKIP_LINKS:
        pytest.skip(f"{post_md_link} in skip list")
    elif post_md_link.startswith("http") or post_md_link.startswith("https"):
        url = post_md_link
    else:
        url = base_url + post_md_link.lower()
    response = requests.get(url)
    assert response.status_code != 404, f"The link {post_md_link} is not found."
    assert response.status_code != 403, f"The link {post_md_link} is forbidden."

Just like the fixtures, I had to update the test functions to use the base_url fixture, instead of the BASE_URL constant. Again, this was done by adding the fixture name as a parameter to the test function, and swapping out the references in the code.

Remove BASE_URL from constants.py, and imports of it in other files.

Lastly, I did some cleanup and removed the BASE_URL definition from the constants.py file, along with any imports of that variable in the other files.

Results

With those changes, what are the results? Well, I can still run my test execution command the same as before, and it will default to running the tests against my CI server:

pipenv run pytest --workers 10 -v .

However, if I want to point the tests at a hugo server different from the default, I can now define it with the --base-url flag:

pipenv run pytest --workers 10 -v --base-url "http://10.0.9.104:1313" .

Conclusion

The best part of this change is that I don’t have to worry about forgetting to switch that config back and forth when pushing to my repo. I simply set the variable when running the tests. Done. I should have made this change years ago.

Custom Resolutions in Gnome Wayland

2022-11-01T00:00:00Z

Downtown Durham NC

The other week, I worked from Red Hat Tower (Raleigh). In that office, IT is testing out a few flex-workstations with an ultra-wide monitor (Samsung S65UA), which I decided to try out. It was the first time I used a real ultrawide monitor for an entire workday. Since then, I have continued to occasionally experiment with working in a 21:9 aspect ratio… on my 16:9 monitors.

My Ultrawide Experience:

The Ultrawide monitor I used at work.

While using the ultrawide monitor for work, I learned that it fits many of my workflows. This is mostly because I always have my obsidian notes opened. The extra width allowed me to have a full window for whatever meeting or code I was working on, with the remaining space a perfect size for my notes. It’s also nice to have 3 full-ish windows for other tasks.

Since then, I’ve wanted to test the 21:9 workflow a bit more… but I don’t have an ultrawide monitor. However, I do still have my old, massive, 42.5" 4k monitor that we now use as a bedroom TV and second workstation.

I’ve done this before…

Looking through my post archives, one might notice that I’ve written about doing something very similar in the past. When the lg ud4279b was my main monitor, I used it as a canvas for what I called ‘sub monitor setups’. I even wrote a script to make doing so easy. Unfortunately, the method I used only works for X11, and not in Gnome+Wayland.

Setting a custom resolutions in Gnome Wayland

Now that I prefer using wayland (on Gnome) instead of x11, I have had a much harder time creating custom resolutions to fake an ultrawide monitor on large displays.

Luckily, I recently found this wonderful post, that finally helped me setup a workable solution on wayland gnome. They get all the credit. Here’s a rough outline of what I did.

My steps:

List/View ‘graphics’ devices and guess which one might be the one for your current display:

➭ ls /sys/class/drm/card*

From that output, you can see the known resolutions for each display by looking at the modes file, if it exists (if it doesn’t exist, you likely picked the wrong device):

➭ cat /sys/class/drm/card1-DP-1/modes
3840x2160
3840x2160
3840x2160
3840x1600
2560x1440
1920x1080
1920x1080
1920x1080
1280x720
1280x720
1024x768
800x600
720x480
720x480
640x480
640x480
640x480

After confirming the monitor name, it can be used to add to the GRUB_CMDLINE_LINUX line in the grub config, adding a fake monitor resolution for that device. On Fedora 36, I had to edit /etc/default/grub. For example, I appended video=DP-1:3840x1600@60" to the line, which added a 3840x1600 @60Hz resolution option in my config:

GRUB_CMDLINE_LINUX="rd.luks.uuid=luks-5a3d262f-9081-41de-ab1d-0266cab3b7fc rhgb quiet video=DP-1:3840x1600@60"

Note: It’s a good idea to first backup this file before editing it.

After modifying the file, remember to rebuild the grub config:

sudo grub2-mkconfig -o /boot/grub2/grub.cfg

Lastly, reboot and the resolution should be available in the display settings.

Things to note

While this works, it isn’t a perfect solution and there are a fewt things to be aware of:

When connected to the configured display, the device will use the resolution you defined when booting.
It seems you can only set one resolution per display device (input). This is kind of annoying and limiting.
I had at least one experience where the display changed my resolution to something weird that I couldn’t fix until I disconnected and reconnected my monitor. So you might experience minor issues like that from time to time.

Conclusion

Fake ultrawide resolution on my large monitor, under Wayland Gnome.

While ultrawides might be well suited for my work tasks, I don’t think they are ideal for the rest of my personal monitor needs. Still, it’s fun to have the option to turn a large 16:9 into a fake ultrawide for when I just want to work with that type of display. And now, I don’t even have to leave my wayland session to do it!

The iPad Magic Keyboard Case

2022-09-30T00:00:00Z

City Hall, Durham NC

As mentioned in my previous post, I found myself trapped on the couch, bed, or generally away from my desk when my son was born. However, I still wanted to work on projects when possible. As a result, I started grabbing my 11" iPad Pro more than ever. Using the iPad for nearly everything, I began to wonder… should I get the Magic Keyboard to go with it?

Background/Why

Keyboard and Trackpad setup with my old iPad case.

In the past, I would sometimes use a keyboard and trackpad with the iPad, but it was clunky and not what I would call ‘portable’. I’ve always wanted a keyboard case for the iPad, but not at the expense of making it an unwieldy device.

Sitting on the couch in the middle of many nights, I conducted a bunch of research on all the various cases out there, and eventually decided on the official Apple Magic keyboard. All of the other options either didn’t have a trackpad, were massive, or just didn’t seem as well matched for my needs.

Yes, the magic keyboard is very expensive, as many reviews state. However, most reviewers also claim that despite the high cost… they love the magic keyboard. This is a ‘buy once, cry once’ item (quite common with Apple hardware). So, one night at 3 AM, in a new parent daze, I ordered one.

Initial Thoughts

New magic keyboard case, with sloth sticker.

When the keyboard arrived several days later, I recorded a few initial thoughts during the first day or two of use:

It felt heavy: All of the reviews I watched said this, but I blew it off thinking they were being a bit dramatic. I knew it weighed more than the iPad, but my iPad without a case feels very light, so I thought that the case couldn’t be that bad. But when I first picked it up … I was surprised at the heft it had.
The keyboard felt great: I liked the ‘pop’ the keys had when typing, and loved how low and flat the keyboard felt when on a desk. I did worry that my palms sat right on the edge and that during longer typing periods, it might be uncomfortable.
It looked plain: So I added a sloth sticker 😄.

Longer term Thoughts

Now that I’ve been using the iPad with the magic keyboard for several months, here are some refined thoughts.

The Good

The floating hinge makes it very easy to pop the iPad off the case for reading.

The weight isn’t that bad: It is something that might surprise you when you first pick it up, but mostly from the sheer density. It’s light for a keyboard case while remaining quite slim. I learned that the iPad and keyboard are lighter than my old 11" netbook I used to love for its portability.
Perfect terminal console device: The portability, satisfying keyboard, and beautiful screen make this a perfect terminal console device. I love to ssh into systems to work from the iPad. I even updated my Neovim config and switched back to using it as my main editor, mostly because it’s much more usable on the iPad compared to VS Code.
Easy Reading Setup: Despite being a keyboard case, the iPad still functions as a great reading device. With my old case, I started to take it off for reading as the case-less iPad feels so light and nice to hold. The floating hinge design of the magic keyboard case makes it effortless to pop off the iPad when I want to read. When done, I plop it down and the magnets snap it back into place.
Multiple functions in one: The magic keyboard serves many roles for the iPad. It’s a keyboard, trackpad, case, magnetic stand, and additional charging port. The iPad is at its best when viewed as a modular computing device. The magic keyboard is the accessory that brings that paradigm to life.

The ‘Could Be Better’

The ability to connect the iPad in portrait orientation would be nice.

Yeah, just as I feared, the edge does cut into my palms over time. It is worst when typing with the iPad on my lap, but hardly a problem when on a table.
As far as I can tell, there is no way to use FN keys with this keyboard. I figured I’d be able to trigger them by combining the numbers with the globe key or another modifier, but that doesn’t seem to be the case. WHY!?!?
There is nowhere to securely attach an Apple Pencil. It still magnetically attaches to the side, but easily falls off when in a bag. As a result, I’ve given my pencil to my wife to use with her iPad mini for now.
The case smudges easily and is hard to clean.
I wish I had the option to set or flip the iPad into a portrait orientation. This would be nice while reading long documents or when writing. Because the iPad only connects and powers the keyboard via the SMART connector, it’s not possible to connect to it in alternative orientations.

Conclusion

My magic keyboard setup.

I love the magic keyboard case and am so happy I took the plunge and bought it. It has transformed my iPad from a video and reading consumption device, to a portable, productivity, powerhouse. I can do all of my portable computing with the iPad now, and it has filled the role that my netbooks and other portable laptops did in the past. All of this, while remaining a perfect tablet for reading and watching videos.

My Obsidian Mobile Setup

2022-08-30T00:00:00Z

Downtown Durham, NC

I hate typing anything on my mobile devices. If I need to write more than a few words, I prefer to sit down at a computer. However… I recently became a parent, and suddenly found myself trapped on a couch at all hours. From my nest, I didn’t have enough space or mobility to use my laptop easily. So, I needed to start using Obsidian on my mobile devices.

Background & Motivation

My Desktop Obsidian Setup.

Two weeks into parenthood, I wanted to restart the habit of writing obsidian daily notes as the day progressed, instead of catching up in bulk every few days. Finding time at a laptop was hard, but I could sometimes find moments during the day to type on my phone, or even my iPad. Unfortunately, while a big motivator for switching to Notion, and later Obsidian was that I could access my notes on mobile devices, I only setup obsidian to read my notes easily. I didn’t have a good setup for writing notes. So, I set out to make some improvements.

Sync/File Setup

My new daily note's folder structure.

Previously, I used working-copy to sync my obsidian files via git. Since writing that post, I have already switched to syncing my personal notes with icloud.

After using icloud sync with obsidian for several months, I’ve learned to open the Files application before even attempting to open Obsidian on my phone or iPad. This helps to ensure icloud has synced all my files, and prevents duplicate Daily Note files from being created.

Another improvement that I learned was that you can set the Daily Note path to include directories, (example: YYYY/MM-MMMM/YYYY-MM-DD), and it will create and store the notes in that folder hierarchy. The only problem I ran into, was that the keyboard shortcuts for the Today, Previous, and Next daily notes didn’t work after the change. After some research, I learned that the periodic notes plugin could fix the problem, as the plugin’s keyboard shortcuts worked. So I switched to the Periodic Notes plugin, and love all the additional functionality it provides, along with fixing my shortcut issue.

UI tweaks

The command toolbar in my Mobile Obsidian.

I next made some changes to the navigation in the mobile app to better fit my workflow. To do this, went to the Mobile section of the settings. From there, items can be dragged and dropped in the “Manage toolbar options” section (Note, I also have the Advanced Mobile Toolbar community plugin installed). Here are the menu bar items I selected, in the order I placed them:

Command palette: Open command palette: run any Obsidian command, which is helpful without having keyboard shortcuts on mobile.
Quick Switcher: Open quick Switcher: search and open any note
Periodic Notes: Open previous daily note: open the previous daily note
Periodic Notes: Open daily note: Open today’s daily note
Periodic Notes: Open next daily note: Open the next daily note
Add internal link: Quickly insert [[]] to add a link with. This is nice on the phone where the [, ] keys are hard to get to.
Toggle bullet list: Toggle having a list or not. Again, more of an ease thing.
Toggle numbered list: Same thing as the bullet list but with numbers.
Toggle checkbox status: Check and un-check check-box items easily (or turn the current line into a check-box item if not already)
Indent list item: Indents the current line
Un-indent list item: Un-indents the current line
Undo: Un-does last action
Redo: Redo previous action
Toggle keyboard: Toggle the onscreen keyboard on and off
Toggle Live Preview/Source module: Switch between the live preview and the raw markdown
Configure mobile toolbar: Open the mobile toolbar settings

These changes made using obsidian on my phone much easier. It was quicker to insert links and indents (since I don’t have a tab key), and having the command button right up front helped me navigate without using the keyboard commands that I normally rely on.

Remote Git snapshot backups

The output from my git backup script.

What I love most about my obsidian-git plugin setup, is that the git commits act as snapshot backups of my vault. If anything gets messed up, I can easily revert to a previous commit. However, the git plugin isn’t available on iOS and iPadOS. Even worse, now that I use icloud sync for my personal vault, it isn’t compatible with working copy either (and stand-alone working copy doesn’t like the sub module setup I use for my vault either).

Even though I am currently unable to use git from my mobile obsidian setups, I still wanted an easy way to kick off these git ‘snapshots’ while writing. So, I automated it into a script that I can execute remotely:

cd /Users/ryan/Library/Mobile\ Documents/iCloud~md~obsidian/Documents/ryan-vault
echo "git pull..."
git pull
echo "\ngit add *..."
git add *
echo "\ngit commit..."
git commit -m "vault-backup -- $(date +%Y%m%d-%H%M) (script)"
echo "\ngit push..."
git push
echo "\nDone!"
echo $(date +%Y%m%d-%H%M)

The script first pulls the latest commits to make sure the repo is up to date. Then, it adds the changed files, creates a time-stamped commit message (similar to what I have obsidian-git write), and pushes the commit. It’s not fancy, but it works.

The git log messages of the backup commits. The `(script)` commits are 1 the ones generated using this script.

I spent a long time trying to kick off the remote script using an Apple Shortcut, but it just didn’t want to work. So now, I just ssh to my MacBook via the blink app, and run a aliased command (git-backup-ryan-vault) to execute the script. It works great!

Remaining Issues

The obsidian sync screen.

While this system has allowed me to use obsidian fully from my iPhone and iPad, there are still a few pain points I’d love to see fixed:

I wish I could use both icloud and working copy in tandem for vault syncing.
The Obsidian mobile client can take awhile to load, as it has to sync all the config files.
Often when I first open the app (especially on the iPhone), it freezes and I need to close it and try again. It can take 2-3 attempts to get the app opened and running on my phone which is very frustrating.

If those issues were fixed, Obsidian mobile would be a much smoother experience.

Conclusion

My iPad Obsidian Setup (In Obsidian Mobile).

All in all, these changes have worked quite well. I am able to easily write in Obsidian from my iPad, and even from my iPhone in a pinch. This has allowed me to access the contents of my vault more readily, and keep up with my notes no matter how busy life gets!

Flashing My Ferris Sweep Using QMK

2022-07-16T00:00:00Z

American Tobacco Campus, Durham NC

In my previous post, I detailed the build process for my ferris sweep keyboard. After building it, I needed to flash the keyboard with firmware. Like many custom keyboards, it uses qmk. Here’s how I flashed my 34 key layout on the ferris sweep.

Create the layout file

The QMK configurator layout tool.

The first step when flashing a QMK keyboard is to build a layout file. I created my layout using the online qmk config tool. From the tool, I was able to select a ferris sweep base layout, and then assign all the keys to match the layout I previously tested on my ergodox (with a few changes). Setting all the keys took some time, but I should only have to do that once. For future changes, the layout file can be uploaded and edited from the configurator. Once completed, I downloaded the layout file and saved it to my dotfile repo.

QMK Setup

Install & Setup `qmk-toolbox`

With a layout file, we can start using QMK to flash the keyboard. In general, there is good documentation about how to do that here. Regardless of your setup, the first step is to install QMK:

On MacOS:

brew install qmk/qmk/qmk

On Linux, it’s suggested to install qmk using pip. So on my Fedora computers:

sudo dnf install python3-pip
python3 -m pip install --user qmk

Note: on Linux I hit a qmk command not found bug. This is because $HOME/.local/bin was not part of the path. So, either add it to your path, or just run the commend directly: ~/.local/bin/qmk COMMAND.

Next, you need to run the setup. This will take care of several setup tasks, like downloading the firmware repo and installing dependencies. It will prompt you with a few questions, but most can remain the default value it suggests.

qmk setup

It will take a few minutes to download and install everything, but afterwards should be all set.

Compile the hex file

To build the firmware file, I first created a new keymap directory and downloaded my layout file to it:

# Copy downloaded keymap file to compile location
cp ryan-test.json ~/Builds/qmk/keyboards/ferris/keymaps/ryan/keymap.json

Next, I compiled the new firmware, using the keymap I specified:

# Compile layout
qmk compile -kb ferris/sweep -km ryan

Lastly, I moved the compiled hex file to a location where the QMK toolbox would be able to see it (out of a hidden directory).

# Move compiled hex file from build directory to one QMK Toolbox can see (non-hidden)
cp ~/Builds/qmk/.build/ferris_sweep_ryan.hex ~/Builds/qmk/ferris_sweep_ryan.hex

Flashing the Ferris Sweep

To flash the ferris sweep, make sure it is connected to one of the halves. From there, open up the QMK Toolbox application.

Load layout file

The QMK toolbox, with my compiled hex file loaded. (Note: the orange device connected text is from hitting the reset, the next step).

In the configuration tool, open your compiled hex file.

Reset

Using a paper clip to activate the reset pins.

In order to flash the keyboard, you have to switch it to reset mode. If you installed a reset switch/button, use that. If not, you can reset the board by connecting the two reset holes using a piece of metal. When I built my sweep, I used a pair of metal tweezers, but I have since graduated to using a bent paper clip (so fancy!). Use whatever works.

Select Half

Selecting the handedness for the firmware.

Due to the reversible PCB for each of my ferris sweep halves, I need to select the handedness for each half when I flash it. Otherwise, it doesn’t know which board is for which hand. To do this in the toolbox, navigate to Tools -> EEPROM in the menubar, and select the handedness you want. You might need to wait a second or two for it to load.

Flash

Flash complete. (Note the text that specifies that it is done).

With everything set, flash the board by pressing the button! You might want to watch the log to verify it’s actually done before unplugging it.

Repeat for the other side

After flashing the first side, repeat the process for the other (load the file, reset, select handedness, and flash). Afterwards, all should be working!

I’m note positive if this step is required, but I always do it…

Some Layout Changes

My current ferris sweep keyboard layout.

Since writing the post about my layout, I have made a few changes after using the ferris sweep for a few more months. Most of these changes revolve around adding different base layers that I can swap in and out using new keys added to the Navigation layer. In addition to my original base layer, I’ve added:

A second default base layer that does not have Home Row Modifiers
A base layer that uses Colemark instead of QWERTY
A mouse toggle (not base) layer that I forget I have

Over time, I’d like to turn my non-home row mod layer into a layout designed for gaming, which is the one thing that this keyboard does not do well right now. But that’s about it.

Conclusion

This process took me a little bit to figure out the first time, but now that I know how it’s done, it isn’t that hard. Also, I don’t need to flash firmware as often now that I have a stable layout. Luckily, when I do finally make that ‘gaming’ layout, I’ll have this post to remind me how to flash it!

Building My Ferris Sweep

2022-05-24T00:00:00Z

My Ferris Sweep, Durham NC

After adapting to the 34 key layout on my ergodox, I felt comfortable enough to start the process of choosing and building my new keyboard. Following some research and deliberation, I selected and built a Ferris Sweep. Here’s how it went.

Why the Ferris Sweep.

I started off by exploring all the different split 34-36 key designs I could find. This included several of the more popular ones, like the gergoplex, corne, ferris, and it’s derivative, the ferris sweep.

After digging into the details of each design, I ended up choosing the sweep for a few reasons. First, I liked the layout. I got used to the 3x5 split with 2 thumb keys on my modified ergodox setup and it worked. I considered the corne, but after observing where my pinkies rest on my ergodox, I realized they didn’t line up with the keys very well. The corne’s columns are much more aligned, whereas the Ferris designs use a much more aggressive stagger across the board that I wanted to try. I figured even if the pronounced column stagger didn’t work for me, it was a chance to confirm that preference.

With the general unavailability of pre-built boards at the time, I knew I would likely have to solder my own. With this being my first keyboard build (and my first time touching a soldering iron), I wanted a design that wouldn’t be too complicated. The Ferris Sweep is special in that each key directly maps to an input on the micro controllers, so you don’t need to solder tiny diodes under each switch, which I wanted to avoid.

So, the sweep won out.

Selecting & Ordering the Components

The Open Source Ferris Sweep PCB I had manufactured

With my design selected, I compiled a list of all the components I needed for the build and picked my specific parts for each one:

PCB: I wanted to customize the shape and screen print on this, but decided to leave that as an option for a more advanced build down the road. I downloaded the sweep half swept plans (one of the options found here) and uploaded them to jlbpcb to be printed. I picked the half swept variation because it is a reversible design. This meant that I could use each of the 5 (minimum for order) PCBs for either half of the board. I choose black purely for looks.

Controllers: I was tempted to get nice!nano controllers for wireless bluetooth, but I didn’t want to figure out batteries, and zmk vs. qmk stuff during my first build (plus, they weren’t easily available at the time). So, I went with 2 usb-c based Elite-C micro controllers instead. I also purchased some TRRS jacks, since the build would be wired.

Switches: This specific PCB design only supports the low profile Kaih Choc switches, but I had no idea which choc switch to get. Historically, I like heavier, tactile switches. I loved the topre switches on my HHKB, and my ergodox has MX browns (which are fine, although I wish it had clears). Even the laptop keyboards I love (thinkpads and my 14" MacBook Pro) all have a distinctive bump at the start of the press. Choc switches however… aren’t really known for good tactile options. I ordered a switch tester and the choc browns were even more unsatisfying than MX Browns. They just felt like scratchy linears. I was very tempted by clickly choc switches. They use a click-bar that feels soooo good. I almost went with clicky white or even heavier navy switches, but I wanted a keyboard that was quieter, not louder than my current one (especially being my portable device). So, I decided to go with linear switches. I actually like to bottom out (softly) when I type, so I thought the linear switches might not be too bad. I also read many people claim using light linear switches on these small, 30-some key, low profile boards pairs very nicely. So, I decided go with the even lighter silver switches instead of the reds I tested (I also thought it would look nicer 😆). Many people use gChoc switches on these small boards, but the 40gf silvers was as light as I was willing to go.

Key Caps: Lastly, I had to pick key caps. There aren’t too many options for choc switches, and availability can be quite low for what is out there. My main option was the MBK Choc keycaps, which come in only black or white. I originally wanted black, but the more I thought about it, the more I leaned towards the white. I thought the contrast of white on the black PCB would look great, and with these being blank PBT keys, I wasn’t as concerned about wearing and yellowing. So, I ordered about 40 1u white caps, and 2 with the positioning bump.

Building the Board

My desk while soldering the keyboard

After ordering all the components, I had to wait a few weeks for them to ship. When everything arrived… I waited a few more weeks to actually do the build 😆. This was mostly due to hosting visiting family and friends a few weekends in a row. The other delaying factor was that I was a bit nervous to start my first build.

When I did finally start, it wasn’t too bad. I took my time soldering, working on the TRRS jacks first to get a hang of it, followed then by the micro controllers. The hardest part of the process was that there wasn’t a good way for me to know if I was soldering correctly. After securing the elite-c controllers, I was able to connect the boards to my laptop, open a keyboard tester app, and use a pair of metal tweezers to check that the ‘keys’ were recognized by the controller (and that I hadn’t fried it, or the pcb). Verifying that I hadn’t horribly destroyed anything yet gave me some much needed confidence.

My completed ferris sweep, before a few add-ons.

From there, I continued on to solder the switches. When they were all done, I once again tested that each switch worked. Finally, I popped on the key caps and was done! (I spent some time figuring out how to flash my layout to the board, but that’s a post for another time). Other than a sensitive u key, the board to worked just as intended!

Additional Post-Build Components

Needed Enhancements

Using the keyboard with the new TRRS cable and rubber feet.

After the finishing the core of my build, there were a few more components I purchased to improve it. First, I wanted to get a new TRRS cable. The ones I had were either too thick for such a small board, or too small in length (they didn’t allow me to split the board to shoulder width like I prefer). So, ordered a 1 meter, braided, sliver and black cable from keeb.io. During the ordering process, I learned that they are based right here in Durham, NC!

Part of what makes the ferris sweep so thin is that doesn’t have a case. It’s just a PCB. The problem with using a case-less board, is that you need a solution to elevate/cover the bottom of the PCB to protect it and the surfaces it’s on. I started with using little rubber feet, but it was very hard to position them so they were all the same length (not on a solder joint), and have the board evenly supported, so edge keys didn’t tip it. I had a case 3D printed… but it was sized too small and ultimately didn’t work. However, while messing with the case, I started to play with tape mods and found that putting painters tape on the bottom of the PCB actually solved the majority my problem. It does slide a bit on a smooth surfaces, but I’ve been putting it on two gripping carpet anti-slip triangles. Eventually, I plan to cut an old mouse pad to fit under each keyboard half. But so far, this is working well.

Nice to Haves

My 'bluetooth' setup at the office, using the magnetic cables, battery, and bluetooth adapter.

For my first ‘nice-to-have’ modification, I made some alterations to my board’s USB-C connectors. First, despite concern about the dangers using of USB-C magnetic adapters, I ordered a few. These are little plugs that I’ve placed in the USB-C port of each keyboard half, paired with a few magnetic adapters I keep on the ends of my usb-c cables for the keyboard. This modification essentially gave my keyboard ‘mag-safe’ cables. Now, I can easily switch which half the board is connected to (useful when flashing it), or immediately disconnect it from my desk to take with me. I don’t have to tediously unplug the cable from the micro-controller (which is a pain and feels so fragile to me). The adapters were an ugly blue color, so I wrapped them in black electrical tape, an now they fit right in.

Lastly, I mentioned previously how I decided to not use the Nice!Nano bluetooth controllers. However, after using the ferris sweep for a short period of time, I realized how wonderful it would be to simply pull out the two keyboard halves without any wires, plop them on a surface, and start typing. While I still agree with my decision to start with a wired board (which works well at my desk), if/when I make another small split board, I think I’ll go for a fully wireless build. To help hold of the desire for a new board, I ordered the only bluetooth adapter of this kind I could find. It’s big and ugly, but works very well. Using the adapter paired with a battery bank, I’m able to wirelessly connect my keyboard to all my bluetooth devices. This helps clean up the wires in my portable setup, and is especially nice when I have my laptop on a stand, as I don’t need a bunch of cables hanging all over the place. I love it.

Conclusion

The Ferris Sweep at my main desk setup.

Overall, I have been very happy with my Ferris Sweep. The column stagger of the ferris layout naturally fits my hands better than any other keyboard I have used, and I have surprisingly become a fan of the lighter linear switches on a low profile board like this one. Lastly, I love how this keyboard works as an efficient daily driver board at my desk, but is still small enough that I can simply pop it off the magnetic cable, slide it in my bag, and use it as my portable keyboard as well.

Despite this being my first full build, I am very pleased with my experience. I was nowhere near perfect putting it together, but I learned that you don’t have to be (for the most part). It wasn’t that hard of project, and I have much more confidence if I ever make a second build. But right now, I am thrilled to be using this keyboard.

Building a 34 Key Keyboard Layout

2022-04-16T00:00:00Z

Cameron Stadium, Duke University, Durham NC

Over the past year and a half, I have been using an Ergodox EZ keyboard and loving every minute of it. It does have a downside tough. As a 60% keyboard lover, it is quite large. So, I have been on the look out for a smaller, split keyboard (I can’t go back to non-split), that I can travel with and use for a portable setup. During this research, I was introduced to 34-36 key split layouts, but wasn’t sure if I could manage one. To test out the idea, I decided experiment with new layouts on my Ergodox…

Background Concepts

Before digging into layout design, lets establish a few key concepts we’ll need to understand for the keyboard layout.

Layers

Like most mobile devices, iOS keyboards utilize layers.

First, lets start with layers. This is something you are probably already familiar with, as most people use keyboard layers every day without realizing it. Your smartphone’s software keyboard likely has a small layout that utilizes layers. For example, my iPhone has a base layer containing the letters of the alphabet, along with a space bar, shift, backspace, enter, and… yep, layer toggle keys! If you want to use numbers or symbols, you tap the 123 key, and iOS switches to the number layer which contains number keys along with some symbols. From here you can use whatever symbols you need, hit ABC to return to the base layer, or tap the #+= key to enter another layer with even more symbols.

Keyboard layers are something we have all become accustomed to. They allow us to use many more keys than a keyboard physically has. With only 34 keys to work with, good use of layers is required for my layout. While toggling layers on a phone is obnoxious and can be annoying, I have found that the task isn’t nearly as bad on a physical keyboard, especially if it has thumb keys.

Tap vs. Holds

The next, but likely first new concept to learn is how key taps differ from holds. In qmk (the open source keyboard firmware), tapping and holding a key can execute different functionality. For example, you can set the Caps Lock key to work as normal when tapped, but function as a CTRL when you press and hold it for a set period of time. I already use this feature on the Ergodox, so it was a conept I’m already familiar and comfortable with.

Home Row modifiers

Testing 34 Key layouts on my Ergodox.

Above all else, what got me realizing that I might actually pull off a 34 key layout, were home row modifiers. Home row mods are when the home row keys (ASDF and JKL; on a QWERTY layout) are used for keyboard modifiers (Control, Shift, Alt/Opt, and Windows/CMD/Super). This can be accomplished using several methods. For example, using tap/holds, the home row keys can function as they normally when tapped, but then double as a modifier when held (ex: a when taped, but CTRL when held). Or, these keys might just be defined as modifiers on a non-base layer, like a symbol layer.

For someone that uses a bunch of keyboard shortcuts, this is a game changer. It’s taking the practice of setting the Caps Lock key to a useful modifer (I always set mine to CTRL), and turning it up to a whole new level.

I started my experimentation by adding home row ‘hold’ modifiers to my normal Ergodox layout, just to try it out for a few days. I knew that if I couldn’t adjust to home row mods, there was no way I would be able handle a 30-something key keyboard. To my surprise, I adapted to it much quicker than I had expected. After taking some additional time to test different orders of the modifiers, I think I’ve settled on (Control, Shift, Alt/Opt, CMD/Super) for now.

Switching to home row mods showed me that not only was a smaller layout possible, but that I might end up preferring it.

Creating the Layout

After adopting home row mods, I felt comfortable enough to try a 34 key split layout. I started by editing several modified layouts I found online from other people.

Layer Toggle Setup

My first 34 key layout on the ergodox.

Influenced by a popular video on 34 key layouts, I started with a toggle layer switching scheme. If I tapped a thumb key, it would switch to another layer, and remain there until I hit a key to switch to a different layer.

This setup worked okay, but I had a real hard time with the layer switching. I just couldn’t flow nicely while typing. In addition, the rest of the layout wasn’t very well thought out, based on how I use my keyboard. Despite these issues, spending time with this setup did help me iron out my pain points for my future revision.

Callum Layout

The Callum Layout.

While struggling with my first design, I continued to do research. Eventually, I found some references to the Callum Layout and decided to try it with my own spin.

The Callum approach handles layers a little bit differently, opting to instead toggle layers using key holds. Rather than tapping a key and locking on a new layer (like CAPs lock), the keyboard switches to the new layer as long as that layer’s key is held (like SHIFT). So, holding a left thumb key switches to the navigation layer, holding a right hand thumb key switches to the symbol layer, and holding both activates a number layer. This method works surprisingly well. It might sound like you need to be an octopus to use the number layer, but because the layers are controlled using thumb keys, all of your typical keyboard fingers are still available. This layer switching technique worked much better than the toggles for me.

Another reason the Callum layout functions so well is that it allocates home row modifiers on each layer. Specifically, the home row contains normal modifiers that can be tapped or held (well sort of, more on that in the next section). Whichever hand is holding the layer key, is the side that the modifiers are on, leaving the whole other half available for most of the layer’s keys (except the number layer, which has modifiers on both sides). This helps you efficiently use both of your hands.

Lastly, the Callum layout enables me to easily use modifiers in other layers, which I use all the time for navigation shortcuts (ex: CTRL/CMD + -> to shift workspaces, or CTRL/CMD + NUM to switch browser tabs).

One Shot Modifiers

However, note that the original Callum arrangement doesn’t include home row modifiers on the base layer (not even holds). So, you might be wondering “how are the most basic keyboard shortcuts like copy/paste (usually CMD/CTRL+c and CMD/CTRL+p respectively) accomplished with this configuration?!”

Modifiers are still available on the base layer using something called One Shot Modifiers. One shot mods function most of the time as normal modifiers (you can tap or hold them like any other ctrl or shift key), but if you tap them without pressing any other keys, it will be queued up to be used with whatever key is pressed next, even if that key is on a different layer. So, if I wanted to use CTRL-c in the Callumn layout, I could tap and let go of the NAV and CTRL keys, which will then return to the base layout. From there, I hit c on the keyboard, and it’s effectively a CTRL-C. It’s not as complicated as it sounds, but it does take getting used to for every day typing.

My Layout

My 34 key layout.

For my layout, I used the Callum design as my base and modified it to better fit my preferences. Here are some of my choices and alterations:

I used One Shot Modifiers on my additional layers, but also added home row modifiers (hold keys) to the base layer, as I find it’s just easier to work with.
I choose to use Space and Backspace for my two non-layer thumb keys. I tried others like Shift and Tab, but I really had a hard time transitioning from my old Ergodox layout without a dedicated backspace key.
I set my Enter key to under my pinky on the navigation layer, again because this was hard for me to adapt to it anywhere else. (This also dictated that I use an ‘upside down T’ arrow key layout instead of VIM-style)
I TAB a lot, so when I couldn’t have it as a dedicated thumb key, I set it to a pointer finger key that’s easy to remember and ergonomically comfortable.
As a programmer, I configured the various bracket pairs to be accessible and easy to remember. I made [] the easiest to use because I mostly code in python and write notes in obsidian, which both extensively use square brackets.
For most of the remaining symbols, I placed them on the opposite hand’s home row, but in the same order they are on a normal keyboard’s number row, so it’s easier to remember (ex: !, @, #…). For the few symbols that extend beyond the 5 keys (excluding () because they’re already defined elsewhere), I just continued the order in the next row.
For numbers, I learned I actually preferred a number row over a number pad in this type of layout. The top row is not hard to reach, now that I only have to extend my fingers up one row. Additionally, giving that the layout is only 10 keys wide, it’s easy to locate each number. I just count my fingers. This setup also made it easy to incorporate the function keys in the same layer in a meaningful order.

Most of the other keys (but not all) were also thoughtfully placed in the layout, but it’s not worth rambling on any more about each one.

Issues/Future Improvements

Testing 34 Key layouts on my Ergodox.

While the layout works great for me most of the time, there are a few issues and areas I’d like to improve over time:

Some common symbol locations aren’t ideal - I said that most, but not all of the symbol locations were well thought out. Over time, I would like to find better locations for symbols that I use a lot, but don’t feel right while using this layout. For example, ~ and ;.

Bad for gaming - The home row mod’s mean that I can’t hold 3 out of 4 ‘WASD’ keys… so that’s problem. I’m thinking that I might be able to add a gaming layer at some point that I can toggle into when playing games.

I’m not able to do some actions one handed - I can do a few tasks like change the volume with only one hand, but many of my navigation shortcuts require two now. This could be worked around if I really did some usage analysis and thought about it. For now it’s not too much of an issue. Just something to think about improving in the future.

Conclusion

I guess I should cut it here before this post gets any longer. I am not done tweaking my 34 key layout, and probably won’t be for some time, as I want to keep learning more about my options using qmk. However, using the Ergodox with 34 keys over the last few weeks has provided me with the confidence needed to take the plunge and order parts to build a 34 key split keyboard. But more on that later!

Gnome Font Scaling Script

2022-02-28T00:00:00Z

Surf City, NC

Previously, I was using the Pantheon desktop environment on Fedora 35 for my work laptop. I liked how well Pantheon handled font scaling with my 4k monitor, and I figured out how to get the accessability menubar item working in Fedora. This allowed me to easily toggle the scaling factor as I connected and reconnected to different monitors. With that said, I have recently starting using Gnome on the laptop, and miss that feature of Pantheon.

Background

4k Display without scaling.

I have been using Fedora Gnome on my work computer for the last month or so. The font scaling works well enough, and it seems to be a little less resource hungry/buggy than my cobbled-together Fedora Pantheon setup.

Most of the day, my laptop is connected to a 32" 4k monitor, which I need at least 125% scaling, and usually prefer to use 150% scaling. However, when I disconnect the laptop and want to work elsewhere, the 14" 1080p screen doesn’t have enough space at 150%, requiring me to tune down the scaling. I wanted a way to once again easily toggle though different scaling factors, without needing to navigate through the tweaks settings menus (which can also be difficult to do when connecting to a monitor that isn’t properly scaling).

So, I figured out how to script it.

Creating the Script

Changing Font Scaling Setting

First, I had to learn out how to change the scaling factor using the command line in Gnome. It turns out this is an item in gsettings, so it is easy to get the current value, and set a new one. I used these to commands to start my script:

SCALE=$(gsettings get org.gnome.desktop.interface text-scaling-factor)
gsettings set org.gnome.desktop.interface text-scaling-factor $SCALE_SWITCH

The first line grabs the current scaling factor and saves it to the SCALE variable (for use later). The second line sets the scaling factor to the value of a SCALE_SWITCH variable (not defined yet).

Simple Logic

I wanted to be able to toggle between my desktop monitor and laptop setups using a keybinding, but realized I ideally wanted 3 options. So, I used some simple logic to flip through the 3 options in order, which would work just as well:

# Set what to toggle to
if [ $SCALE == '1.0' ]; then
    SCALE_SWITCH=1.25
elif [ $SCALE == '1.25' ]; then
    SCALE_SWITCH=1.5
else [ $SCALE == '1.5' ]
    SCALE_SWITCH=1.0
fi

This code uses the $SCALE value we grabbed in the section above, and matches it to determine what the next value (SCALE_SWITCH) should be.

Notify

Notification message.

Working in the CLI on the script, I liked having a debug message print out so I could see the what scale it was switching to and from. After adding the script to a key-binding, I still wanted that feedback. To accomplish this, I added a notify-send line in the script to launch a Gnome notification when it runs.

This has the potential to be a bit annoying… but it can always commented out if I really don’t like it anymore:

# (Optional) Message intentions to CLI and GNOME Notifications
echo -e "Previous Font Scale: $SCALE, Switched to $SCALE_SWITCH"
notify-send "Previous Font Scale: $SCALE, Switched to $SCALE_SWITCH"

These two lines print out a message to both the command line (first line) and a gnome notification (second line) about what the previous scale was and what it is being switched to.

With the press of a key

Adding shortcut to run the script.

Lastly, I wanted the ability to execute the script by hitting a key binding. First, I made a symbolic link to the script location from /usr/bin/toggle-font-scale:

sudo ln -s /home/ryan/dotfiles/scripts/toggle-gnome-font-scale /usr/bin/toggle-font-scale

Next, I added a custom Gnome keyboard shortcut using gnome settings (for me, CTRL+SHIFT+M) to run the script, which toggles the scaling factor.

Conclusion

After some cleanup, I had my completed script:

#!/bin/bash

# Get Current Scailing factor
SCALE=$(gsettings get org.gnome.desktop.interface text-scaling-factor)

# Set what to toggle to
if [ $SCALE == '1.0' ]; then
    SCALE_SWITCH=1.25
elif [ $SCALE == '1.25' ]; then
    SCALE_SWITCH=1.5
else [ $SCALE == '1.5' ]
    SCALE_SWITCH=1.0
fi

# (Optional) Message intentions to CLI and GNOME Notifications
echo -e "Previous Font Scale: $SCALE, Switched to $SCALE_SWITCH"
notify-send "Previous Font Scale: $SCALE, Switched to $SCALE_SWITCH"

# Run switch command
gsettings set org.gnome.desktop.interface text-scaling-factor $SCALE_SWITCH

It’s a simple solution and I’m very happy with it. I use it all the time,and at this point, it feels like it is just a feature baked into Gnome. I’m glad I took the 20 minutes to figure it out!

Intro to Python Type Hinting

2022-01-30T00:00:00Z

Surf City, NC

I have been trying to code daily the last several weeks. Shortly after starting a project in Python, I decided to refactor it to use Python type hinting. Type hinting is a newer feature of the language that I’ve known about, but hadn’t yet used much. So, I gave it a shot.

Why?

Before getting started, you might be asking yourself “Why bother with types in Python? Isn’t dynamic typing a feature that makes python a great language in the first place?”. While dynamic typing can be easier to code, there are some benefits to a more static typing approach. To name a few:

Static types help clarify what a function expects for parameters, and what it returns. You can look at a single line of code and immediately understand what will be returned by a function, without having to skim through and follow the code.
Static typing increases the power compiling/linting has to find errors before run-time.
It forces the developer to fully think through code design a bit deeper. Having to match up data types when chaining functions involves a bit more planning compared to just throwing generic data around.

Basic Type Hinting

By default, python is a strongly typed, dynamic language. This means that while Python does have types for everything, the system generally does all of this under the hood, without requiring the programmer to define and keep track of the types. However, ‘type hinting’ provides the ability to define types in python code.

For an example, lets start by looking at this ~~stupid~~ simple function. It takes an age for input, and returns a string that states how old you will be in the future (it increment’s the age by 1 and adds it to a string 😑. I know, riveting code here…):

def next_year_age(age):
    new_age = age + 1
    return f"In the future, you will be {new_age}!"

It can be assumed that age is likely an int, and that the return value is a str… but thats not a given. So, lets declare the types using type hinting.

To use type hinting in python, just specify the type when defining variables using the format var_name: type. Additionally, to define a return type for a function, add a -> symbol at the end of the function def pointing to the return type. So, our example function would look something like this:

def next_year_age(age: int) -> str:
    new_age: int = age + 1
    return f"In the future, you will be {new_age}!"

Here we defined that the age parameter, as well as the new_age variable inside the function, are both ints. Additionally, we declared that the function expects to return a string, by using the -> str: in the function definition.

MyPy

When using type hinting, it is strongly suggested to use Mypy, an optional static type checker for python, which can be installed using pip:

pip install mypy

To try mypy out with our example function above, lets assume that it is saved in a file named example.py. To run a basic check:

(post_examples) ➜  post_examples mypy example.py
Success: no issues found in 1 source file

Looks good! Now, lets pretend we accidentally wrote that the function would return a float (even though it actually returns a str):

def next_year_age(age: int) -> float:
    new_age: int = age + 1
    return f"In the future, you will be {new_age}!"

Mypy will catch the error for us:

(post_examples) ➜  post_examples mypy example.py
example.py:3: error: Incompatible return value type (got "str", expected "float")
Found 1 error in 1 file (checked 1 source file)

Strict Checking

Up until now, mypy has just told us if there are any issues with the type hinting we added. It hasn’t been too pushy about ensuring we have types declared for everything.

To have mypy enforce that we write code with un-ambiguous types (and get the most out of static typing), we can use the --strict flag. Looking at our original example again, lets say we didn’t declare a return type at all:

def next_year_age(age: int):
    new_age: int = age + 1
    return f"In the future, you will be {new_age}!"

Running mypy normally won’t care about this omission, because none of the types we did declare are wrong:

(post_examples) ➜  post_examples mypy example.py
Success: no issues found in 1 source file

However, using the --strict flag on the same file will let us know that we forgot to define a return type:

(post_examples) ➜  post_examples mypy --strict example.py
example.py:1: error: Function is missing a return type annotation
Found 1 error in 1 file (checked 1 source file)

More Complex Types

If you need to use compound or more complicated types, you might have to import from the typing module to do so. For example, lets say we want to instead have the function return a list of the two ages. We can do this by importing List from typing, and then define the return type as a list of ints:

from typing import List

def next_year_age(age: int) -> List[int]:
    new_age: int = age + 1
    return [age, new_age]

If we want to instead use a 2-element tuple:

from typing import Tuple

def next_year_age(age: int) -> Tuple[int, int]:
    new_age: int = age + 1
    return (age, new_age)

and so on.

Lastly, to define your own class and type hint it:

from typing import Tuple

class Person:
    def __init__(self, name: str, age: int):
        self.name: str = name
        self.age: int = age

def next_year_age(age: int) -> Tuple[int, int]:
    new_age: int = age + 1
    me: Person = Person("Ryan", age)
    return (me.age, new_age)

mypy can check that as well:

(post_examples) ➜  post_examples mypy example.py --strict
Success: no issues found in 1 source file

The typing module also supports types like the Any, Callable, Unions and others.

Issues

As I progressed further in my project, I started hitting more and more limitations/frustrations specifically related to the type checking. Most often, it was related to the crazy work-around I had to do to support Optional parameters and variables. Optionals are supported, but a bit complicated.

The biggest issue I had was being able to tell the system “I checked, and definitely have this object, it’s not None”. I would check for a value and only run a case on it if it existed, but the type system would still refuse to run logic like < on a Optional(int) type. It didn’t catch on to the fact that I had wrapped the variable in a case that can only occur if the value was an int.

I eventually tried using the returns library, which was great and can apparently help get around these issues by introducing Maybe and Some types, but at that point I realized I had added SO much overhead complexity to a very tiny and simple side project, that it really didn’t make sense.

Conclusion

Type hinting in python is probably best if you want to sprinkle it throughout the code without --strict checking, or if you have something where everything is rather straight forward and there aren’t many ‘optional’ variables defined and passed around. For projects where you want really strict type checking… you should probably use a different language (ex: Rust or Go) instead of forcing Python to be something it wasn’t designed to be from the start.

Still, I appreciate that type hinting exists in Python and I think I’ll use it throughout my projects more often. If you haven’t used it before, I recommend giving it a try!

Traded in my M1 MacBook Air for the 14" MacBook Pro

2022-01-23T00:00:00Z

Durham, NC

It’s been about a year since I traded my 16" Macbook Pro for a M1 Macbook Air. Well, I’ve done it again. Three months ago I decided to trade in my M1 MacBook air for a 14" M1 Pro Macbook Pro (Ugh, these names! 😂). Two months ago, the 14" was delivered and I finally made the physical swap. Similar to last time, I didn’t initiate this trade because I had any issues with my current MacBook, or even needed the newer one. But once again… I’m glad I did it.

Background

My previous MacBooks: the M1 MacBook Air (left) and the 2019 16" (Intel) MacBook Pro (right).

When rumors of new Apple Silicon MacBooks Pros started to surface, I really wasn’t too tempted to swap. I have been very happy with my M1 MacBook air. Having said that, as I watched Apple announce the new Pros… it occured to me that the 14" pro was likely my ideal MacBook. A perfect mix of all the things I loved about my m1 air, while adding the few things I missed from my 2019 16" Intel-based Macbook Pro.

What I’ve previously said

To showcase this mingling, lets go through the list of what I have said about my previous two MacBooks, and see how the 14" (m1 pro) stacks up against them:

Keyboard

The 14" M1 Pro's Keyboard.

The 14" Pro has basically the same keyboard as both the 2019 16" and the m1 air. The only difference is that the background behind the keys is all black, which I think looks nice. As on the m1 air, the 14" has a row of physical function keys (full sized this time), which I prefer over the touchbar that my 2019 16" had.

Trackpad

The trackpad remains the same, which is a good thing. It’s a different size than the previous two devices (which makes sense), but it seems to fit this device quite well.

Display

The 14" M1 Pro's Display, including the notch.

While still smaller than the 16", the 14" is a small but noticeable improvement over the 13" m1 air. Additionally, the smaller bezels help minimize any compromises on portability due to the size increase. This size maximizes screen space, without being too unwieldy.

With the new XDR display, the 14" is also technologically better than the previous two. It has a 120hz refresh rate, with 1000 nit sustained and 1600 nit peak brightness when watching HDR content (500 nit peak during normal usage). With 10,000 mini-LEDs, the contrast is also much better (blacks look black, not grey). Lastly, the default resolution is a proper 200% scaling of native (which I’ve been critical about on previous Macbooks). So out of the 3 laptops, the 14" easily has the best display.

‘How can you say that! The 14" has a notch!’

So what.

While not ideal, Apple honestly did a great job making the best of the situation by setting the space under the notch to a 16:10 screen ratio. This means the areas on either side of the notch are extra screen space. If you absolutely hate the notch, you can black out the top part of the screen and have the screen from before (including a massive top bezel). There really isn’t much reason to complain.

Speakers

The speakers on all these laptops are great for laptop speakers. However, it was a real downgrade going from the 2019 16" Pro to the m1 Air. The 16" had amazing speakers. I’m not sure if the 14" speakers are better than the old 16" speakers (I’m assuming not), but they are much better than the air and something that I am very glad to have back.

Ports

The 14" M1 Pro SOC supports two external displays (in addition to the internal display and sidecar).

This was another sacrafice when switching from the old 16" to the m1 air. The 16" had 4 thunderbolt ports, while the m1 air had 2. Thats it. The 14" by comparison has 3 thunderbolt ports… but also a mag safe charger (so the charging cable doesn’t have to consume one of the thunderbolt ports). Additionally, the 14" has an HDMI port, and an SD card reader.

Another connectivity problem on the M1 Air was that it only supported 1 external monitor. While the the base 14" doesn’t quite support the 2019 16"'s 4 4k displays (or 2 6k), it does support up to 2 (6k@60Hz) external displays (note, if you get the M1 MAX chip, that supports 4 external displays but I wasn’t considering that configuration). So, while the 14" M1 Pro doesn’t connect to the most devices, it’s small improvement over the air allows it to fully support my configuration.

Battery Life

The 14" might not beat the M1 air here, but it is more than good enough, and substantially better than the old 16" was. I honestly don’t know how it compares to the air because both exceed my needs.

Portability

Dispite being slightly larger than the M1 air, the 14" M1 Pro is still a very portable workstation.

The 16" is obviously the least portable. That’s a given.

Comparing the 14" to the air is quite interesting though. Yes, it’s a bit heavier and boxier than the tapered body of the air… but it’s in essence just as portable. It’s 3.5 lbs (which is worth it for how solid it feels), with about the same foot print. It’s technically less portable, but in reality… it’s still a very portable device.

Performance

The m1 air was surprisingly on par or better than the 2019 16", at least for the tasks I did. The 14" improves on that. The base model 14" has the same number of CPU cores, but with 2 efficiency and 6 performance cores (compared to the 4 and 4 on the M1 air), which helps increase performance a bit. Additionally, the 14" has almost double the GPU cores (14 vs the 8 that my upgrade air had), which is very noticeable. Lastly, the 14" has crazy fast disk and memory speeds, which contributes much more to general performance than people give credit to.

Heat & Noise

Most of the time, the fans sit at **0** RPM. When they do turn on, I can't hear them.

Again, the 2019 16" is the clear looser here. It was hot, and had noisy fans that ran all the time.

Comparing the 14" to the fanless m1 air… they’re the same. Seriously.

I guess the 14" technically can get louder than the air, but I have hardly noticed the fans turn on, and when they do the only way I know is by reading sensor data. If anything, it means the 14" M1 pro stays cooler than my air did, while still remaining just as silent.

Deciding on the trade

In addition to the 14" being the perfect combination of everything I loved about my previous 2 MacBooks, there were a few other factors that eventually led me to make the trade-in.

Two ‘Upgrades’ for One

When looking up the trade-in value for my M1 Air, I noticed it was only valued about $50 more than my wife’s base model M1 Air (which has half as much RAM and SSD storage). With that, I realized that we could trade in my wife’s computer and “spend” that $50 lost in trade-in value to “upgrade” her M1 Air to 16GB RAM and 512GB storage (which is an amazing value for Apple hardware!). So, we were able to both upgrade our devices with this single trade-in.

Daily Driver

My MacBook setup at my desk. It's become my daily driver computer.

The final reason for this trade-in, was that after switching to my M1 Air, I started daily driving my Mac again… and didn’t stop.

I have used all my MacBooks as my portable device. However, I started using the air as my main computer, even at my desk. I still have my desktop for when I need or want to run Linux, but honestly, most of the time when I’m not working, it’s shut off. I have started to enjoy using MacOS for my desktop computing needs, and Linux for all my sever ones. Another perfect combination.

Being my main computing device, I wanted something that had just a little bit more to offer than the M1 air. And the 14" does. It has amazing interface specs (display, speakers, keyboard, mic), a little bit more GPU power (which I was hitting the limit of on the Air), and still silent fans for when I do push the CPU.

Conclusion

My delivered, base model, 14" MacBook Pro (M1 Pro).

With that, I traded in the m1 air. I decided on the cheapest base model (8 cpu cores/14 gpu cores, 16GB RAM, and 512GB SSD) to not only keep the price down, but also for an easier trade in the future. As I learned during this process, base model devices have a much better value when using Apple trade-in (which is convenient). Once I prove I can keep an MacBook for more than a year… I’ll consider getting devices with upgrades again 😆.

I also selected the base model because, again… I didn’t need to trade the air. It is an amazing computer. However, the 14" seemed to solve the very few problems I had with both the 16 and air, giving it the potential to be my ideal laptop. And so far, it has been.

Still, for most people… just get the MacBook Air. You won’t be let down.

Pantheon Desktop on Fedora 35

2021-12-31T00:00:00Z

Surf City, NC

I recently wrote about how I started using ElementaryOS 6 on my work computers. While I love using elementaryOS, one thing I mentioned in that post still holds true: right now, my ideal Linux setup would be the ElementaryOS UI/UX, but on top of a Fedora base. So, this month I set out to achieve just that.

Background

Elementary OS 6 with several applications opened.

The more I used ElementaryOS for work, the more my desire to have a Fedora base grew. This was especially true for my laptop, as earlier this year I choose the portable option for my laptop refresh. By itself, it is fine. However, it is annoying to partition the laptop’s limited resources for a fedora VM. Running Fedora as the host OS (instead of Elementary) would make it easier when I only have my laptop available (in the office, for example).

But… I still love the pantheon desktop environment. It’s clean, looks great, and has wonderful support for global font-scaling (which is the best option I’ve found for using a 4k monitor with ‘fractional scaling’ on Linux).

I remembered there being a pantheon desktop group in the dnf group list options a bit back, but there isn’t a dedicated Fedora Spin for pantheon. So, here’s how I cobbled together my own solution.

Installing Pantheon Desktop Environment

First, I used dnf to install the Pantheon Desktop package group:

sudo dnf group install `Pantheon Desktop`

That simple step took care of most of what I needed. After installing the group, you should be able to reboot, and select the pantheon desktop environment at the login screen.

Swapping gdm with lightdm

Lightdm Display Manager in Fedora.

Speaking of the login screen…

After installing pantheon, I wanted to take an additional step of switching the display manager (the login screen). Specifically, I swapped gnome’s gdm for lightdm, the display manager that elementaryOS uses. To do this, I first made sure lightdm was installed:

sudo dnf install lightdm

Then, I disabled gdm and enabled lightdm:

sudo systemctl disable gdm
sudo systemctl enable lightdm

After rebooting, I as greeted with a lightdm login window, with the same layout and theming I had on elementaryOS!

Building additional packages

As I started working in pantheon on Fedora, I did notice that there seemed to be a few things were still missing…

Pantheon-Tweaks

Pantheon Tweaks Menu.

First, I wanted to install pantheon-tweaks. Like the ‘Tweaks’ tool for Gnome, this adds a few more settings to further customize pantheon. However, pantheon-tweaks doesn’t appear to be in the Fedora repos, and I could only find deb builds of it. So, I built it from source.

First, I downloaded the source code and entered the directory:

git clone https://github.com/pantheon-tweaks/pantheon-tweaks.git
cd pantheon-tweaks

Next, I determined what the required build dependencies were (and named), when building in Fedora. After some sleuthing, I had the list figured out:

sudo dnf install cmake vala ninja-build meson switchboard-libs switchboard-devel switchboard-plug-a11y granite-devel

With all the dependencies installed, I was ready to run the build and install commands:

meson build --prefix=/usr
cd build
ninja
ninja install

After that, I had the Tweaks page available in the system settings.

Universal Access Settings in Wingpanel

Pantheon Universal Access Settings Page.

Next, I wanted to flip the switch to show the Universal Access settings in the top panel. However, every time I clicked on the Universal Access settings tab, my settings app crashed! By launching the settings application from the command line (io.elementary.switchboard), I was able to observe it was crashing because wingpanel-idicator-a11y wasn’t installed. So, I found that on Github (I love open source!), and prepared to build it:

git clone https://github.com/elementary/wingpanel-indicator-a11y.git
cd wingpanel-indicator-a11y

In addition to the build dependencies I had already installed for pantheon-teaks, I also needed to install the wingpanel libraries:

sudo dnf install wingpanel-devel

Then, I was able to build and install, just like before:

meson build --prefix=/usr
cd build
ninja
sudo ninja install

Once installed, I could view the Universal Access settings, and even enable the dropdown setting in the top panel! (which I use to quickly change my font scaling)

Concerns

Pantheon Settings Menu.

So far, this setup has worked surprisingly well. However, I do have one main concern about the viability of using this long-term…

Will this continue to work? If so, how much of a pain will it be to maintain? I had to build some packages from source, and while I can pull the latest and rebuild, I worry that they might have version compatibility problems with the packages I’m able to pull from the repos, which would be an undesired headache.

I would love to see a team in Fedora grow to make this an official and maintained spin. I saw that there is a pantheon sig, but I’m not sure how current or active it is…

Conclusion

Fedora 35 running the Pantheon Desktop.

Thus far, pantheon on Fedora 35 has worked fine. I’m going to try to keep using it over the next few weeks, and keep an eye on it. In the best case scenario, maybe it will be an offical spin one day. At the very least… I hope my cobbled together version stays stable enough for me to continue using!

Bitwarden CLI

2021-12-10T00:00:00Z

Linville Falls Trail, NC

I have used Bitwarden for a couple of years now, but I surprisingly haven’t utilized the CLI tool for anything. Recently, I was coding a script that needed to store an API auth token to an environment variable, but I didn’t want the token to be stored in plaintext. So, I setup and used the Bitwarden CLI for the first time to solve my problem!

CLI Install

To install the CLI tool (bw), go to the CLI Documentation page. There is a native executable for each major OS, that you can download and install.

Linux

In Linux, I first installed it using node. After installing npm on my Fedora machine, I was able to install bw using the command:

npm install -g @bitwarden/cli

To instead use the prebuilt Linux executable (which I tried later), first download the executable package and unzip it. Next, give the bw file executable permissions (chmod +x bw). Lastly, move or link the bw file to a location in your $PATH variable (ex: sudo ln -s /home/Ryan/Builds/bw /usr/bin/bw).

Mac

I also tested this out on my Mac. Here, I used the install method that I always try use when on macOS: Homebrew:

brew install bitwarden-cli

Login and Sync

With bw installed, you must first login with your account:

☁  ~  bw login email@provider.com
? Master password: [hidden]

Despite what the docs say, you don’t need to provide your password as part of the bw login command. If it is omitted, bw will simply prompt you for the password, which is what I recommend doing, so that you don’t have your password stored in the shell history 😉.

After logging in, you should be greeted with a similar message:

You are logged in!

To unlock your vault, set your session key to the `BW_SESSION` environment variable. ex:
$ export BW_SESSION="WbiVLHrBEb2ktU7xVB4h1y40DhgFwV+vJ/lBw6EFk23wCxhUWsMTasXlXI3OEBS6xwaG638MepXdS7u1+FysNg=="
> $env:BW_SESSION="WbiVLHrBEb2ktU7xVB4h1y40DhgFwV+vJ/lBw6EFk23wCxhUWsMTasXlXI3OEBS6xwaG638MepXdS7u1+FysNg=="

You can also pass the session key to any command with the `--session` option. ex:
$ bw list items --session WbiVLHrBEb2ktU7xVB4h1y40DhgFwV+vJ/lBw6EFk23wCxhUWsMTasXlXI3OEBS6xwaG638MepXdS7u1+FysNg==

The message provides the option of exporting the Bitwarden session token as an environment variable, or appending it to your bw commands using the --session flag. You don’t have to include the session token when using bw, but it will make it much less annoying to work with. If the token is not defined or included, it will prompt for your password with each command.

Also, it’s probably a good idea to run a bw sync after logging in, just to make sure everything is up to date.

Getting a Value

Now that we are logged in, lets work on grabbing some data from our vault. The bw list command is used to list out contents from the password vault. This requires an object to be specified, which can be items, folders, collections, org-collections, org-members, and organizations.

Flags, like --search can be used to help narrow down the wall of results you get. For example, I can use --search to narrow down the results to the specific ‘post-test’ object I want.

☁  ~  bw list items --search post-test
[{"object":"item","id":"aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa","organizationId":null,"folderId":null,"type":1,"reprompt":0,"name":"post-test","notes":null,"favorite":false,"login":{"username":"ryan","password":"terrible-password","totp":null,"passwordRevisionDate":null},"collectionIds":[],"revisionDate":"2021-11-30T11:44:25.846Z"}]

Use in Automation

Once we have found an object we want, we can use it’s id to reference it directly, instead of searching.

☁  ~  bw get item aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa
[{"object":"item","id":"aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa","organizationId":null,"folderId":null,"type":1,"reprompt":0,"name":"post-test","notes":null,"favorite":false,"login":{"username":"ryan","password":"terrible-password","totp":null,"passwordRevisionDate":null},"collectionIds":[],"revisionDate":"2021-11-30T11:44:25.846Z"}]

Knowing the id is helpful when setting up automation. Additionally, the object data is returned as json, so it is easy enough to parse.

As an example, if we were writing a script and wanted to use Bitwarden to supply my post-test password, we could grab it with this one-liner combining jq to grab the value ("terrible-password") from the json output:

☁  ~  bw get item 937b567a-a8a4-4ede-9fc1-adf000c17a4b |jq '.login | .password'
"terrible-password"

Conclusion

I’ve used bw once or twice at this point for automation scripts, and it has worked great! One of the reasons I choose Bitwarden when I last switched password managers was because it also had a CLI client. I’m glad to have finally used it!

Obsidian Basics

2021-11-28T00:00:00Z

Linville Falls Trail, NC

Over 7 months ago, I switched to Obsidian and I love it. Obsidian’s versatility allows it become whatever I need it to be. However, this adaptability comes at the cost of an initial learning curve, which can be daunting for new users. While I can’t setup Obsidian for you, I can at least help with those initial steps by explaining what Obsidian is, and how to navigate around it.

What is Obsidian

Obsidian Graph Viewer.

On the project’s website, it is referred to as a “second brain”, and summarized with the following statement:

Obsidian is a powerful knowledge base on top of a local folder of plain text Markdown files.

At it’s core, Obsidian is simply a markdown editor for your notes. What shapes Obsidian to be different compared to other markdown editors, is that it was designed, and continues to be developed, around 3 fundamental directions (again, from the projects website):

Local-first and plain text

Link as first-class citizen

Make it super extensible

In a world where nearly everything is hosted in the cloud, this first direction reinstates control and portability over one’s notes. While the user can still choose to utilize the cloud to sync or backup notes, the raw data is still under your control. This directly addresses the main reason why I left Notion.

The second direction is what makes Obsidian stand out from other markdown editors. It’s superb support for easy linking and navigating notes raises Obsidian to the top tier of note taking tools. This also enables the use of alternative note-taking systems, like the Zettelkasten method.

Lastly, the third direction (extensibility) is the dream of any power user, and what allows Obsidian to function however you need it to. It also enables a community to blossom around Obsidian, ensuring it’s future growth and success.

Obsidian Vaults

Obsidian Vault Setup Screen.

The first thing to know when getting started, is that obsidian notes are organized into something called vaults. This is a collection of files and folders that make up an obsidian notebook. Multiple vaults can be defined, and the user can switch between each vault as they wish.

When starting Obsidian for the first time, you will be prompted to define a vault location.

Markdown

A note in Obsidian. Left pane is edit mode, the right pane is the rendered preview.

After creating a vault, you can start writing note files. Notes are written in markdown, which is easy to learn, but very powerful. It is a plaintext file that supports formatting with simple markup syntax. For example italic (*italic*), bold (**bold**), and ~~strikethrough~~ (~~strikethrough~~) are all supported. Beyond that, features like web-links ([link](url)), inline and full code blocks(`code`), quote blocks (>This is my quoted text), and internal links (more on this next) are also supported.

Using markdown means that all the notes/data, are in a human-readable and very transferrable format. If you want to read them without using obsidian, you still can by using any other markdown reader (of which there are many), or by simply looking at the raw file. If you decide to switch applications to manage your notes, markdown will be easy to transfer/convert to whatever new system you want.

[[Links]]

An Obsidian note that links to many others (left pane). Right pane shows the *local* graph view of that note's links.

Obsidian is built on the concept of connecting everything using links. This is done using the syntax [[note name]], or [[note name|link text]]. A few things to know about Obsidian links:

You only have to use the name of the note to link, and not the absolute/relative path to it. Obsidian will keep track of where the files actually located. You just worry about connecting everything.
Links don’t have to point to a page that is already created. If you click a link to a page that doesn’t exist yet, it will create that note in your default folder. Obsidian does keep track of all connections between notes, even if the page isn’t created yet. If that note is ever instantiated, it will already have all of the previous back-links to it already setup.

Speaking of back-links…

Back-links

The back-links pane for an Obsidian note. This displays all the linked references *to* the opened note.

When you link to something, that connection is created bi-directionaly. For any given note, you can open up the side pane and see every time another note has linked to the current one.

Combined with something like the ‘Daily notes’ plugin, this is very useful. For example, when I work on a project, I usually mention (and thus link to) the project’s page in my daily note. This allows me later look at the backlinks for that project note, and easily see all the days I worked on it. Additionally, I often define a Logs section in my project notes. If I sub-section the note using links to the daily notes, it has the same effect (linking days to projects, like I did in Notion).

#Tags

A note that uses `#tags` in the header, and the Obsidian tag pane opened on the right.

In addition to links, #tags can also be used while writing notes. There is a tag pane that displays all the tags with the number of occurrences for each one. When a tag in the pane is selected, it pops open the search results for that tag.

Personally, I use tags categorize and mark status on items (ex: #archived or #today). This makes it easier to search and maintain my notes.

Organization (Files & Folders)

Obsidian Files and Folders Navigation.

Like most note applications/computers in general, Obsidian can organize files into folder hierarchies. Notes can be placed in folders, and those folders into other folders.

However, I recommend not being too worried about folder organization in Obsidian compared to other applications you might have used. Focus more on grouping notes using #tags or links, rather that in folders. I know it is a scary thought, but trust me, it will work out.

Searching/Navigation

Obsidian Search Menu.

One reason forgoing a large folder hierarchy in Obsidian is possible, is that the search functionality works great. This is how most navigation will happen. Simply hit the keyboard shortcut (CMD/CTRL-o), and start typing. The pop up window will filter out the results as you type. If the file doesn’t exist, you can create a new one from the navigation.

Plugins

Installed Community Plugins.

One of the fundamental goals of Obsidian is to “Make it super extensible”. This is achieved via plugins.

There are two types of plugins: Core and Community. Core plugins are built into the system, while community plugins are created and maintained by… you guessed it, the community. Plugins allow the user to mold Obsidian into whatever they want.

For example, I use the Day Planner and Templates plugins to customize my setup for keeping daily work notes and logs. I’ve also used a Kanban plugin to better visualize monthly goals.

Tips/Suggestions

I like to set keyboard shortcuts to easily toggle the side bars.

That should be it for the basics. Before I wrap up, here are just few tips and suggestions I have for using Obsidian:

Header information can be used to link up your notes using tags and/or ‘Associations’

If there are other notes that you know will be associated to others, add an ‘Associations’ header at the top. For example, all off the notes I have about posts for this website contain a Associations: [[my website]] as well as a Tags: #post_idea, #website line in a header at the top of the note.

Don’t be afraid to dump most of your notes into one folder

If you’re adding links and utilizing the search, you shouldn’t have any problems locating notes, and it will make it easier to navigate if you ever view the notes in something other than obsidian (ex: Gitlab/Github), as the links will be better supported.

Use keybindings and setup your own.

A few of my suggestions:

CMD+ [/] to toggle opening and closing the side panes
One to insert a timestamp (I think this might be through the Today/Planner plugin?)
If you use the Daily Note plugin, set shortcuts to navigate between the previous and next day
Switch the theme to light/dark modes

Use some plugins

A few core ones (that I don’t think are on by default) that I highly suggest enabling:

Templates
Daily Notes
Word Count
VIM Mode – if you use vim (not a plugin, but a setting)

A few of the community ones I love:

Obsidian Git
Calendar
Day Planner
Kanban

Save/sync your `.obsidian` folder somewhere

This is useful if setting up on multiple machines, or just in case your setup gets whipped out.

Conclusion

My Obsidian Setup.

I could go on for hours about the details and intricacies surrounding my obsidian use over the past months. However, that wouldn’t be very helpful. At some point, you need to just dive in and start using Obsidian for yourself. Hopefully, I’ve covered enough that it shouldn’t be too hard to get started now. Enjoy!

Install Podman on M1 Mac

2021-10-31T00:00:00Z

Blowing Rock, NC

Ever since I started using macOS, and especially after purchasing an Apple Silicon Mac, I have been closely watching podman development around Mac support. After waithing for several patchs to hit upstream dependency projects, everything seems to now be in order. As a result of that work, I finally have podman running on my M1 MacBook Air.

Background

As a Fedora user, I love how well it supports podman. Over in macOS however, I was forced to use docker to run containers, as podman wasn’t well supported yet. This was particularly true for apple silicon Macs, like my m1 air.

Docker Desktop Application.

Additionaly, Docker reacently announced that ‘Docker Desktop’ would no longer be free for large enterprises. For mac users, the solution to this problem isn’t as simple as “just don’t use the desktop app”. In macOS, docker only runs using the desktop app. It is not possible to install just the command line service like on Linux (at least that was my understanding).

To be clear, this change is only for enterprise users. Still, it has prompted many people to look closer at alternatives (finally 😆). Right around the time of this announcement, I noticed much of the updates the podman team had been waiting for upstream, started to release.

So, I gave it a shot, and finally got podman working on my M1 MacBook Air!

Install

Before getting started, first make sure you have Homebrew installed. Then, install podman:

brew install podman

Note: I think you also need qemu and other dependencies installed, but homebrew should take care of that.

Podman Machine

In order to run containers on macOS, podman needs to install a Linux VM to run the containers in (I think docker does something similar on macOS). This is what podman machine is for. To create the VM, initalize podman machine:

podman machine init

Once the VM is created, start it up:

podman machine start

Note, the vm will have to be started to use podman, so this step might be required after a reboot.

Side Note: I had to use the `Next` branch VM

At the time, I had issues with my podman machine setup. I had the required version of podman (3.4.0) installed on my mac, but the Fedora CoreOS image podman machine pulls down for the VM didn’t contain that verson of podman yet. However, the upcoming release did (Fedora-CoreOS pushes an update about every 2 weeks), so I was able instructuct podman machine to use the next branch for the VM image, which worked.

This shouldn’t be an issue anymore, as it has been a few weeks since I initially did this and the current Fedora CoreOS release seems to have the correct version of podman.

To verify, you can ssh into the VM using:

podman machine ssh

and then check the podman version installed inside the VM:

[core@localhost ~]$ podman --version
podman version 3.4.0

This should be at least 3.4.0.

Running Some Containers

To test out podman, I spun up a fedora container and played around in it a bit:

podman run -it --name fedora --network bridge quay.io/fedora/fedora

Inside it, I ran some updates and installed a few packages. Everything seemed fine. Later on after my macbook had rebooted, I ensured that I could still start and podman exec into the same container:

podman start fedora
podman exec fedora /bin/bash

… which also worked perfectly!

Conclusion

Fedora Podman Container running on my M1 MacBook Air.

While it feels like it’s been a long time coming, I am so glad to have podman working on macOS, and specifically on Apple Silicon. This is important to see, especially at a time when Apple is pushing out such impressive hardware. It will be interesting to see if there cotinues to be an increasing shift towards podman over the next few years. I imagine being able to install it locally on Macs can only help…

Pytest Parameter Tests

2021-10-21T00:00:00Z

Moses H. Cone Memorial Park, Blowing Rock, NC

Recently, I’ve been converting a small python script I wrote into an organized project. As part of that work, I started to setup an automated testing framework (like a good QE 😆). While writing tests, I have been parameterizing the code so that each test function can be provided multiple inputs, resulting in multiple tests (based from the same function). I talked about this previously, but used a different method then. This time, I opted to try the parametrize marker.

Conftest.py Method

The last time I wrote about pytest, I used parameterization. However, I implemented it a bit differently, using fixtures defined in the conftest.py file to do the parameterization.

For example, this fixture function loops through a list of post names, and returns the relative url for each post:

@pytest.fixture(params=POST_NAMES)
def post_url(request):
    """Returns the post urls for testing."""
    return BASE_URL + "/post/" + request.param.lower()

Next, I used that fixture as the single parameter in my test function:

def test_post_served(post_url):
    """Checks that the desired posts are available"""
    response = requests.get(post_url)
    assert response.status_code == 200

This causes the test to loop across each item the fixture returns from the POST_NAMES constant list, resulting in a test run for each one.

Using Pytest Parameterize

While that method works, tests can be parameterized without the overhead of setting up fixtures for each test data set. Instead, the parametrize pytest marker can be used.

I started by creating my set of test inputs:

example_resolution_ratios = [
    (1920, 1080, Fraction(16, 9)),
    (2560, 1440, Fraction(16, 9)),
    (3840, 2160, Fraction(16, 9)),
    (1440, 900, Fraction(8, 5)),
    (1600, 900, Fraction(16, 9)),
    (5120, 2880, Fraction(16, 9)),
    (6016, 3384, Fraction(16, 9)),
    (3440, 1440, Fraction(43, 18)),
]

(Note: I did this step in the previous method too. That’s what POST_NAMES was)

Next, I added the @pytest.mark.parametrize marker to my test function:

# Example of parameters marker
@pytest.mark.parametrize(
    "horz_pixels,vert_pixels,expected_ratio",
    example_resolution_ratios,
    ids=[f"{pair[0]}x{pair[1]}" for pair in example_resolution_ratios],
)
def test_resolution_ratio(horz_pixels, vert_pixels, expected_ratio):
    """Tests resolution ratio with set of known calculation results."""
    assert resolution_ratio(horz_pixels, vert_pixels) == expected_ratio

While this marker looks intimidating (It’s larger than the actual test code!), it does a lot for us. It requires at least two parameters, a string containing parameter names to pass to the test ("horz_pixels,vert_pixels,expected_ratio" in this example), and the list of test items to iterate over (example_resolution_ratios). Notice, if the test items are a list of tuples, pytest can match each tuple value to a different parameter to pass to the test function.

Additional options can be provided to the parametrize marker. For example, I used the ids parameter to define how each parametrized test instance is identified when run. This test function verifies a ratio calculator, so using a list comprehension, I was able to have the resolution used for each test displayed in the run output:

...
/tests/test_calcs.py::test_resolution_ratio[1920x1080] PASSED [  3%]
/tests/test_calcs.py::test_resolution_ratio[2560x1440] PASSED [  6%]
/tests/test_calcs.py::test_resolution_ratio[3840x2160] PASSED [ 10%]
/tests/test_calcs.py::test_resolution_ratio[1440x900] PASSED [ 13%]
/tests/test_calcs.py::test_resolution_ratio[1600x900] PASSED [ 16%]
/tests/test_calcs.py::test_resolution_ratio[5120x2880] PASSED [ 20%]
/tests/test_calcs.py::test_resolution_ratio[6016x3384] PASSED [ 23%]
/tests/test_calcs.py::test_resolution_ratio[3440x1440] PASSED [ 26%]
...

This is extremely helpful when a few tests are failing, as you can easily identify what specific inputs are failing.

Conclusion

That’s really all there is to it. If you’re using pytest and want to parameterize the test functions, the parametrize marker is a simple, but powerful tool to do so. We use it often at work for some complicated setups, but I enjoyed getting to try here in a much simpler case!

Importing ZFS Pools on Unraid

2021-09-30T00:00:00Z

Moses H. Cone Memorial Park, Blowing Rock, NC

I’ve been running Ubuntu on my home server, mostly because it supports running zfs out of the box. ZFS has been my filesystem of choice for our home server for years. However, I’ve recently had the itch to switch away from Ubuntu and move to Fedora or CentOS. While comparing distros, a major consideration was if I would be able to transfer my zpools, or if I’d have to wipe and restart. I’ve now switched to Unraid… and got to keep ZFS. Here’s how.

Background

While waiting for the centOS stream 9 to release, I started looking at other distributions built specifically for NAS/Server use. First, I checked out how Truenas is doing these days. While Truenas has great ZFS support (it’s built around it), its VM/container support might be a bit lacking for what I want. I’ll admit, the new TrueNAS Scale alpha caught my eye, but it is still in development and intended for much larger setups than my single HP Microserver.

Next, I decided to look at another home NAS OS I know of, Unraid. By default, Unraid doesn’t use ZFS, but has a much better framework around running VMs and Container applications. In addition, it seemed like creating NFS/SMB shares should be easy, which was my main motivation to try a NAS distro instead of vanilla CentOS/Fedora installs. The downside of Unraid, was that I was afraid I might have to forgo ZFS and wipe my data pools.

The available ZFS plugins to install via the Community App Store.

Fortunately, after reading around, I learned that there are plugins to enable ZFS on Unraid. I decided to at least try it. If ZFS didn’t work, then I could switch to wiping the disks and starting from scratch.

Disclaimer: This might not be the best practice, or even a correct way to do use ZFS on Unraid. It is just how I did it. It is working so far, but I have no guarantees on the long term stability of this setup.

Installing the Plugins

When I first booted into Unraid, I still needed to provide a disk to be used for the array (Note: this might change. See this LTT video to hear more). I started with a test SSD connected via USB, but quickly switched to using the real SSD I have in the sever containing the Ubuntu install (no turning back huh?). After assigning the SSD, I started the array and was able to install plugins.

I started by installing the Community Applications Plugin. I went to the Plugins page in Unraid, clicked the Install Plugins tab, and entered the url for the plugin found on the Community Applications Webpage post. Afterwards, had a new Apps page listed in my Unraid navigation bar.

To Install the ZFS plugins, I went to the Apps page, and searched for “ZFS”. Several plugins appeared, and I installed both ZFS and ZFS Companion.

Importing the ZPools

My ZFS Pools imported, and data available.

After installing the ZFS plugin, I jumped to the command line and poked around. Sure enough, I now had zfs and zpool commands I could use. First, I checked to see what pools the system detected by running zpool import. It returned both my Backups and Data pools, so I proceeded to import them:

zpool import Backups
zpool import Data

With that, the pools were mounted to their normal locations, with all the data intact. On top of that, the ZFS companion widget in my dashboard also listed the two pools, and their status. Much easier than I anticipated.

Linking Shares

I used symbolic links to point Unraid shares to my mounted ZPools.

The next step I wanted to tackle was creating Unraid shares, configured with samba so that my wife and I could mount them on our computers. While making the shares… I didn’t know how to instruct the system to use disk locations on my zpools…

After some investigation, I learned that each share was being created as a directory under /mnt/user/. So, my immediate ‘solution’ was to simply remove each share folder (they were still empty), and replace them with symbolic links to the desired directory on my mounted zpools. For example, my Music share at /mnt/user/Music/ points to /Data/Music/. I didn’t see any alternative solutions online, and so far… it seems to work 🤷🏻‍♂️. I was able to mount and access all the shares from my desktop and laptop computers, so… that’s it I guess?

Issues

I did hit one issue during this process that scared me. Later that night after a reboot, I noticed my containers weren’t starting. Then I saw that the pools weren’t mounting on boot, and worse of all… I couldn’t find them when I tried to import. I thought the system had eaten my data.

Then I remembered that while I was poking around the motherboard settings to switch the default boot drive to the Unraid USB, I also enabled SATA IOMMU (because why not?). Well, ZFS did NOT like that change and refused to recognize my drives with it enabled. Disabling the setting resolved the problem and my zpools auto-mounted on the next boot. All good.

Conclusion

The ZFS Companion shows the health of the ZFS Pools in the Unraid Dashboard.

Thus far, things seem to work well. While ZFS pools aren’t as nicely integrated into the UI compared to the Unraid Array, it isn’t a deal breaker for me. I was previously working with ZFS 100% in the CLI, and I’m fine doing it now. As long as the system persists through updates, and my symlinks hold up, I should be fine. I guess we’ll see…

Trying ElementaryOS 6

2021-09-25T00:00:00Z

Moses H. Cone Memorial Park, Blowing Rock, NC

The recent release of Elementary OS 6 (Odin) really piqued my interest. It has been a few years since I last tried Elementary, (probably around the previous release), and I was curious to see all the improvements that the team have made. So, I installed it to my test laptop… and I was not let down.

Background

Elementary OS 6 with several applications opened.

I’ve used Elementary OS in the past, particularly as a solid Ubuntu-based option. Lately, I have increasingly enjoyed the core system of Fedora so much, that I am rarely persuaded to leave my podman-enriched, selinux-secured distro of choice to try any flashy new Ubutnu (or even Arch) based ones.

With that said, being an “on the edge” distro, the Fedora desktop can sometimes have a few niggles to deal with. It might be a screen-sharing issue with Wayland, sound devices not suddenly not reliably switching in the settings, or bluetooth being finicky after an update. None of these are show-stoppers, and are the small price of being on the latest and greatest, but they can quickly become tiresome during the workday.

To poke around on something beyond Fedora, I installed ElementaryOS 6 onto my test laptop (my T470 Thinkpad)… and I loved what I saw (quite literally).

What I like

Appearance/UX

Close terminal tab notification window.

The first thing I love about ElementaryOS is its appearance, and I don’t just mean “it looks pretty”. The tiniest details of the design are all well reasoned and executed. The window drop shadows, color selection, and even fonts all combine to create a desktop environment that oddly feels easier to use. For example, while messing around in the terminal app, I attempted to close a tab while htop was running and had a notification window pop up to warn me that the htop process would end when the tab is closed. When I saw it the first time, I was very impressed with just how clean the pop up looked. The elementaryOS desktop has a level of polish that, while getting better, can still be quite rare on the Linux desktop.

Additionally, I found the fonts sizing and UI to be exactly what I want. I hate when window chrome has giant gaps wasting space, or when everything is packed so tightly together that it’s hard to read. The pantheon desktop in ElementaryOS finds a superb balance between the two.

System Settings

Keyboard Shortcut Settings.

The control settings in ElementaryOS are easy to navigate and have some additional items I don’t normally see in the standard Gnome settings menu. For example, the notifications section is layed out in a way that is much simpler to understand. ElementaryOS has also added Parental Controls/Screentime options built into the settings menu which I appreciate, as Linux is severely lacking in this area compared to other desktop OSes.

What I like most about the settings though, is their defaults. In particular, I found many of the default keyboard shortcuts already set to what I wanted. For example, I like to switch workspaces and move windows to them using CMD-N and SHIFT-CMD-N respectively (where N is the number of the workspace to move it to). In both KDE and Gnome I have to painstakingly go through and set this up by hand. Even worse, both DEs are hit-or-miss of if they actually respect the changes, and Gnome doesn’t really support setting this past 4 workspaces.

In ElementaryOS, these shortcuts are already set (along with many others I like). For the few that I did have to change, it was simple to do so, and the changes actually worked when applied.

System Coherence

Elementary OS Appearance Settings.

The last item to highlight about Elementary OS, is that the entire system feels cohesive. The default apps all follow the same look and style. Some applications deviate from this, but I understand that these rogue apps are out of the Elementary team’s hands. As a heap of loosely connected software packages, it can be quite difficult to make a Linux desktop seem like it all belongs together. While not perfect, I think ElementaryOS is one of the best examples of this being accomplished.

Everything appears to be well integrated. If I change my sound settings in the menubar, they change. Same if I connect a bluetooth device from there. I can’t always say the same with some of my recent Gnome Fedora installs. Everything is well connected, and there is very little random crap floating around the system.

What I don’t Like

I actually don't like the elementary calendar dropdown compared to Gnome.

This section is a bit smaller because honestly, there wasn’t too much that I didn’t like, and my biggest complaints are mostly petty.

Ubuntu Base: This is ridiculous, but I think the biggest downside I have with using ElementaryOS is that it is based on Ubuntu 😆 . While there are many benefits about being on an Ubuntu based system… I just prefer Fedora. Also, given that I work at Red Hat, using a Fedora/CentOS/RHEL base helps me keep up and passively learn the systems we use at work.

I prefer the Gnome Calendar Dropdown: After using Gnome on Fedora Silverblue for several months, I got used to Gnome’s drop-down calendar menu. In particular, I enjoyed how it displayed the weather and world times from my Clocks app right there. Having the various time zones easily available is actually extremely helpful when working on a world-wide distributed team. I bet this is an area that the Elementary team could easily implement something spectacular.

Limited Customization: There are limited preference options for many of the default applications. For example. the terminal app has 3 pre-configured themes you can select from, but that is all you can set the theme to. I have been able to change it using gogh, but it would be nice to have the ability to deviate from the default settings, even though I think the ElementaryOS defaults are all great.

The Occasional Random Crash: The biggest actual problem I’ve had with ElementaryOS, is that the desktop environment can occasionally crash. However, each crash has happened gracefully. The system will pause, the wallpaper goes black behind the windows, and everything disappears for a second before returning back to normal. So beyond having to wait a few seconds, the crashes have been largely harmless. (No logouts or shutdowns so far)

Conclusion

Elementary OS 6 on my work X1 Carbon.

ElementaryOS feels like a weird middle ground between my normal Linux setup and macOS. When I’m on macOS, I like the user experience, but wish I had full linux under the hood. With elementaryOS, the ends of this spectrum both fall somewhere closer to the middle: The user experience is cleaner than most Linux desktops and generally more supported (due to it’s Ubuntu base), but not as much as on macOS. For example, it is more common to find a deb compared a rpm package, but both are much more rare than finding a dmg installer. On the other end, it is Linux under the hood, but not Fedora, which means some of the system stuff I enjoy isn’t as well supported (podman, toolbox, selinux).

Regardless, I have loved using ElementaryOS 6. In fact, when the Silverblue install on my work laptop started misbehaving, I decided to install ElementaryOS 6 on it instead. To get a better work setup, I also installed a Fedora server VM that auto-starts when the laptop boots 😆.

Using ElementaryOS 6 for my user-land and a Fedora VM for my dev work has been a great Linux combo over the past month!

Obsidian Notes iOS + Working Copy Setup

2021-08-31T00:00:00Z

Eno State Park, Durham NC

After deciding to leave Notion, I have been using obsidian.md for all of my notes and loving every moment of it. The one thing I have missed after the switch has been the ability to view or lightly edit notes on my iPhone and iPad. However, I saw that there was an Obsidian mobile app in development, and I have been eagerly waiting for it. Now that the app has been released, here is how I set it up on my iOS devices (with some help from working copy 😉).

Obsidian Notes

Obsidian Desktop Application.

Compared to Notion, I really appreciate that all of Obsidian’s data is simply a collection of raw markdown files. The downside to this is that I am responsible for setting up a syncing solution. At first, I was using seafile to sync the files, which worked okay (but could be annoying).

However, because obsidian vaults are just text files… they can be stored in version control systems, like a git repository. So, I setup a private Gitlab repo to backup my vault to. This allowed me to snapshot my notes whenever I wanted. Wonderful.

Eventually, I started using the obsidian-git plugin to more easily take ‘snapshots’ of the vault. After a few trial weeks with the plugin, I finally made the jump and configured it to automatically take snapshots throughout the day. Now, I’ve entirely switched to using git to sync the vault across all my devices. The only options for syncing on iOS are to use git or iCloud. Luckily, I’m already set.

Working Copy (Git Repo)

Working Copy Host Setup.

I have used working-copy in the past to pull down git repos on my ipad. You can pull repos with the free version, but I think you need the pro version to push to a repo. For this setup, we ideally want the ability to push changes, so the pro version is recommended.

(If you don’t have a hosted git repo setup for the obsidian vault, you’ll want to do that first.)

Setting up Obsidian Mobile

Creating a new Obsidian vault in iOS.

To start, download and install the obsidian app from the app store. Next, open it up, and select Create New Vault. On the creation screen, give the vault a name and make sure that Store in iCloud is not selected. We’ll come back to obsidian in a minute.

Setup

Working Copy Repo Setup.

Next, switch to working copy and setup the credentials for whatever git host you used for the vault repo (ex: Gitlab for Github). Once that is done, clone the repo.

Open the repo and select the “share menu” at the top right of the application. Select Setup Folder Sync. We want to sync the repo with the Obsidian vault we already made. Under the On My iPad section of the browser, there should be an Obsidian folder that contains an empty sub-folder with the name of the vault created. Select it to sync the repo there. Afterwards, Obsidian and working copy should be linked up!

Using Obsidian Mobile

Obsidian iOS Command Menu.

When using obsidian on mobile, always remember to first pull down the latest repo changes in working copy! Afterwards, you can open up obsidian and use it as you normally would. Just remember to then commit and push any changes you make copy afterward!

The mobile UI can take some getting used to compared to the desktop application. One tip I have is to use the command prompt to do basically everything. Opening notes, closing them, splitting panes. Just use the command prompt for it all.

(And once again, remember to push any changes!)

Conclusion

Note in Obsidian iOS App.

It is still early days, but having a mobile app helps smooth out one of the very few downsides I’ve faced using obsidian. I’m glad to see the project progress, and even happier to be using it now. Congrats to the devs for getting this release out!

New Work Laptop - X1 Carbon (gen7) Thinkpad

2021-08-27T00:00:00Z

A few months ago I became eligible to trade-in my work laptop, a Lenovo p50 Thinkpad, for a newer one. After experiencing problems related to the laptop’s dedicated Nvidia graphics card, I jumped on the opportunity to swap it. Compared to being asked the simple “Thinkpad or Macbook” question I faced when starting at Red Hat, I had some additional decisions to make this time around.

Background Information

The old laptop (p50) outside at the office.

When selecting laptops, we can choose from a few different default options, across a range of size and performance. My p50 for example, was the high performance option, with it’s 32GB of RAM, 512GB nvme disk, and fast i7 cpu. The current high performance Thinkpad is an even more powerful version of the p50, with a newer 6 core, 12 thread CPU. However, I decided to select the compact option *instead - a Lenovo Thinkpad X1 Carbon.

Reasons for Switching

The p50 (left) and x1 carbon (right).

So… why did I switch to the compact, portable option, especially now that I mostly work from home? The main reason is that the carbon has a better screen. It is significantly brighter, with more accurate colors than the p50. While I don’t do much work that requires color accuracy, having a bright screen means I can work in brighter areas. This includes outside on the porch when the weather is pleasant, which is an improvement while working from home. I get to spend a little more time outside, which helps mix up my work environment, even if I rarely leave my house.

The second reason I selected the most portable laptop is shockingly… for the portability! I might not be lugging the laptop to the office every day, but having a smaller, lighter laptop makes it easier to pick up and move to another room when I want to change my environment (as mentioned in the paragraph above). Additionally, when I do return to an office in the future, or visit other Red Hat offices, the lighter laptop will be easier to transport, allowing my to carry less, or have more space in my bag for other traveling gear (like my keyboard).

Lastly, while I did take a hit in performance with the switch, it isn’t much of a problem. Most of my heavy performance work is done in CI/CD pipelines hosted on servers anyway, and the X1 Carbon still has enough power to handle any of the local development I need to do (as long as I’m careful about how many VMs I’m running at the same time).

What I Like

Enjoying the x1 carbon outside.

Thin and very light: It is even lighter than my Macbook Air, despite being slightly larger (It has a larger screen).
Screen: The screen is much better than the p50, and it’s still a matte display, which I prefer.
Keyboard: Even though it is a thin laptop, the x1 carbon still has a wonderful keyboard.
Integrated Intel graphics: Not much GPU power, but I’m not gaming on my work computer. Best of all, it is much simpler to work with than the Nvidia+Intel graphics.
Thunderbolt Dock: The TB dock that works well for my setup. It is smaller than the p50 dock, and being a standardize connection (TB3), I can use my personal laptops with it when I’m not working, as I use the same periphery devices for my personal and work setups.
NVME: Like my old laptop, it still has fast nvme storage.

In summary, it’s small, portable, with a better screen, and still a Thinkpad. That’s all I really need.

What could be better

The x1 carbon system specs.

Heat & Noise: The carbon can get quite hot and will kick up the fans when in video calls. To be fair, it’s not too loud, but I’ve been spoiled by my M1 Macbook Air.
More RAM and Storage: While I can get away with the 256GB HD and 16GB of RAM, it would be nice to have more. This is especially true if I want to be running multiple VMs at the same time, or to have several VM images saved on the device.
Meh CPU: It’s not the most powerful, but I knew that when I selected it. For the most part it’s been fine for everything I do, and I’d rather have a more power efficient CPU than a power hungry one. I wish it had a bit more oomph though for all the heat it gives off 😒.

Essentially, all of my dislikes are simply reasonable trade-offs I knew when selecting this type of device.

Conclusion

The x1 carbon with coffee.

That’s really it. I’m glad I made the trade, as the x1 carbon fits my current work needs much better. I love it’s bright screen, solid keyboard, and how portable it is. When I’m not moving around, I can easily dock it at my desk setup and pretend it’s a desktop. The x1 carbon is a solid, portable, Linux laptop.

Introduction to Toolbox

2021-07-31T00:00:00Z

As I use podman more often, I continue to rely more heavily on toolbox. In particular, on Fedora Silverblue, toolbox becomes the command line environment(s) that I use. Everything is done in a toolbox on Silverblue. So then… what is toolbox and how can you start using it?

About Toolbox

What is it

Podman Logo

At its basic core, toolbox is a fancy wrapper around podman that makes it much simpler to work with. When you create and enter a container using toolbox, you will find your existing username, user permissions, home directory (and a few other locations), system journal and more, are all already setup and waiting for you.

Toolbox does all of this in a single command, without you needing to know all of the crazy podman arguments it would take to do it yourself!

Advantages

I use different toolboxes for each of my dev environments.

What are the advantages of using something like toolbox? Using containers for testing out different tools or programming languages helps keep your system clean, by installing all the dependencies just within that toolbox. For example, if I am trying out a new program that needs to be compiled from source, I will create a toolbox, install the dependencies inside it and build the application. Afterwards, when I’m done testing out that build, I can delete the toolbox, leaving my host system untouched.

Additionally, read-only operating systems like Silverblue, really benefit from utilizing containers. If I want to install a package in Silverblue, I usually have to add it as a layer to the base image, and then reboot into that new image. Having something like toolbox allows me to have a read-only system for my core, but still work in a normal dnf-managed command line environment. It’s the best of both worlds.

How to get it

Toolbox is installed by default on Fedora Silverblue. If it isn’t installed on the base image you are using for some reason, it can be installed as a layer using rpm-ostree:

rpm-ostree install toolbox

On other fedora systems (ex: Workstation or even Server), toolbox can be installed with dnf:

sudo dnf install toolbox

Toolbox Functions

Now that we know what toolbox is and have it installed, lets learn some of it’s commands!

Toolbox Create

First, let’s create a toolbox container:

toolbox create

By default, this will create a container with a default name based on your distro and version. For example, when run on Fedora 34, this creates a toolbox named fedora-toolbox-34.

To create a container with a specific name, the -c flag can be used. For example:

toolbox create -c website

This will create a toolbox container named website (which is where I install hugo to work on this website 😉).

Toolbox image

In addition to the name, a toolbox can be created with a specific container image, using the -i flag (assuming it is a docker/podman image that is compatible with toolbox). For example, the following command will create a toolbox from the Fedora 33 toolbox image:

toolbox create -i fedora-toolbox:33

Instead of providing a full image name, you can also declare a distro with the -d flag (ex: fedora), paired with the -r to specify a release.

toolbox create -d fedora -r 35

This creates a Fedora 35 toolbox (even though I am running on Fedora 34).

Toolbox List

Now that some containers containers have been created, we can list them all with toolbox list:

toolbox list
---
IMAGE ID      IMAGE NAME                                    CREATED
e6d38a7d896c  registry.fedoraproject.org/fedora-toolbox:34  2 weeks ago
30e2dd6cf22e  registry.fedoraproject.org/fedora-toolbox:35  2 weeks ago

CONTAINER ID  CONTAINER NAME     CREATED     STATUS   IMAGE NAME
94a91110021a  fedora-toolbox-34  4 days ago  running  registry.fedoraproject.org/fedora-toolbox:34
f91f8b4a3a51  website            4 days ago  running  registry.fedoraproject.org/fedora-toolbox:34

This command displays all of the toolbox containers, but also the container images that are downloaded. The -i flag can be used with toolbox list to only display the images, or the -c flag to only list the toolbox containers.

Toolbox enter

It’s time to finally enter our toolbox. Like the toolbox create command, the toolbox enter one will enter that default toolbox, and named toolboxes can be entered by using the -c flag. For example:

toolbox enter

Will enter the fedora-toolbox-34 toolbox on my Fedora 34 machine. If I want to enter my website container, I can call:

toolbox enter -c website

When you enter a toolbox, you are dropped into a new shell (as the same user), with access to your home directory. In the toolbox, you can install applications that will then exist in the container, but not on your host system. Toolbox has become so refined over the years, that it even runs GUI apps now!

To exit the container, use the exit command, just like you would to exit a shell.

Toolbox Run

Gitg GUI app running in toolbox

Here’s a fun tip: you don’t actually have to be inside a toolbox to run commands in it. The toolbox run command can be used to pass commands to the toolbox container. Again, no args will run the command in the default toolbox, but the -c flag can be used to run commands in named containers. For example, this command runs guvcview in my default toolbox.

toolbox run guvcview

Remember, I don’t have guvc installed on my host. And yes, it can pass my webcam into the container!

The next example runs the hugo version command in my website toolbox:

➜  ~ toolbox run -c website hugo version
Hugo Static Site Generator v0.80.0/extended linux/amd64 BuildDate: unknown

Easy.

Conclusion

That’s really it! A lot can be done with these few commands. I have been using toolbox for years, and I’ve been very impressed with how far it has come in that time and how much more stable it is now (podman has also become much more stable). So, if you haven’t used toolbox before, why not give it a try?

Website Updates - Moved to Gitlab Pages

2021-07-25T00:00:00Z

Durham Bulls Athletic Park, Durham NC

I recently made some major changes to my website. After playing around testing the site on Gitlab Pages for a few months, I finally sat down to figure out how to make the official switch. In the end, this involved a bunch of additional side projects, including replacing my ‘cloud node’ with a new one, setting up a Gitlab runner, removing all analytic trackers from the website, and completely re-doing the the DNS for my domain name. It’s been quite the overhaul…

Background

An older version of my website.

First, lets start with some background information. I have been using Github pages to host my website for… ummm… a long time. I probably started using it about the time I graduated from college (2014)… way before I switched to hugo, or even jekyll and cryogen before that.

I have used many static website generators to build and deploy the website over the years. However, the one thing has remained consistent the whole time, is that I use two repos, one for the source, and one for the built website. This is something I’ve wanted to consolidate for years now, but just haven’t gotten around to. This was one of the main reasons motivating me to make a switch.

My Old Jenkins Website Pipelines.

Another reason was to have a better CI setup. I’ve used Jenkins off and on for my website testing and CI pipeline, but haven’t maintained a Jenkins server for quite awhile now. This is something I’ve wanted to improve in my workflow, but haven’t had the desire to spin up and host a full CI server*, just for my website. With Gitlab CI… I wouldn’t have to.

*PS: I know this is something I probably don’t have to do with Github pages anymore.

Playing with Gitlab

My Gitlab Page.

I’ve slowly started migrating my personal projects over to gitlab over the last year. This started with wanting one or two repositories to be private (Github didn’t allow free private repos at the time), but eventually I started to prefer how Gitlab has changed over the years compared to Github. Plus, I don’t feel as much pressure to have to be on Github, where discover-ability is higher. Also, I’ve started to use Gitlab more at work so having my personal repositories there helps me learn for work and vice versa.

The only projects that still needed to be on Github, were my website repos. So, I added a remote on Gitlab, and have been pushing the website source there as a backup location, but to additionally start testing out what was needed to make the switch.

Moving to Gitlab

My Gitlab Repos.

Two weeks ago, I finally decided decided to jump in and make the switch. I started by I renaming the website repo in Gitlab to be the default gitlab pages site for my account (ex: username.gitlab.io). This took me an embarrassingly long time to figure out 😆.

Next, I left both websites up for a few days to compare stability. Everything seemed fine. My gitlab.io site was basically the same as the ‘real’ (Github) one. The only issue stopping me from an full switch at this point was that I wanted to configure a CI setup, with tests, running in a Gitlab pipeline. If I could figure that out, it would be worth it to officially switch.

Gitlab Runner

I spent a long time trying to get the tests to work with the shared CI runners. Eventually, I realized it would just be easier if I set up my own private runners, which I have done before. I wanted the runner to be hosted outside my network, so I wouldn’t have to worry or rely on my desktop being on and running the VM when I needed a pipeline to pass.

So, I decided to put it on my cloud node. I had been meaning to migrate my cloud VM from Digital Ocean to Linode, and this seemed like the time to do it. That VM was several years old, and drastically needed to be rebuilt from the ground up. However, I had been dragging my feet for years because I didn’t want to go through the hassle of migrating piwik (what I used for website analytics).

I used Piwik (now Matomo) for web analytics for years.

To solve, the Matomo/Piwik issue… I simply decided to forego all analytics on the website. It’s a decision I’ve been debating recently. Similar to how I don’t feel the need to have my personal projects discoverable by using Github, I don’t need to know how many people are visiting my website, and what posts they are reading. Knowing that information won’t change what I write about, nor should it. So, I made sure to remove all analytics code from the website source: my piwik code, as well as a poorly implemented Google Analytics bit (It never properly linked the data to my account). As a result, this website is now free of any tracking, at least on my part 🙂.

With that ethical and technical mess of my chest, I was able to easily spin up a new linode, and configure a gitlab runner on it. Ideally, I want to use containers as the executors, but for now I am running straight on the host, controlling a nginx service. Switching to a containerized test workflow on the runner might be in the cards for a future post, but currently, this works.

Domain Switch

Moved my domain management to Linode.

With the gitlab runner configured, and my testing CI working, there was only one more thing to do: switch my domain name to point to the Gitlab Pages site. Like the cloud node, I wanted to move management of my domain name from Digital Ocean to Linode too. I figured this was as good of a time as any, as I was already ripping everything up. I switched the nameservers, and started re-creating my records from scratch (they needed to be cleaned up anyway). After starting, I was quickly reminded… messing with DNS sucks.

The problem with moving the DNS for a domain around, is that it can take hours or even days to propagate the change to all of the nameservers around the world. This means that while setting things up, I never knew if something wasn’t configured correctly, or if it just hadn’t taken effect yet. After a day or so, my domain records seemed to be working and back up, pointing to the Gitlab Pages site. Finally.

Conclusion

The only noticeable change on the website is that there is no trackers.

That was a lot. To summarize, I did a bunch work, all around the main task of updating the backend hosting of my website:

Moved the website source repos from Github to Gitlab
Switched from using Github Pages to Gitlab Pages for hosting the actual website
Replaced my persistent cloud instance, on a new provider
Configured that instance as a private gitlab runner
Configured my website testing and deployment CI to run in gitlab pipelines
Removed all trackers from the website
Moved the management of my domain name
Pointed ryan.himmelwright.net to the new Gitlab Pages site.

So, what does all that work mean for you, my wonderful reader? Well, it meant that the site was down for a day or two while the domain switched, but now that everything is finished… it really shouldn’t mean anything. For a reader, the website is basically the same as before. Oh, except no more tracking. So pages might load a little faster. Enjoy!

Multi-hop ssh Connections

2021-06-27T00:00:00Z

Karsh Alumni Center (Duke University), Durham NC

Last weekend, I wanted to ensure that I could work on a personal project using my typical setup, while I was away for a few hours. This consists of running VS Code on my macbook air, but remotely connected to my Linux desktop. All of the coding actually happens on the Linux computer, even though I am running VS code on my macbook. However, when outside of my home network, connecting to the desktop requires multiple ssh hops. Fortunately, configuring ssh to handle a multi-hop setup is actually quite easy and… it works with the VSCode remote plugin. Here’s how.

Problem

To restate the problem: If I am outside of my home network, I need to make multiple ssh ‘hops’ in order to get to my desktop (charmeleon).

Laptop -> Ponyta (SSH Entry Node) -> Charmeleon (Desktop)

Normally, this requires me to first ssh into my home network (ponyta), and then from there, ssh again to whichever device I want to connect to (In this example, Charmeleon). Out of pure laziness, I want to run these steps as a single command. Additionally, for tasks that are automated, or outside of a shell, connecting to my desktop truly needs to be a single step in order to work.

Solution

An simple solution is to create a new host item in the ssh config file. In this file, we can utilize the ProxyCommand option to setup a multi-hop scenario. So, I added the following my ~/.ssh/config (make sure to swap out your PORT and HOSTNAME values accordingly):

Host remote-charmeleon
	Hostname CHAMELEON-IP
	User ryan
	ProxyCommand ssh -p PORT ryan@PUBLIC_FACING_PONYTA_HOSTNAME -W %h:%p

Now, I only need to run ssh remote-charmeleon to kick off the the full Laptop -> (Ponyta) -> Charmeleon sequence. Easy.

Setting it up in VSCode

Connecting to a multi-hop system via ssh in VS Code.

Now, how can this be used in a VS Code session?

All I had to do was add the config sequence to the ssh config file that VSCode uses, which I usually have set to be different from my default one.

To do this, call the Remote-SSH: Open SSH Configuration File in VS Code to open the proper config file. Next, add the same code from above to the config. From then on, you can select that host item when running Remote-SSH: Connect to Host... in VS Code.

For example, I can now select remote-charmeleon when VSCode prompts for the host to connect to.

Conclusion

That’s all. It’s a simple solution that I very much appreciate. I always forget how powerful ssh is, especially when you start setting up custom configs. It’s worth taking a look at! As always, man ssh is a great place to start 😉.

Disable Seafile Notifications on Gnome 40

2021-06-17T00:00:00Z

Jockey's Ridge Stage Park, Nags Head, NC

Quick post here. Basically, I keep figuring out and then forgetting how to disable Seafile notifications on Gnome, because it doesn’t allow tray icons. This time, I’m recording the process.

Background

First, why do I need to disable Seafile notifications specifically? Because I sync my obsidian vaults using Seafile. While this sync method works wonderfully, it unfortunately has the side effect of whenever I type in obsidian, seafile… uh… politely notifies me… that it has synced my changing notes. Which is incredibly annoying, to say the least.

However, I don’t want to solve the problem by simply disabling all system notifications. Luckily, seafile has a setting to do this… if you can get to it.

Install Tray Icons: Reloaded extension

Installing 'Tray Icons: Reloaded' from the Gnome extensions website

The first step is to install an extension that allows application tray icons. One such extension that seems to “work” currently (with a catch, more on that below), is Tray Icons: Reloaded. So, to install it, open up the page in Firefox, and click the slider tab on the top right, to switch it from OFF to ON.

Note: If this is the first extension being installed, Firefox may ask you to install a plugin. This is normal. Just hit accept and it will do it for you.

(Optionally) Install Gnome Extensions app

Installing the Gnome extensions app.

Next, while optional, it is ideal to have the Gnome Extensions app installed. So, open up Software Center, and install it from there. Alternatively, install it as a flatpak from the command line:

sudo flatpak install org.gnome.Extensions

This app allows you to configure and enable gnome extensions. With Gnome Extensions opened, make sure Tray Icons: Reloaded is turned on. Note, icons likely won’t show up yet, even with the extension enabled.

Switch to an Xorg Session

Switching to Xorg Session

This is the step I always forget (and hence why I’m writing this post). While Top Icons: Reloaded “works” in Gnome 40… it only does in Xorg sessions. So, log out, switch the session to GNOME on Xorg (gear icon at the bottom right of the login screen), and log back in. Once logged in, the top icons extension should now work, hopefully displaying the Seafile icon (assuming seafile-client is running).

Edit the Seafile Settings

Accessing seafile settings from toolbar icon.

To open up the Seafile settings, right click on the icon and select Settings.

Changing notification setting.

From there, notifications can be disabled by unchecking the Notify when libraries are synchronized checkbox. Hit OK, and that’s it. Feel free to log out and back into a Wayland session 🙂.

Conclusion

Short post, but I forgot how to do this simple process twice now, so I wrote it. Hopefully this helps that one other person also experiencing this issue, or at least a few more that are confused for similar reasons. Enjoy!

Upgrading My iPad Pro

2021-05-28T00:00:00Z

Bodie Island Lighthouse, Nags Head, NC

A few years ago, I purchased my first iPad, a 10.5" iPad Pro. I use it just about every day. With the release of the newest 2021 iPad Pro, I decided it was finally time to trade mine in for an upgrade. Here’s why.

History

The box of my 10.5" iPad Pro.

When I got my 10.5" iPad Pro, the 2018 11" iPad Pro had already been released. At the time, I didn’t think the extra features justified the increased price of the new iPad. The discount price on the old pro was more appealing to me. I was already upselling myself, after originally looking into the base (non-pro) iPad, so the discounted 10.5" seemed like a good compromise between features and price.

The 10.5" pro was a good starting point for my first iPad. It was a great device, but not too over the top in case I ended up not using it much. If I did use it, I could always upgrade/swap it out in the future.

Why upgrade?

After having my ipad for several months, I started to understand how the improved features of the 2018 pro could be nice to have. In particular, the USB-C port, smaller bezels, and FaceID were all very appealing to me. Still, my 10.5" suited my needs just fine. I did decide though that I would likely upgrade my iPad sometime in the future. I enjoyed having it.

Which brings me to the one thing I did want to add to my iPad: the Apple Pencil. Being able to hand draw is a special feature unique to tablet computers that I cannot easily reproduce on my other devices. It’s a foundational reason to even consider owning a tablet. Additionally, in the years since I purchased my iPad, new iPadOS releases have started to polish the use of the ‘magic’ pencil with the iPad.

Unfortunately, my 10.5" ipad only supported the 1st generation Apple Pencil. Knowing that I would eventually upgrade my iPad, I didn’t want to buy an old pencil, only to have to also replace it when I upgraded. So, I decided to wait to get the Pencil until it was time to upgrade my device. Now feels like a good time.

Why this generation?

The 2021 iPad Pro includes a ultra wide front camera, to use with the new center stage feature.

There are a few factors that pushed me to upgrade to this generation of the iPad Pro. First, I can still get a decent trade-in value for it. Considering it’s been about 4 years since the 10.5" model was released, I’m not sure how long that value last with my iPad. Second, if I want to trade it in with Apple (and I do, it’s probably the easiest method), I am limited to spending the credit value in their store. While I could try to get a refurbished device, there hasn’t been many used iPad Pros in stock since the pandemic started. So the latest model is my best bet.

Beyond ease of trade-in, there are a few new features in this latest generation that I am interested in:

The M1 chip is complete overkill, but I wouldn’t mind having one in my iPad considering how much I love it my MacBook Air. Additionally, it provides more computational overhead (and increased RAM) incase Apple ever decides to finally push the capabilities of iPadOS further (which they need to do).
While I thought it was a bit of a gimmick at first, the wide-angle front-facing camera paired with the new “Center Stage” feature on the 2021 iPad Pro could actually be useful. My wife and I use FaceTime to video chat with my family (we live in another state from everyone), and my ipad is usually what we use to do it. Being able to more easily fit the two of us in frame, and have it self-adjust as my wife comes and goes would be delightful.
Lastly, while USB-C is great, Thunderbolt is better. I already have all of my periphery gear hooked up to a thunderbolt dock to connect my work and personal laptops to. The increased bandwidth also helps to “future proof” (again, assuming Apple can improve the OS).

Why the 11" over the 12.9"?

The iPad Pro Size Comparison.

Whenever I would think about upgrading my iPad Pro, I always figured I would switch to a 12.9" version. So why then have I instead opted for the 11" when the 12.9" is clearly the more exciting option this year, having the new “Liquid Retina XDR” display? Well, as much as I’d enjoy seeing that new display, that price bump makes it a bit more unreasonable for a feature I truly do not need. I don’t do any visual work, and the display on the 11" Pro is more than enough for anything I need.

The real reason for the 11", is that I now have a great portable laptop. I was previously looking at the bigger ipad to be my travel/portable device, but now that I have an m1 air, a 12.9" ipad with a keyboard would actually weigh more. I want my ipad to be more portable than my mac. As a bonus, if the combined size and weight remains low, it’s easy enough to travel with both devices. For reference, a 11" iPad Pro (without a case) and my macbook Air have a combined weight of only 3.6 lbs, whereas my previous 16" Macbook Pro alone weighed 4.3lbs! I would like to keep this portability.

Why not the new iPad Air?

The iPad Air Color Options.

Lastly, if I am contemplating getting the 11" iPad Pro, why not consider the 2020 iPad Air? I struggled with this decision for awhile, particularly before the latest generation pro was announced. In general, I think the air would be fine and meet all of my needs. The reality is that both of these devices are currently overkill for most iPad tasks. With that said… I think the additional features of the 2021 pro are actually worth the $200 difference (in my case). Here are a few of those feature differences, and my justification for the pro:

The Display: I like the 120hz pro-motion display. I read all the time on my ipad, and I love how smooth it is. In addition, if my plan is to get an Apple pencil right away, the high refresh rate can help the pencil’s responsiveness also feel better.
- (furthermore, while not a big deal, the screen is a little brighter on the pro, which is appreciated when I want to read outside)
Cameras: As I said above, the ultra wide front camera is great for FaceTime, as it is my FaceTime device.
Better speakers: I love having nice speakers on my systems, especially my travel devices. I’ve already downgraded my laptop speakers going from the 16" to the m1 air, and I don’t want to do it again. (The pro has 4 speakers, whereas the air only has 2).
FaceID: TouchID is great, but my finger tips tend to peel due to allergies, and FaceID simply works more consistently for me.
Thunderbolt: Not a deal breaker, but if I want to play around with testing an ipad as a modular computer at all, I already have TB3 docks setup to use.
Storage: The base version pro has twice as much storage as the air. I didn’t think this was an issue, but then I looked at the usage on my ipad and saw I had consumed 60/64GB. I expect my usages will only continue to grow if I begin to sketch diagrams and notes more often. The air also starts at 64GB, where the pro begins with 128GB of storage. To upgrade the air, you need to jump to all the way up to the 256gb hard drive. After that upgrade, the air is less than $50 away from the price of the pro. Considering everything else I listed above, I think it is definitely worth that additional cost.
LiDar (seriously 😆): This is more of a novelty than a feature for most people. Still, as someone that likes to play with AR, uses their iPad to scan floor plans of their house, or to take rough measurements… the LiDar would help the iPad improve all those tasks.

Conclusion

My iPad Pro order status: Processing.

There you go. Those were most of my reasons for why I concluded to finally trade in my 10.5" iPad Pro, for a new 11" model. I have ordered the 2021 11" iPad Pro (in silver), along with an Apple Pencil 2, and selected to trade in my old one with the purchase. Now, all I can do is wait for it to ship. It’s been stuck processing for over a week, but could arrive as early as two days ago 😐. Hopefully it ships soon…

Leaving Notion

2021-05-20T00:00:00Z

Kill Devil Hills, NC

After using Notion for my notes and daily planning over the last year, there are a few issues with it that have worn on me. Over time, my uneasiness with the application has built up to the point where I have decided to switch away Notion completely. Here’s why.

Background

My Notion Dashboard Page.

Before getting into it, I want to first state that for the most part, I have loved using Notion. It took a few iterations of figuring out how I wanted everything organized, but after my most recent system overhaul, I have it arranged perfectly.

I have everything stored in a handful of databases: Areas, Projects, Tasks, and Notes. The items in each of these DBs are all linked. For example, tasks and notes are usually attached to a project, which is all associated with a particular area (Home, Learning, *Social, and so on).

In the most recent iteration, I moved away from linking tasks to a daily journal object. Instead, I assigned everything a date, and created dashboard pages with date filters to see the Weekly and Daily boards of my tasks. All in all, it was a remarkably efficient system… most of the time…

What I liked about Notion

My linked tasks in a goal project.

I’ll run through these quickly, as many of the statements echo my original notion post…

Easy to Setup and Use

I can have Notion easily configured on all my devices. All I have to do for setup, is install the app and log in. I even have the option to access it in a web browser, if I cannot install it as an app on a device.

Flexibility

As described in the Background section above, the flexibility of Notion allowed me to tailor it to whatever I needed. Notion wasn’t a system itself, as much as it was a tool that let me build my own system. It felt like a modern evolution of emacs org-mode for ‘normal’ people, which is a good thing.

Templates

With such a malleable system, it helps when the more intricate components can be automated and simplified. Luckily, Notion has a template system, which is what really allowed my to custom system to be usable. My frequent tasks were all templated to save time when entering them into the system. Without templates, Notion would have been a giant pain to maintain.

Everything can be linked

I enjoyed being able to link everything together. Seeing all the associated tasks and notes of a project in a dashboard was game changing for how I think about organizing information in a system. As an example, I would plan out monthly goals and link each task to it. That way, at the end of a month I was able to go back and see exactly if and when I had completed each task for that goal.

What I didn’t like about Notion

Exporting Notion Data.

Requires Notion Servers Being Available Notion is completely hosted online

and does not have a true solution for working offline. While this can be convienient because I don’t have to worry about syncing data, it does mean that if the Notion service goes down… I can’t access anything.

I used Notion for my tasks, notes, and everything else. When it went down, I couldn’t do basic actions like check my TODO list, or jot down a quick note in my system. I had to record everything somewhere else, and then go back later to transfer it into Notion after the service was available. While Notion didn’t go down often, I still faced this problem whenever my internet went out, or I was traveling and didn’t have a connection. Combined, this happened often enough to be annoyed by it.

Access to the data

On a similar note, all of the data and content resides on Notion servers, and in a non-standard format. As a result, any important data-related responsibility fall’s entirely on their shoulders, and I have to accept whatever level of security they choose.

While data security is very important, my biggest concern is that the database data is in a weird format. The page content is in markdown-ish, but the only way to view everything organized properly in the databases (without hassle, as far as I know), is within Notion. There are no raw files I can easily open or convert. They have an exporting system, but every time I tried to use it, it froze. So even while in theory you can export everything to md and csv files… I couldn’t. (Update on this below)

In particular, I am most worried about data formatting. If Notion ever disbands as a company, or is acquired by a company that slowly shifts direction (or honestly, if Notion themselves shift the product over time)… I’m stuck. If I want to move to a different system, I can’t easily reformat/import the data into a new system. This alone has been a nagging hesitancy in the back of my head, the entire time I’ve used Notion.

Update:

Exported Notion Data in Obsidian Notes.

So while taking screenshots, I tried again to export the data. This time, it made it all the way through the process successfully. I was then able to take the output and run it through the Notion-to-Obsidian-Converter script (spoiler for what I’m using now 😆), and import it into an Obsidian vault. While this is nice to have as an archive, by default it isn’t useful in how it’s formatted and would be a lot of work to truly convert all the data. This is mostly due to how I’ve organized by Notion notes, but still… it is a mess and I don’t even know where to begin in understanding it.

Buggy Issues. Particularly With Templates

My Notion templates when creating a new task.

Lastly, what ultimately pushed me to the edge and confirmed my decision to switch away from Notion, was the annoying bugs and glitches I kept experiencing. Unfortunately, some of my favorite features of Notion, were also my biggest obstacles.

There were many times when templates just did not work. I would create a new item and apply a template… but the content would not load. This was an issue I experienced very frequently. Considering how much I relied on templates, I would have to either wait till a later time when it would work again, or manually setup everything. Neither of these choices were productive.

The last straw, finally inciting action to switch, happened at the end of a week not too long ago. I noticed mid-week that all of the database links for my week’s items (linking tasks to their projects, or even the link for what area a task belonged to)… were missing. Nothing was connected.

Normally, this is set automatically with the templates, so I chalked it up to template issues, again, and manually connected everything. That Friday, I had off from work, so I sat down to plan out my vacation day. I quickly noticed that the items weren’t linked… again. That was enough. I spent the rest of my vacation day researching alternatives for Notion. (and trying to export my data… but I already stated how that (originally) went 😖).

Conclusion

My Notion Task Board.

I think Notion is a great product, but with several trade-offs. I recognize that many of the items I feel strongly about, won’t be concerns for others. In fact, I was able to overlook many of them because of how great a tool it was.

Over the past year however, my concerns about access to the data have really been nagging in the back of my mind. Every day that passed, I felt the guilt of having an ever-growing collection of notes that was equally growing in how painful it would be to loose/convert if I switched away from Notion. On top of that, every time I experienced issues with my templates, I wanted to replace Notion. So now I finally have.

It’s been a great year using a great tool. Notion is an excellent tool if you can manage not always having access to it, and if you are okay with the chance it might all go away someday. But for my planning and notes system, I just can’t afford to use something that doesn’t give me ownership of the raw files it uses.

Using kid3 Audio Tag Editor

2021-04-30T00:00:00Z

Jockey's Ridge Stage Park, Nags Head, NC

A few months ago, I set out to finally clean up and organize my music library. My goal was to get rid of all the random files, and ensure that every item was properly tagged so that it would accurately self-organize in the music players I use. This would not have been possible without the open source tool, kid3.

The Problem

In order to clean up my music library, I needed a method to edit the metadata tags of the files. Music applications use these tags to grab information about a song. For example, a song’s title, artist, album name, and release year are all contained in the a music file’s tags. Using this information, the music player is able to organize and sort the music by artist, album, year, genre, or by any other bit of data it knows.

In the past, I used a tool called easytag to edit the metadata of my files. However, after not being able to initially find it (I had the wrong name 😆), I started to look for a new tool. I needed something easy to use, as I did not want to waste a bunch of time learning a complicated new tool. After very little searching, I found kid3.

Kid3

kid3 window.

As a Fedora Plasma user, I was happy to learn that kid3 appears to be maintained directly by the KDE team. To use kid3, just drag and drop the files into it, add/edit any of the tag info, and save. New tags can also be added if they aren’t listed.

I started using kid3 on my Linux desktop… but one night I decided to work in the other room on my macBook. I sadly assumed I would have to find a different tool to use on macOS. However, while searching a new tool, I learned that kid3 has a macOS build! In fact, in addition to Linux, the KDE team has official kid3 builds for macOS, Windows, and Android! I love a good cross-platform tool.

Efficiently Editing Album Tags

Editing multiple files in kid3.

After using kid3 it for a bit, I fell into a workflow that made the tedious task of grouping multiple files into single albums easier to manage. Specifically, I learned that I could select multiple files, and edit a field to apply to all of the selected items at one time. Using this technique, I was able to write the artist, album, and year only once, instead of 10+ times for each album. This made it much more efficient than methods I had previously used while editing metadata.

Conclusion

A 'home-made' file tagged in kid3, now properly sorted in my music player.

That’s all I have. This was a short post, but I think it was worth it. I love when I find a great open source, cross platform tool that does exactly what I need. Kid3 is a gem application. If you need to edit some audio tags, check it out. It certainly made my life much easier 🙂.

Setting up NFS

2021-04-21T00:00:00Z

Pocosin Lakes Wildlife Refuge, NC

Thanks to VFIO passthrough, I find myself sitting in front for a virtualized Linux system on a daily basis. While this setup is unbelievable, it does come with a few complications. One such hurdle is sharing files located on the host system with the VMs. After trying a few methods, I determined that nfs was the simplest to get up and running. Here’s how.

The Problem

There's no music files in the VM and the player errors looking for the missing files.

My desktop is my main workstation. It is the hub that holds all of the data I use, including documents, music, and videos. When I am working in a VM, I often want access to that data. I like listening to my music while working in VMs 😉.

It would be inefficient, both in time and disk space, to copy all the files I desired to each VM that I use. Setting up a network filesystem is an can get around this. After some trial and error, I eventually concluded that for my use (linux guests on a linux host), NFS worked. If you want to share with Windows guests, something like samba might work better.

Server Setup

First, we want to setup the sever. Be sure to run the following steps on the host machine.

Install Dependencies

Being on a Fedora host, I installed the dependencies using the following dnf command:

sudo dnf -y install nfs-utils

Configure The Domain Name

Next, open up the file /etc/idmapd.conf and make sure that the Domain = ... line (usually line 5) is un-commented, and set to your machine’s hostname. If not, edit it to configure the idmapd domain name:

Domain = charmeleon

Define the Exports File

After everything is installed, we need to create the exports file. This file will define all of the nfs shares we want to make available. Create and open the file /etc/exports, and add something similar for each directory you want to share:

/home/ryan/Seafile      10.0.7.0/24(rw,all_squash,insecure)
/home/ryan/Music        10.0.7.0/24(rw)
/Data/Videos            10.0.7.0/24(rw)

I like to share my Music and Video folders, along with my Seafile library (so I don’t have to configure it on each VM).

Each line breaks down as follows:

The directory location to share (ex: /home/ryan/Music)
The range of addresses that are allowed to access the share (I use 10.0.7.0/24, which allows devices on my home network)
The properties for each share. I use (rw) to allow read and write permissions.

That should be it. Feel free to dig deeper into all the nfs share settings if you need them, especially if you want to be a bit more secure in your setup!

Firewall/other permissions

After creating the exports file, we need to allow the service through the firewall. To do so, permanently allow the nfs service:

sudo firewall-cmd --add-service=nfs --permanent

Don’t forget to reload the firewall!

sudo firewall-cmd --reload

Start the service

Last but not least, make sure the rpcbind and nfs-sever services are started and enabled to autostart after a reboot.

sudo systemctl enable --now rpcbind nfs-server

VM Client Setup

With the server setup, we should be able to mount the shares in the VMs now. First, ensure the nfs-utils package is installed there too.

Mount

Now, use the mount command with nfs for the -t flag to mount the shares:

sudo mount -t nfs 10.0.7.82:/home/ryan/Music /home/ryan/Network/Music

If you experience weird issues when mount the shares, a reboot may help. I occasionally hit permission issues that seem to be resolved after a restart.

fstab

It is possible to add the share to your /etc/fstab file, so that it auto-mounts during boot. However, I don’t usually do this. I like to make the conscious decision to mount data from the host machine, as I don’t always need it available in the VMs.

Conclusion

Listening to music in a VM, mounted from a nfs share on the host.

That’s about it. This was just the basics of setting up a nfs share, and I’m sure I do something wrong, but so far… it has worked great for my simple use case of sharing data between the host and guests on the same computer. Enjoy!

Using Remote VSCode

2021-03-31T00:00:00Z

Duke University, Durham NC

I mostly work on my desktop, but usually from another computer, like my laptop. Historically, the most common method I use for this has been combining neovim and tmux (with tmuxp, for ease). However, more often than not, I now find myself using VS Code with it’s remote development plugin. Here’s how.

Install the Remote Development Plugin

To get started, first make sure that VS code is installed on your machine. This can be found on the project’s website, or even as a flathub flatpak

Next, to instal the plugin, click the Extensions tab in the side bar (The icon with four squares). From there, search for the extension by using something like ‘remote development’ as the search term. The Remote Development extension should appear in the list. Select it, and click Install.

(Click for an example video)

This extension is actually a package that bundles three extensions: Remote - SSH, Remote - WSL, and Remote - Containers. Alternatively, you can just install the one you need.

Connect to the Server

With the remote plugin installed, we can now connect to a remote machine. I mostly do this using ssh. Make sure you first have ssh keys copied to the device you want to link to. Then, open the command pallet (CMD/CTRL-Shift-P), and search for the Remote-SSH: Connect to Host... command, and run it.

This will prompt you for an ssh host to connect to. Just enter a host like you would when using ssh. For example, ryan@example.com. If it’s the first time adding a host, the plugin will ask where it can save a config file for the hosts. I usually just select one of the defaults it offers me.

(Click for example video)

The connected status is displayed in the lower left of the window (top window: remote, bottom: local).

When connecting to a remote machine, VS code will usually pop open a new window. To verify, the ip address of the machine that the window is connected to can be found in lower left corner.

Working on Projects, Remotely

Opening a directory in the remote window.

When working in a remote VS Code window, everything is passed on from the remote machine, as if you were sitting down at that computer. When opening a new file or folder, you browse the contents of the remote filesystem, not the local one.

Even the built in terminal runs on the remote machine. I still work in tmux, so I have my vscode terminal attach whatever tmux session I need. This provides me with the flexibility to switch to a new machine and pickup where I left off. Using this setup, I get a full graphical IDE window, while still reaping the benefits of vim + tmux.

Working on this post, with `vim` opened in `tmux` in the built-in terminal.

And yes… you can run vim, inside tmux, in the built-in VScode terminal, in a VScode remote window. This is something I actually do quite often XD.

Conclusion

I am almost always working from a different machine than the one which my files and work are on. This setup has been idea for how I work. It is easy to use, yet very powerful.

Sitting in front a gruvbox-themed VSCode window, connected to my desktop, and typing using vim keybindings (via a vim plugin) has become my new default, no matter what computer (or VM) I am physically working from. I love it.

Jenkins Parallel Stashing

2021-03-19T00:00:00Z

Pennsburg, PA

I recently hit a snag while developing a Jenkins pipeline at work. I was having difficulty preserving files across the entire pipeline run. In the end, my solution was to use the stash feature. However, I found little support for my specific type of issue online, so I figured I might as well write a short post about my experience.

Background

The pipeline runs an integration test suite I am working on. It contains mostly sequential stages, but there are a few that run in parallel. Specifically, there are provisioning and tear-down stages, one for each of the components we want to use during the testing.

Pipeline stages.

The Jenkins instance is hosted on Openshift, and uses containers for the job nodes. The pipeline uses several containers across the different stages, which means that the local filesystem does not persist throughout the entire pipeline. This was a problem, because we wanted to maintain metadata files from each of the provisioning stages, to use in dynamic tests, as well as during each tear down stage.

The issue is further complicated by the parallel provisioning stages. Each provisioner runs in its own container, all at the same time. On top of that, the number of provisioner/tear-down stages is dynamic. The pipeline might run using data provisioners for components A, C, and D, in one run, and A, B, C, G, and H for during next. It all depends on what integration tests we want to perform. This meant that my solution also had to be dynamic and flexible.

My Plan

I looked briefly into changing the containers’ volume configuration, but my proof of concept pipeline hit a silent failure right away (the pipeline would endlessly hang, with no error). I decided to revisit that approach if my next plan (implementing stash and un-stash) didn’t work.

To simplify what to stash, and to have it work with the dynamically changing parallel stages, I decided to try a single stash location. I made a metadata directory that the stages could all write to, and stashed it. My thought was that I could then un-stash it at the start of the first stage in each subsequent node, and re-stash the contents at the end of a node.

sh "mkdir metadata"
sh "touch metadata/metadata_init" // Stash won't work with empty dirs
stash includes: 'metadata/', name: 'metadata', allowEmpty: true

unstash 'metadata'

I added the calls to my pipeline, having everything write to a metadata stash. It was a good idea, but there was a problem…

My Problem

Because the stash is executed individually in parallel, only the files from the latest parallel stage were being saved and stored.

The Solution

As many issues in software, both the problem and the fix were rather simple. All I really did was change my stash call to used as documented. Still, I had trouble finding anything online talking about using stash the way I needed to, so I thought I’d share.

My solution was to continue to use the single directory method, but dynamically change the name of the stash during that initial parallel stage. It turns out, that the data was overwritten because each stash call had the same stash name, not because it was stashing the same directory.

I only need to parallel stash during the initial parallel stage, as the second one was at the end of the pipleline where the metadata would no longer be appended. So my plan was to dynamically stash each provisioner metadata, and then merge all the stashes into a single metadata stash to be used throughout the rest of the pipeline.

// Stash each builder's metadata if success
stash includes: 'metadata/', name: 'metadata-${testLabel}', allowEmpty: true

// Un-stash all successful builder metadata into single dir
for(label in testLabel) {
    unstash "metadata-${label}"
}

Before making things complicated by unmerging to a different location from where I wanted metadata to be, I first decided to see what happens if I simply unstashed everything directly to the metadata dir. Luckily, everything merged just fine with no overwrites. I even didn’t have to write a merging function! Problem solved.

Conclusion

So that’s it. Often, the solution to a seemingly complicated problem is a simple one. This can be made more difficult if all of the online discussion seems tangential, but not exactly what you need. In those moments, it is probably best to pull aside a buddy to bounce ideas off of. Even if they don’t know how to immediately solve your problem, they can highlight problem areas that you might be able to find a solution in.

That is exactly how this solution surfaced. My friend Elyezer pointed out that the stashes were likely being over-written because they had the same stash name. Even though I knew that, having him re-emphasize it helped me zone in and work out the idea of dynamically making different stashes and merging them for the rest of the pipeline.

Teamwork. It makes a difference.

(P.S. Thanks Elyezer!)

2020 M1 Macbook Air Initial Thoughts

2021-03-02T00:00:00Z

Durham, NC

As discussed in my previous post, I decided to trade-in my 2019 16" MacBook Pro and replace it with a new 2020 M1 MacBook Air. So far, I think making the swap was a terrific idea. Here are my initial thoughts…

Background (what I ordered)

The MacBook Air Box.

To recap, I got the ‘second tier’ 2020 M1 MacBook Air (Silver, 8 gpu cores, 512GB ssd) and upgraded the RAM to 16Gb. I wanted to maintain the same RAM and storage that my MBP had. After an eternity (3+ weeks), the air was delivered.

What I Like

Side of the MacBook Air.

First, lets start with the good. There is a lot to say here, but these are my main thoughts:

It’s cool and ~~quiet~~ silent. It hardly heats up, and when it does, it is barely noticeable.
It is very portable. I can easily pick it up with one hand, and set it on my lap nearly anywhere.
I like having physical function keys again. I even learned how to utilize BetterTouchTool for window management using keyboard shortcuts, instead of the touch bar.

Click for A video comparing instant wake

Instant wake is actually delightful. I initially thought my sleep settings were misconfigured, and that the laptop wasn’t turning off when I shut the lid. It’s a feature that I thought “who cares” when it was announced, but after I tried it, my 16" suddenly felt unbearably slow for the few days I had both.
Rosetta II is amazing at what it does. Most apps just work because of it, even if they haven’t been ported to Apple Silicon yet. This has made the transition rather seamless.
It works surprisingly well at my desk on the laptop stand, possibly even better than the 16". The laptop is about the same size as the stand, so I’m able to push it further back, out of the way, and it is much sturdier. The smaller screen hasn’t been an issue with the content I usually play on it at my desk (background videos or music). With how cool it runs, I also feel safer closing it to focus solely on my external monitor, as if I was using a mac mini.

Click for a video comparing opening teminal tabs

My terminal instantly opens! For some reason on my mac, it always took a second to load the shell prompt in iterm with my config. On the air, it is instant.
Garage band works much better. It opens immediately and is ready. My 16" would freeze while loading, then sometimes beach-ball and crash.
The battery life is great. I don’t really think about it.

What Could be Better

The Air has thicker bezels compared to my old 16-inch MacBook Pro.

The screen bezels are rather thick, even compared to the 16" and could use some slimming.
More TB ports. The air has two Thunderbolt ports, on only one side of the laptop. I use hubs, so I don’t usually require more than the two ports, but it can be an issue to only have them on one side depending on where I want to place my laptop. Having one port on either side wouldn’t solve the issue either, because I wouldn’t like wrapping the power cord all the way around to the other port when my hub is plugged in. The only true solution is to have more than 2 TB ports, split across both sides of the laptop, even if you don’t need them all at the same time.
Some applications are still tricky to install. For example, I had to compile neofetch in homebrew (which was simple enough to do). I imagine this issue should resolve with time. Overall though, I haven’t really had any major installation issues.

What’s… Different?

The MacBook Air does not have a touchbar.

There were a few things that were neither good nor bad, just different coming from an Intel 16" MBP:

Surprisingly, not having the touch bar. Not that I miss it, but I apparently got used to checking the time there and had to slowly break that habit.
The Efficiency cores are weird at first. I would look at my CPU usage when only running background tasks and wonder why the first four cores had such high usage, especially when compared to the other four. Eventually, I realized it was because the tasks were running using mostly the efficiency cores. So while it looked like there was a heavy load that it wasn’t being properly balanced across the cores… the work was actually shifted to be more efficient. This is the desired functionality, but weird to get used to at first if you’re someone that is constantly watching CPU usage like I do.
The smaller screen space. I got used to the smaller screen quicker than I anticipated, and I think the trade-off is worth it for the portability of this laptop. But it is different.
The speakers. The air has great speakers… but the 16" had truely amazing speakers. Like the screen resolution, this is mostly a physical size limitation, and I think the trade-off is well worth it here as well.

Conclusions

The MacBook Air with my coffee.

This laptop has been remarkable. It reminds me of some of my previous portable laptops, like the x230 thinkpad and my x201e. That is a good thing. I loved both of those computers and their ability to be thrown in a bag on the go. It is a setup that pairs nicely with a powerful workstation, which I currently have. While the air has about the same footprint as both of those previous laptops, it is thinner, lighter, has more screen space, and is magnitudes more powerful. I can’t complain with that.

Traded my 2019 16" MacBook Pro for a M1 MacBook Air

2021-02-11T00:00:00Z

Durham, NC

Over a year has passed since I purchased a base model 2019 16" MacBook Pro. My first Mac. With it’s large “retina” display, resurrected magic keyboard, and truly amazing speakers, it might be the most pleasurable laptop I have ever owned. However, I decided to trade it in… for a new MacBook air. Here’s why.

Background

Installing macOS Catalina on my wife's old MacBook Air for my Mac challenge.

In the Fall of 2019, I decided to do a 30 day macOS challenge. During the challenge, I learned that while I think Linux is still my go-to and indispensable OS, I did enjoy having a mac available for doing more “normal life computing”. Web browsing, writing emails, managing calendars, personal finance, planning home projects with my wife, ssh’ing into servers to code… that type of stuff.

After the challenge, I decided to acquire a mac for myself. I choose the 16" MBP mostly because it was the only MacBook at the time with the new magic keyboard. That was a hard requirement I had. I wouldn’t even consider buying a butterfly MacBook. Keyboard aside, as my only Mac, I did want it to be powerful enough to be my ‘workstation’ mac/laptop. And it was.

While the 16" MacBook Pro did improve cooling compared to it's predecessor, it was still too warm for my tastes.

After a few months of use, a problem emerged regarding that power… I didn’t utilize it. As I discovered the last time I had a more “workstation laptop”, I don’t like when my computers overheat and spin up their fans. The 16" macbook was relatively thin for its specs (did you see the size of my previous workstaion laptop!?!?). However, that sleek design came at a cost. While the re-designed cooling on the 16" was considered quite good for a MacBook… it didn’t meet my standards.

While using the CPU (importing photos), the temps stayed at 80C with turbo boost disabled. After enabling it, the temps shot up to 100C!

I eventually purchased and installed turbo boost switcher, to disable turbo boost on my cpu This runs the cpu at it’s base clock, without boosting it up for more ‘bursty’ performance. This didn’t seem to hinder my small day-to-day usage, and in my mind, was more than worth the trade to keep the computer a little bit cooler, and prevent the fans from spinning up all the time. It was just a shame to pay so much money for such powerful computer… and then purposefully hinder it.

The M1 Macs are Released

Apple announced it's first Apple Silicon CPU, the M1.

Then, in November, Apple announced the new M1 Macs, the first computers in the company’s planned 2 transition to switch from Intel CPUs to Apple’s own “Apple Silicon” arm processors. And they were a hit.

While I was skeptical about the smoothness of a cpu architecture switch, early reviewers indicated that Apple handled it extremely well. This had me tempted to immediately swap my MBP in for one of the newer ones. But, I decided to hold out for a bit, reasoning that my 16" computer was still an amazing laptop that was more than capable of what I needed it for.

Still, the seed of questioning how I use a mac had been planted.

How I use a Mac

I started to map out my Mac use-cases in two broad categories: What I use my mac for, and where I use it. After some brainstorming, I had the following lists:

What I use a Mac For

I most commonly use my Mac for:

Utilizing Apple apps and services (Messages, FaceTime, Notes, Reminders, Photos, GarageBand, Music, iCloud, Etc)
Web Browsing
Online Courses/Proctored Exams
SSH and/or VS Code (I do coding, writing, and all those other tasks on my workstation, and remote in when on my MacBook)
Personal Planning/organizing, emails
Listen to podcasts or music
Watching videos or livestreams
Chatting with Friends and Family
Maybe a light game if I’m on the go or not at my desk (possibly streamed from my gaming VM)

Where I use a MacBook

As my main laptop, my MacBook occasionally has to serve as my portable workstation.

Being both my only Mac, as well as my main portable computer, there is actually quite a range of where and how I setup my MacBook. Some examples include:

Portable: I use it as my portable machine, in a few types of locations:
- Sitting on the couch
- At a Table, by itself
- At a table on a stand w/ Keyboard, and possibly other external displays
As my single portable workstation if I travel or visit family
As my Mac ‘desktop’, connected to all the devices at my desk via a thunderbolt hub.

Best Tool for the Job

Using my MBP while running benchmarks on my desktop after [some upgrades](/post/selecting-charmeleons-upgrades/). Dispite its pwer, my Mac is a *secondary* computer.

A disparity stood out to me while makings these lists. I generally try to use appropriate hardware for my need (at least within reason). Looking over what I wrote, it felt weird to have a $2000+ laptop for what I use it for. I did not need that much for my secondary computer, even if I loved the speakers and screen.

If this was my main computer, it would be fine. But it isn’t. I have a powerful linux workstation for my heavy lifting and complicated, virtual workloads. Hell, macOS isn’t even my main OS. I do all my power-user computing on Linux… even if it’s from my mac (as previously stated, I ssh into my desktop for nearly everything… including writing these posts). The MBP met the role I needed it for, but it didn’t necessarily fit that role.

16" MBP vs. Air

The 2019 16" MacBook Pro and the 2020 M1 Macbook Air.

After hammering out my MacBook use cases, I theorized how a M1 MacBook Air might compare to my 16" MBP in meeting those needs. I came up with the following conclusions:

While I do occasionally use some of the mac’s heavier software (garageband, imovie, Photos, and others), the performance of the M1 is more than enough for what I would need it to do, and even beats my Intel MBP on some tasks.
It wouldn’t benefit me to wait for more powerful Apple Silicon Macs, as my needs are already passed for what I need this device to do. Even if an Apple Silicon 16" MBP was available, tempting as it might be, I’ve already established it isn’t a good fit for this device.
While I would miss the screen space and speakers, the air would be much more portable. Not only would this be great when traveling, but it would also be appreciated when picking it up with one hand to take into another room.
The dedicated GPU in the MBP doesn’t really benefit me, as I do all my gaming in a Windows VM on my workstation. In fact, it may actually hurt my experience by generating excess heat and kicking up the fans.
And on that note… the air doesn’t even have a fan to spin up, and doesn’t appear to get too warm either. Unlike the M1 MBP, it will throttle to cool the CPU during sustained workloads, but we’ve already established that I am currently doing that on my 16" anyway ☺︎.

Overall, I concluded that the Air should be able to do most of the tasks my 16" MBP does, at around the same level of perfomrance, if not better. In the case of portability and noise, the air would be much better suited than my Pro. Additionally, most of the workloads the MBP is better at, aren’t as important for my needs.

Decided to Go for it

I knew my trade-in value would only get worse, and I didn't need to wait for the new 16" Pro to come out.

Eventually, I decided to make the trade-in. My main reasons were:

The current trade-in price would cover the cost of the “higher end” M1 Macbook Air, and I would only have to pay tax and for any other updgrades I wanted to make.
I knew I would slowly get less and less for my MBP. I figured once a new 16" MBP was released, the value for mine would likely plummet.

So, I ordered the higher tier Air ( with 8gpu cores + 512GB of storage), and bumped up the RAM to 16GB. If I was going to be be “downgrading” from a MBP to an Air, I wanted to at least keep the same amount of RAM and storage that I had before. It took almost a month to arrive, but now it’s here, and I’m packing up the MBP to send back.

I need this secondary computer to function as my portable laptop, and only (physical) Mac. I think the m1 air will fill both of these roles extremely well. If I needed a laptop to be my main machine… I would have probably kept the MBP and upgrade after a newer Apple Silicon version is released.

Conclusion

My old 16" MacBook Pro.

The 2019 16" MacBook Pro was, and still is, an amazing machine. I will miss it. Ultimately though, a combination of the M1 Macs being very good, and the fact that a smaller, thin and light laptop fit my use case best, finalized my decision to switch.

Automatically Create and Run a Podman Container Using Systemd

2021-01-31T00:00:00Z

Emerald Outback, Beech Mountain, NC

No, this is not the same as my last post, but is a continuation of it. While the basic podman generate systemd generated file works for many cases… it wasn’t a good long-term solution for my jellyfin container. So, I made a small tweak.

The Issue

My issue was that the container initially worked, but would occasionally break when it grew old. By itself, it isn’t a problem. Containers are designed to be ephemeral, and arguably should be blown away before being instantiated again. The difficulty is that creating a new container results in having to edit the container uuids in the service file, otherwise the service breaks… which seems to sometimes then break the new container.

This requires me to then disable the service, blow-away and make the new container, update the service file, and finally start the service… again.

A Different Approach

I started to think about how I could better the process. I knew that a newly created container seemed to work each time, as I previously used the --rm flag when starting it manually. From there, I wondered if I could write my own systemd service file using the podman run command instead. This would both create and kick off a new container after boot, instead of re-starting a persistent one.

Testing it out

To test out my plan, I decided to edit the file created by the podman generate system command in the last post and go from there. I opened the service file (~/.config/systemd/user/jellyfin.service) in my editor and changed the following line under the [Service] section:

ExecStart=/usr/bin/podman start 2ba0f86b0fc53cb2fe43abb20215680982800c1bf53421e1a3a90855fa79f030

I swapped ExecStart to use my manual podman run command, with a --rm flag:

ExecStart=/usr/bin/podman run --name jellyfin --rm -d -v /home/ryan/Network/jellyfin/config:/config -v /home/ryan/Network/jellyfin/cache:/cache -v /home/ryan/Music:/media/music:ro -v /home/ryan/Videos:/media/videos:ro --net=host --privileged jellyfin/jellyfin:latest

I also edited the ExecStop and ExecStopPost values:

ExecStop=/usr/bin/podman stop -t 10 2ba0f86b0fc53cb2fe43abb20215680982800c1bf53421e1a3a90855fa79f030
ExecStopPost=/usr/bin/podman stop -t 10 2ba0f86b0fc53cb2fe43abb20215680982800c1bf53421e1a3a90855fa79f030

I switched them to use the container name (jellyfin) instead of uuids:

ExecStop=/usr/bin/podman stop -t 10 jellyfin
ExecStopPost=/usr/bin/podman stop -t jellyfin

After saving the changes, I removed my old jellyfin container, reloaded the unit files (using systemctl --user daemon-reload, as described in the previous post), and restarted the machine to determine what elements would need tweaking when writing my service file.

Another Shortened Post…

However, I never made my own service file, or even edited the generated one again. It appeared to work fine after making those few tweaks. For the second time in a row, I have a much simpler post than originally intended due to the tooling around podman. I’m okay with that.

Conclusion

While the solution still isn’t 100% perfect (containers sometimes don’t destroy themselves, and I still have to login for the service to autostart) it has overall been working great for me. I no longer have to worry about uuids. When I encounter an issue, I stop the container, it kills itself, and then the service starts up a fresh one. Much better.

Create Podman Systemd Services

2021-01-15T00:00:00Z

Emerald Outback, Beech Mountain, NC

On my Linux workstation, I have started to host a jellyfin server using podman. I have also started to shutdown my computer when I go to bed, as all of our important services (ex: home automation) are hosted on my home server. With that said, there is one remaining problem with this configuration. When I boot up my computer the next morning, my containers do not automatically start… and I always forget to start them up myself. Let’s fix that, using systemd.

Create the container

In order to define a systemd service for a container, the container needs to already be created and running. The podman run command is used to start a new container. For example, to run my jellyfin server, I used the following command:

podman run --name jellyfin -d -v /home/ryan/Network/jellyfin/config:/config -v /home/ryan/Network/jellyfin/cache:/cache -v /home/ryan/Music:/media/music:ro -v /home/ryan/Videos:/media/videos:ro --net=host --privileged jellyfin/jellyfin:latest

It should be noted, that because we will want the container to persist, even if it is stopped, the --rm flag should not be used here. I normally add this flag to keeps things clean, but had to remove it for the service file.

Root vs User containers?

Before getting started, I just want to mention user vs. root containers. In my first attempt to start podman containers with systemd, I hit some errors and then I realized… systemd was running as root but the container ran under my username. For example, running sudo podman ps -a didn’t list my container, but podman ps -a did.

Once solution could be to switch over and run the container as root using sudo, but that didn’t feel right. A benefit of podman is that it is able to run rootless, and to not take advantage of that feature would be a shame. So, I started running the systemd steps as the user, but providing the --user flag and it resolved my issues.

Creating the Service File

At first, I started creating the systemd service files manually, just as I have done in previous posts. Although, after reading an example in this article, I was reminded that there’s actually a podman generate systemd command. This command will assemble and output a unit file for a container. I ran it, and saved the generated service file to my user local systemd location.

podman generate systemd jellyfin > ~/.config/systemd/user/jellyfin.service

Simple! This file could be altered if needed, but after quickly skimming it I thought it looked good.

Starting & Enabling the Service

Before starting the service, it is a good idea to have systemd reload the user unit files:

systemctl --user daemon-reload

Afterwards, I started the service, and checked the status to confirm that the service started up without issue.

systemctl --user start jellyfin.service
systemctl --user status jellyfin.service

Lastly, I enabled the service, again with the --user flag so that it would automatically start on boot:

systemctl --user enable jellyfin.service

Testing it out

Jellyfin service running in a podman container auto-started at boot.

With the service setup and enabled, I rebooted my computer to test it out. After booting up, I used podman ps to prove that the container was started:

podman ps -a

With the container running… I next opened up my web browser to verify that jellyfin was actually working… and it was!

Caveats

While this solution works for the most part, I did hit two small annoyances:

Running as a user service, the service won’t start until the user has logged in. This makes sense, and can be resolved by quickly ssh’ing into the machine. However, it should be known and worked around if using something like WOL startup, as sending the magic packet won’t be enough to get the service containers up and running.
If something breaks with the container and it has to be reset, the service file should also be regenerated and replaced. The file references container uuids, and if that changes, the file needs to reflect that. It’s not a big problem. Containers break and that’s okay (remember, they’re designed to be ephemeral). Just remember to remove and regenerate the service file when it happens.

Conclusion

When I set out to write this post, I though there would be a bit more to it, requiring me to create the service file manually. However, I quickly stumbled on the podman generate systemd command and I am glad I did. It is one more feature to add to the ever-growing list of reasons while I love podman and the other container commandos. Enjoy!

Selecting Charmeleon's Upgrades

2020-12-28T00:00:00Z

Durham, NC

After months of planning and price tracking, I have finished upgrading several major components in my Linux workstation. What started as a planned cpu and ram upgrade, eventually ballooned to also include a new motherboard and a secondary gpu. Let me explain…

Background

Charmeleon Internals before the upgrades.

Over two years ago, I built and designed charmeleon, my Linux desktop computer. In that post, I explained how I wanted to upgrade charmeleon over time, and designed it with that in mind. Since the initial build, I have only made a few minor enhancements: I added a nvme ssd and upgraded the gpu after the original one stopped working. The next change I hoped to make was adding more RAM. However, as I started down that path, things became a bit more… complicated…

Part Selection

RAM

RAM upgrade: 2x32 GB of 3200Mhz DDR4, CL 16.

An easy upgrade I prepared for during the inital build was RAM. My motherboard had 4 slots with a max capacity of 64GB. So, I made sure to populate it with 2x16gb sticks instead of 4x8gb ones, so the capacity could be effortlessly doubled when RAM became cheaper. But, the release of Ryzen 3000 cpus changed things. Overnight, a Ryzen 3000 serries processor paired with my motherboard could support up to 128gb of RAM.

This made me wonder if I should instead fill those blank spaces with the new max dimm size sticks. The only problem was, if I wanted to upgrade RAM to the new max size, I also needed to upgrade to a newer CPU. Oh, and also buy twice the anticipated capacity of RAM.

Spoiler alert: I did also buy a new cpu. More on that below.

Eventually, I settled on a Ripjaws V 2x32GB 3200Mhz kit with 16-18-18-38 timings. There wasn’t a 2x32gb pair that matched the model of my original RAM, and but this one had all the same timings while still being cheaper than other alternatives. A total of 96GB of RAM is definitely overkill for most uses (arguably, even my own). However, I tend to run large, automated VM deployments and often use my /tmp/ dir as a large working folder, so the extra overhead is nice to have.

TLDR: I always planned to expand my RAM. Ryzen 3000 allowed an even higher max RAM capacity with my MB, but I would have to also upgrade my CPU. I upgraded both, and now have 96GB of RAM.

CPU

CPU upgrade: Ryzen 9 3900x, 12 core/24 thread cpu.

My origonal plan when I built charmeleon was to get the 2600, and later upgrade to a 8 core 2700x or a 3000 cpu that was rumored to potentially have even more cores. I made sure my motherboard had good VRMs for the cost, so it could support a future upgrade with more cores.

Months later the 3000 cpus launched, and I was set on getting a 3900x. However, I didn’t need a new cpu yet and I knew AMD was going to be releasing the “4000” cpus in the Fall. I figured I would wait until they were released, and then buy a new one, or a discounted 3000 serries one.

When the 5000 series cpus were released, I thought the 5900x would be perfect. Unfortunately, like ever other tech release of 2020, it was nearly impossible to buy one. I didn’t feel like waiting even longer, and I don’t actually need the increased performance. The new gains are particularly observed in gamming… but I’m rocking an rx580 and am clearly gpu bottlenecked anyway 😆. I can manage just fine with slower single core speeds…

With the 5900x not available anytime soon, I became tempted by some crazier options… like the 3950x. The 3950x is an amazing cpu, and honestly I’d still love to have one. However, even if I could get it on sale… I knew it would still cost a ton and I couldn’t justify the price difference. While a 16 core cpu is amazing, a 12 core still easily covers my needs. The money “saved” on the price difference could then be used to get the bigger ram upgrade I wanted.

So, I looked back at the 3900x and wondered if it would go on sale. It did occasionally, but seemed like there might start to be shortage issues with people jumping to it after not being able to get a 5900x. The next time I saw it go on sale, I snagged one just as they sold out again. I didn’t get it for the cheapest it’s ever been, but the sale was quite close to the all time low. And it was much cheaper than what it has remained at ever since.

TLDR: I planned to upgrade to a higher core count CPU. After waiting, I wanted a 5900x, but you can’t buy them. After being tempted by the 3950x, I purchased the more reasonably priced 3900x on sale.

CPU Cooler

CPU Cooler upgrade: Noctua NH-U14s.

Over the summer I upgraded all the fans in my case to Noctua ones and love them. At the time, I wanted to also get a new cpu cooler to match, but with plans to upgrade my cpu soon, I figured I would wait and order both at the same time so I wouldn’t have to go through the hassel of repasting the CPU twice.

I switched what specific model I wanted once or twice, after being confused about compatibility. I didn’t want the massive NH-D15, but did want a cooler that used a 140mm fans instead of 120mm ones, because I much prefer their lower sounding “woosh”. After I confirmed it should fit in the Meshify C and with my RAM, I selected the NH-U14s.

TLDR: I wanted a new CPU Cooler with a 140mm fan. I love Noctua.

Motherboard

Motherboard upgrade: x570 Aorus Elite Wifi.

Unlike the CPU and RAM, I never intended to get a new motherboard when building Charmeleon. When I was looking at getting a 5000s cpu, I knew my B450 thomahawk wouldn’t get the required bios upgrade until January… maybe. I also knew the upgrade might not be reversable which I didn’t like, so I started to look at B550 and x570 boards.

I realized x570 fit my needs a bit better when compared to my b450, and even the new b550 (which I think most people should probably get). The x570 paired well with the other upgrades I was making. Particular features I liked compared to the b450 tomahawk were:

Gen4 PCIe support
Better iommu groupings for VFIO passthrough
Often had a second m.2
More usable sata ports (2 of mine were disabled because they shared lanes with my m.2),
wifi (mostly for bluetooth)
In general, more pcie lanes

I looked at mother boards for months while waiting for Ryzen 5000 to officially be announced. I decided that I wanted a x570 Gigabyte Aorus board, but wasn’t sure which one. The Aorus Master seemed to be one of the defacto recommend boards for VFIO builds, but it was very expensive. Much of what made it so good however for iommu seemed to still be present in the cheaper Aorus models. In particular, the Aorus boards had great iommu groupings and support, and a bios feature that allows you select which x16 slot to use for the primary gpu. This makes it easier to boot into a secondary gpu, saving the beefier one to pass through to VMs.

I went back and forth trying to decide between the elite wifi, pro, ultra… and sometimes even the master, for weeks. When I started making purchases, the elite wifi went on sale and I decided to grab it. It had everything I needed, and most of the upgraded features on the other boards I either wouldn’t use or were just small nice to haves.

TLDR: Looking at Ryzen 5000 CPUs had me looking at newer motherboards. Even after dropping back down to a 3000 CPU, I still wanted a motherboard upgrade for better VFIO support and expandability.

GPU

Second GPU: RX 550 4GB.

With a motherboard that better supported vfio passthrough, I wanted to get a smaller gpu that I could use for host machine’s graphics. I read that the rx550 and 1030gt were both good, sub $100 gpus often used for the host.

I wasn’t thrilled about using a nvida gpu on a linux host, but I loved how small it was (although that could translate more fan moise). In comparison, the 550 seemed to be slightly more powerful, was amd, and well rated. In fact, I had read several gpu passthrough guides using the rx550 and passing through a rx580 (my current gpu). I purchased the 550.

TLDR: I wanted an easier time with VFIO GPU passthrough. Having a second card for the host or a second VM makes this easier. The rx550 is a common card for this use.

Conclusion

Charmeleon Internals after upgrades.

Overall, I am extremely happy with my selections. It’s surreal how powerful of a machine can be built with a modest budget these days. With the upgrades, Charmelon’s specs now stand at:

Ryzen 9 3900x [3.8 GHz (4.6GHz Boost), 12 Cores, 24 Threads)
Noctua NH-U14s CPU Cooler
Aorus Elite Wifi x570 Motherboard
96 GB (2x16GB, 2x32GB) DDR4-3200 Mhz, CL 16 RAM
Sapphire Radeon Pulse RX 580 8GB GPU
Sapphire Radeon Pulse RX 550 4GB GPU
500 GB Samsung 970 EVO NVME SSD
500 GB Samsung 850 EVO SATA SSD
120 GB Kingston SSD (Dedicated windows VM)
EVGA SuperNOVA G4 650w 80+ Gold, fully modular PSU
Fractal Design Meshify C Dark TG ATX Mid Tower Case
2 x Noctua NF-A14 PWM 140mm Fans
1 x Noctua NF-F12 PWM 120mm Fan
Fedora 33 (KDE Plasma)

Not only have these changes drastically improved the performance of my workstation, but they have allowed me to run virtual machines at the next level. When passing hardware directly to my VMs, I cannot tell that the computer I’m sitting at is virtualized. I have been living inside Vms, and I love it.

That’s it! I still want to get a second nvme sdd, but other than that I think upgrades to Charmeleon will slow down for a bit…

Opening Ports with firewall-cmd

2020-11-30T00:00:00Z

Emerald Outback, Beech Mountain, NC

I use a handful containerized services on my workstation. Jellyfin and minecraft servers are two examples. Many of these self-hosted applications require ports to be opened in order to work. However, I often forget this. While I am getting better about remembering to open the ports… sometimes I forget how. Not anymore.

The Problem

Firefox unable to connect to my hugo page.

To reiterate: The problem is simple. The services need a port opened, but my firewall is blocking it. The solution is just as simple: ~~disable the firewall~~ open the port.

Determine Zones

Before opening ports, lets first determine what firewalld zone to apply the change to. The following command will show all active zones, and which devices are in each one:

sudo firewall-cmd --get-active-zones

Additionally, I use the next command to help figure out what zone is my default.

sudo firewall-cmd --list-all

It is usually public. The --list-all command will also show which ports are already opened in the ports section, so use it to verify that they aren’t already opened.

Opening the Port

Connected to Hugo after opening the port.

Lets open that port. Change the port value in the following command to whatever you need. I used 1313 here to open a port for hugo. Remember to also set the zone to whatever was found in the previous step:

sudo firewall-cmd --zone=public --add-port=1313/tcp

To verify:

firewall-cmd –-list-ports

Afterwards, try connecting to the service again. If it still does not work, it is possible that additional or different ports may need to be opened.

Make it Persistant

If everything does work, the change can be made persistent by running the same command again, but this time using the --permanent flag:

sudo firewall-cmd --zone=public --permanent --add-port=1313/tcp

Reload

When running the command with the --permanent flag from the start, the changes might not take affect until firewalld is reloaded. This command should apply the changes:

sudo firewall-cmd --reload

Conclusion

That’s it. It’s a short post, but it’s one I will use. I am trying not rage sudo systemctl stop firewalld anymore. This should help with that.

My new Ergodox-EZ Keyboard -- Initial Thoughts

2020-11-27T00:00:00Z

Durham, NC

Several weeks ago, I wrote about how I decided to purchase an ergodox-ez split keyboard. In that post, I omitted any details about what configuration I ordered, as well my impressions about the board. Now that I have been an ergodox owner for… wow, over two months… I guess this post is due. Here are my ‘initial’ thoughts of the ergodox-ez…

Ordering and shipping

The boxes the ergodox-ez shipped in.

So, what ergodox did I order? After a few hours toggling the configuration tool and endless internal debating, I eventually settled on an ergodox-ez glow (backlit keys), with a black body, and black printed, LED compatible, key caps. With a narrowed selection of switch options (to maintain backlit key compatibility), I landed on Cherry MX Browns as my best choice. I wanted a noticeable tactile bump, but didn’t want “click-y” keys. I prefer a lower “Clack”, while typing and find the higher pitch of MX Blues annoying. I decided that I might as well go all in with this keyboard, so I ordered the tenting poles and black wrist rests as well.

With everything decided, I placed my order and prepared myself to wait in anticipated glee for… an average of 3 weeks shipping time ಠ_ಠ. Luckily, I was fortunate and about a week later, I had a large, flat box show up at my door!

Opening the main box.

Everything was very well packaged. The keyboard came in one box, and the wrist rests had their own. The two boxes had the same width, so they could be shipped as one unit. Lifting the lid on the larger box, I was greeted by the keyboard looking back at me through a plastic panel, with it’s accessories (cables, key switch puller, screws) neatly pinned around it.

Setup

The initial physical setup was quite simple. I took everything out of the box, connected the two halves of the board together with the included TRRS cable, and then used the mini usb cable (also included) to connect right half of the board to my laptop’s usb port. I positioned the wrist-rests and tweaked the tenting poles to some arbitrary position, acting as if I knew what I was doing. That was it. After that point, I was able to type… stuff… into my computer.

Initial Layout Changes

The default ergodox layout.

I say stuff, because I wasn’t able to just pick up the board and feel 100% comfortable like one might with a “normal” keyboard. There was definitely a learning curve (more on that later). However, the main problem was that I didn’t even know the default layout. The back-lit QWERTY keys were labeled, but all of the modifier keys on the sides and in the thumb clusters were not. So, I navigated online to the oryx configurator with two goals in mind:

To learn the standard layout that the board shipped with, and
To alter some of the defaults with changes I knew I would want (caps lock as a ctrl, for example)

After a few minutes of scanning and tweaking, I had my first layout to flash. I downloaded the file… and ran into my only true issue I’ve experienced with the ergodox…

Issues with Wally

Using the ergodox-ez with my MacBook Pro.

While I was able to flash the firmware using my macbook, for some reason my Fedora laptops and desktop had issues at first. I would open up wally (ZSA’s firmware flashing tool), load the firmware file, and press the tiny ‘reset’ button on my keyboard with a paper-clip… and it would freeze. The back lights stopped their colorful dancing and key presses rendered no response. The only way to get the board to respond again was to unplug and replug it back in.

I had heard that ZSA had good customer support, so I emailed them about my problem. They responded right away and helped to debug what might be happening. It appeared to be some sort of permissions issue, although to this day I’m not 100% sure what it actually was. Eventually, it started working. Since then I have updated most of my computers to Fedora 33, and I use a newer version of Wally. I haven’t had trouble since, even on fresh OS installs. It just works.

Oh well, problem resolved I guess? My main takeaways from the experience were that 1) wally definitely requires sudo root permissions, and 2) that ZSA’s customer support is wonderful. Oh, and open source firmware is the best, but I already knew that.

(Note: ZSA has since updated the Linux README instructions to contain the udev instructions they gave me)

Getting used to and learning the board

As previously stated, I knew I would have to put some effort into learning how to type on the ergodox. When I got it, I did a fair bit of typing practice and used it for work everyday. Within a few days, I was at a passable typing speed, and after a few more, and I reached a decent speed. I am not sure if I am any faster than on my old keyboard, but I can type fast for much longer (I could only burst speed-type on the HHKB before starting to cramp up, mostly due to poor form).

The Oryx Training Tool.

I haven’t done any typing practice the last few weeks and I’ve started to notice some of my bad habits creeping back in. I want to start up again and might try to use the trainer that is included in the Oryx tool. It is able to connect to the keyboard to show your layout, and provides several types of content to type through (code, books, symbols). I might use it to help determine an optimized coding layer places my most used symbols in easy to reach places.

What I like

Customizability

This board is fully customizable on both the hardware and software side.

Firmware

One of my custom layer layouts.

Being able to tweak the layout of multiple layers means there are an endless amount of ways I can tailor my keyboard to fit my needs. My tweaking has slowed down a bit, but I love that I can make quick changes to my layout if I decide there is a better way to do something. For example, today I realized I wanted to add a key on one of my alternate layers, so I popped open oryx and wally, and had the change finished in about a minute.

Having an open firmware keyboard is reminiscent of when I used vanilla arch linux. I would go weeks assuming I had everything required for a typical desktop environment, when all of sudden, I would try to open a pdf and realize I never installed a pdf reader… or a file browser. Last week while working, I noticed that my ergodox layout didn’t have a ‘delete’ key. So I quickly added one. Done.

(Note: Be aware that I haven’t even touched on the advanced layout options, like macros and long vs short key presses)

Hardware

Testing swapping out a switch with one from my switch tester.

In addition to the firmware, It’s nice to know that there is quite a bit I can physically change about my keyboard. Specifically, the key caps can be changed and the switches themselves are hot-swappable.

I’ve already looked around for some of the parts I could swap out on my board in a few years if I want to mix things up. For example, I am interesting in swapping to mx clear switches someday. The problem with clears is the lights would not (ironically) shine through anymore. I guess that wouldn’t be too big of an issue though, because many of the alternative key caps I like are opaque ツ.

Anyway… Despite talking about all the hardware upgrades I could make, I actually do enjoy it’s current configuration. But I do love that I have the option to change it over time.

Ortholinear

While it was more difficult to get used to than I expected, I have found using an orthlinear layout to be considerably easier to type on. My fingers no longer fight and trip over each other like Black-Friday shoppers trying to grab the last hot-ticket item. The worst thing about using an ortholinear layout is that I now want it to become a standard option when buying laptops: ISO or ANSI, Ortho or Staggard? Unfortunately I don’t see that happening anytime soon…

Split Design

A split design keyboard pairs nicely with a laptop stand.

Like the ortholinear layout, a fully split keyboard is a considerable ergonomic boost. I can lay my arms on the desk in whichever way is most comfortable… and then position the keyboard to match them.

As a bonus, the split design pairs nicely with laptop stands, allowing me to slide the stand closer. This makes it easier to view smaller screens at a proper height.

Thumb cluster keys

The thumb cluster of keys has allowed me to better utilize my thumbs, which has been a nice change.

Another feature I immediately miss on other keyboards, is thumb keys. It is a relief to provide my thumbs control over more keys than just the spacebar. The thumb clusters enable me to offload responsibility for backspace, tab, enter, delete, and a few other keys from my weakest fingers (pinkies), to my strongest (thumbs). This makes a huge difference while typing, but is probably the biggest difficulty I have when switching to a laptop keyboard from my ergodox. I keep finding myself smashing the spacebar when trying to backspace or enter.

What I don’t love

Honestly, there is not much that I don’t like about the ergodox-ez. That being said, if forced to be picky, here is a thing or two that I don’t love about it…

Learning curve

Results from one of my typing practices.

The biggest con to getting a keyboard like the ergodox is the learning curve. It isn’t something that you will be able to pick up and immediately start using. It will require time and deliberate daily practice to really start to feel comfortable. If you know this when purchasing, it isn’t that big of a problem, but can be extremely frustrating at first.

Cost

It is expensive. Especially for a unique board that you might end up hating. With that said, if the ergodox works for you, the cost isn’t as bad as it might initially appear. The ergodox-ez is a solid board, with the ability to customize both it’s hardware and firmware over a span of many years. On top of that, when compared to other ‘ergonomic’ split keyboards, the ergodox is priced rather fairly.

It’s difficult to switch back to my HHKB…

Switching to keyboards on my laptops is mostly fine, but for whatever reason … I can’t type on my HHKB. It feels so squished, and I can’t seem to naturally position my fingers on the correct keys anymore. I’m sure if I dedicate a few hours to using the HHKB this might go away, but I haven’t done that yet.

Conclusion

The ergodox-ez in my home office.

Overall, I love this keyboard and am very satisfied with my choice. Now that I am used to it, it feels normal to type on and I don’t have to think about it. The Split design and ortholinear layout make it the most natural feeling and comfortable board I have used. As a bonus, any layout issues can be stomped out with a 2 minute config change and flash, ensuring that I should continue to love this board even as my preferences change. As long as you are aware of what you are getting into, I highly recommend the ergodox-ez.

Automated Nativefier App builds

2020-10-31T00:00:00Z

Kill Devil Hills, NC

Most applications we use today are fancy web pages, wrapped up in a desktop shell. Many people even forgo desktop builds, instead opting to run webapps simply as another tab in their web browser (Ex: Slack, Discord, Notion). Personally, I prefer to have dedicated windows opened for my essential tools. As a result, I love using nativefier to create desktop versions of my favorite web-based applicatons. The only problem is… it can be a pain to setup. Lets fix that.

What Apps I use nativefier it for

Using the Pocketcasts web app in a desktop window via nativefier

I use nativefier for all of the web services that are missing a Linux client, or web tools that I use often and want a dedicated desktop client for. On most days, I use the following tools via a nativefier desktop build:

Pocket - to read articles
Pocketcasts - to listend to synced podcasts between my phone and computer
Fastmail - For my personal email and calendar
homeassistant - To control parts of my house (mostly lights and temp)
Notion - My notes and planning tool
Soundcloud - For music while working

Also, while not used every day, I have made nativefier builds for twitch and icloud. These are both apps that exist on other desktops but don’t have an official Linux build.

How I make them feel like normal apps

There are a few steps I take to help make these builds feel more like native ones.

Desktop Files

The Pocket and Pocketcasts builds showing as normal applications in search

The first, and argualy most important step I take, is creating a desktop entity for each nativefier build. A desktop entity defines an application in Gnome and KDE Plasma desktop environments, allowing it to show up in menus and launchers. Defining an entity for the nativefier builds tells the system to to treat the build as a normal application, which is what we want.

Icons

The second action I’ve started doing is making sure that I use proper icons when I run nativefier. I already needed the icon for creating the desktop entity, but I noticed the icon displayed in the running app often didn’t match the one defined in the desktop file. This extra step resolves this issue and helps make everything look more cohesive.

How I usually create nativefier builds

Coping to a new computer often requires editing all the application files

While these additional steps help nativefier builds behave like real desktop applications, it adds a bunch extra work to the creation process. When I want to make a new nativefier app, I usually have to:

Install nativefier and it’s dependencies (usually in a podman container)
Build the application to a build directory
Find an icon for the application and add it to my ~/.local/share/icons/ folder
Create a new .desktop file for the application, which involves:
- Filling out the description
- Changing the exec paths.
- Adding the icon path
Verify it works (something is usually missing)

To save some time when setting up a new system, I often copy the build directories along with the desktop and icon files. However, even that can be quite tedious. For example, if the new system uses a different username (ryan vs. rhimmew), I have to update all of the desktop files to reflect the change.

We can do better. Lets automate it.

Using a podman container instead

Before diving into the automating the steps, there is one improvement to my normal process I want to make. Instead of installing nativefier from a npm install, I want to switch to using an ephemeral nativefier container for the builds. This has several benefits, including not having to install dependencies, and always using the latest version for the automation. Plus… containers are fun.

After tweaking the suggested docker instructions, I was able to get a nativefier container building apps on my desktop using podman:

# Pull the Image
podman pull jiahaog/nativefier

# Run a build for a container
podman run --rm  -v ICONS_DIR src -v NATIVEFIER_BUILD_DIR target jiahaog/nativefier --icon /src/ICON --name NAME -p linux -a x64 URL /target

This command had a few parameters to swap:

ICONS_DIR: the directory that contains the application icons to build with.
NATIVEFIER_BUILD_DIR: the directory to save the application builds to
ICON: The filename of the image to use for the icon
NAME: The name of the application (ex: pocket or homeassistant)
URL: The url of the webapp to build

This command spins up a podman container, passes our variables through, and builds a nativefier app. After finishing, the container automatically deletes itself.

Automating the process

Now that we’ve figured out how to run nativefier with podman, we can focus on writing the automation. Like I have done previously, I will be implementing the automation by creating a new role to use in my ansible playbooks. If you are unfamilar with roles, checkout the ansible quickstart post I wrote earlier this year.

To start, I’ll create a folder for the new role, with tasks, defaults, files, and templates sub directories.

Vars

First, lets define default values for the variables we will use in tasks and template files:

user: ""  # User to run as
nativefier_build_dir: "/tmp/"  # Location to build apps at
nativefier_dir: "/home//Builds/" # Location to move completed builds
icons_dir: "/home//.local/share/icons/" # App Icon location
applications_dir: "/home//.local/share/applications/" # desktop entry location

We will define one more variable which won’t have a default, nativefier_apps. However, I’ll talk about that later when we use the role in a playbook.

Templates

Next, lets create the one template file we need for this role: an application.desktop file. Create a new file in the templates directory named nativefier.desktop.j2 and paste in the following contents:

[Desktop Entry]
Name=
Comment=Nativefier app for 
Exec=/-linux-x64/
Terminal=false
Type=Application
Icon=
Categories=Multimedia;
TryExec=/-linux-x64/

This template will take the variables for each nativefier app we define in the playbook, and use them to fill out a desktop file for each app.

Files

The role's files folder, containing the app icons

Before writing tasks, there is one more role directory to fill: files. We want to include the icon files for each application in this role. So, fill the files sub-directory with the icon files to use.

Tasks

Time to write the tasks. Create and open up a main.yml file in the tasks sub-directory.

The first few tasks will check and ensure that the directories we intend to use exist. This is generally a good practice to prevent playbooks from breaking due to missing folders:

- name: "Ensure Icon dir exists"
  file:
    path: ""
    state: directory

- name: "Ensure applications dir exists"
  file:
    path: ""
    state: directory

- name: "Ensure  exists"
  file:
    path: ""
    state: directory

- name: "Ensure  exists"
  file:
    path: ""
    state: directory

Managing Icon Files

Next, add a small task that will copy the icons we included with the role, to the user’s local icon folder:

- name: Move icons to local folder
  copy: src="" dest="/"
  loop: ""

This will ensure that all the icons are already in place when we build both the nativefier app and the .deskop files.

Automating the podman builds

We can now define a few tasks that will pull and run the nativefier container using podman. (Note: This could be done with docker… but I prefer podman XD):

- name: Pull down nativefier container image
  shell: "podman pull jiahaog/nativefier"

- name: Build Nativefier app via podman
  shell: "podman run --rm --security-opt label=disable -v :/src -v :/target jiahaog/nativefier --icon /src/ --name  -p linux -a x64  /target"
  loop: ""

- name: Clean out nativefier dir
  become: true
  file:
    path: "/-linux-x64"
    state: absent
  loop: ""

  # I don't love this, but copy was too slow
- name: Move Builds to Nativefier Location
  become: True
  shell: "mv /-linux-x64 /-linux-x64"
  loop: ""

These tasks also move the built app to the final location.

Manaing Application Files

Lastly, with icons in place and the application builds complete, we can create the application entry files by adding this final task:

- name: Generate Application Desktop Files
  template:
    src: templates/nativefier.desktop.j2
    dest: "/.desktop"
  loop: ""

And there we go, that’s it! Well… sort of.

Selinux woes

While this ‘worked on my computer’, when I tested it on my laptop and in some VMs… it failed.

Issues

Every time I ran the playbook, I hit this error:

Error during build. Run with --verbose for details. [Error: EACCES: permission denied, mkdir '/target/linux-x64-template'] {
  errno: -13,
  code: 'EACCES',
  syscall: 'mkdir',
  path: '/target/linux-x64-template'
}

I was able to tell the error occurred while running the podman container. I assumed it might be selinux related, but was unable to sort out a solution right away.

The Fix

Eventually after browsing the internet, I learned that my easiest solution was to add the option --security-opt label=disable to my podman run command, to turn off label separation for the container. I’m sure there is a better, more secure soltion I could do. However, I figured this was at least a good compromise of being easy to implement, but more secure than the common (and wrong) suggestion of “just disable selinux”.

One last fix…

drwxr-xr-x. 1 100999 100999  560 Oct 25 20:40 jellyfin-linux-x64

Lastly, I wanted to add one more cleanup task. My builds had the ugly uid/gid pair of 100999 100999, so I a task to change ownership to the user:

- name: Change Permissons of Nativefier Dirs
  become: True
  file:
    path: "/-linux-x64"
    owner: ""
    group: ""
  loop: ""

After that change:

drwxr-xr-x. 1 ryan ryan  560 Oct 25 20:40 jellyfin-linux-x64

Much better!

Example: adding it to my playbooks

With the role complete, it is finally time to add it to a playbook. I define playbooks to provision all of my machines, so I will just add it to them. First make sure the role is added to the list of roles used by the playbook. For example:

  roles:
    - apps/nativefier

Next, define a new var named nativefier_apps. This variable is a list of dictionaries, with each dictionary providing the values for a different nativefier application. Each nativefier build requires three variables to be defined:

name: The name of the application
icon: The filename (including ext) of the icon file to for the application
url: The address for the webpage to build as a nativefier app

So, to build my pocket, fastmail, and homeassistant apps, I added the following to my playbook:

nativefier_apps:
  - name: pocket
    icon: pocket.png
    url: "https://app.getpocket.com"
  - name: fastmail
    icon: fastmail.png
    url: "https://www.fastmail.com"
  - name: homeassistant
    icon: homeassistant.png
    url: "http://homeassistant.local:8123"

That should be it! Afterwards, my playbooks could build and configure all my nativefier applications automatically!

Conclusion

My desktop covered in some of my nativefier apps.

I have wanted to create this role for a very long time and am glad I finally did. Nativefier is such an amazing tool, and paring it with podman and ansible has somehow managed to make it shine even more. Enjoy!

Decided to get an Ergodox

2020-10-20T00:00:00Z

Durham, NC

For the past month, I have been learning to type… on my new keyboard! I decided to purchase an ergodox-ex and have been using it ever since. I still love my HHKB, so why did I get a new keyboard? And why… the ergodox?

Previous Keyboard

My Happy Hacking Pro Keyboard

For a few years, I have been gladly using a Happy Hacking Pro 2 keyboard. Before that, I was on a Happy Hacking Lite, which I started using while in college. I love the layout of HHK and still think it’s an amazing keyboard. The years I spent on HKBs are why I think every keyboard should have ctrl instead of a caps key, or at least switch to two.

Rationale for a new one

While I love my HHKB and have no real complaints, there were a few things that pushed me to start looking at new boards again.

I was getting the itch to build one

My initial motivation was the slow, growing itch to build a mechanical keyboard. It is an itch that I’ve felt before. I purchased my HHKB Pro as a backup plan when the order for a planck keyboard kit via mass.com was unfortunately cancelled. My original intention was to learn how to solder (I even brought a soldering iron), and build the keyboard from scratch. Ultimately buying the HHKB, I never got to scratch that itch. While I technically still haven’t… this is what had me looking at keyboards and browsing /r/MechanicalKeyboards again in the first place…

Wanted a program-able board

More important than physically building a keyboard, I wanted to be able to programatically build it. That is to say, play with custom firmware. This is something that my HHKB doesn’t have, but the plank did. The pok3r, which I’ve considered in the past, also has programmable layers. Even some of System76’s laptops now allow the ability to remap their keyboards using open firmware (which is awesome!). The biggest thing I wanted from a new keyboard was the ability to customize and program the layers and layouts.

After starting to learn proper touch typing, I wanted to try an ortholinear layout

Working in mostly the same space for much of this year, I’ve noticed that my wrists and arms have really started to tighten up. So, I put my foot down and decided it was time once again to try and force myself to proper touch type… using all of my fingers. Normally, I slide my hands across the board and have just a few fingers do the bulk of the work. This not only creates unnecessary movement, but also over-strains parts of my hands, building up tension in my wrists.

'Proper' finger key map for touch typing on a standard layout

As I started typing with proper finger use, I noticed my hands really have a hard time hitting some keys. Particularly the bottom row, especially with my left hand, gave me trouble. My other fingers commonly blocked the path that the “correct” finger needed to reach a key. It became obvious why I naturally adapted to sliding across the board over the years. I wondered if an ortholinear layout would better allow me to touch type properly, using all of my fingers. My hope was that the straight grid would let my fingers slide directly up and down to the other rows, without crossing over and impeding my other fingers.

My shoulder pain increased

The last straw, promoting my keyboard urgency to something I needed to order now, was that my shoulder was beginning to hurt… all the time. During my search, I noticed split keyboards and thought they might help to alleviate some of my pain. The ability to type with my arms positioned at any width would at least allow me to vary my shoulder angle as it got sore. So, I switched from looking at DIY kits, to pre-built split keyboards that I could have shipped and under my fingers much quicker than I could order and build one.

Keyboard Considerations

With requirements decided, I started searching for my keyboard. Both of the keyboards that I considered are made and sold from the same company, ZSA. From what I read online, they were usually highly recommended, with people rating both ZSA’s products and customer service very well. Personally, I enjoyed reading about their focus on sustainability and creating open source products.

Moonlander

The Moonlander

The first keyboard that caught my eye was the new Moonlander. When I first saw it, it had just been released. I admired it’s floating keys and slim profile. Apparently, it was designed to be at least somewhat portable, as it comes with a carrying case. The board seemed to be a fresh new take on the ergodox, with a clever folding design and some newer features like usb-c.

I had to dig deep to learn anything about the moonlander as there were only a handful of people that had them at the time. As I dug, I passively learned more and more about the ergodox…

Ergodox-EZ

The Ergodox-EZ Configurator

In addition to the moonlander, ZSA also builds and sells the ergodox-ez (and the planck ez, but I wanted a split board). The ergodox-ez is a pre-built version of the well-known (to keyboard nerds…), open-source, ortholinear, split keyboard, the ergodox.

I loved the Moonlander’s features and design, but I ultimately decided to go with the ergodox-ez for a couple of reasons:

The Moonlander is brand-new and there wasn’t much feedback for it yet, whereas there is plenty of information about the ergodox-ez.
On a similar note, because the ergodox has been a popular design for years now, 3rd-party parts like specialized keycaps sets can easily be found on mass.com and other websites. This makes the ergodox a bit easier to customize than the moonlander, at least currently.
The ergodox is great starting board for testing out the orthlinear+split design. If there were parts of it I really hated over the next few years, I could always switch to a moonlander or another split board that better fits my needs, based on what I learn from the ergodox.
However, if I instead love the ergodox design and want another in a few years for some reason, I still have the option to build one… and finally scratch that itch XD.

It’s a win-win situation.

Conclusion

I ordered the Ergodox-ez and have had it for a over a month now. If you are wondering what configuration I selected, and what I think about it so far… stay tuned for my next post ;) .

Setup Optfine with Minecraft Flatpak Install

2020-09-30T00:00:00Z

Kitty Hawk, NC

Over the past few months, I started to play Minecraft (Java) on my desktop again. After upgrading my graphics card, I wanted to install some shaders. However, unlike when I was in college, I now install and play minecraft using flatpak. While flatpak makes installing minecraft convenient, it also complicates enabling mods like Optifine. So… here’s how it’s done :) .

Note: Before we start, I have only done this on Fedora Linux. It is possible that paths and locations may differ on other distros.

Install the Minecraft Flatpak

The Minecraft Flathub page.

First, lets install the minecraft flatpak. Ensure that flathub is enabled, then run the following command to install Minecraft:

flatpak install com.mojang.Minecraft

When it finishes, open up the launcher and login to verify that everything is working.

(Logging into the official launcher at least once is required to install Optifine in the later steps).

Optifine

Optifine is a Minecraft optimiziation mod which supports installing shader and texture packs. Simply put, this makes minecraft look better. Additionally, some shaders can optimize the game to perform better too. I use a shader that renders some textures more realistically from the default, but nothing too fancy.

Download

Optifine Download page.

To obtain optifine, go to the downloads page and download the version which corresponds to the minecraft version you are using. If you are running a version which is more recent, you might have to try a Preview version of optifine. In my experience, the previews have worked without issue.

Finding the minecraft folder

Before we install Optifine, there is one piece of information we need to know: the .minecraft folder location. This is where running a flatpak verion diverges from a normal minecraft install, as the folder will not be at ~/.minecraft/.

This is because flatpak applications are sandboxed from the system. While good for security, it means that the ‘home directory’ observed inside the application is different from the user’s (like a chroot).

The flatpak minecraft folder.

On my computer, the flatpak applications are located at /home/ryan/.var/app/, making my ‘/home/ryan/.minecraft/’ folder actually at /home/ryan/.var/app/com.mojang.Minecraft/data/minecraft/. Find and remember this location. (Hint: it should be similar to mine)

Install Optifine

Back to Optifine…

To run the optifine installer, open a terminal, navigate to the downloaded jar file, and execute it using java:

## If java is not installed:
sudo dnf install java-openjdk-latest
## Then:
cd ~/Downloads
java -jar preview_OptiFine_1.16.3_HD_U_G3_pre1.jar

This will open the installer.

The Optifine installer window.

Switch the Folder to the minecraft one we found previously, and click Install. Note: If the official minecraft launcher was not already logged into, this step will not work as it will fail to find minecraft (the launcher downloads content on the first login).

Configure The Optifine Launcher

Select Optifine in the Minecraft Launcher.

With Optifine installed, we need to select it in the minecraft launcher. If Optifine was correctly installed, it should now be an option at the bottom left of the window (Next to settings).

Add your own installation

Creating a Minecraft launcher for Optifine.

If for some reason optifine is not an option, a custom launcher can be added. To create one, click Installations at the top of the launcher, and hit the New button. From that window, give the install a name and select your latest Optifine version from the drop-down menu. If optifine isn’t an option, it likely wasn’t installed correctly or in the right location.

Lastly, remember to once again switch the Game Directory to the location of the flatpak minecraft folder we’ve been using. Hit Create and switch to the new profile.

Shaders

The main motivation behind adding Optifine is the ability to use shaders. With Optifine all configured, lets finish what we came here for.

Download

First, find a shader. I have been using the BSL shader pack and love it. It keeps a more ‘classic’ minecraft style, without changing too much. However, it improves key visuals like the lighting, water, and swaying plants.

When you find a shader you want, just download the package and move it to the shaders folder in our flatpak minecraft folder.

FYI, not every shader I tried worked.

Enable in game

Minecraft Video Settings, with Shaders setting.

With the shader package in place, we should be able to enable it in the game. Fire up minecraft (using the optifine install), and select Options, then Video Settings. From there, you should now see a newly available Shaders option. Click it to open up the shaders menu.

Minecraft Shader Settings.

If the shader package was moved to the correct location, it should show up here. If so, click the desired shader (the laucher might restart when a new shader is selected). If not, open the Shaders Folder to double check the location.

After picking the shader, hit Done and check it out!

Conlusion

Minecraft using the BLS Shaders.

Much better! Shaders can make minecraft feel like a whole new game, and are a blast to play with. While they might seem difficult to setup with a flatpak install, it really isn’t bad if you know (and remember XD) where the minecraft folder is located. Enjoy!

PS: For those that read my previous post: yes, I did use a virgil VM to get clean install screenshots for this post… including the gameplay one!

Running VMs with VirtIO 3D Acceleration

2020-09-10T00:00:00Z

Kitty Hawk, NC

While on vacation the other week, the only laptop I brought along was my macbook pro. That being said, I still wanted the ability to jump into Linux while I was away. I needed to virtualize. I’ve used VirtualBox in the past, and while it’s fine for running headless installs, I find that it is not a great experience on macOS when trying to run a full desktop environment. So I downloaded the free trial of parallels desktop 15, and honestly… it was great. Afterwards, I was motivated to improve the experience of VMs running on my (Linux) desktop.

Parallels desktop on MacOS.

Parallels handled graphics on both my Windows and Linux guests wonderfully. I would full-screen the VM and it felt like I was running that OS directly on my macbook (for the most part). I was sad when the trial was over, but decided not to buy the subscription. My main reason being… my VM computer is not my macbook, it’s my Linux desktop.

I love libvirt and virt-manager, but using parallels showed me that I really needed to figure out how to improve the graphics situation of my VMs to catch up. I wanted to be able to work in guest VMs, without a lagging UI or feeling like I was in a VM.

VFIO - GPU Passthrough

Prior to my trip, I had been experimenting once again with VFIO… and actually got it working* for the first time. If you don’t know what this means, basically in Linux it is possible to pass a full device, like a GPU, through to a VM to be used directly. The technology was made more widely known a few years ago from the “X Gamers, 1 CPU” serries of videos Linux Tech Tips made.

I was able to temporarily run both Proxmox and Unraid on my desktop, and pass my gpu directly to Fedora VMs. While it was a neat experiment, I realized it can also be quite a pain.

I currently only have one GPU in my desktop, so to pass it to the VM I had to disable it on the host. Not having a GPU for the host system meant that I had to start the VMs from another device, which honestly isn’t that bad. The much worse issue however, was that the gpu would often get stuck while being passed around, so I had to reboot my host whenever the VM restarted… which defeated the point of using a VM instead of multi-booting.

So to properly utilize VFIO and have a more stable setup, I would want a second GPU. In truth though, I realized this setup wasn’t what I was looking for. KVM gpu passthrough is a great option if you run Linux as your main computer, but want to be able to boot into a Windows VM to play games. But GPU passthrough is slowly becoming overkill even for that, as gaming support on Linux keeps getting better.

Aside from having an occasional Windows VM to play around with, I mostly want to run Linux VMs on a Linux host. I needed to figure out how to let my VMs utilize the power of my GPU, without the complications of GPU passthrough…

Virgil

Introducing, virgil. According to the project’s website:

Virgil is a research project to investigate the possibility of creating a virtual 3D GPU for use inside qemu virtual machines, that allows the guest operating system to use the capabilities of the host GPU to accelerate 3D rendering. The plan is to have a guest GPU that is fully independent of the host GPU.

So, Virgil essentially allows libvirt to create virtual 3D GPUs, that work with the host GPU for 3D acceleratuon. This means, that I should be able to get high performing VMs, including the graphics, on my Linux desktop.

Setup

Enabling virgil for a VM using virt-manager actually ended up be much easier that I had anticipated. Here is how to do it:

Create a VM

Newly created Fedora VM.

First, create and install a new virtual machine using virt-manager. I recommend going through your normal install process and then boot into the installed system once, just to ensure that everything works as expected. If things look good, shutdown the VM and open up the settings window.

Enable VirtIO and 3D Acceleration - Spice Settings

In the settings menu, open up the Display Spice section.

In the VM's 'Display Spice' config, select 'None' for Listen type, and check the box for OpenGL.

Select Spice server for Type:, and None for Listen type: (virgil only works on local VMs right now). Lastly, make sure that the OpenGL checkbox is checked. Hit Apply.

Virtio Settings

Next, select the Video Virtio section.

In the VM's video settings, switch to Virtio and select 3D acceleration.

Switch to Virtio for Model:, and make sure to check the 3D acceleration checkbox. That’s it! Hit apply, start up the VM, and verify that it is working.

Testing and Comparisons

Youtube Video Playback

While configuring this for the first time, there were a few benchmarks I used to tell if and how well it was working. The first one, was simply opening up Firefox and trying to play a fullscreen Youtube Video (at 1440p).

Now to be fair, the video playback was still okay before I made this change. This wasn’t a good measure, but rather an initial test.

Unigine Heaven Benchmark

My next step was to give the VM a real GPU test, so I downloaded the Unigine Heaven Benchmark. First I got my baseline by running the benchmark directly on my host system. It should be noted that I did have other applications running during all of these benchmarks.

Unigine Heaven Benchmark running directly on the host (my desktop).

So not bad, and roughly what I expected. Next, I tried running it in the VM…

Unigine Heaven Benchmark running inside the VM with *shared* graphics.

While not as good as on the host, it is a very respectable result. Especially after remembering that the VM had less cores and RAM assigned to it compared to the host.

Additionally, my default VM wouldn’t even run the benchmark. I was able to open it, but the load screen ran frame-by-frame. Turning on the 3D acceleration settings definitely made a difference.

Portal 2

With the Unigine Heaven benchmark working in my VM, I decided to attempt the next test… playing a video game. Now, I have zero intentions of running games in my VMs, but this is the most common use case for having a GPU-passthrough setup, so I wanted to see how this system compared.

I decided to try running a classic game: Portal 2. It’s not a very resource intensive game, but one that certainly does not normally run well in my VMs. I installed the game, started a level… and was immediately forced to stare at the ground.

Staring at my character’s feet, I was reminded that I was playing in a VM, and that the game didn’t actually have my proper mouse capture. It only had the input clicks from my host to the VM spice window, which wasn’t enough to actually play the game.

Playing Portal2 at 2560x1440 resolution, inside my Fedora VM.(I had it full screen but windowed it to show it was indeed in the VM).

Anyway, after one VM shutdown and usb mouse pasthrough later… I was playing Portal 2! It was a little glitchy for the first few seconds, but afterwards ran well. I think the initial lag was likely due to the system heavily hitting the virtualized disk while loading the level. Regardless, I was thrilled.

Drawbacks/Limitations

While I’m very happy with the performance of my virtio VM, there are a few drawbacks or limitations to keep in mind:

Need to pass through hardware devices: … like a mouse when gaming XD. This could also be a webcam, microphone, flash drive, or even a full ssd if you need better IO in the VM.

Linux Guests Only: While researching, I saw conference talks about getting virgl working on Windows , but I think it currently only works with Linux guests.

Both my Manjaro and Pop_OS! VMs worked just fine with virgl.

Even on Linux, some Distros might freak out a bit: I had weird issues when trying it on some of my RHEL 8 VMs. The mouse pointer wouldn’t show and the screen would glitch and flicker when running high graphics tasks. I’m Not sure why it’s happening, but I’m guessing it might be related to older software packages/kernel. I have tried it with Pop_OS! and Manjaro VMs and they seem to work just fine.

My Fedora VMs on my T470 Thinkpad (intel graphics only) didn't seem to like virgl.

Not entirely sure about hardware support: So far my working VMs have all been on my desktop, which has an AMD RX580 GPU. I tried using a Fedora guest on my T470 Thinkpad which has integrated Intel graphics, but the screen was all crazy at login. I don’t know what hardware is supported and what isn’t. It is possible that different hardware might just need different settings selected, but I have not played around enough yet to know for sure.

Conclusion

Overall, I love this setup. It allows me to utilie my desktop to it’s full potential, by running VMs I can jump into and forget that I am in a virtalized system. This works well when testing fedora packages, or maintaining a Rawhide machine. Virgl is an awesome projects and I hope it continues to progress over time. If you haven’t tried it yet, give it a shot!

PIA Client Interfering with Podman Containers

2020-08-23T00:00:00Z

U.S. Route 64, Jamesville, NC

Earlier this month, I woke up and tried to start working on my previous post, but quickly hit a snag. I was unable to start the toolbox container I use while working on my website. In fact, none of my podman containers would start.

Was Podman Broken?

Specifically, when I tried to start a container I encountered this error message:

podman start website
Error: unable to start container "f8ab31d42b9d04d051b23c65604e19748a9496f17bd3baab8e6f947eee8f3692": creating cgroup directory `/sys/fs/cgroup/net_cls/user.slice/user-1000.slice/user@1000.service/user.slice/libpod-f8ab31d42b9d04d051b23c65604e19748a9496f17bd3baab8e6f947eee8f3692.scope/container`: No such file or directory: OCI runtime command not found error

So, I attempted to use podman on a different computer. It worked fine. I compared version numbers and noticed that the second computer had a newer version of podman installed. I figured that I had hit a bug that must now be fixed, so I waited for the update to reach my desktop (it wasn’t available on that machine yet for some reason).

A day later when I ran my updates, the new version of podman was installed, which I thought would surely fix my problem. It didn’t. (ಠ_ಠ)

The Issue

I started to scour the internet again to look for answers. Eventually, I found this reddit post. While reading it, the poster’s experience sounded very similar to my own. After reading some of the comments that connected the private internet access client to the original poster’s issues, I suddenly remembered… I had installed the PIA client on my machine earlier that week!

Sure enough, when I checked the ownership of my net_cls files (as suggested in the thread), it looked like piavpn was claiming group ownership of the files:

➜  ~ ll /sys/fs/cgroup/net_cls
total 0
-rw-r--r--. 1 root piavpn 0 Aug  4 21:21 cgroup.clone_children
-rw-r--r--. 1 root piavpn 0 Aug  4 21:21 cgroup.procs
-r--r--r--. 1 root piavpn 0 Aug  4 21:21 cgroup.sane_behavior
drwxr-xr-x. 6 root root   0 Aug  4 21:21 machine.slice
-rw-r--r--. 1 root piavpn 0 Aug  4 21:21 net_cls.classid
-rw-r--r--. 1 root piavpn 0 Aug  4 21:21 notify_on_release
-rw-r--r--. 1 root piavpn 0 Aug  4 21:21 release_agent
-rw-r--r--. 1 root piavpn 0 Aug  4 21:21 tasks

Some commenters in the thread stated that the conflict went away after they removed the PIA client.

Removing the PIA Client

As a result, I decided to un-install my PIA client. It wasn’t a major loss for me, as I hadn’t used it in months. I only installed it to double check if it was a service I wanted, or if I should cancel my subscription before it auto-renewed later that month.

At first, I couldn’t find an un-install option, but eventually found it deep in the settings. After removing the client, the piavpn group went away… sort of. It still had a 1004 gid, which I’m guessing was the previous piavpn group.

➜  ~ ll /sys/fs/cgroup/net_cls
total 0
-rw-r--r--. 1 root 1004 0 Aug  4 21:21 cgroup.clone_children
-rw-r--r--. 1 root 1004 0 Aug  4 21:21 cgroup.procs
-r--r--r--. 1 root 1004 0 Aug  4 21:21 cgroup.sane_behavior
drwxr-xr-x. 6 root root 0 Aug  4 21:21 machine.slice
-rw-r--r--. 1 root 1004 0 Aug  4 21:21 net_cls.classid
-rw-r--r--. 1 root 1004 0 Aug  4 21:21 notify_on_release
-rw-r--r--. 1 root 1004 0 Aug  4 21:21 release_agent
-rw-r--r--. 1 root 1004 0 Aug  4 21:21 tasks

Whatever the case… podman still didn’t work.

… Don’t forget to Reboot!

I was furious. After calming down, I reasoned it probably still wasn’t working due to cruft from the client lingering on my system (like the 1004 group for example), so I rebooted my desktop… and it worked!

Conclusion

In conclusion… why did I write this post? This complication was a huge pain to troubleshoot. It was only by chance that I stumbled on that reddit post, and would have had an even harder time without it. I assume having at least one more page on the internet stating that podman and the PIA client don’t play nice, might help others find the solution quicker. Hence, this post.

Before I end, it is worth nothing that some users reported that configuring the PIA openvpn profiles and using them to connect to the VPN works without issue. It is just the client that breaks containers. So if you want to still use PIA (I let mine expire. I don’t use it enough), give it a try!

Running my Website Tests in Parallel with Pytest-Parallel

2020-08-16T00:00:00Z

Durham, NC

Not too long ago, I added some basic testing for my website. Now when I push changes to my website’s source repo, automated tests run to verify that the site’s pages are being served, and that the (markdown) links are not broken. It works well enough through the 450 or so generated tests. However, one recent afternoon I realized… I should parallelize them.

Why Parallelize?

Not all tests can be parallelized, but if they can handle concurrent runs, it might be worth trying to figure it out. All of my current website tests do a single thing: check if a page can be reached, or not. Furthermore, each check does not interfere with the others. Checking if one link is available does not change the state of a second one. Ultimately, this means that I could in theory, run all of the tests at the same time.

Adding pytest-parallel

It turns out getting my website tests to run in parallel wasn’t actually that hard. I just added a new pip package. I found a few, but the most promising one seemed to be pytest-parallel, which simply adds the option to start pytest runs using parallel workers. To test it out, I installed the package using the following pip command:

pip install pytest-parallel --user

Once I verified that it worked, I updated the Pipfile in the website test repo to include the pytest-parallel package:

[packages]
pytest = "*"
requests = "*"
pytest-parallel = "*"

That’s it! To run the tests in parallel, just add the --workers flag with a number to the pytest call. For example:

pytest --workers 12 -vv .

This will run my normal test run (pytest -vv .), across 12 parallel workers.

Improvements

So, did this actually improve anything? Yep, quite a bit ☺.

Time decreases in Jenkins Test Pipeline.

Using 12 workers, my test pipeline in Jenkins went from taking around 4.5-5 minutes, all the way down to under 1.5 on average, and sometimes only a minute! These times include some other steps, but the tests themselves went from taking a few minutes to under 30 seconds! All 450+ of them!

Conclusion

That’s all I have. One day I thought I should parallelize my website tests, and it turned out to be quite easy to do. Considering the results, I’m glad I did. Again, while it is not always possible to run tests concurrently, if it is for yours… try it out!

Creating Systemd Unit Files using Ansible

2020-07-29T00:00:00Z

UNC-Chapel Hill Campus, Chapel Hill, NC

In my previous post, I created a systemd unit file to define an application as a service, and configured it to auto-start on my server. I’ve been making a big push to define the provisioning of all my homelab machines/VMs using automation. So the last step in setting up my FoundryVTT server, is to automate the process. Fortunately, creating a systemd unit file is quite easy to do using ansible.

Creating the Ansible Role

Let’s start by creating a role (if you don’t know what an ansible role is, checkout this guide I wrote not too long ago). First, create some directories:

cd roles
mkdir -p foundryvtt/{defaults,tasks,templates}

One difference from roles I’ve created previously, is that this one contains a templates directory. That is because this role will use a j2 template to define the systemd unit file, but more on that later.

Defining Variables

Lets create some files, starting with one to define the default variables. Open roles/foundryvtt/defaults/main.yaml and add the following:

---
user: ""
service_description: "A service to run the Foundry VTT node app"
foundryvtt_dir: "/home//foundryvtt/"
foundrydata_dir: "/home//foundrydata"

This defines a few default veriables that will be used in the service file template, as well as in the tasks file. The variables can optionally be over-ridden when running a playbook, but they will default to these values if not specified.

Making a Template

Now that the varibles are defined, we can create the unit file template. So, open a new file (roles/foundryvtt/templates/foundryvtt.service.j2 in my case), and insert the unit file from the previous post.

Next, walk through the file and substitute any values for the variables defined in the previous section:

[Unit]
Description=
Documentation=https://foundryvtt.com
After=network.target

[Service]
Environment=NODE_PORT=30000
Type=simple
User=
ExecStart=/usr/bin/node /resources/app/main.js --dataPath=
Restart=on-failure

[Install]
WantedBy=multi-user.target

Great! This template is now ready to be used in our role.

Ansible Tasks

Last but not least, time to write some Ansible tasks. Open a new file (roles/foundryvtt/tasks/main.yaml), and lets start by adding any additional tasks this particular role needs, outside of the service file. For my example, this includes creating the defined data directories, unzipping the application source package, and opening required ports in the firewall:

## Note, you likely don't need these tasks. They are just for my particular
## example...
- name: Create foundryvtt dir at 
  become_user: ""
  file:
    path: ""
    state: directory

- name: Create foundrydata dir at 
  become_user: ""
  file:
    path: ""
    state: directory

- name: Send Foundry Package
  when: foundryvtt_send_src is defined and foundryvtt_send_src is not none
  copy:
    src: ""
    dest: ""

- name: Extract package
  unarchive:
    src: ""
    dest: ""
    remote_src: yes

- name: Start firewalld
  systemd:
    name: firewalld
    state: started

- name: Open FoundryVTT Ports (firewalld)
  firewalld:
    port: 30000/tcp
    permanent: yes
    state: enabled

- name: reload service firewalld, in all cases
  systemd:
    name: firewalld
    state: reloaded

With all that defined, lets finally define a task to create our unit file using the template:

- name: Create foundryvtt systemd service file
  template:
    src: templates/foundryvtt.service.j2
    dest: /lib/systemd/system/foundryvtt.service

The src is set to the relative location (to the role) of the template we defined earlier, and the dest is set to where we would like the generated file to be copied to. This template is a unit file for a systemd service, so I’m going to copy it to /lib/systemd/system/foundryvtt.service.

Last but not least, lets start the newly created service:

- name: Start foundryvtt service
  systemd:
    name: foundryvtt
    state: started

… and… our role is finished!

Playbook

To run the role, it is easiest to have it run in a playbook. I try to define the provisioning of all my systems in their own playbooks, including my foundry server, so I use this role there. Just remember to call it in a roles: section of the playbook, like so:

  roles:
    - foundryvtt

Conclusion

That’s it. We’ve easily automated setting up the systemd unit file from the previous post using ansible. This makes defining and reproducing unit roles very simple. In addition, knowing how to use templates in ansible is very powerful, and I will definitely be utilizing them much more moving forward. Enjoy!

Auto-starting Applications with Systemd Services

2020-07-14T00:00:00Z

UNC-Chapel Hill Campus, Chapel Hill, NC

Recently, I’ve been hosting a Foundry VTT server (a nodejs app) in a virtual machine on my home network. I would start the application inside a tmux session, by executing a CLI command which worked… fine. However, if the VM restarted or the applications crashed, I had to ssh in and manually run the command again. So, to better automate this tedious task, I created a unit file to define the foundry server as a systemd service. Here’s how.

Unit Files

With so many distributions utilizing systemd, unit files have become a new standard for auto-starting, or re-starting, applications in Linux. Simply put, unit files are used to define resources to be managed by systemd. This includes services. So to run FoundryVTT as a service, we need to create a new systemd .service unit file.

Creating the service file

To create a unit file, I opened /lib/systemd/system/foundryvtt.service in vim and filled it with the following contents:

[Unit]
Description=A service to run the Foundry VTT node app
Documentation=https://foundryvtt.com
After=network.target

[Service]
Environment=NODE_PORT=30000
Type=simple
User=ryan
ExecStart=/usr/bin/node /home/user/foundryvtt/resources/app/main.js --dataPath=/home/user/foundrydata
Restart=on-failure

[Install]
WantedBy=multi-user.target

This file tells systemd all the information it needs to run the service. The contents of the [Unit] section define some basic information about the unit file. The variables which define our service, are appropriately listed in the [Service] section and include:

Environment=NODE_PORT=30000 sets an environment variable
Type=simple states that our service is executing a single command, and is “started” when that command runs.
User=ryan tells the service to run under the ryan user
ExecStart defines which command to run when the service is started (this is the command I had to manually type in tmux)
Restart=on-failure tells the service to automatically restart on any failures.

Save and exit the file.

Start and Enable the Service

With the unit file created, the new service can be started:

sudo systemctl start foundryvtt

In addition to starting the service, I also enabled it so that it will automatically launch whenever the system reboots:

sudo systemctl enable foundryvtt

Stopping, Restarting, and Status

To check that the service is running, use the command systemctl status foundryvtt:

sudo systemctl status foundryvtt
[ryan@magmar dotfiles]$ sudo systemctl status foundryvtt
● foundryvtt.service - A service to run the Foundry VTT node app
     Loaded: loaded (/usr/lib/systemd/system/foundryvtt.service; disabled; vendor preset: disabled)
     Active: active (running) since Sun 2020-07-12 16:34:29 EDT; 2s ago
       Docs: https://foundryvtt.com
   Main PID: 1070 (node)
      Tasks: 11 (limit: 2327)
     Memory: 93.8M
        CPU: 1.227s
     CGroup: /system.slice/foundryvtt.service
             └─1070 /usr/bin/node /home/ryan/foundryvtt/resources/app/main.js --dataPath=/home/ryan/foundrydata

... *A Bunch of Logs I removed*...

The output contains Active: active (running), which means the service is running. We can also restart or stop the service using the systemctl restart foundryvtt and systemctl stop foundryvtt commands, respectively.

For example, I can stop the service and then check the status to verify it is killed (Note the Active: inactive (dead) in the status output).

[ryan@magmar]$ sudo systemctl stop foundryvtt
[ryan@magmar]$ sudo systemctl status foundryvtt
● foundryvtt.service - A service to run the Foundry VTT node app
     Loaded: loaded (/usr/lib/systemd/system/foundryvtt.service; disabled; vendor preset: disabled)
     Active: inactive (dead)
       Docs: https://foundryvtt.com

Conclusion

That’s about it. Systemd unit files might seem complicated at first, but after writing, one they aren’t that bad. Additionally, it turns out that using automation to create them isn’t too difficult either… but I’ll show that in another post. Until then, enjoy!

My Three Required Gnome Extensions

2020-06-30T00:00:00Z

Gnome is the default desktop environment for many Linux distributions, including my distro of choice, Fedora. When Gnome 3 was first released, I had to install a bunch of extensions to get a functional experience. These days, between becoming more familar with the Gnome workflow, and the Gnome team ironing out the rougher edges of the DE, I don’t need nearly as many extensions to get going. However, there are still three which I do not think I could live without.

Gnome Extensions

The Gnome Extensions website.

Gnome extensions are installable enhancements to the Gnome desktop environment. They are obtained by going to the gnome extensions website with a supported browser, installing the plugin, and switching the tab of desired extensions to “On”. Extensions can be further configured using the Tweaks desktop application.

My Must-Have 3

Caffeine

My first required extension is Caffeine, which adds a little coffee cup icon to the status bar. When the cup is empty, the system behaves as normal. However, when clicked, the cup ‘fills up’ and caffeine will prevent the system from suspending the display.

This is useful when giving presentations to stop the screen from dimming off during longer slides, or during Q&A time. I also use this if I have a secondary computer displaying information, to prevent it from repeatedly logging me out. I can temporarily pause my sleep settings, without having to actually go in and change them.

Top Icons Plus

The right side of my top bar, without extensions (top), and with the *caffeine* and *top icons plus* extensions enabled (bottom).

Many applications still expect an icon to be visible, in a task bar, where the application can to store settings in. However, Gnome will often hide these icons, making them less accessible. Top icons plus moves these icons back to the top status bar, like many people expect.

I use this extension so that I can more easily see and control applications on my system which assume the icons to be visible. Top Icons plus also allows me to choose where on the bar I want them located (left, center, right). I usually just set them to the ‘right’ side of the screen, with all the other junk.

Sound Input & Output Device Chooser

Top bar dropdown menu, with (right) and without (left) the *Sound Input & Output Device Chooser* extension enabled.

Lastly, the Sound Input & Output Device Chooser plugin adds a drop-down menu to the top bar where audio input and output devices can be easily switched. Otherwise, I usually need to do this from the sound section of the system settings.

As someone that switches between speakers and headphones all day, has 100% remote meetings, and usually more than one microphone device connected to their computer, this is probably my most required plug-in. It makes my life easier, and allows me to react quicker in a meeting if I notice that an incorrect audio device is selected.

Conclusion

While I still use other extensions, these are the big three. I think default gnome has gotten surprisingly usable, but I would probably struggle to use it if these extensions were no longer available to me. So, to the Gnome team… please consider adding the functionality of these items as defaults in Gnome. If not, I understand. However, if that is the case… Gnome community, please don’t let these extensions become deprecated! Thanks :)

Trying out Notion

2020-06-28T00:00:00Z

Over the past year or two, my notes and planning systems have been a bit of a jumble. It started when I attempted to switch everything to joplin notes… only to eventually switch back to org-mode (for my work notes) a few months later. Around that same time, I also started using trello to organize my personal and home life task boards. This system has worked well, but feels very disjointed. My notes, goals, and tasks are spread all over the place. This might all be about to change. I’ve started using Notion.

Quick History

Before diving into my experience with Notion, lets first describe in more detail the system I’m coming from.

Previous System

My Trello Weekly Task Board.

Most recently, my personal and professional notes, and task planning systems have consisted of three applications: Trello, Emacs (org-mode), and Joplin.

Trello: I had several boards for planning my weekly, monthly, and multi-month personal tasks (including household ones). I also maintained a few boards to keep track of goals and projects ideas (ex: future website posts, books to read, training courses to take, etc).

Emacs org-mode: For my professional work life, I completed all the same tasks I listed under the trello section, using emacs instead. I would write down my tasks, and log quick notes and thoughts under each one while I was working on it. Logging helps me think through problems, and serves as a reference later. While I could log notes into Trello cards, it wasn’t nearly as easy to quickly open up and record a stream of consciousness, as it is was in org mode.

Joplin: I previously attempted to make Joplin my universal planning system, but it failed. It was a bit messy for a universal system, and lacked a board style layout for my goals and task planning (what Trello or even my ry-org-scrum package provide). That said, it was a great cross-platform notes application… so I continued to use it for that. Any personal or work note that wasn’t directly related to a single task card was stored in Joplin.

Issues with the system

My Emacs org-mode task/notes, using the task-board package I wrote.

While this system worked, it does some issues.

Couldn’t share items across applications

The biggest issue I had with this framework was that it was made from several disconnected systems. This meant that my notes and information were spread all over the place, and not linkable. I tried to keep notes and task logs separate, but while working on projects I often wanted to create longer, stand-alone, reference notes related to a task. With this setup, I couldn’t easily have the two items reference each other.

Had to configure each application… on every device

I have automated my emacs setup, but both joplin and emacs still required me to manually configure their sync solutions. Additionally, while Trello’s minimal setup of ‘just login’ seems simple enough, even this can quickly become tedious when doing it for multiple applications at a time… on several devices… some of them phones.

Not accessible beyond my computers

Lastly, my main issue was that apart from Trello, these systems weren’t really accessable from devices beside my own. In fact Emacs wasn’t even available on my mobile devices. While good from a security standpoint, I don’t like having all my notes and information only available when I sit down at a desktop. If I’m working on a family member’s machine, or in a fresh VM (without a shared clipboard), I want to be able to access my notes from a private browser tab. I also like to move cards across my task board from my phone as I complete them throughout the day.

Desires

With these issues defined, what is it that I actually want to see in my note taking and task organization system?

Joplin Notes.

Consolidated into one system: I don’t want to setup multiple apps and logins on all my systems.
Cross platform: Despite only wanting a single system, I want it supported on all my devices, including mobile.
Task board support: I really enjoy moving and displaying my task items in a board view.
Normal notes too: In addition to task cards, I also want support for traditional, stand-alone notes. Ideally tasks can be linked to notes, but the notes don’t have to be task-bound.
Easy to jump in, add a log note, and get out: I love this in org-mode. I need the ability to quickly jump into a task, hit a keyboard shortcut to auto-insert a time-stamp, and write a log note.

Notion

Notion window in macOS, displaying my page for this post. The UI looks the same on the web and in my 'Nativfier' app in Linux.

Notion might be my solution. The Notion website states that it is an “All-in-one workspace. Write, plan, collaborate, and get organized — all in one tool”. Having used it as my lone system over the past several weeks, I think agree.

What I like so far

FLEXIBLE: This cannot be emphasized enough. Notion is extremely flexible, providing the tools to setup your own system, rather than being a defined system itself. Want just a pile of markdown-ish notes? Fine. Want to create endless clusters of relational databases? Go ahead. This endless flexibility makes Notion what I would consider to be the Emacs org-mode for ~~normal~~ reasonable people.
Easy logging using @now:

Easy logging using `@now`.

Notion has the ‘easy logging’ feature I wanted. I just have to type @now, hit enter, and it auto auto-inserts a dynamic date and time (the ‘today’ changes to ‘yesterday’, and eventually the date as time passes).
Databases: Databases are a compelling tool that are the backbone of a powerful notion setup. They allow collections of data to be linked, sorted, and filtered. Databases themselves have several notable features worth mentioning:
- Multiple view methods for the same data: Database items can be displayed in different views. These include tables, boards, galleries, calendar, and lists.
- Views can be filtered and sorted using property rules: Properties can be also marked as hidden, and any custom changes to the data view can be saved. This makes it easy to flip through different views to get a better picture of the information.
- Linkable: A database can be embedded in a page as a ‘linked database’. This is a database that points to an already existing one, and any changes to it also occur in the original DB. However, the linked databases have their own saved and default views.
Templates: Templates are another convenient feature. A specific page layout can be defined as a template, which new pages can then be created from. For example, I have weekly templates that contain all the tasks and linked databases for that week automatically configured. So when setting up a new week, I can simply create a new page from the template and fill in some information. Done.
Accessible from all my devices:

Notion views in iOS. Navigation, my 'Areas' card view, a daily log page' properties, and a task 'board' view.

Notion has desktop applications for macOS and Windows, as well as mobile apps for iOS and Andriod. Additionally, it functions well as a webapp, which made it easy for me to create a nativefier build of it to run on my Linux systems.
Free, and affordable for pro: When I started using notion, the free tier was limited to 1000 blocks, but since then they have made the personal tier free without a use limit! There is still a Pro version with fancy features (like revision history), but even that is offered at a reasonable price.

Downsides/Concerns

Notion makes it easy to export an entire workspace.

All my eggs in one basket: My biggest concern with Notion, is that I am throwing everything into a single service which I now rely on. That is a huge risk. Fortunately, Notion does make it extremely easy to export a workspace, which eases my fears a bit. I need to test this out, and maybe make it a habit to periodically do it as a backup.
A bit of a steep learning curve: All the flexibility and power means it can take a bit of trial and error to setup a system that works for you. While overwhelming at first, I don’t find it to be not that big of an problem. I am coming from using mostly emacs after all :P
Lacks a cheaper ‘Family’ Plan: I’m ecstatic that they made the personal plan free, but I would still love to see a ‘Family’ plan that would allow my wife and I to have shared collaborative pages. I know this is what the teams version provides, but we don’t need all the other advanced features, and the price ends up being a bit steep for what we would want.

Conclusion

In summary, I’ve found Notion to be amazing. I’ve had to redo my setup twice already as I’ve started to better understand more powerful features, but every time it has been worth it. Notion appears to be fitting my needs perfectly, and it keeps getting better every day as I continue to tweak and perfect it.

Switching Hugo's Markdown Handler

2020-05-31T00:00:00Z

Duke Park, Durham NC

While writing my previous post, I hit a frustrating issue. After saving a large chunk of the draft, it appeared that hugo wasn’t rendering the new additions. I verified on several computers, including fresh installs. None of them would generate the updated post, despite the source files containing the changes. Ugh.

The Actual Issue

When I first noticed the problem, I though hugo wasn’t rendering any of my recent changes, which consisted of adding some images and a paragraph or two. However, after inspecting the page in greater detail, I did see that the updated text was present. Only the images were missing.

With this information, I was able to search for a solution. After spending a few minutes reading some irrelevant stackoverflow posts, I eventually found this wonderful blog post. The problem it described matched my issue exactly.

It explained that recent versions of hugo had switched from using blackfriday to goldmark, to render the markdown content. Unlike blackfriday, by default goldmark does not render raw html tags. I use raw html to define the images in my post, and thus they were no longer rendering.

A Simple Solution

The post offered two solutions to fix the problem. For now, I just applied the easier of the two: reverting back to blackfriday to render my markdown. To make the switch, I added the following line to my config.toml:

[markup]
  defaultMarkdownHandler = "blackfriday"

The alternative solution, is to override goldmark’s default setting and allow raw html tag rendering. This can be accomplished by instead adding these lines to the config.toml:

[markup]
  defaultMarkdownHandler = "goldmark"
  [markup.goldmark]
    [markup.goldmark.renderer]
      unsafe = true

I’m sure that the hugo team had valid reasons for the switch. With that said, until I have time to read up on those reasons, and to verify that goldmark doesn’t break rendering anything else in my website, I’m going to stick with blackfriday.

Conclusion

With that change the problem was solved. While the solution was easy, it took a few days of head banging to figure out exactly what was happening. Hopefully this post will help others figure out the fix quicker than I did. Or, at the very least… it will remind me what I did when I eventually switch to goldmark.

Rx-580 Upgrade

2020-05-28T00:00:00Z

Earlier this year, I noticed that my desktop seemingly did not connect to my monitor. I used the computer mostly as a server, remoting in via ssh, and didn’t think much of it. I assumed it was either a mis-plugged cable or a configuration issue. A few months later, I started to game a bit more and wanted to use my desktop as a gaming machine again. That is when I realized… it was an actual issue. Long story short, my desktop now has an rx580 instead of it’s old rx560.

My Old Card - The rx560

Before diving into the issues, lets recap. Why did I select the rx560 in the first place? What did I like about it?

I wanted an AMD GPU

When I designed and built my desktop, I knew I wanted to get an AMD GPU because their open source drivers were great, and would allow me to skip all the headaches I’ve had over the years messing around with the proprietary Nvidia drivers. I remain very pleased with that decision, and have had zero issues getting games up and running. No hassle, period.

I needed a card to handle modest gaming

In the post about my desktop, I stated that while I wasn’t a big pc gamer, I did enjoy the occasional game. I play many games on consoles, especially the ones I play with my wife, but desktop gaming is still my favorite. That said, I don’t need all my graphics settings tuned to ultra, and I tend to not play the newest games.

The rx560 was a perfect little card that give me enough performance to play most games, without costing too much. GPUs were still coming down from steep prices due to crypto mining at the time, and saving some money on the GPU allowed me to buy more RAM (which I needed more), while remaining under budget.

I planned to eventually upgrade

The rx560 (bottom) next to my new rx580 (top).

One of the reasons I wanted to build a desktop computer again was for the ability to upgrade it down the road. The build I designed had enough power to fit all of my needs, but I configured it so that if any part lacked enough performance over time, I could easily upgrade it.

I knew one of the components I might eventually want to upgrade would be the GPU. I figured that the rx560 should last me awhile, but if I started playing more games, I could easily get a better one, and hopefully prices would be lower. I could always upgrade to an rx580, or an even newer card if needed.

Issues

Debugging

Okay, back to the problem. As I said before, my desktop wouldn’t connect to my monitor. When I ssh’ed into it, everything worked as normal, and all of my VMs and containers spun up without issue, as if there was nothing wrong. I ran commands like lspci | grep 'VGA' and neofetch, and they still correctly detected and listed the rx560, confirming the system knew the card was there. I did some more testing and confirmed that I couldn’t get a signal on any monitor in the house, using a large assortment of cables.

No display would 'connect', even when plugged in.

I attempted to boot into live cds and other os installs, but the system wouldn’t even display the BIOS when starting. Investigating further, I saw in the logs that the $DISPLAY was trying to connect via all the various inputs (DVI, HDMI, etc) during bootup, but would eventually fail stating that there was No Display Attached, even when it was plugged in. I opened up the case, ensured everything was connected properly, and even tried different cables. Nothing.

The Possible ‘Solution’

I eventually narrowed down that the problem was being caused by issues with the GPU, despite it showing up fine in my pci devices. If it wasn’t the GPU, it was likely the motherboard… So I figured the next best (and easier) step was to swap in another card and see if it worked. Not having a spare graphics card on hand, and not being able to borrow one due to COVID-19 lockdown, my only option was to order one.

Fixing the display issues on my workstation was not an urgent issue. However, without the ability to connect to a display I couldn’t easily re-install or debug my system if something went wrong. This was compounded by the fact that I was running Fedora Silverblue, and needed to reboot every time I wanted to update my packages, never quite sure if I would be able to ssh back in when it was done. Lastly, I didn’t know what availability or shipping times would be, given the pandemic. So, I decided it was best to just order a new card now, rather than wait until the problem did become urgent.

My New Card: The RX-580

The rx580 box arrived a few days after ordering.

After some brief researching, I decided on the rx580. I also considered the 570 and 590, as well as glanced at some of the newer AMD cards. The truth is, my rx560 worked fine for what I needed. If I was getting a new card though, a small spec bump would be nice. The 580 seemed to provide more than enough performance.

So far, my experience confirms this. I am able to play my games (mostly Divinity Original Sin II, Minecraft, and City Skylines, right now) generally above 60 fps, and at mostly high or better settings. Additionally, because the card isn’t being too taxed during gameplay, it doesn’t seem to be heating up too badly… which I also appreciate.

rx580 price history.

Lastly, the card dropped in price since I built my desktop. I purchased the rx580 for only a little bit more than I originally paid for the rx560, and over $100 cheaper than what it cost when I built the desktop. For what I need, the price and performance are perfect.

Conclusion

So far, I’m loving it. The rx580 does everything that I need it to and honestly doesn’t give me any problems. Once again, I was able to plug in the card, boot up the computer, open Steam, and start playing my games. No additional drivers, no weird configuration, no bloatware. All on Linux. I am very happy with the purchase, and it has once again reminded me why I truly do love having a desktop/workstation under my desk, even if it is used as a server most of the time.

Ansible Quickstart

2020-04-30T00:00:00Z

Ansible Office Balcony, Durham NC

A long time ago, I briefly explained how to configure Ansible, in a post about building a raspberry pi cluster. All in all… it was by no means a great introduction to the basics of ansible.

A month ago, I drafted a progression of examples with notes, to teach a co-worker the basics of writing and using ansible roles and playbooks. After reading through them, I realized it wouldn’t take much to turn them into an actual Ansible quickstart post. So here we are.

I am not an Ansible genius, and reading this will not make you one either. However, the goal of this post is to provide enough understating to get started with writing some ansible playbooks.

Installing

Lets start by installing ansible. It should be in most distro’s main repos these days:

Fedora Linux:

sudo dnf install ansible

MacOS:

… I have no idea. I usually always ssh to Linux boxes from my macbook.

I think it can be installed with pip though, so possibly:

pip3 install ansible

Remote Node Requirements

In order for ansible to connect to a remote node, that node usually needs 3 things:

1. Python installed
1. password-less sudo permissions
1. ssh keys configured (if running against remote hosts. Not needed if just running playbooks against localhost)

python

Python should already be installed on most systems. If not, check your package manager, or try searching the documentation on python.org to learn the best install method for your system.

Passwordless `sudo`

This will allow a user to run sudo commands, without having to type in a password each time. I shouldn’t have to say this, but… please use with care!

Granting password-less sudo permissions are most easily accomplished with visudo:

sudo visudo

This will open up the sudo settings in your $EDITOR. Once opened, find the following line and uncomment it (it’s usually near the bottom of the file).

## Same thing without a password
%wheel        ALL=(ALL)       NOPASSWD: ALL

ssh

Lastly, exchange ssh-keys with the remote node. This will allow ansible to ssh into the node without having to deal with those pesky passwords. The easiest way to exchange keys is using the ssh-copy-id command, as such:

ssh-copy-id username@hostname

Ansible Basics

Hosts File

A host inventory file is a yaml file that defines hosts ansible can connect to. The default file is located at /etc/ansible/hosts. An alternative inventory file may be provided using the -i flag.

Example file:

[VMs]
## Server VMs
192.168.10.50
192.168.10.71
192.168.10.118

[Hosts]
## Hosts
192.168.10.12

[cluster]
192.168.112.205
192.168.112.206
192.168.112.207

Modules

Modules are premade functionality used in ansible that can be imported into a playbook. Simply, they do what you want done. Some examples are ping, dnf, apt, redhat_subscription.

Feel free to search the ansible documentation to learn more.

Ad-hoc Ansible Commands

Simple and straight ansible executions can be called with the ansible command. Ad-hoc commands are usually called with a module, using the -m flag. For example, ping:

➜  ansible -m ping localhost

localhost | SUCCESS => {
    "changed": false,
    "ping": "pong"
}

For a more complicated example, lets use the dnf module to install htop:

ansible -m dnf -a "name=htop state=latest" localhost --become

This module requires some parameters to be defined. We are able to supply them using the -a flag, followed by a string of the key/values pairs.

Also, because the dnf module requires root permissions to function, we supply the --become flag, to become root.

Note, if I want to run this against another machine (beyond localhost), it has to be defined in whatever inventory file we are using.

So, if I define an inventory file (./hosts.yaml) containing my desktop computer:

[charmelon]
192.168.1.5

I can install htop on my desktop, using:

ansible -i ./hosts -m dnf -a "name=htop state=latest" charmeleon --become

… and it works!

192.168.1.5 | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    },
    "changed": true,
    "msg": "",
    "rc": 0,
    "results": [
        "Installed: htop-2.2.0-8.fc32.x86_64"
    ]
}

Playbooks

As you can imagine, doing everything from the command line isn’t always helpful, or easily reproducible. That’s what playbooks are for. In a nutshell, playbooks are ansible scripts. They are a yaml file which ansible runs, instead of running a series of ad-hoc commands.

To demonstrate, lets convert the dnf command from above, into a simple playbook named install-htop.yaml.

---
- hosts: charmeleon
  become: true

  tasks:
    - name: Install Htop
      dnf:
        name: htop
        state: latest

Being a yaml file, the first line starts with ---. Next, we define some meta information for the entire playbook. For example, this is were we put the --become flag, by turning it into become: true. This is also where we define what hosts the playbook will run against. If I’m providing a hosts file, I can alternatively use hosts: all to run against all hosts defined in the inventory file.

local connections

If the playbook is to run only locally, the connection type can be set to local (by default, it is set to ssh.)

  hosts: 127.0.0.1
  connection: local

tasks

Below the header information, we can define a set of tasks to run. In the tasks section, a block is defined for each task, usually by calling a module with parameters. It is best practice to describe each task using name:. This will make it easier to trace the logs.

For example, lets add theping module to the playbook so we have more than one task…

---
- hosts: 127.0.0.1
  connection: local
  become: true

  tasks:
	- name: Ping host first...
	  ping:

    - name: Install Htop
      dnf:
        name: htop
        state: latest

Now the playbook will run both tasks, using name as the header for the output of each one:

➜  /tmp ansible-playbook install-htop.yaml

PLAY [all] ****************************************

TASK [Gathering Facts] ****************************************
ok: [192.168.1.5]

TASK [Ping host first...] ****************************************
ok: [192.168.1.5]

TASK [Install Htop] ****************************************
ok: [192.168.1.5]

PLAY RECAP ****************************************
192.168.1.5                : ok=3    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Variables

We can define sections other than tasks. A useful section to add is vars:, which defines variables for use in the playbook. To illustrate, lets replace the hard-coded htop in the dnf task, to a variable named package. We can even use the package variable in the name string, to dynamically change the output in the log:

---
- hosts: 127.0.0.1
  connection: local
  become: true

  vars:
    package: htop

  tasks:
    - name: Ping host first...
      ping:

    - name: Install 
      dnf:
        name: ""
        state: latest

And the output:

➜  ansible-playbook install-htop.yaml

PLAY [127.0.0.1] ****************************************

TASK [Gathering Facts] ****************************************
ok: [127.0.0.1]

TASK [Ping host first...] ****************************************
ok: [127.0.0.1]

TASK [Install htop] ****************************************
changed: [127.0.0.1]

PLAY RECAP ****************************************
127.0.0.1                  : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

One great feature of variables is that they can be swapped out when calling the playbook. The -e flag allows you to provide an alternative value for a variable. For example, lets say we want to install nano instead of htop:

➜  ansible-playbook install-htop.yaml -e package=nano

PLAY [127.0.0.1] ****************************************

TASK [Gathering Facts] ****************************************
ok: [127.0.0.1]

TASK [Ping host first...] ****************************************
ok: [127.0.0.1]

TASK [Install nano] ****************************************
changed: [127.0.0.1]

PLAY RECAP ****************************************
127.0.0.1                  : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Note how the task name has changed accordingly in the output. So Fancy!

Creating some structure - Roles

As nice as scripts are, they don’t scale well. To combat impending chaos, we break functionality down into roles. Roles are collections of tasks, variables, and other resources that can be mixed and matched in playbooks.

A role is defined by a directory of it’s name, and usually contains a tasks sub-directory, where all of it’s tasks are defined. Each sub-directory requires a main.yaml to be the root file for that directory. So, at the vary least, a tasks directory will have a file named tasks/main.yaml which contains the role’s tasks.

If there are a BUNCH of tasks defined, they can be broken out into seperate files, and included in the main.yaml task file.

In addition to tasks, a role might include a defaults or vars sub directory. These are again structured with a main.yaml file that may, or may not, import other files, depending on the size and organization of the role.

## Example structure of a 'subscriptions' role
roles
└── subscriptions
    ├── defaults
    │   └── main.yaml
    ├── README.md
    └── tasks
        └── main.yaml

It is important to note that these yaml files contain just their item. For example, the task files contain just tasks. This is because when a role is imported into a playbook, its items are simply inserted accordingly.

ansible.cfg

Before we start writing some roles, it is important to know that if you are using roles, you need to tell ansible where to find them. The easiest way to do this is to define an ansible.cfg file in the directory you will run ansible-playbook from. For example:

[defaults]
roles_path = roles/

Our role

As it stands, our example playbook is a massive 13 lines long! I can hardly open the file without crashing my text editor. So, lets try to break up the functionality into roles.

First, lets make the directories:

mkdir -p roles/install-htop/{tasks,defaults}

Next, we can add our variables to a default file, roles/install-htop/defaults/main.yaml:

---
package: htop

With the package variable set, lets create the tasks. To demonstrate including other files in the main.yaml, I’m going to be overly-complicated and extract our ping task into its own file, and then include it in the main.yaml.

So first, roles/install-htop/tasks/ping.yaml

---
- name: Ping host first...
  ping:

And then, roles/install-htop/tasks/main.yaml, which will also include our dnf install task…

---
- include_tasks: ping.yaml

- name: Install 
  dnf:
    name: ""
    state: latest

Congrats, we have an install-htop role defined!

Including roles in playbooks

Just as we included vars and tasks in the playbook, if we already have tasks and vars defined in a role, we can instead include that role:

---
- hosts: 127.0.0.1
  connection: local
  become: true

  roles:
    - install-htop

Easy. Let’s run it:

➜  ansible-playbook install-htop.yaml
[WARNING]: Ansible is being run in a world writable directory (/tmp), ignoring it as an ansible.cfg source. For more information see
https://docs.ansible.com/ansible/devel/reference_appendices/config.html#cfg-in-world-writable-dir

PLAY [127.0.0.1] ****************************************

TASK [Gathering Facts] ****************************************
ok: [127.0.0.1]

TASK [install-htop : include_tasks] ****************************************
included: /tmp/roles/install-htop/tasks/ping.yaml for 127.0.0.1

TASK [install-htop : Ping host first...] ****************************************
ok: [127.0.0.1]

TASK [install-htop : Install htop] ****************************************
ok: [127.0.0.1]

PLAY RECAP ****************************************
127.0.0.1                  : ok=4    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

and it still works :)

Conclusion

While only the tip of the iceburg, I think we have covered enough basics to make something useful. Using this small amount of Ansible knowledge, I have been able to create playbooks that configure applications, update all my computers, and setup each of my machines when I reformat them. However, don’t let that stop you from learning even more! Ansible is a powerful tool and worth any amount of time invested into it. Enjoy!

TSQA 2020

2020-04-22T00:00:00Z

TSQA 2020 - Durham Convention Center, Durham NC

Several weeks ago I attend TSQA 2020, a conference presented every two years by the Triangle Software Quality Association (TSQA). Despite being hosted by my local software testing group, the speakers and attendees were from all over the country. While only a single-day conference, it was packed full with solid advice and ideas I left with. Here are a few.

Just to clarify: TSQA occured in February, right before COVID-19 really started spreading in the US. I’m just very late in this post.

TSQA 2020

Getting There

TSQA was held at the Durham Convention Center, which is located in the middle of downtown Durham (NC, USA). Being so close to our office, I managed to help convince/remind several of my co-workers to register last minute. On the morning of the event, I drove the one mile to the office parking garage, then hustled coat-less through the cold morning wind to the conference center.

As I walked inside, I immediately saw my manager at the entrance handing out bandages, and knew I was in the right place (he was a TSQA volunteer). After saying hello and checking in, I made my way to the main ballroom to grab some food. After awhile, my co-workers started to trickle in, so we found a table near the front and gathered for the opening statements.

Keynote

The Jetsons.

They keynote, presented by Angie Jones was an entertaining look at the history and future of technology. She took ideas that were shown in The Jetsons, and compared them to how close (or far) we have come to many of them. She then used this as an example of how we should be planning to test future technology, because like it or not, it’s coming. She concluded by providing some examples of what testing in the future could look like.

Talks

For the rest of the day, I attended several talks interspaced with a lunch and snack break. I mainly focused on attending talks around automation, but also went to a few about developing test cases or UI testing tools. Similar to my All Things Open 2019 post, instead of narrating each talk I went to, I picked out few lessons I learned and will share them below.

After the conference ended, my co-workers and I made our way back to the office to share experiences and debate our thoughts over a drink. It’s always great to hash out ideas with others after a conference, while they are still fresh. Also… it’s fun :) .

Lessons Learned/Strengthened

Now to summarize a few of the many lessons I picked up while at TSQA 2020. I’ve heard many of these suggestions before, but the speakers presented them so well, I really want to ensure I start implementing the at work. Lets get started.

No failing tests

It is all too easy to let a backlog of failing tests build up. This may be due to the test being out of date, a low priority issue, or worse of all… just a flaky test. Regardless of the reason, failing tests really should be mitigated immediately, for several reasons.

Leaving failing tests to continuously run causes failure fatigue. This normalizes the failing tests and causes a team to ignore other failing tests in the future. Basically, it decreases the competence in the test suite, and thus the QE team as a whole.

This is supposed to be a joke, but it's actually the point I'm sharing. If tests are failing, disable or get rid of them, *AFTER* filing an issue!.

So, if a project has continuously failing tests, try to fix them as quick as possible. If there currently isn’t time to fix it (we’ve all been there), that’s fine, but file an issue to remember to fix it later. Then disable it. It will make a test suite more meaningful, because each failure means something important.

One last statement here: No flaky tests either. If we can’t depend on them, they aren’t worth having. Either figure out how to fix them so they are consistent, or remove them.

Write tests to pass for known issues

Now, lets expand that suggestion of disabling failing tests, and write tests that pass when a known defect happens. While sounding counter-intuitive at first, this idea makes more sense if you accept the fact that tests don’t actually find defects. Rather, automated tests depict the state of a system, and fail when something has changed. A passed or failed test simply acts as a data point. When a quality engineer goes and investigates the test results, they determine if there is a defect based on the data.

For example, lets assume I have a test that asserts that an api call returns a 200 status. However, due to a known issue, it is currently returning 404s. I know that in its current state, the system returns a 404 for that api call, so I can change/add a test to assert that it is indeed, returning 404s. This technique removes a failing test (as suggest in the previous section), while allowing us to maintain that data point. The test will pass while it continues to return 404’s, but will fail and notify us of any related state changes.

Those changes could be from the developers merging a bug fix… or a new defect popping up, resulting in the api call now returning 500 errors. Regardless, we know that something has changed again, and we should investigate. By comparison, an always-failing or fully removed test would not have demanded our attention so easily.

Better Documentation

So… if we have our tests setup to pass for known defects, lets make one thing clear… we need to make sure we have amazing documentation. This is imperative.

Any test designed to pass on a known defect should contain a few pieces of information in its documentation:

Why the test is there: If a test is checking for undesired behavior, it is a good idea to quickly document something along the lines of ‘This test is checking for behavior we don’t want. It is just here until an issue is fixed’.
The issue number. If a test exists until a known issue is resolved… please include the issue/bug number in the documentation. This will make it much easier for others (or you!) to find more information, or check if the issue is already closed once the test starts failing (due to a fix).
What we expect from the test: Try to document why the test is currently passing, and what we expect when it starts failing again.

Continuing with our example from above:

“This test has been altered until issue #23483 is resolved. Currently, this api call returns a 404 status. If the issue is resolved, it should return a 200 status, at which point this test needs to be updated.”

Lastly, remember to periodically clean up the documentation. When filing an issue, state were a failing test can be found if it is being updated to match the behavior. This way when the issue is closed, it will be easier to go update the test. Temporary test states can also be marked with a tag, such as TODO to make them easier to search through. Honestly, whatever works for your team. The goal is to create enough context so that someone else is able to know what is going on and fix it without having to find you.

Conclusion

Overall, TSQA 2020 turned out to be a wonderful conference and was a desirable size. It was not crazy and over-crowded, but also not so small that it felt awkward. It had a diverse mix of people from all over the country, and the highest percentage of women I have ever seen at a tech conference. I had a great time and definitely plan to attend the next one… even if I have to travel more than a mile next time.

Creating Tests For This Website - Docker Jenkins Nodes

2020-03-31T00:00:00Z

PNC Arena (Parking Lot), Raleigh NC

Okay, quick post! Previously, I wrote about how I automated my website tests using Jenkins. When I wrote that post, I had the tests run on any node. I wanted to have the tests run inside a fedora docker container, but ran into issues configuring it. With the problem now long fixed, I decided I would write a quick update post about switching the pipeline to use container nodes.

Why Switch

When I first defined the pipeline, I had it use any node for the agent:

pipeline {
    agent any

    stages {

        // This is where the stages will be defined //

    }
}

This just runs the job on any available node, which for me was just the same VM server I was running Jenkins on. This was fine, but for testing I want to make sure everything in my automation is configured and up to date. The best way to ensure that, is with clean runs.

When using docker for the pipeline agent, a new container is created to run the pipeline in, and then destroyed when completed. This means all packages and dependencies must be defined correctly, or the run will fail. This is what we want.

Using Docker Nodes

First, the obvious: make sure docker is installed on the desired Jenkins nodes. I won’t cover this as it can be different for every user (and I already had docker installed on my Jenkins host).

With docker installed, next make sure the Jenkins server has the Docker Slaves (and possibly Docker Pipeline) plug-in(s) installed.

Lastly, with some docker plug-ins enabled, switch the agent statement to use a container image. I choose to use the fedora:31 image:

pipeline {
    agent {
        docker {
            image 'fedora:31'
        }
    }
    ... // rest of the pipeline
}

Fixing root/sudo error

When I first set the pipeline to use the fedora image, it kept failing. Specifically, the sudo dnf steps would fail because the sudo command didn’t exist in the container. If I removed sudo from the command… it still failed because I didn’t have permissions to run dnf inside the container ಠ_ಠ (yes, the user was root).

After some research, I learned that it wasn’t passing the root permissions to the container, and I could “solve” this issue by providing the -u flag with 0:0 as an arg to the docker agent:

pipeline {
    agent {
        docker {
            image 'fedora:31'
            args '-u 0:0'
        }
    }
    ... // rest of the pipeline
}

I don’t love this solution… but it seems to work.

Conclusion

Like I said, this was just a quick update about switching my test nodes to use docker containers. Honestly, I’d much rather try to use podman containers for my test agents, but I’m sure that would be much more complicated currently. Maybe in the future…

Creating Tests For This Website - Links

2020-03-30T00:00:00Z

PNC Arena, Raleigh NC

In my previous two posts, I created a test framework for my website, and automated it using Jenkins. But we can do better. One of the most annoying things when maintaining (or even reading) something on the internet, are broken links. While I cannot control the availability of content outside the website, I can choose to remove links if they are broken. So, in this post, we will add tests to ensure that links in our posts are working. Well, at least the markdown ones.

What to Test

Google 404 Error Page

For this test set, we will be scanning the content files of all of the posts, and grabbing every markdown link defined in them. With the links known, we will then make a request to each one to check that it is available. If we can connect, the test passes. If not (ex: we get a 404), it fails.

Adding Utility Functions

Before we are able to write the test function, we first need to add to the utility functions. These will allow us to get the post’s file paths, grab their content, and extract all the markdown links from that content.

get_file_paths

First, lets define a new helper function, get_file_paths:

def get_file_paths(src, extension=None):
    """Collects the paths of all files of a directory"""
    file_list = []
    root_path = path.expanduser(src)
    for file in listdir(root_path):
        # If extension provided, check file has that extension
        if extension:
            if file.endswith(extension):
                file_list.append(path.join(root_path, file))
        # Otherwise, add everything
        else:
            file_list.append(path.join(root_path, file))
    return file_list

When provided a directory path (src), this function will return a list of all the file paths in that directory. Optionally, the extension parameter can be supplied to only return files of that extension type (in our case, md). This will be used to grab the paths of all of the website post source files.

get_file_content

Now lets define get_file_content. This function will take the file lists generated from get_file_paths, grab the content from those files, and return a dictionary of all the data.

def get_file_content(file_list):
    """Grabs all the content from a list of file paths."""
    content_all_files = {}
    for file in file_list:
        f = open(file, "r")
        file_content = f.read()
        content_all_files[path.basename(file)] = file_content
    return content_all_files

The returned dictionary uses the filename as the key, and the content set to the value. For example:

{
  'post1.md': 'This is the text of post1.',
  'post2.md': "This is the text of post2. Basically the same ol' stuff."
}

get_md_links

Lastly, lets define get_md_links. This function takes the content dictionary returned by get_file_content, and uses some regular expression magic to match the markdown links:

def get_md_links(content_dict, regex="\[.*?\]\((.*?)\)"):
    """Parses the dictionary of content strings, and pulls out the url of any links."""
    p = re.compile(regex)
    all_links = []
    for file in content_dict:
        content = content_dict[file].replace("\n", "")
        match_iter = p.finditer(content)
        for match in match_iter:
            # Regex can't properly match urls with parens in them, so skip.
            if "(" not in match.group(1):
                all_links.append(match.group(1))
    return all_links

First, the function compiles the regular expression defined by the regex parameter. Next, it loops through all the data in the content dictionary, strips the newline characters, and then grabs all the regex matches.

Unfortunately, the regex expression can’t properly match markdown formated urls with parenthesis in them, so we have to check if each match has a ( in it. If it does, the url is thrown away because we cannot be sure we matched the full url. If there are no parenthesis, the url is added to our saved list. After parsing all the values of the content dictionary, a list of the matched urls is returned.

Adding to conftest.py

With our new utility functions defined, we can next add a new fixture (and it’s helper function) to the conftest.py file. Lets start with the fixture’s helper function, post_md_link():

def post_md_links():
    """Returns the md_link object of the md links in all the posts."""
    all_post_files = get_file_paths(POST_DIR)
    all_post_contents = get_file_content(all_post_files)
    all_post_md_links = get_md_links(all_post_contents)
    # Return de-dup list
    return list(set(all_post_md_links))

This function uses the utility functions we just wrote above, to extract all of the markdown links from the post files found at the POST_DIR constant location. It then returns a de-duplicated list of all the links.

Now, we can define the fixture, post_md_link:

@pytest.fixture(params=post_md_links())
def post_md_link(request):
    """Returns the md_link object for a md link found in a post."""
    return request.param

Similar to the fixtures in the pages tests, this one will allow tests to map across all the links found in the markdown pages, so a test will run for each link.

Writing the markdown link test

Finally, time to write the one and only test function in this post:

def test_md_links(post_md_link):
    """Checks that the markdown links are not broken."""
    if post_md_link.startswith("http") or post_md_link.startswith("https"):
        url = post_md_link
    else:
        url = BASE_URL + post_md_link.lower()
    response = requests.get(url)
    assert response.status_code != 404, f"The link {post_md_link} is not found."
    assert response.status_code != 403, f"The link {post_md_link} is forbidden."

Because I link to both internal and external pages in my posts, I have to prep my urls a bit. So, I first check if the link starts with http (which would also match ones starting with https). If it does, we can leave the link as is. If it doesn’t, we can assume the link is an internal one (ex: /post/creating-website-tests-links/), and we need to prepend it with the BASE_URL constant.

With a proper url, we can use requests.get() to attempt to retrieve a response code from the page. If we get a response, I then assert that the status_code is not 404 or 403.

Side Note: I started by asserting that each link returned a 200 status, but quickly learned that it was a bad idea, because I was testing mostly external links. I never got all the tests to pass because they would often return odd 500-level errors for issues that quite frankly, doesn’t matter to me. For example, one site kept return a 500-level error I think because their servers were ‘under a slightly higher load’… but when I went to the link, the page loaded fine.

In the end, I decided I wasn’t trying to test the issues the websites I linked to were having, but instead just wanted to make sure that my links weren’t broken. So, I now just ensure that I’m not getting 404 or 403’s, and I’m happy with that.

Limitations

While I am very happy with the coverage these tests provide, they do have some limitations to keep in mind:

They cannot match urls with parentheses
Currently, they only check that pages do not return 403 and 404 errors. This means I could possibly still have broken links due to permission errors or other issues. I plan to expand this assert list in the future to cover more cases.
I’m currently only testing markdown links. This doesn’t grab any html links I have in my posts.
- On a similar note, because I link most of my images with html, it also isn’t testing if my images are broken.
- I’d like to add tests for both of these issues eventually, but decided testing the markdown links was the best place to start.
Sometimes tests fail because a site is down. No biggie. I just usually wait a bit and then run the tests again before deciding to remove the link.

Conclusion

Passing tests, including the new markdown link tests

That’s it. By adding a few easy helper functions, a new fixture, and a single test function, We’ve expanded my test results from 70 to over 420 tests (and growing).

More important than the number of tests, is what the results tell us. A failing test tells me that one of my markdown links might be broken. These tests have already been beneficial to me, as I ended updating/removing probably about 100 or so bad links from my archived posts while implementing this test. So, I’d say it was worthwhile!

Creating Tests For This Website - CI

2020-02-28T00:00:00Z

PNC Arena, Raleigh NC

In the last post, I setup some simple testing for my website builds to that ensure that pages were being served correctly. However, I can’t trust myself to always manually run the tests before merging a branch into master. Luckily, I have Jenkins to take care of all the “responsible” tasks. In this post, we will take the test framework created in the previous post… and automate it.

What I’m Using

The Jenkins Logo

I have started using Gitlab for more of my projects recently, but have decided to keep my website source hosted on Github for the time being. So, I won’t be using Gitlab’s CI/CD tools for this, but I wanted it to be known that this automation is very straight forward and could be easily accomplished there as well.

For this project, because my website is hosted on Github, and I already have my own Jenkins server configured… I will be using a Jenkins pipeline. First, we will create the pipeline file to add to the git repo. Then, we will use the pipeline to configure a new multi-branch pipeline in Jenkins.

Jenkinsfile

Lets start by creating the Jenkinsfile. Create a new file named Jenkinsfile in the project directory. Next, lets create a new pipeline and get ready to add stages:

pipeline {
    agent any

    stages {

        // This is where the stages will be defined //

    }
}

(Make sure to add the closing }'s!)

Stages

Now, lets add the stages. A stage is a named, functional chunk in our pipeline. Breaking the pipeline into stages will help keep all the various steps organized, as well as make it easier to follow along as the pipeline runs.

Each of the following stage definitions will be placed inside the stages { .. } section we defined above (in the same order as they are listed!).

Setup Deps

stage("Setup Deps") {
    steps {
        sh 'sudo yum update -y'
        sh 'sudo yum install -y epel-release'
        sh 'sudo yum install -y hugo python36-pytest'
    }
}

The first stage is ‘Setup Deps’. This stage handles installing any dependencies our Jenkins node will need installed to run the tests. Our tests will require hugo, pytest, and python3. It will also require some pip packages, but we will get to that later.

Note: My current Jenkins node is running CentOS, so my package manager commands are specific to that. As always, adjust accordingly.

Start Hugo Server

stage("Start Hugo Server") {
    steps {
        sh 'hugo serve &'
    }
}

With hugo installed (and presumably being inside the website’s git repo…), we can start the web server. This is done with hugo serve. The & is used to have the server run as a background process so that it won’t be killed when the pipeline to continues on.

Note: Make sure the tests are set to point to localhost:1313, as that is where hugo will try to serve the website by default.

Setup Python

stage("Setup Tests") {
    steps {
        sh 'pip3 install pipenv --user'
        sh 'python3 -m pipenv install'
    }
}

Next, lets configure python by setting up an environment and installing the packages we need in it. First, I use pip3 to install pipenv. Then, I have pipenv install the tests’ required python packages, which are defined in the repo’s Pipfile.

Run Tests

stage("Run Tests") {
    steps {
        sh '''
            set +e
            python3 -m pipenv run pytest -v --junit-xml himmallright-source-test-report.xml .
            set -e
        '''.stripIndent()
    }
}

Test time. I again utilize pipenv here, by having it call the test command (pytest -v --junit-xml himmallright-source-test-report.xml .) so that it runs in the pipenv virtual environment.

Two things to note here:

The --junit-xml flag defines a xml filename to write the junit test report to. This will be used by Jenkins to collect the test results.
The test command is wrapped between set +e and set -e commands, which allows tests to fail but without triggering a pipeline failure in Jenkins. This way even if tests fail, we make it all the way through collection so we can see what tests failed and why.

Collect Test Results

stage("Collect Test Resuts") {
    steps {
        archiveArtifacts "himmallright-source-test-report.xml"
        junit "himmallright-source-test-report.xml"
    }
}

Next, we archive the junit report xml file as a Jenkins artifact (just in case). Finally, we have the junit plugin collect the report.

Save & Commit

That should be it for the pipeline file! Commit and push it to the git repo, and we can start working with it in Jenkins!

Multibranch Pipelines

With the Jenkinsfile in the repo, we can create the pipeline! Specifically, we will be creating a mult-branch pipeline. A multi-branch job scans a git project, and creates a separate pipeline for each branch or PR in the repo. This is beneficial for testing, as it will automatically instantiate a test pipeline against a newly created PR, so we can verify that the PR passes the tests before merging it into the master branch. Additionally, it lets us make sure we aren’t breaking anything while working in a new branch.

Creating the Pipeline

Select Multibranch Pipeline from the new item menu in Jenkins.

To create a Multi-Branch pipeline, select New Item in the menu on the left. Next, enter a name for the pipeline and select Multibranch Pipeline at the bottom. Click Ok.

Configuring the Pipeline

Configuration options when creating the multibranch pipeline.

On the pipelines configuration page, start by filling out the Display Name and Description text boxes. Next, go down to Branch Sources and click on Add source. My website is currently hosted on Github, so I will select that. However, select Git if your project is hosted on another git service.

Add the Repository URL and choose the pipeline Behaviors. The Behaviors define how the pipeline will split up branches. For example, it can be selected to only discover branches that are also PRs.

Next, in the Build Configuration section, be sure that the Script Path defines the path where the Jenkinsfile is located. If it’s in the root directory (and named Jenkinsfile), the default should work.

Lastly, select an interval to automatically check the repo for changes in the Scan Repository Triggers section. This step is optional, but I highly recommend it.

That’s all we need to setup, but feel free to research more options. I mostly have defaults selected for the rest. When complete, hit Save.

Running Pipelines

The multibranch pipeline overview page.

Once the multibranch pipeline is created, it should scan the repo to detect any branches or pull requests that have defined Jenkinsfiles. It will create an job item in the list for each branch/PR it detects (and kick off runs for each).

To manually start a scan, select Scan Repository Now in the menu on the left, and it will scan all the branches again, looking for changes, and kicking off pipeline runs for any branch or pr that has changed.

The overview page of a single branch.

To manually start a specific branch run, click on the branch name to enter the branch’s job overview page. Then, simply click Build Now on the left. Once the run starts, it runs like a normal jenkins job and can be viewed by clicking the job’s run number to the bottom left. The job’s progress can then be viewed on that page, or using Blue Ocean (Recommended).

Viewing Results

The overview page of a single branch.

Personally, I prefer to always view the test results using the Blue Ocean viewer. Once a job completes, select the Tests tab at the top of the viewer to see the collected test results (this is what the junit steps in our pipeline does). If all the tests passed, the page will be green and list all the completed tests. If some failed, it will be yellow. When there are failed tests, they can be clicked, and the row will expand to show the failed test’s error message and stack trace. For my test set, this makes it easy to see which page failed, and even know what status code was actually returned in the response (by looking at the stack trace). Very helpful!

Conclusion

There we go! Not only do we now have the website tests automated as a pipeline, but as a multibranch pipeline. This should help automatically ensure that nothing breaks as I edit and add to the website. It will even run the tests against all my PRs, so I can be confident that when I merge to master, it won’t slowly degrade my website over time. I have one more post about these tests planned, but in the meantime… enjoy Jenkins!

Creating Tests For This Website - Pages

2020-02-23T00:00:00Z

PNC Arena, Raleigh NC

As this website grows, there is an increasing amount of complexity. More posts, more images, and more links. I’ve gotten better at breaking work up into separate branches (instead of pushing everything straight to master), but even that isn’t enough to ensure everything works as expected when publishing something new. Then, I thought of something obvious… I could setup some simple testing… for my website.

What to Test

After editing a page or drafting a new post, I often wonder “how can I be sure everything will still work when I publish this change”? I question if every post file is actually being served as a web page. Or worse… I fear that a post that isn’t ready to be published might accidentally get pushed with an unrelated website fix.

(Yes, this is a completely unreasonable fear given that ALL of my website source files, drafts included, are publicly hosted on Github. Nonetheless, the fear exists)

This will be a multi-post serries, so in this first one we will focus on:

Configuring the test environment
Building up the testing framework
Writing some basic tests to ensure:
- The pages I want to be served are
- Pages and posts that are not ready, are not being served

As my website is currently compiled using hugo, the tests will be centered around that framework. However, most of the information can be applied to testing websites using other static website generators, are they are all quite similar.

Setting up the env

I will be using pytest for the testing framework, and to make all the python dependencies a bit easier to manage, I will also use pipenv. Lastly, I usually work on a Fedora computer, VM, or at the very least in a Fedora podman container. So, some of my instructions use dnf, but feel free to adjust to your package manager accordingly.

Install `pipenv`

First, lets install pipenv, which is easy enough in Fedora:

sudo dnf install pipenv

Install needed packages inside `pipenv shell`

After installing, create a pipenv shell and enter it:

pipenv shell

Install pytest and requests in the pip environment:

pip install pytest requests

Creating the Test Framework

With the environment setup, we can start building up the test framework. We will start by defining come constants, then use those when building helper functions. Lastly, we will use the helper functions to piece together the conftest.py and test_pages.py files.

Defining Constants

First, lets define some constants we can use throughout the test framework. In the future, I might switch these to be set optionally with CLI arguments, but for now… they’re just static constant variables defined in a file.

So first, create a new file in the tests directory named constants.py. In that file, lets dump our contants:

BASE_URL = "http://localhost:1313"

SITE_PAGES = ["/", "/pages/about/", "/pages/homelab/"]

POST_DIR = "./content/post/"
POST_NAMES = [
    "25-days-of-c",
    ...
    <Removed middle of list because it's long>
    ...
    "ZFS-Backups-To-LUKS-External",
]

As you can see, in my constants.py file I have 4 variables defined:

BASE_URL: this is the base url for the website when running hugo serve. For most, it will default to http://localhost:1313, but I have this as a constant because I usually run my hugo serve command with the -b to change it to an ip address so I can view it from other computers.
SITE_PAGES: This is a list of paths that come after the baseurl for pages that we well be testing. For example, I want to make sure that my “about me” page is being served, which is at baseurl/pages/about/, so /pages/about/ is one of the values in this constant.
POST_DIR: This is the directory for where the post files are located.
POST_NAMES: This is a list of the names of the post files (without the .md)

Add in your values for the variables, and remember to save the file.

Writing Some Helper Utility Functions

With those constants defined, we should be ready to write some helper functions. These are normal python functions that will be called from tests or even test fixture functions.

First, lets create utils.py. The helper functions will need to use listdir, as well as the path function from the os module. They will also need the regex functions. So, lets make those imports at the top of the file:

from os import listdir, path
import re

get_file_names

Lets define a helper function named get_file_names:

def get_file_names(src, extension=None):
    """Collects the names of all files of a directory"""
    file_list = []
    root_path = path.expanduser(src)
    for file in listdir(root_path):
        # If extension provided, check file has that extension
        if extension:
            if file.endswith(extension):
                file_list.append(file)
        # Otherwise, add everything
        else:
            file_list.append(file)
    return file_list

When provided a file path (src), this function will return a list of all the file names in the directory. Optionally, the extension parameter can be supplied to only return files of that extension type (for exapmple, md). This function will be used to grab the names of all the post source files.

… and that’s all we need in utils.py… for now!

Conftest

Now lets start digging into test-related stuff, by first creating a conftest.py file. This file will mostly hold the fixtures we will use for the tests. In our particular setup, they will gather lists of pages to run multiple calls of each test against by using @pytest.fixture(params).

But first, lets import a few things at the top of conftest.py:

import pytest
from os import path

from constants import BASE_URL, SITE_PAGES, POST_DIR, POST_NAMES
from utils import get_file_names

The imports include the os.path() function, some of the constants we defined, and the get_file_names() helper function. Oh, And of course pytest ;) .

page_url

@pytest.fixture(params=SITE_PAGES)
def page_url(request):
    """Returns the page urls for testing."""
    return BASE_URL + request.param

The first fixture, page_url, is very basic. It creates a list of all of the website pages (not posts), by combining the BASE_URL with each of the values defined in the SITE_PAGES constant. This list will later be used to paramaterize a single test across all of the page links.

post_url

@pytest.fixture(params=POST_NAMES)
def post_url(request):
    """Returns the post urls for testing."""
    return BASE_URL + "/post/" + request.param.lower()

The next fixture, post_url is basically the same as page_url, except it creates a list of all the posts using the POST_NAMES constant. Again, this will be used to expand a single test into many, one for each post.

non_live_post_url

@pytest.fixture(params=non_live_post_urls())
def non_live_post_url(request):
    """Returns the url of a non-defined post file"""
    return BASE_URL + "/post/" + request.param.lower()

Lastly, we have non_live_post_url with its accompanying helper function, non_live_post_urls. This pair creates a list of posts that have a markdown file in the /post/ directory, but are not listed in the POST_NAMES constant (so in practice, not really to be published).

def non_live_post_urls():
    """Returns the urls of md files that should not be live."""
    all_post_md_names = list(
        map(lambda name: name.lower().split(".md")[0], get_file_names(POST_DIR))
    )
    live_post_names = list(map(lambda name: name.lower(), POST_NAMES))
    non_live_post_names = set(all_post_md_names).difference(set(live_post_names))
    return list(non_live_post_names)

First, the non_live_post_urls helper function returns a list of non-listed post files. That list is then used in non_live_post_url as the pytest.fixture(params) object, much like SITE_PAGES and POST_NAMES were for the previous fixtures.

Finally… Some Tests!

Phew. Okay. With all of that defined… lets create the first test file. When pytest runs, it will try to grab tests recursively from all the files down the current directory, starting with test. This first set of tests will be checking whether a web page is being served (or not), so lets name the file test_pages.py. Again, start with the required imports. This time we only need pytest and requests.

import pytest
import requests

Testing Pages

The first test will check that each page defined in the SITE_PAGES constant is being served. More specifically, we will use the requests module to ensure not only that the page is served, but returns a 200 status. This actually requires very little code to accomplish (Gotta love python) :

def test_page_served(page_url):
    """Checks that the website pages are available"""
    response = requests.get(page_url)
    assert response.status_code == 200

We simply define a function, test_page_served(), and because it is in test_pages.py, it will be assumed to be a test by pytest. We provide the page_url fixture we previously defined in conftest.py as the only parameter. This will call the test_page_served test for each url in the list generated by page_url. Next, we use requests.get() to make a page request. Lastly, we assert that the status_code from our response is 200. If it is, the test passes, if not, it fails.

Testing Posts

Next, lets test that all of the posts are being served. This test works exactly the same as test_page_served, except we are using the post_url fixture instead of page_url to supply the links to test:

def test_post_served(post_url):
    """Checks that the desired posts are available"""
    response = requests.get(post_url)
    assert response.status_code == 200

(While I could combine these cases into a single test function, I decided to keep them separate for flexibility in the future)

Lets Get Fancy: Testing Unapproved Posts Are NOT Served

For the last test, lets get a little bit more complicated and ensure that post files not listed in the approved list are not being served. Well… it turns out all the “fancy” code required for this test case already occured in the non_live_post_urls helper function. The test function itself, is essentially the same as what we’ve already encountered except that we are now checking for a 404 return status instead of 200:

def test_non_defined_posts_not_served(non_live_post_url):
    """Checks that a non-defined post is NOT available"""
    response = requests.get(non_live_post_url)
    assert response.status_code == 404

That defines all of the tests for this first set! Don’t let only having three test functions fool you, they should generate over 70 test results when run! (For my website, at the time of writing this post)

Lets Run Some Tests!

Finally, we should be able to run the tests. To do so, first ensure that you are in the pipenv by running pipenv shell, or you can run the tests from outside the pipenv using pipenv run COMMAND. Next, call:

pytest -v .

The -v flag runs pytest in ‘verbose’ mode, which I like to do as it shows the results for each test run, rather than each file.

Example video of tests running and passing

So it looks like all the tests are passing! To be sure, Lets do a quick check to make sure they work as expected… I’ll mark this post with draft = "False", but not add it to the approved lists, and the test for this page should fail…

Checking that a test fails when we want it to.

Awesome, it failed! I guess all there is left to do is to finish up this post, so I can add it to the approved posts lists and publish it! Stay tuned!

Authorizing Thunderbolt 3 on Fedora Plasma

2020-01-25T00:00:00Z

Durham, NC

After buying a 16" MacBook Pro the other month, I’ve been using a Thunderbolt 3 hub to connect it to my periphery devices. Luckily, in addition to the macbook, I am able to use the hub with my my work laptop when working from home. Normally, it works fine. However, last week I reformatted the work laptop with the KDE Plasma spin of Fedora Workstation 31… and my TB3 hub stopped playing nicely with it. Here’s why.

The Issue

Due to a NVIDIA Quadro M1000M in my lenovo P50 work laptop, I’ve been using Gnome with Wayland, because that combination seems to have the least amount of annoying trade-offs. While doing a system refresh, I figured I would see if things are any different with a clean Fedora 31 Plasma install (they weren’t, but that’s beyond this post).

The Gnome desktop environment now has a thunderbolt pane in the settings GUI.

Booting into the live cd, and later the installed system, I noticed that my Thunderbolt 3 hub did not work while I was in Plasma. That was fine. I expected it really, because for security reasons Linux distros tend to disable thunderbolt ports by default. In gnome, there are now settings to authorize it (which I guess I had done on my previous install, and forgot about). In Plasma however, I needed to install and configure bolt.

Bolt

`bolt` is in the Fedora repos and is easily installed with `dnf`

In Fedora, bolt is in the repos (at least in Fedora 31+), so installing it is easy:

sudo dnf install bolt

Once bolt is installed, it isn’t a bad idea to check to make sure it is running: sudo systemctl bolt.

Checking `bolt` is running using `systemctl status`

If the status shows that it isn’t running for some reason, it can be started using systemctl start bolt. Check the status again, and if it it still is not running… sorry?

With bolt installed, the command boltctl is available to use. The base command will show information about attached thunderbolt devices. To see more boltctl commands, run boltctl --help.

Authorizing

Authorizing the device.

In order to use the thunderbolt 3 device, it needs to be authorized. To authorize a device, first use a plain boltctl command to get the uuid of the device. Next, call boltctl authorize using the uuid to authorize it:

boltctl authorize <UUID HERE>

The device should now be authorized. When I authorized mine, some of my devices started showing up automatically, while others needed to be disconnected and replugged.

Enrolling

Enrolling the device for permanent use.

Authorizing a device enables it to be used, but doesn’t guarantee persistence. While this may be desired to grant temporary authorization to a device, many users tend want their device to always work, without having to manually authorize it. In this case, the device should be enrolled (like systemctl enable). Simply call boltctl enroll, with the device UUID:

boltctl enroll <UUID HERE>

Once enrolled, the device’s UUID will be recorded and added to a database. By default, the device will now automatically be authorized whenever it is connected.

Conclusion

That’s it. While the solution isn’t very difficult, it can be frustrating to figure out when it appears that the thunderbolt device simply is not working. Regardless, I’m glad that I now know about bolt and how to use it!

My New 16-inch Macbook Pro - Initial Thoughts

2020-01-12T00:00:00Z

In my previous post, I mentioned that I had purchased a new 2019 16" Macbook Pro by the end of my macOS challenge. I have had the laptop for over a month now, but I made sure to take notes on some of my initial thoughts during the first few days. Here are those thoughts.

Selection

But first, a word from our sponsor!.. Just kidding. However, I do want to take a minute to talk about how I selected my first macbook.

New 16" MBP on my desk.

While completing the macOS challenge, I decided I wanted to buy a macbook for personal use. However, I wanted to hold out and wait to hear if the rumors of a new scissor-switch macbook pro were true, or if I should be considering a used 2015 Macbook Pro on eBay. Fortunately, the rumor turned to fact, and Apple released the new 16" Macbook Pro. This news coupled with a $300-off, Thanksgiving (US) sale, meant that I had a new macbook shipping to me the first week of December.

The Base Model Specs

16" Macbook Pro closed on the table.

I chose the base-spec MBP (the cheapest one) for several reasons. First, it saved me a bunch of money because 1) Apple upgrades tend to be expensive, and 2) the base models can be found on sale more easily.

Additionally, the “low-end” base model specs of the 16" Macbook Pro were increased to include 16 GB of RAM, and a 512GB SSD. This meant that it already had everything I would normally need to upgrade. So, the specs:

- 2.6 GHz (4.5 GHz Turbo Boost) 6-Core (12 thread) i7 CPU
- 16 GB 2666MHz DDR4 RAM
- 512 GB SS
- 16" "Retina" display w/ True Tone
- New Keyboard design, with touchbar and Touch ID
- Four Thunderbold 3 ports
- Headphone Jack
- New speakers and mic
- Silver

In a perfect world, I might have wanted 32 GB of RAM. But if I think about it, I mostly need RAM to run clusters of VMs… which I do on Linux workstation/servers… not my laptop. So 16GB should be plenty for the needs of this machine.

Initial Thoughts

Okay, now lets get to my initial thoughts. Remember, these are the (reformatted) notes I took within about a day or so of receiving the Macbook.

I love both the new keyboard and speakers on the 16" Macbook Pro.

Keyboard - Well, I’m surprised. I think I actually really like it. The first few minutes were a little weird, because I was used to the (2014) MacBook Air (Also, I had been using my HHKB all day). After using it for about an hour now, I think it feels close to the air (which I like), with enough travel. It definitely does not have as much travel as my Thinkpads, but it is acceptable. The keys feel like they “pop” as I type on them, with a very satisfying bump. Overall, I very much enjoy the typing experience. As long as it proves to be sturdy and reliable, I’d say this is a great laptop keyboard. I am very happy with it.
Trackpad - Little weird at first (I had trouble sliding my fingers across it. I guess I needed time to oil it a bit lol), but now it seems to be working well. It is massive, but doesn’t seem to register accidental touches much. Really, there isn’t much to say. It does what I need it to, basically all the time.
Screen - Wonderful. It’s big, bright and the colors look great. Using the default scaling still, but actually think it’s big enough. It’s a beautiful screen large enough to get work done, on a still small-ish laptop.
Touch Bar - Meh. It’s whatever. I have already installed better touch tool 😂. I’ll play with that later to figure out how I can switch the audio/brightness to buttons instead of slides (the main thing I don’t like so far), and maybe add weather. In summary, with better touch tool, I don’t mind having the touch bar, but probably wouldn’t miss it much if it was gone.
Speakers - Amazing. Simple as that. They aren’t going to replace a high-end pair of studio monitors, but are probably the best laptop speakers out there, and likely better than most cheap desktop ones.
Ports - Just charging so far. I like being able to charge on both sides. Side note: the power brick feels like an actual brick. It is solid and heavy, which is fine, but did surprise me a bit.
Performance - As far as I can tell, great. It seems like it can handle most of the work loads I will need it for.

Issues/Annoyances/Concerns

As for ports, the 16" Macbook Pro has two USB C/Thunderbolt3 ports and a headphone jack on one side (pictured), and 2 more USB C/Thunderbolt3 ports on the other side.

Heat - As I’m typing this, I do notice the metal at the top of the laptop (above the Touch Bar) is getting quite toasty. In fact, the keys on the number row are even a bit warm. I understand that this is where the heat is exhausted, and that the cooling system probably relies on the aluminum body a bit as a heat sink. I just hope it’s normal. I should note that this heat up happened while I was installing Xcode… which seems to be a beast. Once Xcode finished installing, it did cool down fast. The entire time, the palm rest remained nice and cool to the touch. Also, the fans never kicked up for this. So it seems that the cooling system is very good on this laptop. I just wish it would kick in a little bit sooner to prevent it from heating up.
Ports/Adaptor Annoyances - Everyone is complaining that there still isn’t an SD card reader, but honestly I could care less. I generally think having powerful TB3 ports is a good plan, as it allows the user to craft their system to have whatever ports they need. For example, I don’t usually need an SD card reader, but I do use USB3 and HDMI daily. With a dongle/adaptor, I can have that, and the TB3 connector has enough bandwidth to support it. However, the annoying part is buying into the TB3 life. I had to buy an expensive dongle for my new expensive laptop in order to use anything with it. Also, the functionality of the adaptors seems to be hit or miss. I got one that works okay, but only after having to return the first one I bought because it just wouldn’t connect my devices. Very Annoying.
Wrists Rest - While I really enjoy the solid feel of the aluminum my hands rest on (and how it tends to stay cool even when the computer is heating up), the sharp edge of the laptop cuts into my wrist over time. I will admit that this is something I have experience on other laptops too. Maybe I just type wrong.

Conclusion

The 16" Macbook Pro is a great all in one portable workstation.

In conclusion, I love this laptop. It is powerful enough to do everything I need, and I think Apple has done a great job addressing the complaints users have had with Macbooks over the last few years. Considering the speaker, microphone, keyboard, and screen upgrades this is an amazing portable workstation. As much as I love having a full desk setup, the “built-in periphery devices” on this laptop are so good, I really don’t mind working straight from the laptop. Which coming from me, is saying something.

Challenge - 30 Days with macOS

2019-12-31T00:00:00Z

American Tobacco Campus, Durham, NC

I have used many operating systems on my computers over the years, including decades worth of Windows versions, tons of Linux distros, several BSDs, iOS, Android, and even chromeOS. While I am familiar with using macOS (I often used the school Macs in college, and my wife had a Macbook Air throughout medical school), it is the one OS I have never tried as my personal daily driver. After switching to an iPhone and getting an iPad this year… I decided to give macOS a spin.

(Fun fact: I first learned about Linux while attempting to Hackintosh my laptop in high school.)

The Challenge

Using a mac as the interface device during my personal computing, for 30 days.

Here, “interface computer” means the device that I sit at and interact with. For example, I might be sitting at macOS and using it for my web browsing and music, but working on a project on my linux workstation via a mosh/tmux session in iTerm. I can use remote Linux computers as much as I please, but should connect to them through macOS.

One thing to note is that I still used Desktop Linux as my “interfacing computer” for work.

Why

At home, I don’t mind iOS and macOS teaming up to make my “normal person” computing (web browsing, video chatting, emails, and online courses) easier. I still deeply care about working with Linux at home, but exclusively on the backend. So having the front end get out of the way, and not tempt me with the ability to fine-tune it, is a good thing (for me, currently).

(While I’ve slowed down distro hopping the last few years,… I have still been swapping my desktop environment every other month).

Right now, I am really only interested in using macOS for my personal computing. At work, I don’t need my computer to be in step with my personal phone and/or ipad. In fact, I like having that barrier there. Also, Linux works REALLY well for the work I do, especially when it comes to running VMs (I love libvirt). So, there currently isn’t a reason (or even a desire) to switch there.

Side Note: I actually wouldn’t even consider this challenge if I didn’t get to use Linux so much at work every day, or if I didn’t have my Linux desktop/workstation at home. I already ssh into my desktop to work from my other devices. In fact, I technically remote into my desktop even when I’m working at it, since I do everything in a container.

Observations

Now that I’ve spent a few weeks in macOS, lets break down some observations I had.

Stuff I didn’t like

The elephant in the room… it’s very proprietary. I still love open source and using something that’s as locked down and secretive as macOS (and Apple products in general) feels, well… dirty.

Error while trying to run Joplin.

I had trouble getting some applications like Joplin to work. I later learned that it was likely due to issues with signing the applications in Catalina (I think). Apparently, there is a work-around for this.
There surprisingly isn’t a great solution for virt-manager. I was able to install it using homebrew, but have to launch it from the terminal. I guess I might be able to setup an application launcher for it? I’m just surprised there isn’t a better solution on mac, or at least one that’s easy to find…
On that note, it is wonderful that brew exists, but it confuses the hell out of me (What’s a cask? Is it like a dnf group install?). This is most likely because I am new to it, but in my opinion, I still think it is nowhere near as nice as default Linux package managers. This is one area where using Linux for so long has definitely spoiled me.
Working with filesystems is a pain, especially when connecting to network devices. I couldn’t use Finder to do an easy network connection without first setting up samba or nfs on my Linux computers. While I ideally want to setup a proper samba/nfs server on my desktop/server in the long run anyway, it was annoying that it couldn’t just mount the system in the file browser, using something like sftp under the hood like Linux does…

Click for example video connecting to a remote file system in Gnome

Window management isn’t great out of the box. I had to buy magnet to snap/tile my windows. (Alternatively for touch-bar mac users, note that better touch tool has window snapping and other features built in). Also, I can’t seem to hold a key and click a window anywhere to resize/move like I can in most linux DEs.
My desktop workspaces keep changing their order on me (ಠ_ಠ).
Deviating from the stock experience can become painful. Don’t like using a global menubar, and a dock for applications?.. sorry? I’m sure there are third-party ways to get around it, but they are likely hacky and not guaranteed to always work across new versions of macOS.

Stuff I liked

I am actually starting to like the CMD-centric shortcuts. They work everywhere and are nice when working in a terminal (which I do all the time).
Default keybindings, while different from what I’m used to are intuitive, and consistent across the system. There are not many that are crazy. For example, on Windows/Linux systems, the keybinding to close a window usually defaults to Alt+F4, so I normally change it to Super+Shift+Q. While not the exact same as what I do, the macOS default of CMD+Q is close enough that I’ve been able to easily adapt to it, without feeling the need to figure out how to change it. It’s a similar story across most of the default keybindings. They aren’t exactly how I’ve been setting mine up, but close enough that I’m fine with accepting them as is.
iCloud syncing has been nice. Specifically for photos, TODOs, and notes.

I like being able to answer all phone calls from my computer. I hate talking on the phone. Oddly enough, I don’t mind voice chat on the computer, so this works out really well for me. Synced iMessage and Facetime are also nice, but universal normal phone calls is what I particularly enjoy.
Profiles. Specifically, setting up something like my Fastmail account in the mail app was SUPER easy because of this. I basically just downloaded the profile from fastmail and opened it with mail, and everytihng was setup. That’s it.
I find the UI to be really space respecting. In addition to a global menu, title bars on windows are thin and minimal. Combined with scaled font sizes, a lot less space is wasted, especially compared to Gnome.

I find macOS to be much more space efficient than Gnome.

Videos just work without configuring anything/switching back and forth between Wayland and X sessions. I have faith that wayland will mature to this point eventually, but its nice to already have it in macOS.
Gestures are fluid and intuitive. Even on the old MacBook air, the trackpad is amazing.
Settings like screentime work across all my “front-end devices”. If I set a downtime for 8:30pm, it boots me off all my devices… and I get to bed earlier.
macOS comes with a set of well-made core apps (calendar, mail, notes, music, photos, keynote, garage band, etc). They work for everything I need. If I want something more advanced, I generally have the full selection of open source and proprietary apps at my disposal. For example, I’ve been using Gimp just fine on the macbook.
Security does actually seem to be a focus on the system. For example, someone tried to log into my icloud account from a random location the other week. In under a minute, from my phone, I was able to unlink my account from all my devices and change the password.

Future Plans

Overall, I’d say this challenge was a success. In fact, it was so successfull, that I’m actually writing up this post on a new base model 16" MacBook Pro (but more on that at another time).

[ Well, I’m really writing it on my desktop, but from the macbook pro ;) ]

My plan is to use macOS and my iOS devices to handle my interactive/client computing. I’ve enjoyed having my apple devices take care of all my “normal” person computer needs over the last few weeks. It has allowed me to finish up required, but mundane tasks easily and efficiently, so I can better focus when diving into iTerm to work on complicated Linux server projects.

While I do have some reservations about this plan (mostly that my desktop Linux skills will fade, and that I’ll loose respect in the Linux community), I don’t think it should be too much of an issue. At the end of the day, it is most important that I am able to spend my limited time learning and building in the areas I want to focus on, which is back-end, rather than front-end Linux. I want to love all technology, and be able to help people find the best solution to their problems, no matter what they use. Maybe one day the OS flame wars will die down and we can all just get along and build cool shit together… Till then, wish me luck!

Resolving Issue

2019-12-15T00:00:00Z

Lowes, Durham, NC

While working on a post a couple of weeks ago, I noticed that for some unknown reason, my website wasn’t rendering correctly. After some back-tracking, I remembered that I had updated the container I work in to a Fedora 31 base image, which has a newer version of hugo. So, I filed the problem as issue #26, and finished my post in a Fedora 30 container for the time being. Here is a quick explanation of how I eventually came back and resolved issue #26.

The Problem

What the website homepage is *supposed* to look like.

My website has two types of content: posts, and pages. Pages are the ‘normal’ content of the website, like the About and Homelab pages. Posts on the other hand, are the dated ‘blog’ posts I write (like the one you are currently reading). When everything is working, my hugo template takes the newest x number of post files from the posts directory, and lists them on the homepage with a small summary. (See image above)

The broken homepage.

However, after the hugo version in my website container updated from v0.56.6 to v0.58.3, the post list on the homepage broke. Specifically, instead of listing the recent posts, the homepage just listed a single “post” named… “Posts” (see image above).

This indicated that something changed between those two versions that made my template out of date. However, I wasn’t sure if the problem was in the website’s page content, or if it was from an issue in the template I used.

Finding a Fix

To find out, I grabbed some other hugo templates, and temporarily switched my config to render the website with them. This was actually a bit more complicated than I thought it would be. Hugo templates rely on a all sorts of configuration variables being defined in order to work properly. Additionally, not every template had a recent post section like I did, so they didn’t help to isolate the issue at all. Lastly, if the template was older… it had the same rendering issue mine did. Eventually, I found a newer, but rather bare-bones template that had a similar home page layout to my own. When I rendered my site with it… it had a working recent posts list on the home page!

This proved 1) the issue was somewhere in my template code and 2) a fix was possible.

Coding a Solution

Studying the layouts/index.html file in each template, I noticed the working template used a different method to gather and display the recent posts. My template used a basic in-line piece of code that grabbed a range of the latest posts to be rendered, right then and there.

<div class="container">
      <div class="row">
        <div class="col-lg-8 col-lg-offset-2 col-md-10 col-md-offset-1">
          {{ $posts := or .Site.Params.PostSummariesFrontPage 4 }}
          {{ range first $posts (where .Data.Pages "Type" "post") }}
              {{ .Render "summary"}}
          {{ end }}

By comparison, the the newer template used an improved setup. It first defined variables at the top of the layout page to calculate all the recent post lists, and then simply referred to that when rendering the summaries. So, I read up a bit on the hugo documentation to better learn how variables and some of the basic functions worked.

Eventually I was able to come up with my own solution, defining the following variables at the top of the page:

{{ $mainSections := .Site.Params.mainSections | default (slice "post") }}
{{ $section := where .Site.RegularPages "Section" "in" $mainSections }}
{{ $section_count := len $section }}

I then referred to the variables further down where I render the recent post summaries:

<div class="container">
  <div class="row">
    <div class="col-lg-8 col-lg-offset-2 col-md-10 col-md-offset-1">
      {{ range (first $n_posts $section) }}
          {{ .Render "summary"}}
      {{ end }}

and… it worked! Done!

Conclusion

For as quick and easy as this post implies, the fix was actually a giant pain. It took a long time to figure out where the problem was occurring, but in the end I’m happy with the solution. In fact, I used a similar approach afterwards to fix issue #28! Assuming hugo doesn’t suddenly break me again, see you in the next post!

Setup Mosh

2019-12-04T00:00:00Z

North Buchanan Boulevard, Durham, NC

Since builing my desktop, whenever I work on another machine, I usually end up ssh’ing back to it to work remotely. It has my files, more power, and much of my work flow is done from a terminal window anyway, so why not? The only issue I have with ssh is that if I have a spotty internet connection, or if I sleep/suspend my laptop while moving around, the ssh session will occasionally time out. Tmux and tmuxinator make this less of an issue, since I can re-attach my session, but I still wish my remote sessions were a bit more seamless. They can be… using mosh.

Mosh

Using `mosh` to connect to one of my servers.

Mosh is a more robust replacement for interactive ssh terminals. It automatically roams and continues to work even as the computer switches networks or is put to sleep. It also responds to typing, even on a bad connection, which cuts down on lag. Lastly, it’s free and open source software, licenced under the GPLv3.

Mosh Install

Mosh should be in most Linux repos, and is also available on BSD, Mac, Windows, and basically everything else. For more information on how to install it on your platform, head over to the getting mosh page. For me, it was a simple dnf/yum install to get it on both my laptop and server:

sudo dnf install mosh

*Note: My centos server required me to first enable the epel repos to get access to mosh. Fedora might also, but I already had it enabled on my laptop.

sudo yum install -y epel-release
sudo yum install -y mosh

Open Firewall Ports

After installing mosh… it might not immediately work. If so, it is likely due to not having the required ports open. Mosh uses UDP ports 60000-61000 for it’s connections. Enable these ports and optionally restart the firewall, before trying mosh again.

Note: If you want to connect from outside the network, remember to also forward these ports on the network.

Connect

We should now be able to connect to the server using mosh. Typical connections look very similar to ssh:

mosh ryan@centos-server

If a specific mosh UDP port needs to be specified (for port-forwarding, for example) use the -p flag:

mosh -p 1234 ryan@centos-server

SSH Options

Mosh uses ssh for the initial connection. Occasionally, particular ssh options might be required in order for mosh to initialize a connection. For example, I typically ssh home on a particular port, so that my router knows which VM to transfer me to. Options like this can be passed to mosh using the --ssh flag:

mosh --ssh="ssh -p 1234" ryan@centos-network

Conclusion

That’s really it. There is a bunch of cool technical stuff going on under the hood of mosh, but on the surface… it is simply useful. For an even better experience with mosh, be sure to check out tmux. Enjoy!

All Things Open 2019

2019-11-09T00:00:00Z

Downtown Raleigh, NC

Two weeks ago, I attended All Things Open (2019) for the second year in a row. ATO is an annual conference that explores and celebrates… well, all things open. Open source, open tech, and open government are all main topics at the conference. Best of all, it’s right here in the triangle (Raleigh NC). Here are some of my overall take-aways from this year.

While the talks are always full of information, the main benefit I receive from attending the conference is leaving with a list of technologies and ideas I want to test out. So, rather than go into boring detail about each talk I went to, I will elaborate on some common topics/themes I experienced and what I am excited to dig into as a result.

(Sorry there aren’t any visuals in this post. I’m still a terrible event participant and didn’t take any pictures)

Tech

Let’s get tech out of the way, right up front. Here are some ideas I left the various technical talks with:

I am going to keep using ZFS for my server

It happened again. Over the last few months I’ve entertained the idea of moving off of ZFS on my home server for something like mergerfs + snapraid. Then I went to a Jim Salter ZFS talk, and just like last time… I want to ZFS all the things. So, it looks like I’ll be sticking with it on the server. Also… it might be time for a ZFS tune-up to make sure I have the appropriate options configured correctly…

I want to start building Flatpaks

As a Fedora Silverblue user, I rely heavily on flatpaks for most of my desktop applications these days. While I have only built the “hello world” demo when playing with flatpak in the past, this is something I think I want to dig deeper into. Knowing how to build and tweak flatpaks would be extremely useful. It would enable me to package up software I need to use, but then also contribute my efforts back to the community.

`sudo` is awesome.

I already knew this, but I always forget how much sudo can do. Like ssh, the possibilities of being well informed about the advanced options of sudo are endless. I think I’m going to dig deeper into some of the settings, and see what I can find.

I need to automate my cloud provisioning

This has been on my list for awhile now, but after seeing so many demonstrations of people using terraform and chef to spin up their cloud environments… I need to start adding some provisioning roles to my ansible playbooks. I think using the Digital Ocean module to spin up/tear down temporary project droplets might be a good start…

Mentoring/Teaching

Large Scale Mentoring

One of my favorite talks of the conference was co-done by two kubernetes contributers (from Red Hat and Google). They explained how the kubernetes project tackles the big problem of mentoring… specifically how it can even be done in a several-thousand contributer project that also has hundreds of fly-by contributors.

What I enjoyed most about their mentoring system was how it was organized. First, they went through and defined all the leadership roles, documenting what the role was, and how it is done. Next, they made a mentoring chart for all the defined roles, listing the current person for each role in one column, and the person(s) next-in-line that are training for that role in the next column. I think this is a great system for communities, and I might see if I can take some lessons from this structure and help apply it to some volunteer organizations I work with.

Gamifing Education

Another interesting education talk I went to was about gamifying education. In the past, whenever I encountered topics trying to “gamify” education, they usually involved how to add gaming aspects to current methods of teaching (points for answering questions, leveling up in difficulty, etc). This talk however, focused on actually making games (ex: board games) that just happen to teach children as they play. It went in depth about what to keep in mind while designing such a game, and gave great examples about challenges I wouldn’t normally think of. For instance, the presenter suggested designing the games so that any random items could be used as the game pieces (rocks, beans, spoons). This lowers the barrier of entry so that children in all types of living situations are able to play, and I think it is great advice.

Best of all, the talk concluded with the suggestion to host all game sources/documentation in a public repo on a website like Github. This not only makes the game accessible to people all over the world, but allows the community to improve and expand on it. If there is something confusing in the documentation, another person can clarify and fix it. Open sourcing the game also enables the community to modify and invent new variations, improving it over time. The talk showcased how we can take the best aspects of open source, and use them to hopefully make learning a bit more fun and accessible.

Those who can Do, Should Also Teach

As someone that struggled to focus in traditional lecture-style classes, but thrived in hands-on lab courses, I am always looking for new methods and ways to help others learn. One talk I observed, encouraged the audience to start teaching others as they continue grow their skills. The talk provided many useful tips and approaches for teaching both technical and non-technical “students”.

A big motivation behind this website is that it allows me to share my experiences so others can (hopefully) learn from them. As I sat listening to the talk, I started to brain-storm what I could do to better extend my ability to teach. I first wondered what changes I could make to the website, but then started thinking about even new avenues I could pursue. I haven’t finalized any ideas yet, but it is something I will keep in mind as I start planning my goals for the new year.

Communities/Diversity and Inclusion

I first listed “Communities” and “Diversity and Inclusion” as too separate topics. However, as I started to sort talks into the categories, I realized these two overlapped all the time. So I merged them.

Honestly, this topic was so prevalent at ATO, that it’s hard to summarize even a little bit. From learning about the curb cutting effect, to diversifying recruiters and continually switching up who leads meetings, there were all sorts of facts and tips about how to build more diverse teams and even communities.

This topic sparked my curiosity, and I ended up walking away with two new books added to my reading list: The Culture Map by Erin Meyer, and Jono Bacon’s upcoming book, People Powered.

Burn-out/Team Dynamics

Last but not least, there were talks about another related topic pair: team dynamics and burn-out. Every talk I went to that focused on either these topics was really well delivered, which was a relief as both topics are very important in the tech industry. To summarize, I’ll list 3 items which I already subscribe to, but came up again and again in the presentations:

Multi-tasking isn’t real
Stop blaming others
A balanced life outside of work is required to be productive at work (eat healthy, exercise, get more sleep, have a hobby)

Conclusion

With my second year at All Things Open finished… would I recommend it? Absolutely. No matter why someone is interested in open source (the tech, the community aspects, sharing with others), there will be something there for them. See you next year!

My Experience with the Large Display Paradox

2019-10-21T00:00:00Z

Crimson View Rooftop Bar, Washington DC

About a year ago, I switched to the LG ud4379b, a 42.5" 4k IPS monitor. My initial motivation for upgrading was to convert to an IPS panel. However, my selection of monitor was an attempt to simplify my hardware, by pairing down my dual 1080p setup. While I wanted a single monitor, I didn’t want to drop my total resolution because everyone knows that multiple monitors are required to get any real work done, especially programming. Although, after a year of using my massive display, I started to tire from the large display paradox.

Background

Before I dive into what exactly the large display paradox is, lets quickly refresh on what I have said in the past. In my post about choosing the 43" monitor, I went into depth about my history of using dual monitors, and why I wanted to switch to a single monitor:

While I like having the screen real estate of two monitors, I was never able to optimally use them because of the bezels between the monitors. While working, I would naturally shift over and use one monitor as a primary display, and the other as a secondary display.

I wanted to be able to view my monitor straight on, without any bezels getting in the way. I also knew that a single 1080p monitor was just a bit too cramped.

In that same paragraph, I provided an additional reason for the switch. If I had been paying closer attention, I might have noticed it foreshadowing issues I would face with a large display:

This was okay for some tasks, but it usually meant that I just used the primary monitor, and half of the secondary one, because it was too hard to see the far end of the screen.

I was excited to use a single monitor as a 4x 21-inch 1080p bezel-less grid.

I was aware that a large screen may not be optimal, but the alure of having four monitors in one was too strong. This is emphasized in the section where I compare the LG 43" with a Dell ultrawide:

Being so large, I could still have auxiliary windows opened on either side. One advantage the ud4379 had over the ultra-wide however, was that being a massive 16:9 screen, it could easily be divided into 4 1080p screens.

Despite the nagging voice in the back of my head saying the ultrawide might be more reasonable, or reminding me that a single 27" 1440p monitor was also an option… I ignored it and picked up the 43" LG (which nonetheless, is a great monitor!).

The Large Display Paradox

So then. What is the “Large Display Paradox”? As far as I can find, the term was coined by Jeff Atwood in a blog post titled with the same name. In the post, Atwood refers to another post in which Dan Rutter states the following while reviewing his new Dell UltraSharp 3007WFP-HC monitor:

Users of 30-inch monitors face the terrible, terrible problem of how to effectively use all of that space. You don’t often want to maximise a folder or document window on a screen this big; either you’ll end up with a lot of white space and important program buttons separated by a vast expanse of nothing, or you’ll get lines of text 300 or more characters long, which are difficult to read.

Atwood summarizes that this is a perfect example of what he deems to be the Large Display Paradox:

That’s the large display paradox. Having all that space can make you less productive due to all the window manipulation excise you have to deal with to make effective use of it.

He further explains how once a display grows beyond the point of being able to efficiently maximize windows without too much whitespace (above 1600x1200, by his estimates), it becomes less productive due to the user having to continuously manage the windows.

While these posts are on the older side (2007!), I think most people are more likely to experience this issue today, due to cheaper large monitor options.

My Experience

With the Large Display Paradox finally defined… did I experience it using my 43" monitor? Yes.

At first, it was great. I loved throwing everything I wanted up on a single screen, without any applications overlapping.

In particular, during the first few months using the monitor, I still worked remotely at my previous job and needed to use a Windows machine. I appreciated that I could RDP to my work computer using an ultrawide resolution (3440x1440), but still have extra pixels to keep my notes and chat windows below it.

Remoting into my Windows work computer using an ultrawide resolution, and still having space for my notes and chat applications below.

Then, I switched jobs and used Linux 100% for home and work. Instead of having to RDP into a second machine… I could just ssh using a small terminal window. I no longer had to view multiple machine desktops to get work done.

Issues

At first, I didn’t notice any issues, especially when using it on my personal computer which I mostly used for leisure anyway. It didn’t matter much if I had a video and chat window next to a personal project I was working on.

Regardless, over time I started to notice that when I tried to actually focus on a task, the “usefulness” of the large screen really started to fall apart. Below are a few issues I started to run up against when trying to be productive with what was seemingly one of “the best productivity monitors on the market”.

Always wanting to fill the space.

The tricky thing about having all that space… is you want to fill it, even if you don’t need to for the task at hand. So for example, if I wanted to work on writing a blog post, all I need is a text editor. In addition, I like to have a preview of the post open in a browser so I can see how it renders. I sometimes like to also have a photo editor and picture folders open, but I find it best if I can have them in a separate virtual workspace. With the large display, I would open up all those windows on the same screen. In reality, having anything beyond my text editor and post preview in front of me, really starts to detract from my ability to focus on writing the post.

With so much space, I felt obligated to fill it with windows unrelated to the task at hand.

People like filling space. How often do you find that a surface in the house seems to always collect random crap? Large monitors are the same. Even when I knew I only needed a one small terminal in from of me… the vast space surrounding it made me feel uneasy. So, I would fill the space by either extending the editor window to a comical size, or litter the surround space with random applications, unrelated to the work I was trying to focus on.

I didn’t use the entire space, just focused on what was in my field of view in front of me.

Even though I felt obligated to fill all the space, I still didn’t use all of it. When I had two monitors, I would center myself on one to prevent looking at bezels, and then usually only utilize half of the secondary monitor, because the far end of it was out of my sight. This didn’t change with a single large monitor. I usually found my self using an “appropriately sized” window, smack in the middle of the screen. If I needed to switch to working in a different application, I would move that application to the center and focus on it. The outer spaces of the monitor were never fully used.

I couldn’t fullscreen (or even split-screen) my windows

Fullscreen, and even split screen windows were unwieldy. Most of the time I just wanted to work from a normal sized window.

Speaking of focusing on a single application… fullscreen… it isn’t really an option on such a big monitor. While a maximized window might look awesome at first, it wasn’t very practical for day-to-day use. This issue was compounded by the “shadow” that prevented me from seeing the first few characters near the edge of the screen. The size of the screen combined with how far I sat from it, meant that all but the center was out of my field of view. As such, I didn’t maximize my windows, and usually just wanted a normal-sized one in the center.

Bad Keyboard Window Management Experience

Without being able to view windows in full or split screen configurations, all of the quick window-snapping key bindings I’ve become accustomed to were useless. I was forced to switch over to my mouse more than I would like to. While some might point to using window management software, or a tiling window manager, I don’t think this would solve my issues. You might be able to get away with using a window manager like i3-gaps (or sway) on a 1440p ultrawide monitor, but a full unscaled 4k space is just too much. Unless you always have 5+ tiled windows open at a time, they’re just too big.

Conclusion

I think the large display paradox could be a real modern tech issue. Well, more accurately, a major first world problem. Given the increased affordability of large displays, what might seem like an great setup, might not actually be optimal for one’s workload. At it’s core, technology is a tool. When choosing tech hardware, we should always focus on what the best tool for the job is, and not just what is the best tool.

Switching to Bitwarden

2019-09-13T00:00:00Z

TROSA Thrift Store and Donation Center, Durham NC

After years of using pass as my password manager, I am mixing things up. Over vacation, I started importing all of my passwords into Bitwarden. With all my passwords finally transferred, I have now switched to using Bitwarden full-time. Here are my thoughts so far.

Why Switch?

Before I start, I want to point out that I had already started my switch before the large Bitwarden segment on the recent episode of Linux Unplugged. That was just a re-assuring coincidence :) .

So, why did I decide to switch?

Easier Setup

Bitwarden on Flathub.

The main reason is that when configuring a new system, Bitwarden is simply easier to setup. All I need to do is install a Bitwarden client and login. In fact, using the hosted service I don’t even need to do that. I can just login using the web vault. Installing is made even easier by the fact that Bitwarden is packaged as a Flatpak on flathub. So on my Fedora Silverblue computers, it works out of the box and already fits in with the design philosophy behind Silverblue (running all user apps in containers). Perfect.

Mobile Support

Over the past few months I have been switching up the tools I use, at least the ones I use outside of work. The big driving force behind many of these switches is better mobile support. While I still prefer to keep as much data off of my phone as possible, there are some things I need to have on the go. My password manager is unfortunately one of them.

Additionally, as I start to test out different workloads on alternative devices like my ipad, having a mobil-friendly work flow is a godsend. In fact, trying to setup pass on my iPad is what eventually frustrated me enough to give Bitwarden a chance.

Setup on both my phone and ipad was dead-simple. Install the app, and login. Done.

Wayland Friendly

One of the best features of my pass system was how well it was integrated with my desktop environment. However, I want to switch to wayland now more than ever, but my pass desktop integration is the biggest blocker preventing me from using it on my machines. I access pass using launcher applications like dmenu and rofi… but they do not function correctly in wayland. This forced me to use pass from a terminal window while in a wayland session. Bitwarden, at least the limited GUI interactions I’ve used thus far, appears to work just fine in wayland.

Easier to share/expand

Lastly, if my wife eventually wants to switch to Bitwarden, I think we can share our joint passwords with each other, even being on just the free personal plan. If not, we can upgrade to the “family plan” which is currently $1/mo and allows unlimited collections across 5 accounts and optional self hosting. For shared family password management… that’s a great deal.

What I am still learning

Normally, this is where I list the few things that I “don’t like”. However, this still is all new and I think the majority of my current issues are solvable, but I just haven’t figured it out yet. So it’s a “What I’m still learning” section this time (•‿•) .

Better keyboard/DE integration

The main feature I still seem to be missing is better keyboard support/integration with the desktop environment. The GUI app is nice, and Bitwarden has much better support for browser extensions and tools. Yet, I haven’t found a good way to open up Bitwarden (give my master password), search for an item, and copy the password… all from the keyboard.

By comparison, pass let me integrate with launchers like dmenu and rofi to efficiently grab my passwords which was awesome. As mentioned a few sections above, passmenu unfortunately falls apart for me these days now that I am mostly on wayland… but I don’t need a full launcher. Honestly, all I need is the ability to copy a password (and maybe other fields) using my keyboard once I have selected an item. I can already search using ctrl-f + tab, and then arrow my way down the list, but once there I cannot copy unless I continue to tab through all the options. This is not efficient and defeats the purpose of remaining on the keyboard in the first place.

This is a highly requested feature, and I’m sure someone has hacked some solution together… I just have yet to find it… or implement it myself.

Conclusion

That is it. TLDR; I’ve switched to Bitwarden and have been using it as my main password manager for almost a month and I love it. It works on all my devices and doesn’t give me too many headaches. My only real complaint is that I wish it had better keyboard support, but I’m sure I’ll be able to figure out a work-around eventually.

Setup a Runner VM for Gitlab

2019-09-04T00:00:00Z

Sugar Creek Restaurant, Nags Head NC

I play around with CI/CD pipelines quite a bit, both at home and at work. I have mostly used Jenkins, but I wanted to see how Gitlab’s CI/CD tooling has progressed over the last year. So, I decided to try to use Gitlab to manage the automated build and deployments of a personal project I’ve been working on. The first step of the process was to setup a runner my Gitlab instance could use for the builds.

Setup a Machine/VM

Installing a new Fedora 30 VM in Virt-Manager for my runner.

This will be a BYOG post (bring your own Gitlab). I already “had one laying around”, so I won’t cover setting that up.

Your runner needs may differ, but in this post I am installing runner on a Fedora 30 VM. I will also be using both buildah and podman for this project.

Some things to note/consider during VM setup:

Install packages required for pipeline tasks (ex: podman and buildah)
If sudo is required, manage the gitlab-runner user/group using visudo
If using docker runners, docker-machine needs to be installed

Install runner

First, install the gitlab-runner package. This can be done using the instructions found here. However, I encountered issues installing it on my Fedora VMs, as this install method isn’t supported for 30 yet. (Check out this issue for more info).

Add GitLab’s Repo

curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.rpm.sh | sudo bash

Install gitlab runner

sudo dnf install gitlab-runner

(Alternative) Copr install

For now, I have been using the copr install posted in the comments of that issue (linked above). I recommend checking if the issue is resolved first, as it might change from the time of writing this post. To install:

First enable the copr repo:

sudo dnf copr enable snecker/gitlab-runner -y

Next, install:

sudo dnf install gitlab-runner -y

Register the Runner

Once installed, register the runner. Instructions on how to register a runner can be found here.

sudo gitlab-runner register

Enter the coordinator URL (ex: https://gitlab.com)

Next, a gitlab-ci token must be shared with the runner.

Gitlab Runners Settings Page.

To obtain a gitlab-ci token, got to Admin Area -> Overview -> Runners. On the right, there should be a token to use during setup.

When the runner registrations asks for the token, use the “registration token” listed in the “Set up a shared Runner manually” section.

Next, provide a short description, and add a tag or two (when prompted).

Lastly, enter the executor (the system on the runner that executes commands). For now, I’ve been using "shell" for my needs, as these VMs are fully dedicated to be used as the runners for a single project.

Congrats, the runner should be registered! Now to set it up…

Link to CI/CD Builds

It is time to link up the runner to a CI/CD job. This can be done with tagging, but I currently just have one pipeline using my runners, so haven’t used the tags as much. Edit the runner by clicking its edit icon.

Gitlab Runner Edit Page

In the runner edit menu, ensure that the “Active” checkbox is checked. I’ve also checked the “Run untagged jobs” box for this runner, which will allow it to pick up any job that does not have a tag. If the runner is to be assigned to a specific project, that can be enabled/assigned below in the “Restrict projects for this Runner” section.

Test Run

To test out the runner, start a new build in a project! (Note, if there are several runners already setup, 1. why are you reading this, and 2. it might be a good idea to pause the others to ensure the new one will run with the test).

I won’t detail how to write a gitlab-ci.yml now, but for my test I made an empty demo repo with the following pipeline:

before_script:
  - whoami
  - pwd
  - sudo dnf update -y

build-base:
  stage: build
  script:
    - echo "Hello world!"

After committing it, a build kicked off with the new runner and finished successfully!

Notice that the job indeed ran on post-runner, the runner I setup specifically for this post

Gitlab Demo Job Run Results

If the job is more complicated, more runs might have to be manually started after tweaking the runner settings again. Pipelines can be started by going to the project’s CI/CD->Pipelines page via the side menu, and hitting the Run Pipeline button.

Conclusion

That’s it. We should now have a connected runner! So far, the runners have been working (mostly) fine. When they do break, it is usually because I’ve let the disk fill up or allowed some other system-related negligence to build up ¯\_(ツ)_/¯. I might add some ‘runner maintenance’ steps to my pipeline… but some other time. Enjoy!

Back On org-mode For Work

2019-08-07T00:00:00Z

Eno State Park, Durham NC

Last month, I wrote about my switch to Joplin for both my personal and work notes. While I enjoyed many features in Joplin, I also had a few concerns about using the system long-term for all my notes. As of last week, I am still using Joplin for all of my personal notes, but have switched back to org-mode for my notes at work. Why?

Use Cases

To better understand why Joplin works well for me at home but can’t effectively replace org-mode at work, lets first list some different features between Joplin and emacs org-mode.

Joplin Features

“Notes”-Style Focused

Joplin uses a traditional 'note' layout, with blank text boxes organized into notebooks.

Joplin organizes notes in a traditional “notes” style. Each note starts as a big, blank text-box that the user fills in. These notes are further organized into “notebooks”, which are simply a list of one or more notes. The notes are formatted with markdown, but the application doesn’t seem to support features like collapsible sections.

Simple Markdown Format – Sharable

The markdown editor in Joplin may not support collapsing sections, but it does a great job at supporting other features, like simple code highlighting and web links. Additionally, being formated in markdown, the note contents can be easily exported, transferred, or even shared with other applications and/or people. This is handy when grabbing snippets from the web, which may already be in a markdown friendly format.

Shared across Computer and Mobile Devices

The Notebook and Notes views on the Android client.

Lastly, the Joplin application is available on all platforms (including mobile), and manages the system to sync the notes across all devices. This makes it a breeze to write a note at a computer, and then have it immediately available on a phone afterwards.

Org-mode use cases

Logging-Style Notes

By comparison, Emacs’ org-mode can be very task oriented, allowing for a more ‘log-style’ note-taking system. It encourages outline organization of notes, organized under “tasks” which can be collapsed in a hierarchy. The tasks can be marked “TODO” or “DONE” using the built in TODO system, or tagged for organization. While org-mode can be used when writing “normal” notes, it really shines when working as a task/note system.

Quick Notes, Key-bindings

Another feature in org mode is the note drawer. By pressing a key combination (as one tends to do in Emacs), a new buffer will open up to take a note in. The contents of the note may be text, or even a code snippet (in the emacs mode of that language). When done, I can simply press C-c C-c and the buffer will close, inserting the note with a time-stamped entry as part of the logbook for the task I opened the note in.

Click for a demo animation logging a quick note

This quick logging feature, along with the power of all the key-binding customization, makes note logging fast, efficient, and enjoyable.

TODO/SCRUM Board tasks

I wrote a package that lets me view org tasks in a scrum board.

As mentioned, org-mode has a powerful todo system. Users can configure how task items should be grouped beyond just a basic TODO/DONE. Using this flexability (and some lisp knowledge), I wrote an scrum board generator for org-mode. This package takes all of my TODO items, and organizes them into scrum task board. This feature helps organize my daily and weekly task lists.

Exporting/Archiving Weekly Logs

The big “feature” org mode has that I find lacking in Joplin, is the ability to better organize long-term sets of notes. With a logging style of note-taking, notes are often organized by a specific range in time (day, week, month). These systems are naturally structured hierarchically and Joplin’s notebook/note organization only allows for systems to be 2 levels deep. However, org files are stored as text files on a traditional file system.

Lastly, org files can be linked (and exported as html files), so any note can still be quickly navigated to, even if the archive spans hundreds of note files over several years.

Home vs Work Notes

Now with some of each note system’s features broken down, how do they fit with my preferences across different note taking use-cases? (mostly personal vs. work)

Home

My personal notes tend to be organized as more traditional notes. I make large groups, such as ‘Home’, ‘Tech’, ‘Travel’, and stash notes into each. In a note, I tend to organize information on a topic and store it for later. For example, I may have a notebook for online classes, and then a note for each class.

For another example, I have a notebook called ‘Travel’ which contains a note for each trip I plan. In the note, I store information that I will need, such as flight numbers, or hotel addresses. Below important information, I jot down ideas while planning and researching before leaving. This may include places to to eat, or activities I want to do (with links to the websites).

Joplin’s mobile platform support is particularly useful here, as I am able to write up notes on my desktop, and then read them from my phone on the go.

Work

By contrast, my work notes are a mix of task-list and logbook for each week. I plan each week by moving TODO tasks from one day to the next, and changing the status as I work on each one. As I work, I log notes under each task about my progress, as well as any issues/solutions I experience on the way. While the notes are mostly rambling, logging thoughts as I work helps me to quickly identify issues, and solve problems faster.

The index page of my exported weekly work notes.

Most importantly, due to organizing notes by week rather than topic, it is best if I can archive the notes in more of a tree layout. Org notes are plain text files, so I can save them on my hard drive any way I want. For example, I have my work notes organized as ./work/archive/{YEAR}/{MONTH}{DAY}/week-of-{date}.org. Each week, I also export the finished notes to linked html files, which is very useful for going back and looking at my previous notes.

Conclusion

That’s it. I’ve switched back to emacs org-notes for my work tasks, but remain on Joplin for my personal and home notes. So far… I’m loving it. I think that I switched to Joplin because org-mode wasn’t ideal for my personal notes. However, Joplin was lacking for my work notes. That’s okay. I learned that my home vs. work notes are two completely different use cases, and as a result should use different tools.

So, moving forward, I plan to keep using Joplin for my personal notes, and emacs org-mode for my work tasks. Perfect.

New LG-32QK500-w Monitor

2019-07-24T00:00:00Z

At the beginning of summer, I was entertaining the idea of improving my wife’s computer setup by adding a new external monitor. She agreed. As a result, we have purchased, recieved, and even mounted, the LG 32QK500w monitor. Here are my thoughts.

Reasoning

Recently, my wife has had to spend more time prepping at night for her next workday, which she does on her laptop. This made it a good time to finally setup her office space in the guest bedroom. She has an old 21.5" 1080p monitor from college, but we decided it might not be the best fit for the space.

We thought that it would be nice to have a larger screen in that room, guests could watch video content. Additionally, my wife and I have a habit of falling asleep on the couch watching TV most weekend nights… but we want to keep our bedroom TV free. Having a spot where we can lay down on a bed and prep for the inevitable while watching a movie, is probably a good idea (and our backs agree).

I started looking for slightly larger monitors and quickly noticed something… the 2560x1440 32" monitors were cheaper than their 27" counterparts. This made sense, as the dpi is lower. However, after doing the math, 2560x1440 pixels @ 32" is the same dpi of as a 24" 1920x1080 monitor (109 dots per inch). While this isn’t the sharpest configuration, it is very common and reasonable for productivity setups.

2560x1440 pixels at 32" has a dpi of 91.

1920x1080 pixels at 24" also has a dpi of 91.

In fact, my wife doesn’t have the best vision (especially at night when she tends to work), and tends to actually prefer the larger scale of 109 dpi. We even purposefully picked a 1600x900 resolution screen over a FHD one when buying her Thinkpad for the same reason.

Lastly, while it is scaled up a bit on a physically larger display, these monitors still provide a 2560x1440 resolution. Recently, I have decided that this particular resolution, may be the best for productivity , at least in the type of workflows I use. I even figured out how to configure a 2560x1440 “sub-monitor” on my 43" QHD monitor when I need to focus. Having an actual 2560x1440 monitor in the house would be wonderful for both my wife’s and my own productivity.

Selection

The new monitor in it's box.

So I wanted a 32", 2560x1440 IPS monitor… at a decent price. After browsing around amazon, I came across the LG 32QK500W. It was listed at a very reasonable price (on sale), had a few bonus features (75 Hz refresh + freesync options), and was an LG IPS panel, which I felt comfortable purchasing considering how much I enjoyed my LG monitor’s panel.

In summary, the LG 32QK500-W has the following specs/features:

32 inch, 2560 x 1440 pixel resolution
IPS Panel
mDP and 2 HDMI imports
VESA mount compatible
Decently thin bezels
60 and 75 Hertz refresh rates
Freesync
On sale for less than $250 (USD)

We also purchased a cheap Amazon basics mount to wall mount it, but in hindsight it might of been worth paying a little extra for a better one.

What I Like

Resolution

The resolution is perfect for productivity work.

The “2k” resolution is perfect. It gives just enough space to get work done, but not so much space that it is distracting. I can perfectly fit two full windows next to each other on the screen. At the same time, it is still a small enough resolution that a single focused full screen window doesn’t look odd.

Colors

The colors look great. They are vivid, and the blacks are dark. I think I may even enjoy them better than on my own monitor…

75 Hz refresh rate (and Freesync!)

Lastly, the 75 Hz refresh rate and Freesync was a nice little bonus. While the only computer we have in the house that can really take advantage of these features is my desktop (which is in another room)… it’s nice to know we have the option if I want to test it out.

What I Don’t Like

No speakers… and audio jack is noisy

Normally, I could care less if a monitor has speakers or not. They usually sound terrible, and I end up plugging external headphones/speakers to the computer directly anyway. However, as we intend to also use this monitor as a “TV” via an Amazon Firestick, it would be nice to have even basic speakers to pump audio through by default. It does have a 3.5mm headphone jack, but there seemed to be a bit of noise when I tried it.

While dpi is perfect for it’s use, it isn’t the best

As mentioned earlier, the dpi isn’t the best, and the image isn’t quite as “crisp” as you might find on a high resolution display. Also, at such a large screen size, it is almost required to sit back from the screen a little bit further than normal. However, as I already described in the sections above… for our use case, this was actually preferred.

Conclusion

The monitor all set up mounted above my wife's new desk. (Next step... hide the wires!)

There isn’t much left to say. At this price point, and for what we wanted this monitor for, I think it was a perfect match. While it might not be the absolute best monitor out there, it is a great value. If you’re looking for a large 1440p monitor to get some work done on, that could also be a TV replacement at a good price (I’m looking at you college students), I highly recommend it.

Switched to Joplin Notes

2019-06-30T00:00:00Z

Pardee Hall - Lafayette College, Easton PA

As a massive fan of emacs’s org-mode, it should be no surprise that I’ve been using it for my personal and work planning/notes over the last few years. However, as my daily emacs usage has slowly dropped, and the support for .org files outside of emacs remains low (other than Github README files), I started to look for a more standard system. The last few months, I’ve been using Joplin. Here are my thoughts.

Previous System - Org-mode

Emacs with a note opened in org-mode.

Previously, I had been taking all of my notes using org-mode in emacs. Each week, I made a new .org file for my work notes, and another for my personal planning/notes. I would then list my todo items, usually grouped by day, category, or both. I would also tag each item, for better organization.

As I worked on each task, I could quickly add time-stamped notes using the built-in logbook drawer shortcut. In these notes, I would ramble, or paste a code snippet to save for later. As work on each item progressed, I could set the TODO status to Working On Finished, Removed, or any other state I had pre-defined. Eventually, I wrote an emacs-lisp script/package that took the todo items in my org file, and dumped them into a SCRUM board at the top of the file.

org-notes allowed me to link and export my notes to HTML.

Lastly, org files can be exported to all sorts of file types (md, html, pdf, latex). By combining the export feature with internal linking, I was able to create an organized system for my archived notes. I created an index.org file which linked to each week’s notes. Every week, I would archive the html export of a note, and add it to the index list. This meant I could browse my old notes as a website.

What I liked

org-mode is great.
I could take all of my notes in a simple markdown language (org)
Good code-block support
Note linking
Exported to html (or txt, md, pdf, or anything really)
My previous notes were all exported to an archive website

Issues

I had to use emacs for any edits
Not supported on mobile devices
As I used more .md syntax for other applications, I found myself fumbling between org and md syntax often.
Syncing issues (mostly Nextcloud issues. Seafile helped, but it still occurred)

All in all, I loved taking notes in org-mode. However, I wanted something that could be used on all my devices, and was more easily translatable outside of “emacsOS”.

The Switch - Joplin

Joplin GUI window (in 'split mode').

About Joplin

Joplin is an open source note-taking and to-do application. It synchronizes and organizes markdown notes into notebooks, across many platforms (Android, iOS, macOS, Linux, and Windows). It even has both a GUI and terminal client. So, I decided to give it a try.

My Setup

When configuring joplin, a sync system has to be defined (well, should. Notes can also be saved locally). There are a few supported options including using Nextcloud. I had previously synced my org notes using Nextcloud, but eventually moved away from it due to repeated file conflict issues. So I wanted to try something different.

Joplin GUI window (in 'split mode').

While researching Joplin, I learned that I could sync notes to the WebDav server on the fastmail account I have for email. I wasn’t using any of the account’s file storage, so I figured I this would be a good way to get use out of a service I’m already paying for. It works great!

What I like

Syncs on all my devices

What I have enjoyed most about this setup is that my notes sync to all of my devices. Most importantly, I can read the notes on every device. With previous systems, the files would sync everywhere, but I still couldn’t always easily read them.

CLI and GUI versions

Similarly, I like that Joplin has both a GUI and CLI client. I mostly use the GUI, but enjoy having the CLI as an option because it means I can always ssh into a machine and take notes. Additionally, the cli client allows me to take/edit notes using my own terminal editors such as emacs or vim. This is quite convenient, and I should probably start using Joplin this way more often.

Markdown Note Exports

Export options in Joplin GUI.

I like that notes are formatted in markdown. This provides some consistency when writing, but ensures that my notes are in a format that is standardized. Additionally, joplin allows easy exporting of the notes to other formats such as json or as pdfs.

Easy enough to use

Lastly, Joplin has been easy to use. After setup, I just have to open the app, and start typing away (maybe with a sync or two). While my last setup required the user to be somewhat of an emacs/org guru… this doesn’t.

Issues I’m still figuring out/Anticipate

Joplin CLI window.

Setup is a pain

While configuring sync can be a pain, it’s not my biggest gripe with the Joplin setup process. It’s adding a new system. By default, if I configure a new device with my sync solution (my laptop or phone for example) and then sync, it will load up all of the crap default example tags and notes, which then pull down to all of my systems. This is extremely frustrating because I have to keep deleting them. This is multiplied by the fact that it happens with each client I setup. So if I setup the gui and cli on my laptop, it will happen twice (2x each default tags/notes will show up on all my systems).

Semi-Solution:

A slight work-around I have been able to use when setting up a new computer is to simply copy the config files from an existing system to the new one. These folders can be found at ~/.config/joplin/, ~/.config/Joplin/, and/or ~/.config/joplin-desktop/. This also means that the new system should have less it needs to sync down from the server on the first sync.

Long term archiving

The big problem I have… and think I will continue to have with Joplin, is the long-term organization of notes. Part of this is my inability to figure out an efficient and organized way to handle my “log” notes.

This problem partially stems from Joplin’s shallow organizational structure. There are notebooks, and notes. That’s it. Unlike having a file structure where I can do something like archive/2019/June/Weekly_Plans/, I can’t really go that deep with Joplin, so I end up just having a single notebook with all my weekly plan notes listed in it.

One remaining solution might be to get more aggressive with the tagging feature, and see if I can create a more organized system using that. I might be able to tag each note with a month/year, and then truncate my log notes each month. The only problem with this is that it doesn’t seem like I can select multiple tags at a time, which would mean all of my notes for a month will be shown under the tag. That would be a deal breaker for me :( .

Conclusion

In summary, I think that Joplin is a great open source notes solution. If someone needs a system to keep a bunch of normal notes synced across all their devices, this is it. I however, might need to try out something else long-term. On the bright side, there are a bunch of features in Joplin that love and will definitely look for… in my next solution.

SELF 2019

2019-06-20T00:00:00Z

Sheraton Charlotte Airport, Charlotte, NC

The past few years I have watched the South East Linux Fest (SELF) from a distance, often via listreams run by Jupiter Broadcasting and/or the Ask Noah Show. Last year, I decided to finally travel to Charlotte for SELF 2018. While my wife and I did drive down to North Carolina that weekend, it was instead to Durham… with a moving truck. This year, being only two hours away, I decided there was no excuse. I did a day-trip… but I went. I’m glad I did.

SELF

I finalized my plans to attend SELF last minute, about two weeks ago. I convinced a friend from work to attend with me one day over lunch, and we registered right after. So, this past Saturday morning we left Durham around 6:30 am for the 2-ish hour drive to Charlotte.

The first thing I saw when I passed through the hotel doors.

When we got to the hotel parkng lot, we weren’t entirely sure where to go. Then, we noticed a person walking towards a door carrying a old mechanical keyboard under their arm… a clue. Sure enough, after following that person through the same door, we found ourselves in the middle of a hallway filled with Linux booths and face-to-face with a “RUN GCC” T-shirt on display. We found SELF.

My SELF 2019 Badge.

After walking around a bit, we worked our way to registration and received our badges. They were little booklets on a lanyard that contained event information and the schedule. However, it did confuse me at first because the one side was upside-down compared to the other side… but then I realized it was so it could function as a name-tag, but still be easily read while still around one’s neck. Very practical.

Talks

Throughout the day, I went to several talks with an array of topics, including building personal clouds, 3D printers, security compliance testing, network backups, and even building a tiny house. The talks were generally interesting, and I left with a bunch of notes on topics and software I might want to dig into at some point. For example, I think it might be useful to look into chef inspec (or at least similar alternatives to it), for verifying build/test environments at work.

The biggest lesson I learned from the talks at SELF wasn’t from their content, but rather the environment they were presented in. Overall, it seemed like a very knowledgeable, but supportive audience.

For example, one of the speakers was a little flustered as it was his first time giving a talk. When the speaker stated he was a bit nervous, the room responded with “don’t worry, you’re doing great so far! This is how you learn”. Then… the live demo experienced some issues (as they tend to do). The audience responded by laughing with the speaker and shouting out statements like “That always happens to me! See, you are one of us!”. People then started discussing related tips and tricks they’ve learned over the years, while we all worked to troubleshoot the demo.

The room was able to take what could have been a disaster for the speaker, and turn it into an engaging experience where everyone in the room connected and learned from one another. As a result, a “failed” talk ended up being a better experience than originally planned.

Thoughts

I think SELF displayed glimmers of what the Linux and Open Source community can be when at their best. Did I encounter people I disagreed with, or thought were a bit too abrasive? You bet. Did it ultimately matter? Not really. When we disagree, if we can muster up the maturity to work together, we will learn and grow as a community, even when everything is going wrong. Just like in the talk I described above. </soapbox>

I am fully aware that I “did SELF wrong” by only going for a single day and mostly watching talks. I didn’t spend too much time in the “hallway track”, and missed out on the late-night antics and philosophical discussions I heard about. Regardless, I still learned a bunch and had a great time. Now that I’ve scoped out SELF, in the future I think I plan to stay for the whole event… and maybe even submit a talk. See you there!

Creating a RAM Disk with a tmpfs Mount

2019-06-05T00:00:00Z

Riverwalk, Wilmington NC

RAM is fun. If a computer has extra memory, it can be used for fun beyond opening extra chrome tabs, or firing up Slack. Want to mount a partition that is fast and can be entirely wiped out just by rebooting? Or are you just bored (guilty)? Regardless of the reason, lets create and mount a RAM-disk!

RAM disks and tmpfs

Creating a RAM disk in Linux is actually quite easy. Unlike other OS’s, it doesn’t require special third-party software or digging deep into the registry. This is because RAM-based file systems are already heavily used in a Linux system. Many root sub-directories are actually mounted tmpfs objects, most notably /tmp. To see some tmpfs mounts on a system, grep the df command for tmpfs:

➜  ~ df -h | grep tmpfs
devtmpfs                  16G     0   16G   0% /dev
tmpfs                     16G  299M   16G   2% /dev/shm
tmpfs                     16G  2.0M   16G   1% /run
tmpfs                     16G     0   16G   0% /sys/fs/cgroup
tmpfs                     16G  5.9M   16G   1% /tmp
tmpfs                    3.2G  184K  3.2G   1% /run/user/1000
tmpfs                     12G     0   12G   0% /var/ramdisk

All the contents of a tmpfs filesystem reside in system memory, typically RAM. This allows file access that is fast. However, like RAM, tmpfs is also volatile, meaning it will all be erased if the computer restarts or shuts down. So don’t store anything important in tmpfs!

fstab

The easiest way to add a new ‘RAM disk’ to our system is by adding it as a new mount in the /etc/fstab file. I’m currently running Fedora Silverblue, so I had to place it under /var/ because I don’t have write access elsewhere, but feel free to place your mount wherever you want. Just make sure the location exists, or the system won’t boot correctly when loading the fstab. After adding the following line to /etc/fstab, a 10G tmpfs ‘device’ will be mounted at /var/ramdisk during each boot:

# /etc/fstab
...
tmpfs      /var/ramdisk       tmpfs   rw,nodev,nosuid,size=10G	0  0

By default, Linux allocates half the RAM available on the system to a new tmpfs mount. However, I wanted to specify my disk size, so I used the size argument to allocate 10G.

Mounting

With the fstab edited, it is time to mount everything. The arch wiki page cautions against running mount -a to mount the new tmpfs disks, but that is because any files that might already be in the directories will be removed during the mount. While this may be a concern when editing any of the already “naturally occurring” tmpfs mounts on a system (such as /tmp and /run), our use case is adding a new one. So, we should be safe!

sudo mount -a

To verify that the RAM disk is mounted, check the df output. (Note: It should be empty since it was just allocated)

$ df -h
...
tmpfs                     10G     0   10G   0% /var/ramdisk
...

Testing

With the tmpfs mount, how can we be sure it’s actually functioning as a RAM disk? The simplest way to test it is… move some files over to the mount location and see if it starts to use up RAM!

For my quick test, I quickly copied two large iso files to the new mount:

Link: Demo video of RAM usage increasing as file copied to RAM disk

sudo cp -r ./*.iso /var/ramdisk/

During the copy, I had the gnome System Monitor application opened so I could see if the RAM usage slowly climbed as the files copied… and it did! Additionally, when I deleted the copied files, the RAM usage went back down. Looks like everything worked!

Conclusion

While RAM disks are less useful these days due to faster and faster storage, they can still be fun. I’ve placed a full Linux file system in mine, chroot’ed into it, and ran a full DE with xorg and everything! It was quite snappy. I also use it as a ‘scratch’ disk where I can work on a temporary project and blow it away when I’m done. The options are endless!

Setting up Tmuxinator

2019-05-23T00:00:00Z

Port Angeles, WA

Several weeks ago I wrote about scripting a tmux session’s initialization. At the end of the post I mentioned that while writing it, I had learned about tmuxinator. I even teased that it looked so good, I may switch to it eventually. Well… that didn’t take long. I’ve scrapped my tmux shell scripts, and have converted to using tmuxinator. Here’s how.

What is Tmuxinator?

Tmuxinator is a command line application that makes it easy to create and manage tmux sessions. It allows a user to define how to setup a session, including naming windows, splitting panes, and what commands initially run in each pane. The configuration is done in yaml, so it is very easy to figure out and get started.

Install

Most distros probably have tmuxinator in their offical repos. These days, I mostly run Fedora on all of my machines and I was able to find it there:

sudo dnf install termixinator

However, I found that this version was quite out of date (0.6.11) even on my new Fedora 30 install, so I recommend using the method stated on the project’s Github page (requires rubygems) to install a much more current version (1.1.0):

gem install tmuxinator

Create

To create a new tmuxinator profile, pass the desired project name to tmuxinator new:

tmuxinator new Website

This will open up a new tmuxinator template inside your default editor. The default template contains a bunch of comments that help guide how to configure it.

Header

The first thing to set at the top of the template is the name field, followed by the root directory. The tmux session name is defined with name, and root will be the directory which all of the windows/panes open up in by default. For example, when converting my website script, I set name to Website and root to the location where I keep my website’s git working directory, ~/Documents/himmAllRight-source:

# /home/ryan/.config/tmuxinator/Website.yml

name: Website
root: ~/Documents/himmAllRight-source

Windows

Next, I needed to configure my windows. To convert the script from my previous post, I wanted a dedicated ‘Main’ window, one for the server, one for vim, and one for an extra shell. Additionally, since that last post I’ve added a window that launches my hugo-served website in a web browser. To recreate all of that with tmuxinator:

windows:
  - Main:
      - zsh
      - clear
  - Server: hugo serve -D -F
  - Write: nvim
  - Shell: zsh; clear
  - Web: qutebrowser localhost:1313

Done. Save the file and that’s it! See the benefit of using tmuxinator??

Pane splits

While I don’t use them in this example configuration, it should be noted that setting up pane splits is also quite easy with tmuxinator. For example, if I wanted to split the server window into two panes, one running the hugo server and one running htop, I could use:

- Server:
    layout: even-vertical
    panes:
        - hugo
        - htop

The layout key defines which tmux layout to use when splitting the panes, and panes is a list that defines what to run in each pane.

Launch

To start up the newly defined tmuxinator project, use tmuxinator start:

tmuxinator start Website

Click for a demo animation of tmuxinator

Manage

All of the tmuxinator profile templates are stored at ~/.config/tmuxinator, which means they can easily be copied to a new machine, or even saved/maintained in source control.

Note: I think older versions stored them at ~/.tmuxinator/, so check there if they are missing.

Conclusion

That’s it. While scripting my own method was a good bash exercise and helped me learn the details of tmux a little bit better, I ultimately think that tmuxinator is the way to go. This is especially true when using tmux to work on several projects or even across multiple machines. If you still haven’t tried tmux… I highly recommend checking it out!

Custom Neofetch ASCII Art

2019-04-30T00:00:00Z

Downtown Raleigh NC

I have been using neofetch as my command-line system information tool for a few years now. While browsing /r/unixporn, I noticed that several of the submissions did not have distro logos in their neofetch/screenfetch ascii art. I knew that neofetch was customizable, but have never dug into it myself. So I decided to change that.

Custom Neofetch ASCII Art

I started my neofetch investigation the same way one should start researching any Linux command-line tool: man neofetch. Inside the man pages, I was greeted with all sorts of customization options. Eventually, I scrolled down to the section I wanted, IMAGE BACKEND.

There I learned that a custom image could be specified using the --source flag. For example, to set the art to an ascii charmeleon file I just happen to have on my desktop, I tried:

neofetch --source /home/ryan/Documents/Pokemon-ascii/5.txt

AAAAAND… It didn’t work. It was still the Fedora logo. Clearly, I was still missing something.

After some browsing, I found the ASCII art page on neofetch’s Github wiki. Reading it, I noticed that I was missing the ${c1} in my pokemon ascii files. So, I made a copy of the file and added S{c1} to the top:

➜  welcome-messages git:(master) ✗ head charmeleon
${c1}
                      ,-'`\
                  _,"'    j
           __....+       /               .
       ,-'"             /               ; `-._.'.
      /                (              ,'       .'
     |            _.    \             \   ---._ `-.
     ,|    ,   _.'  Y    \             `- ,'   \   `.`.
     l'    \ ,'._,\ `.    .              /       ,--. l
  .,-        `._  |  |    |              \       _   l .

AAAAAND… It worked!

Adding a Shell Alias

Custom Charmeleon ASCII art for my desktop's neofetch output.

Lastly, I wanted the charmeleon ascii art to be the default neofetch output on my desktop. While I’m sure that I could have accomplished this properly by editing the ~/.config/neofetch/config.conf file I learned about, for now… creating an alias seemed easier XD. So I added the following line to my ~/.zshrc:

alias neofetch='neofetch --source /home/ryan/Documents/ascii/pokemon/charmeleon'

Now whenever I call plain neofetch on my desktop, it runs with the charmeleon art.

Conclusion

So that’s it. A nice short post for once! Despite the size, I hope if was useful. If you haven’t heard of neofetch before, go check it out. If you have, I encourage you to go digging through the man pages. I promise you won’t regret it. Enjoy!

Creating "Sub"-Monitor Workflows Using xrandr

2019-04-29T00:00:00Z

NC State Campus, Raleigh NC

A couple months ago I swapped out my dual monitor setup for a single (but massive), 42.5" UHD IPS display. I love it, but admit that sometimes, it has too much workspace. While I still think that the larger display was better choice, I sometimes wish that I had the more limited, but focused workspace of an ultrawide or 1440p monitor. Maybe I still can…

Reasoning

Okay. I understand that this post might seem ridiculous to most people.

“But Ryan, if you have a great display with such a large resolution, why would you want to intentionally scale it down smaller?!?!?!”.

The truth is, there are a number of occasions when having a single, smaller display is helpful.

Focused Work

First, when trying to focus deeply on a single task, I like to have the windows that I need for the activity opened, and nothing else. Furthermore, I like to have these windows opened at a reasonable size (Note: a full screen 3840x2160 single terminal looks stupid). With such a big screen, it is just too tempting to drag chat windows, monitoring apps, and videos to the sides of the workspace. While 1080p is a little cramped, I think a single 1440p resolution is more ideal for focused work (especially because I tend to take advantage of virtual desktops).

Tiling Window Managers

I actually wrote most of the draft for this post in i3.

Second, when working (especially programming), I often like to use tiling window managers ( i3 for example ). This lets me work without having to manually move around the windows, or even leave my keyboard. In tiling window managers, applications tend to open up full screen by default, which again… is just obnoxious on such a large display. Scaling down the display allows me to still use tiling window managers without compromise.

Gaming

Lastly, a half-reason is gaming. I’m not a huge gamer so I didn’t opt for a crazy graphics card when I designed and built my desktop. When I do game, I usually play in windowed-mode (which is usually a better experience anyway… again for field of view reasons). However, sometimes a game won’t support windowed mode, or I want to play full screen at a lower resolution so that my GPU can handle it.

How

Now that it is (hopefully) understood why I want to setup a “sub-display” inside my monitor, lets switch to how I did it.

Problem 1: Scaling

To get “sub”-resolutions working, I had to fix two problems. The first was that even if I select a smaller resolution on my computer, most monitors will display that resolution, but scaled to the size of the display. For example, if I set my computer’s display settings to 1920x1080 on my 3840x2160 monitor, by default it will double up the pixels to get the smaller resolution to fit full-size on the monitor. This makes everything appear gigantic… which is the exact opposite of what I wanted.

After playing around in my monitor’s input settings, I noticed that there is an “Aspect Ratio” field, with a 1:1 option. When selected, the monitor displays an image at its pixel density. So a 1920x1080 pixel display shows as a smaller image in the middle of the screen, but with a 1:1 pixel density. Problem #1 solved!

Problem 2: New Resolution Sets

The second problem was that by default, many of the resolutions I wanted to use out do not show up in the display settings. This makes sense, as most people won’t be selecting 21:9 resolutions on a 16:9 panel. I needed to add new options using xrandr.

After spending sometime in a terminal with xrandr, I was able to create new xrandr modes, and set my monitor to use them.

Creating a new xrandr mode

To create a new xrandr mode, I first needed to calculate a new modeline. This can be done by using the gtf tool. To calculate a modeline for a 3440x1440 resolution at 59.9 hertz for example, use the following:

➜  ~ gtf 3440 1440 59.9

  # 3440x1440 @ 59.90 Hz (GTF) hsync: 89.25 kHz; pclk: 418.41 MHz
  Modeline "3440x1440_59.90"  418.41  3440 3688 4064 4688  1440 1441 1444 1490  -HSync +Vsync

➜  ~

The line that starts with Modeline (but not including “Modeline”) is what we want. Copy that and give it to xrandr with the --newmode flag to create a new mode:

xrandr --newmode "3440x1440_59.90"  418.41  3440 3688 4064 4688  1440 1441 1444 1490  -HSync +Vsync

Lastly, to add that mode to a display, use that same line but with the --addmode flag, and the monitor to switch (DP-1 in my case):

xrandr --addmode DP-1 "3440x1440_59.90"  418.41  3440 3688 4064 4688  1440 1441 1444 1490  -HSync +Vsync

Note: To find the available monitor names, enter a plain xrandr command, and it will spit out all the available outputs.

Switching to the new mode

To switch to the new mode, I like to use a GUI tool named arandr. Simply right click on the display’s rectangle, and select the new mode name from the “Resolution” list (Or Outputs -> Monitor Name -> Resolution -> New MODE NAME in the menu bar).

Using `arandr` to select new mode (SO many resolutions XD ).

If you are too 1337 to use a GUI app, never fear! The display can be switched to the new mode using xrandr with the --output and --mode flags:

xrandr --output DP-1 --mode "3440x1440_59.90"  418.41  3440 3688 4064 4688  1440 1441 1444 1490  -HSync +Vsync

But Wait, There’s More! Scripting it

After running these commands twice… I realized it would be easy enough to automate. So I did with this script:

# add-xrandr.sh
# --------------
#!/bin/bash

# A function to prompt the user if they want to switch to the new mode now.
switch_to_new_mode () {
 echo -n "Switch to mode $modename now? [y/n]: "
 read change
 if [ "$change" == "y" ]
 then
  echo "switching monitor..."
  xrandr --output $MONITOR --mode $modename
 else
  echo "Okay, no switch. Enjoy!"
 fi
}

# Create a new mode for the display
create_new_mode () {
 echo "Adding new mode: $gtf_output"
 xrandr --newmode $gtf_output
 echo "Adding new mode $modename to display $MONITOR"
 xrandr --addmode $MONITOR $modename
 echo "Done!"
}

# Message if the mode appears to already exist
mode_already_exists () {
 echo "Hmmm... I think the mode $modename already exists."
}

## Main Function to set vars and code
main () {
 # Input Vars
 WIDTH=$1
 HEIGHT=$2
 MONITOR=$3
 ESLEEP=3
 # Fancier Vars :P
 gtf_output=$(gtf $WIDTH $HEIGHT 59.9 | grep -i "modeline" | sed -e 's/\<Modeline\>//g')
 modename=$(echo $gtf_output | grep -o  "\(\".*\"\)")
 modeexists=$(xrandr | grep -i $modename)

 # Run
 if [ "$modeexists" == "" ]
 then
  create_new_mode
     switch_to_new_mode
 else
  mode_already_exists
     switch_to_new_mode
 fi
}

## Execute Main
main "$@"

Basically, the script is run by providing it the desired width, height, and xrandr display to apply the new mode to. For example, to create a new 3440x1440 mode for my DP-1 display, I would run the following command:

./add-xrandr.sh 3440 1440 DP-1.

The script will generate a new mode based on the parameters fed it it, and then check to see if the mode already exists. If it doesn’t, it will create it, add it to the display name passed in, and will ask the user if they would like to switch to the new mode. If the mode was already detected, the script will just ask the user if they wish to switch to it.

Note: The script is hard-coded to always create modes at 59.9 Hertz, because I don’t have any fancy fast monitors. If you want that option, just add another parameter and swap out the 59.9 with it.

Pros/Cons

Monitor in 'ultrawide mode'. It is set to 3440x1440px and measures ~36" diagonally.

What this solves

When I have the monitor set to a sub-resolution, I don’t experience any of the “edge shadowing” issues I mentioned in the review post.
I can test out any resolution setup less than 2840x2160. (Including Ultrawide setups)
Using a smaller resolution is friendlier to my tiling wm setups
I can use a better “single focus” monitor setup. For example, the equivalent of a 28" 1440p display.
I can set my monitor to a smaller resolution to better play full screen games (so that the resolution better fits my GPU performance and the game is in my field of view)

What it doesn’t fix

The physical size of my monitor stays the same. (MASSIVE Bezels :P )
I can’t really do multiple displays setups. Even though my monitor supports multiple inputs and picture-by-picture, I still can’t do a good horizontal + vertical 1080p setup, because it will center each one.
The DPI is slightly bigger than if I got the common size/pixel monitors.
Any curving or other physical attributes another monitor form factor has.

Conclusion

In conclusion, I love this setup. With this work-around, I was able to take a few use cases where my monitor didn’t fully fit my needs, and fix it. I am even happier with my monitor selection now, as I can enjoy a massive IPS display when I want it, but also have the ability to tone it down when I want to focus in more. So far, it’s working great!

Switching to a Mesh Network

2019-04-01T00:00:00Z

American Tobacco Campus, Durham NC

Not too long ago, I updated our home networking equipment to an Ubiqui setup. However, we recently moved into a new house and I switched to a mesh wifi network. No, the switch was not just to increase wifi coverage throughout the entire house. The real motivator (ironically)… was that I needed a better ethernet connection…

History

About two years ago, I replaced my Linksys wifi router(s) with the tiny-but-mighty Ubiquity Edge Router X, paired with an Ubiqui AP AC lite wifi access point. I loved it. It handled our web traffic flawlessly, and unlike the previous Linksys routers, I never had to randomly restart it. It just worked for months on end.

The Google Fiber "network box".

I enjoyed the setup so much in fact, that while I was thrilled to have a Google Fiber internet connection when we moved to Durham NC last year, I was a little sad that setting up the edgerouter on the fiber modem required a little bit of tweaking. I looked into it but ultimately decided that figuring it out wasn’t worth the hassle for the short period of time which we’d likely live in the apartment. So, I reluctantly used the Google provided box which worked okay.

We Moved

The Google fiber was indeed short lived, as my wife and I just moved into a house this past month. When working with Spectrum to setup our internet (yea… I know), we discovered that while there were coax plugs in both the living room and master bedroom, we couldn’t actually find any hook-ups to connect the house to the external cable/internet service line.

Cable-line in Bedroom

After searching with multiple techs, we decided just to have them drill a line into the master bedroom (which is at the back of the house and allows the connection box to be with the other utility meters). Our house isn’t massive, and having the router in the bedroom works well enough, except for one issue.

I have several servers and desktop computers that require an ethernet connection, either because they don’t have wifi cards, and/or because I use Wake-On-Lan with them. This means that I need an ethernet connection in my office… at the front of the house. Now that the only ethernet router/switch was in the bedroom, and the rest of the house relied on wifi… I needed to figure something out.

Power-line Issues

In past apartments, I had used power-line ethernet adapters to extend ethernet from a router in a different room to my computers. The speeds weren’t great, but then again, neither was our connection so it wasn’t a huge problem. This time though, I ran some tests…

Different internet Speed test results after hooking up internet to the house.

So, while it worked, this setup wasn’t ideal. More importantly, I found out a few days later there was a massive problem that completely prevents me from being able to use the power-line adapters. The house’s electrical panel is between the master bedroom and the office, which is the worst case scenario. One day, we turned on the washing machine and our internet went down. It wasn’t just the power-line connections that were effected… the entire network, wifi and router was essentially being dos’d by the washer. When I unplugged the power-line adapters, it worked again. Power-line was not an option anymore.

Mesh Network

I wondered if there was another way to connect ethernet in my office, beyond having to drop cat6 cable across the house. I looked into wifi bridges, but most of the devices were designed for large distances, and ones for home use were not much better than a power-line adapters when it came to performance/reliability. While looking at wifi bridges, a thought occurred to me… would a mesh network system work? If the satellite devices had ethernet ports… maybe.

Netgear Orbi

When I started researching wifi mesh networks, the orbi systems were consistently rated the highest, and people generally reported that they experienced great performance. When I dug further into it, I learned that this high level of performance was likely due to the orbi’s “tri-band wifi technology”. Basically, in addition to the normal 2.4 and 5 Ghz network bands, the orbi system has a second 5 Ghz band dedicated to connecting the orbi router and satilites together. In other words… a dedicated wifi bridge :D.

The RBK50 system comes with 1 router, and 1 satilite device. Both have 4 etherent ports.

I started looking at the orbi hardware and saw that there were several types of satellites. Some were just wall plugs, but a few had 2-4 ethernet ports. Perfect. Realizing this, I ended up ordering the RBK50 system, which contained a 4-port orbi router, and a 4-port satellite. My hope was that I could set up the router in the master bedroom with our modem, and then put the satellite in my office on the other end of the house. I could then connect my hardwired devices, or even a network switch, into the ethernet ports on the satellite. Problem sovled.

Conclusion

I have ordered and setup my orbi system, and will write a review post about it sometime in the future. This post’s intention was mostly to just explain why I ended up getting an orbi system.

Many people have started to use home wifi “mesh” systems because they provide a simple solution to cover an entire house with a strong wifi connection. While this is a great bonus I get from having the mesh system, as this post demonstrates, my main purpose for purchasing the orbi was to be a solid “ethernet over wifi” system in the new house. So far… it’s working great.

Scripting A Tmux Work-space Start-up

2019-03-27T00:00:00Z

Liberty Warehouse, Durham NC

I love the terminal multiplexer, tmux. It adds functionality to the terminal, such as multiple tabs, pane splitting, and the ability to detach and re-attach everything later (which is amazing when combined with ssh). I have been utilizing tmux even more at work, and recently started to script the start up/configuration a tmux session for each project. The other day, I decided to write a script to spin up a session for working on my website… and thought it would be a great tutorial!

Tmux

When working on a project, I like to use tmux to maintain all of the terminal windows related to it. This keeps everything together, and even allows me to detach the session, switch to another computer, ssh into the previous computer, and re-attach my working tmux session. Paired with the fact that I’ve started using VIM again, it works seamlessly. Scripting the initialization makes getting started even smoother.

Tmux Demo Animation

Automating tmux initialization for working on my website

New Script

First, lets create a new script. Start by opening a new file, and adding a bash shebang line to the top:

#!/bin/bash

This line tells the system that the following text will be a bash script.

Optional: Save and close the file, then re-open it to take advantage the text editor’s bash syntax highlighting if it didn’t automatically switch it on.

Create a new tmux session (with var name)

Next, lets define a variable to store the tmux session name. This will make it easier to change the session name later on. Using the session variable, a new tmux-session with our desired name can be created.

# Session Name
session="Website"

# Start New Session with our name
tmux new-session -d -s $session

Name initial default Window (and switch to `zsh`)

Let’s customize the default window and give it a new name. Lets call this first tmux window "Main", and have it simply run a zsh shell. After creating a new session, there is only one window, so I know it will be identified with the number 0. I can use this with -t to rename the window.

# Name first Window and start zsh
tmux rename-window -t 0 'Main'
tmux send-keys -t 'Main' 'zsh' C-m 'clear' C-m

I can then use the send-keys command with the new window name to start zsh. This is the equivalent of typing zsh, [Enter], clear, [Enter] into the command line.

Add a new (named) window for hugo server

With the main tmux window setup, I want to add a few more for different tasks. First, I want a window that can run the hugo server as I’m writing a post. With a session already created, I can name the window as I create it, using just the new-window comamnd with the -n flag.

# Create and setup pane for hugo server
tmux new-window -t $session:1 -n 'Hugo Server'
tmux send-keys -t 'Hugo Server' 'hugo serve -D -F' C-m

Again, I used tmux send-keys, this time to send the hugo serve -D -F command to start up a hugo server for local draft editing.

Add a new (name) pane for vim

Now, I need a place to write website posts… so lets fire up a new tmux window, and open up neovim inside of it.

# setup Writing window
tmux new-window -t $SESSION:2 -n 'Writing'
tmux send-keys -t 'Writing' "nvim" C-m

Another shell

Lastly, lets create one more shell window, just in case it’s needed. Why not?

# Setup an additional shell
tmux new-window -t $SESSION:3 -n 'Shell'
tmux send-keys -t 'Shell' "zsh" C-m 'clear' C-m

Attach Session

With the tmux session all configured and customized, we can tell the script to go ahead and attach it, using the attach-session command with the session name variable.

# Attach Session, on the Main window
tmux attach-session -t $SESSION:0

Note: the 0 tells tmux to attach the first window (Main). Using another index value will attach a different window. For example, 1 would open the Hugo Server window when attaching

Bonus: Minor improvement

At this point, I was done with the tmux script. It worked well for creating my session and attaching it. However, there was one issue I still had. If I ran the script when there was a tmux session with the same name already, it would just double up the windows in that session. If I accidentally did this, I would have to go through each window, close out whatever and was running, and then close the window.

To fix this issue, I decided to wrap the initialization commands inside of an if statement, and only run them if the tmux session didn’t already exist.

Duplicate windows demo animation

Checking if the Session Already Exists

First, I needed a way to check whether the desired tmux session already existed or not. This can be done by “grep’ing” the output of tmux list-sessions for the session name, which we’ve already conveniently stored in our $SESSION variable. For Cleanliness, I took the output of that process and saved it in a SESSIONEXISTS variable, defined directly under SESSION in the script.

SESSIONEXISTS=$(tmux list-sessions | grep $SESSION)

If the session exists, SESSIONEXISTS will be a string of the line which grep matched. Otherwise, it will just be an empty string ("").

Now, I can use the SESSIONEXISTS variable with an if condition to wrap the tmux setup code so that it only runs if SESSIONEXISTS is "".

if [ "$SESSIONEXISTS" = "" ]
then
...
fi

The attach-session command should be outside of the if body, because it will be run in both cases (even if the session doesn’t have to be created, we still want to attach the one that already exists).

Conclusion

That is it. Here is the script with everything all put together:

#!/bin/sh

# Set Session Name
SESSION="Website"
SESSIONEXISTS=$(tmux list-sessions | grep $SESSION)

# Only create tmux session if it doesn't already exist
if [ "$SESSIONEXISTS" = "" ]
then
    # Start New Session with our name
    tmux new-session -d -s $SESSION

    # Name first Pane and start zsh
    tmux rename-window -t 0 'Main'
    tmux send-keys -t 'Main' 'zsh' C-m 'clear' C-m # Switch to bind script?

    # Create and setup pane for hugo server
    tmux new-window -t $SESSION:1 -n 'Hugo Server'
    tmux send-keys -t 'Hugo Server' 'hugo serve -D -F' C-m # Switch to bind script?

    # setup Writing window
    tmux new-window -t $SESSION:2 -n 'Writing'
    tmux send-keys -t 'Writing' "nvim" C-m

    # Setup an additional shell
    tmux new-window -t $SESSION:3 -n 'Shell'
    tmux send-keys -t 'Shell' "zsh" C-m 'clear' C-m
fi

# Attach Session, on the Main window
tmux attach-session -t $SESSION:0

I hope this post has been helpful! Although, I have to admit that this post may be moot if I decide check out tmuxinator (which a co-worker recommended) … oh well. I guess this endeavor was still a good exercise in some bash scripting :P.

Why I Got an iPad

2019-02-28T00:00:00Z

My Desk, Durham, NC

Over the past few months, I’ve been considering getting a tablet device for the first time since I was in college. After researching all sorts of options, I had myself convinced that I would likely be picking up a 2018 9.7" iPad at some point. Then I ordered an iPad Pro 10.5" (2017 model). Here’s why.

Why a Tablet?

When I use technology, I try to plan out and use devices which best fit the tasks I want to accomplish. After evaluating my work habits, I started to think that a tablet might function well in many of my use cases.

My blackberry keyONE

The main reason? I don’t like using smart phones. I think modern phones are a wonderful piece of technology, but they are often misused. While some smart phone apps can greatly enhance one’s life, many are convenient distractions.

My phone is always with me, so I keep the set of apps installed on it to a minimum. I try to use apps that are most needed “on the go” (maps/lyft, music/podcasts, and various communication apps). I find that this helps prevent me from easily distracting myself throughout the day.

Recently, I realized that the experience of some tasks on the phone isn’t great, and that they would likely be best completed on a tablet. Additionally, setting up a tablet for these apps meant I could still enjoy them in a more portable way, but maintain a clean, distraction-free phone. Some of the activities I intended to offload from my phone are:

Portable Media Consumption

I usually consume the majority of my media either on our TV, or at my computer. That being said, I did start to notice myself watching Youtube, Hulu, and Netflix, more often from my phone. This usually occurred when the TV was in use, or I simply wanted to watch from a different location (ex: my bed). Having a small screen and tinny speakers, watching video on a phone is terrible. This is especially true on my Blackberry KEYone’s even smaller screen (Which I choose because it is a great communication device, with a long battery life. Regardless, even on a large, bezel-less phone screen, watching video on a smart phone is still sub-optimal).

In college, I used my previous tablet (an Asus Transformer T101, with no dock) to watch videos in my dorm or around campus. I loved its convenient, but yet still enticing viewing experience. I assumed that with a modern tablet, it would only be better.

Reading

Being such a visual learner, I tend to gravitate towards visual media. Reading however, is much less passive than video. When watching a movie, it will continue to progress whether the viewer is paying attention or not. Reading is much more self-driven, which I find helps to build my concentration. As a result, reading is something I am always trying to do more of.

I have owned a kindle paperwhite for years, and it is perfect for reading books. However, there are some types of reading material that my kindle falls simply does not handle well.

I have a bunch of dev books in my reading collection.

For example, thanks to Humble Book Bundles, I have acquired a collection of programming, DevOps, system administration, and general technology ebooks. You know… the type of book that is technically classified as a “textbook”… but does not sound nearly as cool when I call it that… Anyway the point is… they are horrendous to read on the kindle. I would love the ability to flip through a textbook on a large, color screen, and a tablet enables that.

Lastly, the internet is full of all sorts of “articles”. News stories, blog posts, product announcements, opinion pieces, tutorials, reviews… all litter the internet. Recently, I’ve been starting to corral this craziness by using tools such as pocket to save articles I want to read. Then, I can come back and batch-read them later. This helps me break the endless rabbit-hole of digging through, and semi-reading “related” articles (half of which aren’t even good). This practice has worked alright on my computer, but I do think it would be ideal to sit down in an armchair with a tablet in my lap, for my batch reads.

Offload Some of My Smart Phone Use

As previously stated, my biggest motivation for buying a tablet is to offload my phone usage to it. This is particularly true for apps which I am required to use… but don’t need to have access to at all times.

The best example of this? Banking and other financial apps. I appreciate the ease of modern banking, and love that I can simply take a picture to deposit a check. However, I don’t need this on my phone. Not only am I unlikely to be scanning checks and moving money around my bank accounts in public… it is not really safe to be doing so. I’d rather just wait until I get home on my own network, and do my banking from a tablet (which stays at home much more frequently than my phone).

Other Options

Android Tablets

When I started looking at tablets, I quickly noticed that the market had changed dramatically from a just few years ago. While there are still android tablets available, the future of Google supporting android on tablets is a bit unclear. They appear to be trying to shift everything to ChromeOS, as demonstrated with the Pixel Slate. Additionally, Android still tends to treat tablets as just a large screened phone, and applications are often not optimized for it. Understandably, I was a little leery about Android tablets. As for ChromOS tablets, the Slate is currently the only real contender, but with it’s high price point, and lack-luster reviews, it was out of the running.

Amazon Fire Tablets

In contrast to the pixel slate, at the low end of the tablet market was the Amazon fire tablet family. These devices are quite affordable and would work well for anyone that just wants to read/watch video on a tablet (especially Amazon content). However, I just felt that the Fire tablet might be a bit too limiting for me, as I tend to push technology as far as I can. Beyond that, the cheaper iterations of each tablet are the “special offers” ones, which show amazon ads on the tablet. By the time I had selected a 10" tablet without special offers, the price point started approaching (well… almost) another contender… the 2018 iPad.

Why an iPad?

My Apple Device History

I originally wanted to do a full multi-paragraph section detailing my limited use of apple products throughout my life, and how I only really buy them when they obviously stand out above the competition. But that would be pretentious, extremely boring, and quite honestly, annoying. So in short: the only Apple devices that I have owned are 1) an iPod Video, and 2) iPod Touch. So why am I willing to get an iPad as my tablet device?

iPad

While researching, I found that the Apple iPad was consistently recommended as a great-value tablet-device on the market today. Most notably, the 2018 entry-level iPad, which sold for $330 USD, but could often be found on sale for under $300.

Aimed at the education market, the 2018 iPad saw a cpu upgrade, and support for the Apple pencil. The only con I ran into people reporting was that because it didn’t have a laminated screen, there was a bit of a “gap” between it and the glass. Despite the minor screen issue, this was still the best device I could find around that price range.

Why the 10.5" Pro?

A 10.5" iPad Pro

So… why did I end up with the 10.5" iPad Pro? The switch happened last minute the morning I ended up ordering it. My upgrade happened because of two factors:

Sale: The iPad Pro 10.5" had gone on a sale at BestBuy, and the configuration I liked dropped to $499 (from $650). While this was a bit more expensive, I thought it was still a good deal, and the 2018 iPad wasn’t on sale anywhere so it was the full $330.
Features: Essentially, the Pro had a slightly larger and much better display, better speakers, and double the storage space (64GB compared to 32GB). These are features that I don’t think most people need to care about, I knew I would really appreciate those features, especially now that I could get them for only a ~$170 increase over the iPad, compared to the normal $310 difference.

Summary

So, that’s the history and reasoning as to why I decided to get a tablet device, and why that device ultimately ended up being an iPad 10.5" Pro. It arrived a couple weeks ago, and I have been enjoying it so far. I think I’ll give myself a bit more time with it, before I even attempt to write any sort of review.

Extending a VM Virtual Disk

2019-02-07T00:00:00Z

Foster Street, Durham, NC

Last week, I extended the virtual disk of the VM hosting my jenkins server. Shortly after, I increased the maximum disk size of one of the job’s docker containers, maxing out the disk. This meant that I needed to extend the drive*… again*. If you ever do something twice, it is best to have it documented for the potential third time. So, here we are.

Assumptions

Before getting started, I want to point out that this method is specific to the environment I currently have for my VMs. Specifically, I am using kvm/qemu and virt-manager, with qcow2 images for the virtual disks. Additionally, the specific VM I was extending was installed with LVM and it’s main partition was formatted with a xfs file system. Just note that some steps may differ elsewhere. This is what worked for me.

Clone VM

Cloning the VM in Virt Manager.

While not required, it isn’t a bad idea to first clone the VM (just in case anything becomes damaged). If using virt-manager, cloning a VM is as simple as right clicking a powered down VM, and selecting “Clone…”. A window will pop up with options for cloning the VM. Make the desired name changes and hit Clone.

Extend qcow2 file

The first step in resizing the virtual drive is to first expand the qcow2 image. By default, the images tend to be stored at /var/lib/libvirt/images/ and will require root privileges to access. Virt-Manager can be used to double check which image the VM is using for its disk. To resize the qcow2 image, use the qemu-img resize command, providing it the image file path/name and then the size to expand it. For example, I used +40G in my command (qemu-img resize Jenkins.qcow2 +40G) to extend the image by 40GB.

root@ninetales:/var/lib/libvirt/images# qemu-img info Jenkins.qcow2
image: Jenkins.qcow2
file format: qcow2
virtual size: 20G (21474836480 bytes)
disk size: 20G
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: true
    refcount bits: 16
    corrupt: false

root@ninetales:/var/lib/libvirt/images# qemu-img resize Jenkins.qcow2 +40G
Image resized.

root@ninetales:/var/lib/libvirt/images# qemu-img info Jenkins.qcow2
image: Jenkins.qcow2
file format: qcow2
virtual size: 60G (64424509440 bytes)
disk size: 20G
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: true
    refcount bits: 16
    corrupt: false

root@ninetales:/var/lib/libvirt/images#

The command qemu-image info can be helpful in verifying that the resize worked, by checking the size of the image.

Gparted Live ISO

For the next few steps, it is a good idea to boot the system from a live CD. This will run the OS in RAM, allowing the disk to be fully unmounted. With access to the VM’s display, an ISO such as the gparted live CD can be used to resize the partitions, as it contains the amazing graphical tool, gparted (duh).

Note: If you boot up the VM and don’t see the new unallocated space available in the volume… make sure you didn’t accidentally boot the backup VM… Not that I made such a mistake… :P

LVM Resize

My VM is installed using LVM volumes, so I had to resize them before I could resize the file system. Gparted will do this automatically when resizing a partition.

Booting into the Gparted live ISO.

To resize the partition, first verify that the correct virtual disk is selected in the top right drop-down the window. Next, select the partition to expand, and click the “Resize/Move” icon at the top.

Resizing the partition in Gparted.

In the resize window, I changed the Free space following (MiB) value to 0, to expand the partition to use all of the unallocated space. Lastly, I hit the Resize button and let Gparted do it’s magic.

Grow XFS

Finally, with the lvm volume expanded, I just had to grow my file system to use the new space. I booted up the VM and logged in. This VM uses an xfs file system, so I was able to use the xfs_growfs command to expand the partition:

[ryan@mr-mime ~]$ sudo xfs_growfs /dev/centos/root
(... Ryan removed output for the post...)
data blocks changed from 4851712 to 15322112

[ryan@mr-mime ~]$ df -h
Filesystem               Size  Used Avail Use% Mounted on
/dev/mapper/centos-root   59G   19G   40G  32% /
devtmpfs                 1.9G     0  1.9G   0% /dev
tmpfs                    1.9G  8.0K  1.9G   1% /dev/shm
tmpfs                    1.9G  8.7M  1.9G   1% /run
tmpfs                    1.9G     0  1.9G   0% /sys/fs/cgroup
/dev/vda1                497M  231M  267M  47% /boot
tmpfs                    379M     0  379M   0% /run/user/1000

Note: My VM’s disk space was COMPLETELY full. This meant that I couldn’t use auto tab complete in my shell because it spit out there’s no disk space. Typing the command out fully by hand, still worked.

With the xfs partition resized, I rebooted the VM for good measure, and everything was up and running again.

That’s about it. Remember, the steps I took might need to be altered for other environments, but this post should still be a good starting point. I know it will help me when I acidently overfill this VM again… Enjoy!

Replacing a Drive in My ZFS Mirror

2019-01-15T00:00:00Z

My Desk, Durham, NC

Right before Thanksgiving, one of the hard drives in my server started get noisy… very noisy. Fearing the worst, I did a backup, and shutdown the server until I had time to investigate further… and likely replace the drive. That time came this past week.

Verifying the Drive Failed

Before throwing money at the problem, I wanted to see if ZFS was detecting any issues. When I ran a zpool status on my Data pool, it warned me that one of the devices had experienced an error, but that I had not (yet) encountered any data errors. Time to buy a new drive.

λ ninetales ~ → zpool status Data
  pool: Data
 state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
	attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
	using 'zpool clear' or replace the device with 'zpool replace'.
   see: http://zfsonlinux.org/msg/ZFS-8000-9P
  scan: resilvered 1.14M in 0h0m with 0 errors on Sat Jan 12 10:49:31 2019
config:

	NAME                                  STATE     READ WRITE CKSUM
	Data                                  ONLINE       0     0     0
	  mirror-0                            ONLINE       0     0     0
	    ata-TOSHIBA_DT01ACA300_365XDT3KS  ONLINE       0     0     2
	    ata-TOSHIBA_DT01ACA300_365XDR5KS  ONLINE       0     0     0

errors: No known data errors

Ordering a New Drive

When I started shopping for hard drives, I decided to replace my broken 7200 RPM one a 5400 RPM one. I’d rather have the drives last longer and run quieter, than whatever marginal speed difference the faster spinning disks may provide. I decided to finally go with a 3TB Western Digital RED drive this time, even tough it’s a bit more expensive… mostly to try out.

Replacing the Drive

Swapping the hot-swap caddy from the broken hard drive (left) with my new WD Red drive (right).

Physically swapping the hard drives was a breeze. I could easily tell which drive was the defective one… as it caused the entire server to rumble (ಠ_ಠ) . So I slid it out, unscrewed the drive from its caddy, and screwed in the new one. Lastly, I slide the caddy back into the server and booted it up. I love hot-swap drive bays.

Figuring Out Which Disk To Replace

Determining which disk was being replaced in software was a bit more difficult. In order to add the new drive to the Data pool, I needed to tell ZFS which one had been replaced. This was made more complicated by the fact that previously, the two drives in the mirror were the same model and both showed up as /dev/disk/by-id/ata-TOSHIBA_DT01ACA300_365XDR5KS. So, I needed to find the guid for each drive, because it would differ between them. I used the command zdb to spit out information about of each of my pools:

λ ninetales ~ → zdb
... (other pool output)...
Data:
    version: 5000
    name: 'Data'
    state: 0
    txg: 15996848
    pool_guid: 2285339125999939520
    errata: 0
    comment: 'iocage'
    hostname: 'ninetales'
    com.delphix:has_per_vdev_zaps
    vdev_children: 1
    vdev_tree:
        type: 'root'
        id: 0
        guid: 2285339125999939520
        children[0]:
            type: 'mirror'
            id: 0
            guid: 15171243251753521949
            metaslab_array: 34
            metaslab_shift: 34
            ashift: 12
            asize: 3000588042240
            is_log: 0
            create_txg: 4
            com.delphix:vdev_zap_top: 125
            children[0]:
                type: 'disk'
                id: 0
                guid: 4676737554230074290
                path: '/dev/disk/by-id/ata-TOSHIBA_DT01ACA300_365XDT3KS'
                phys_path: '/dev/ada1'
                whole_disk: 1
                not_present: 1
                DTL: 123
                create_txg: 4
                com.delphix:vdev_zap_leaf: 126
            children[1]:
                type: 'disk'
                id: 1
                guid: 13442522248687181242
                path: '/dev/disk/by-id/ata-TOSHIBA_DT01ACA300_365XDR5KS'
                phys_path: '/dev/ada3'
                whole_disk: 1
                DTL: 122
                create_txg: 4
                com.delphix:vdev_zap_leaf: 159
    features_for_read:
        com.delphix:hole_birth
        com.delphix:embedded_data
...

At first, I still didn’t know which drive was which. However, after looking deeper, I noticed that one of the Toshiba drives listed had the line not_present: 1, indicating that it was the broken drive I had removed!

Replacing the drive

With the guid of the broken drive known, I was able to start the process of replacing it in the pool with the new one. I issued a zpool replace command with the following arguments:

sudo zpool replace Data 4676737554230074290 /dev/sdd

Data - the name of the pool
4676737554230074290 - the guid of the previous drive
/dev/sdd - the path to the new drive

Afterwards, the resilvering process started (rebuilding the mirror by copying data from one drive to the other). I was able to check the status of the process using zpool status Data.

λ ninetales by-uuid → zpool status Data
  pool: Data
 state: DEGRADED
status: One or more devices is currently being resilvered.  The pool will
	continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
  scan: resilver in progress since Sat Jan 12 11:29:36 2019
	72.6M scanned out of 1.03T at 2.42M/s, 123h32m to go
	72.2M resilvered, 0.01% done
config:

	NAME                                  STATE     READ WRITE CKSUM
	Data                                  DEGRADED     0     0     0
	  mirror-0                            DEGRADED     0     0     0
	    replacing-0                       DEGRADED     0     0     0
	      4676737554230074290             UNAVAIL      0     0     0  was /dev/disk/by-id/ata-TOSHIBA_DT01ACA300_365XDT3KS
	      sdd                             ONLINE       0     0     0  (resilvering)
	    ata-TOSHIBA_DT01ACA300_365XDR5KS  ONLINE       0     0     0

errors: No known data errors

Resilvering can take a long time. Luckily, I only had about ~1 TB of data to rebuild, so I hoped it wouldn’t actually take the 123.5 hours that the first status predicted! Regardless, during the resilvering process, the only thing to do is wait (and hope that the other drive doesn’t break in the process!). So I did.

Resilver Complete

In just over 4 hours, my pool had rebuilt and was back online.

λ ninetales ~ → zpool status Data
  pool: Data
 state: ONLINE
  scan: resilvered 1.03T in 4h8m with 0 errors on Sat Jan 12 15:38:26 2019
config:

	NAME                                  STATE     READ WRITE CKSUM
	Data                                  ONLINE       0     0     0
	  mirror-0                            ONLINE       0     0     0
	    sdd                               ONLINE       0     0     0
	    ata-TOSHIBA_DT01ACA300_365XDR5KS  ONLINE       0     0     0

errors: No known data errors

Looking at this output now, I realize I probably should have added the new drive by uuid instead of pathname…hmmm…

Oh well. That is a post for another day. At least my broken drive has finally been replaced!

Shell Stuff - Easy File Cleanup

2019-01-05T00:00:00Z

World War I Memorial Park, North Attleboro, MA

Applications can leave their junk files all over the place. While I appreciate that all of the .swp, .retry, and "conflict" files are there to help me when things go wrong… sometimes I just want to clean up my file system. So… here is a simple string of commands I often use to declutter my files.

DISCLAIMER: I know there are MANY ways to accomplish this. The method described in this post is to share one efficient solution I use, that might help someone that currently knows zero.

The commands

First, lets quickly meet the commands we will be using:

Find

find is a classic UNIX command that searches for files in a directory hierarchy. By default, it writes out the file path for each file/directory that it finds.

Example:

➜  tree
.
├── dirA
│   ├── file3
│   ├── file4
│   └── file5
├── dirB
│   └── file6
├── file1
└── file2

2 directories, 6 files

➜  find .
.
./file2
./file1
./dirB
./dirB/file6
./dirA
./dirA/file5
./dirA/file4
./dirA/file3

Grep

Another classic. Basically, grep searches for a pattern in each file provided. In addition to files, it can search text passed through a pipe (this is important for our use, but more on that later).

Example:

➜  cat file1
This is a fake file
with a few lines of content.

However, I want search for something
without opening it...

Secret: 12345

I wonder if I will be able to get it...


➜  grep Secret file1
Secret: 12345

Command Substitution

Lastly, command substitution is taking one command, and using it’s output as part of another command. Traditionally, this was done by calling the substitution command `inside backticks`, but it is now preferred to use $(COMMAND) instead of backticks.

Example:

➜ echo I am at: `pwd`
I am at: /tmp/demo

or (preferred):

➜ echo I am at: $(pwd)
I am at: /tmp/demo

Pipes

An unix pipe (|) directs the output of one command, to be used as the input for another command. Pipes can be used to chain together several commands, forming a pipeline.

Example:

The output of ls can be fed as input to wc (word count) to create a pipeline command that returns the number of files/directories in the current directory.

➜ ls
dirA  dirB  file1  file2

➜ ls | wc -l
4

Putting It All Together

Now that we know all the parts, how does it all fit together? One particular shell chain I find convenient is pairing find and grep to recursively get all the paths of a particular file type, and then use it in a command substitution to pass that result on to another command (such as rm).

COMMAND $(find . | grep SEARCHSTRING)

This is the combination I use to clean up my directories. While working on writing ansible playbooks, I can generate a few *.retry files, as well as some *.swp files from editing in vim.

Example:

➜ find . | grep .retry          ## Find *.retry files
./file1.retry
./dirA/file5.retry
./dirA/file3.retry

➜ rm $(find . | grep .retry)    ## Delete *.retry files

➜ find . | grep .retry          ## Check that they were deleted

➜

Summary

That’s it. A small post for a simple but powerful command line set. If you haven’t used this team of commands before, give it a try sometime! Have fun!

Configuring Pass, the Standard Unix Password Manager

2018-12-12T00:00:00Z

Liberty Warehouse Apartments, Durham, NC

I’ve been using pass to manage my passwords for quite some time. During the early days of use, I occasionally had difficulty configuring it on new machines, but those days appear to be long gone. It is a simple, generic, yet flexible system. Here’s how to get started.

(but first, some background… feel free to skip ahead)

My Password Manager History

For the longest time, I didn’t use a password manager (in my defence, not many people did). Then in college, I started using LastPass. It was simple and made it easy to switch all of my passwords to randomly generated ones. I had a good system that worked for a few years, and was even able to integrate my yubikey with it.

Then in 2015, Lastpass was acquired by LogMeIn, which had a questionable past of Linux support. Like many others in the open source community… I started looking for alternatives.

I had already been searching for a LastPass replacement even before the acquisition. My search was mainly fueled by one big issue I had with LastPass… it required a web browser to use. Additionally, to utilize it’s full feature set, it needed to run as a Chrome or Firefox plugin. As someone who often uses alternative web browsers (like qutebrowser), or works on headless machines, I try not to use applications that exist solely as a FireFox/Chome app. I am also not a fan of pure website-apps in general.

So, as the I watched others switch password managers amongst the acquisition hype, one switch I remember seeing was Chris Fischer of the Linux Action Show. In episode 387 of LAS, Chris and Noah (his co-host) discussed LastPass alternatives, and Chris highlighted his switch to pass. While I don’t think he kept with the system long-term… I have.

What I like about pass

Unix Philosophy “Simplicity”

Okay. The average computer user will not think pass is “simple”. I agree. However, being designed to follow the Unix philosophy, pass’s architecture is. Basically, pass is just a nice wrapper around a bunch of gpg encrypted text files. It is a minimal, but tested solution. This model makes pass easily compatible with many other great tools, such as bash, git, dmenu, xmonad and emacs.

Command Line Tool

As a command-line tool, I can use pass anywhere. It doesn’t matter how conventional or strange the setup may be. I can have it on my desktop, on a headless server, or even inside a container. It makes no difference. Even if I am on a public computer, if I can ssh into one of my servers, I can access my passwords.

Flexible

By default, pass assumes the first line of a store file is the password. However, the multi-line contents of a pass file can be anything. For example, pass could be used to securely store encrypted notes. This gives the system a ton of flexibility, as the password items don’t have to conform to any sort of template.

Installing Pass & Help Packages

On Fedora, pass can be installed using dnf. For other systems, check out the “Download” section of the pass website.

sudo dnf install pass

Configuring Pass

After installing pass, there are few steps to configure it. First, we need to create a gpg key if one doesn’t already exist. Then, we need to initialize a password-store using that key.

Note: I went a little heavy with the animation images in the remainder of the post. Sorry. I hope they are more useful than annoying. Being a visual learner, at the very least they are helpful for me when I reference this post in the future. …

New GPG Key

To create a gpg key, the gpg2 --gen-key command is normally used. However, I opted to use gpg2 --full-gen-key, which allows for a bit more control during setup. The command will prompt for several bits of information, and the default selections are generally fine for most of the options (Personally, I use a 4096-bit key, because… why not?). At the end it will ask for a name, Password, and optional comment.

It should be noted that gpg2 most likely needs to be used instead of gpg for pass. However, it may vary depending on distribution and the package versions.

Pass Init

After a gpg key has been generated, it can be used with pass. First, find the key’s ID by using gpg2 --list-secret-keys. Then, configure pass with pass init GPG-KEY-ID. This will create a password-store directory, located by default at ~/.password-store/.

Add some items

With pass initialized, lets start adding passwords to it! Here are some of the most common commands to do so:

insert

Simply put, pass insert … inserts a password. Call it with the desired folder/file structure for the password, and pass will then prompt for the password to save. That’s it.

pass insert Shopping/amazon.com/ryan

pass generate

In addition to inserting existing passwords, pass can also generate new ones using pass generate. Just provide the password path, and optionally the length of the password. Pass will then generate a random password, spit it out on screen, and insert the entry to the password-store.

pass generate Shopping/SomeFakeStore/ryan 35

pass edit

Generating passwords is great, but being a forgetful person, I like to keep additional information in my pass entries (username, email, website url). This is where pass edit comes in. When called, pass edit will open up the contents of the entry in the default editor. From there, make the changes, and save.

For example:

pass edit Shopping/SomeFakeStore/ryan

Then, in vim:

<&DdU1x<&~&{;w7w"kvsWdHAF-\Vi"I9Q)I
---
Username: ryan
Password: <&DdU1x<&~&{;w7w"kvsWdHAF-\Vi"I9Q)I
URL: https://www.some-bs-store.com
Notes: I love this place!

pass

Lastly, to retrieve stored passwords, call pass with the password entry. Optionally, use the -c flag to copy the password (first line if a multi-line entry) to the clipboard instead of spewing it into the terminal.

pass Shopping/SomeFakeStore/ryan
## or ##
pass -c Shopping/SomeFakeStore/ryan

Making Pass Better

With pass’s flexibility, there are many additional features to help improve it for each user’s needs. For me, there are two extensions that make my pass experience much more enjoyable.

Pass Git

Password-Store items are text files, which allows them to be easily version controlled. Consequently, pass has built in support for git with the pass git command. If a password-store is linked up to a git repo, normal git commands (add, mv, rm…) can be used with the store.

Additionally, when modifying the store’s contents, pass git will automatically create commits that reflect the changes. After adding or modifying a password, issue the command pass git push on the updated machine, and then pass git pull on others to sync the changes.

Passmeu

While having a CLI password manager is nice when working with headless systems, it can be a bit cumbersome for normal day-to-day use. Hence, passmenu.

Passmenu is a script (now built into the upstream project) that wraps dmenu around pass. When passmenu is run, dmenu opens up with all the password-store items to search/filter from. When an item is selected in dmenu, the user is prompted for the gpg password (if it hasn’t been unlocked recently), after which the password is then temporarily added to the user’s clipboard.

On all my computers, I bind the command passmenu to the keys SUPER + SHIFT + P. Whenever I need a password, I just hit those three keys, and dmenu pops up so I can search for the password I want. After typing in my master passphrase, I can paste the password wherever I need it. Passmenu makes pass much more reasonable to use.

See Also: rofi-pass

Setting up your pass setup on a new system

Now that I’ve done it over a hundred times, setting up a new system is easy. Here’s my usual steps:

Export GPG Key

First, export the password-store’s gpg key. To do that, use gpg2 --list-secret-keys to confirm the key’s ID, then export that key to a file with the following command:

gpg2 --export-secret-keys KEY-ID >> key-filename.gpg

Next, transfer that file to the new machine.

Import GPG Key

On the new machine, import the gpg key using the following command (note, you will be required to enter the key’s passphrase):

gpg2 --import key-filename.gpg

After the key is imported, its trust level will have to be set to ultimate. Use the command gpg2 --edit-key KEY-ID to enter the edit prompt. From there, type trust and hit ENTER. The various levels will be shown on screen. Enter and confirm 5, to select ‘Ultimate’. Lastly, use quit to leave the gpg key editor.

Pull Pass Repo

With the keys configured, the last step is to pull down the password-store to the new machine. If using git, this can be done with pass git clone… but if I’m being honest, I usually just do a normal git clone, and then move the folder to ~/.password-store/. If not using git, just copy the store’s directory and files to the new machine. The important thing is that the store can be found at ~/.password-store (by default, this of course can be changed using pass init).

Conclusion

That’s about it. As I previously stated, I’ve been loving pass for years, and I don’t plan to be switching off of it any time soon. At this point, if there is something I want to improve with my password setup… I’m sure the community has already figured out how to do it with pass!

Designing my new Ryzen Workstation

2018-11-26T00:00:00Z

Red Hat Tower, Raleigh NC

Over the past few years (particularly after moving to the T470 as my main device), I have been anticipating what my next desktop workstation build will look like. For a period of time, I was convinced that I would build a muti-socket, used xeon build. Then, in 2017 AMD released their Ryzen series CPUs…

Why a Desktop?

My T470 has been a little tight on resources.

My laptops have served the majority of my needs these last few years, but I have been feeling a bit restricted when trying to do more demanding workloads. While I still side with a “use low-powered, portable computers, and remote into more powerful ones when needed” mentality, I enjoy having one of those power computers being my physical, main workstation desktop.

Workstation Goals

My T470 has been a little tight on resources.

While building a new workstation may be fun, I have particular goals I want it to achieve. Additionally, it seems like 98% of online build guides are for designing gaming desktops… which is not exactly what I want. So, the features I wanted in my build, likely deviate from much of the suggested guidance found online, which tends to focus entirely on gaming performance (more FPS). Here is the breakdown of what I was looking for:

More Cores

I tend to have workloads that benefit from multiple cores (and threads). For example, I run a bunch of virtual machines, often at the same time. While my server’s 4 core Xeon handles day to day VMs just fine, it is nice to have a workstation with several cores to spin up a virtualized cluster across. Beyond VMs, I compile code, compress/decompress file packages, and occasionally encode audio/video files. These are all tasks that love having many threads at their disposal.

Lots of (fast) Ram

When running VMs, the amount of available RAM is usually more limiting than the number of cpu cores. For example, when running 6 VM’s, my server’s 4-core Xeon will typically be 15-35% utilized, whereas it’s 20GB of RAM is nearly full. If I want more cores, I should supplement it with lots of RAM. In addition… I use electron apps (Slack, Spotify, etc.) and modern web browsers… so yea… more RAM.

Being a Ryzen build, the RAM also needs to be fast. More than other CPU architectures, Ryzen CPUs actually run better with faster RAM.

Fast Storage

This main goal of this build is for it to be a multi-tasking beast. If I am running several VMs and a bunch of applications, utilizing a stack of fast RAM to move data around… I need good fast storage with MOAR IOPs to support of these operations, which will all be competing for disk access. While I wouldn’t mind having space to eventually throw in a pair 3.5" rust drives to configure as a ZFS data pool, the goal for the initial primary drive of this machine is the most IOPs I can afford. So, I ideally want a m.2 NVMe SSD, but need a normal 2.5" SATA SSD at the very least.

Some Gaming Ability

While it doesn’t have to be a gaming machine… I would like to be able to play the occasional game. I don’t mind playing games with lower graphic settings on my laptops, but I do mind that even with the lower settings, the CPUs nearly max out, which spikes up the temperature. Increased graphics performance would also better support my large monitor. A discrete graphics card would be nice, ideally an AMD one due to their recent work on improving their open source drivers.

Ability to Upgrade

Most importantly, I want this computer to be a solid base that I can upgrade over the next several years. Upgradability is hands-down the main advantage of switching to a desktop over a laptop for my main rig. I want this build designed so that as my needs grow, I can easily max out the RAM, add storage, or upgrade the CPU and/or GPU.

Final Build Selections

After spending weeks researching all the parts on the market and building probably 50 theoretical builds, I eventually settled on my final part list.

Name: Charmeleon

Since my last desktop build was named “Charmander”, I figured “Charmeleon” was fitting. Also, I’m then free to rename it to “Charzard” in the future if I evolve the build to a next-generation Ryzen 7…

CPU: Ryzen 5 2600

While I initially wanted a Ryzen 2700, it pushed the build out of budget. Coming from a 4 core/4 thread system… it’s likely that I don’t need 8 cores/16 threads right off the bat. Honestly, 4 cores/8 threads would be a noticeable improvement from what I’ve been using… which is why I first considered the Ryzen 5 2400g. I planned to start with the 2400g and upgrade to the 2700 and a GPU once I had more cash. However, someone on the level1tech forums reminded me that the 2600 was the same price as the 2400g, and I could get that plus a GPU for not much more cost (considering I don’t need an expensive GPU). After thinking it over, I realized that was my best option, and should actually fit my needs for quite awhile.

Motherboard: MSI B450 Tomahawk

I had selected the MSI B450 Tomahawk after Wendell reviewed it. Several other reviews confirmed that it was a solid board, especially for the price. When I was considering the 2400g, I had to switch to the MSI B450 Gaming Pro Carbon AC, because the Tomahawk didn’t have a display output that the integrated graphics could properly use with my UHD monitor’s resolution (I need a display port). When I switched to the 2600, I was able to go back to the TOMAHAWK, and funnel the cost savings towards getting my GPU.

GPU: Sapphire 1024 4GB PULSE Radeon RX 560

As discussed above, I don’t need a killer GPU as I don’t play very demanding games. However, I did want something that could perform at least a little bit. So, I picked the rx560 over the rx550. At first I planned to get something like a rx580, which I still might eventually do… but I don’t think I will need to for a long time.

RAM: G.Skill - Trident Z 32GB (2 x 16GB) DDR4-3200Mhz RAM

I wanted to start out with 32GB of RAM, with the ability to easily upgrade to 64GB down the road. This meant that I needed a 2x16GB pack. Additionally, I needed the RAM to be fast, but still not overpriced. Some research showed that at least right now, 3200Mhz seems to be the sweet spot, especially for my particular motherboard. I originally picked another kit, but the reviews for it weren’t great, so I switched to the G.Skill kit for only a few dollars more, but with much better reviews. I also briefly searched for kits with tighter timings, but they were way more expensive and not worth it.

Case: Fractal Design - Meshify C Dark TG ATX Mid Tower

I wanted a case that was sturdy, sleek, but not too flashy or expensive. During my search, the Fractal Design cases kept catching eye. Eventually, I found the Meshify C TG Dark, which seemed to be universally loved by reviewers. Additionally, it appeared to have good thermals, which I liked. This was my easiest part to pick.

PSU: EVGA - SuperNOVA G4 650W 80+ Gold, fully modular ATX PSU

For the power supply, I wanted one that was at least gold rated, and fully modular. So I searched for the highest rated PSUs fitting those requirements… and SuperNova popped up all over the place. At first I picked a 550w model, but upped to the 650w at the last minute because it was about the same price and gives me a little more room as I upgrade.

Storage: One of my Samsung 850 Evo SSDs (temp)

Lastly, storage. For now I am just going to use a spare 250GB Samsung 850 Evo SSD I had in my test laptop. I plan to eventually upgrade to a 500GB Samsung 970 Evo m.2 NVMe drive, but the 850 will work for now.

Conclusion

So after months of planing and research, my desktop parts have all been ordered. Hopefully, it meets the goals I set out to solve, and I will be able to upgrade it over the next few years

Reset Plasma 5 Monitor Configuration

2018-11-14T00:00:00Z

North Carolina Arboretum (Pisgah National Forest), Asheville, NC

Extending a laptop’s screen to an external display can often be a nightmare. Sometimes, it might be best to just hit the reset button and configure from scratch. With KDE Plasma 5, you can. Here’s how.

Issue

While I love having a dock-able laptop and my 43" 4k LG monitor, getting them configured and agreeing with each other can sometimes be a headache. Luckily, KDE Plasma 5 (the desktop environment I’ve been using for awhile now) is fantastic and remembers previous display configurations.

However, when I am playing around with resolutions or multiple inputs, sometimes I set something that doesn’t work… and it is remembered, causing more problems.

The Fix

Luckily, there is an easy fix: just delete the saved profile, and re-setup the display again. However, I often forget these very simple steps… hence, this post (and to help anyone else that needs it, of course :P).

Delete old profile

The first step is to delete the plasma profile that saves the screen settings. Note, this may remove the profiles for all previously configured displays, but I am okay with it for my use case.

cd ~/.local.share
rm -rf kscreen

Extend monitors when docked

The KDE Plasma 5 extend display GUI.

After deleting the config, dock or reconnect the laptop to the display. A GUI with extension layout options should pop up. If it does, select the preferred layout option.

Close laptop lid (don’t disable)

After extending the display, I usually try to make the monitor the primary display and disable the laptop screen. This doesn’t seem to work (It causes my monitor to blank). However, I have learned that if I simply close my laptop lid once it is connected… it does both of those things for me. I always forget this… but now it’s documented!

(Optional) Reset kscreen

After flipping around the displays so much, plasma shell may have gotten a bit frazzled. If so, just reset it using the following command:

kquitapp5 plasmashell && kstart plasmashell

I like to execute this in krunner (alt-F2) so that it runs in the background, and because I can usually still get to krunner, even if something goes terribly wrong with plasmashell during the process.

Conclusion

That’s it. While this post is rather short (for once), hopefully it will be useful. Enjoy!

Monitor Selection - My New LG-UD4329b

2018-11-12T00:00:00Z

After a decade of using several combinations of dual monitors, I have decided to upgrade to using a single, but better, monitor. After two years of debating, I have finally made my choice.

Monitor History

My monitor setup right before moving to college.

Half way through high school, I purchased my first laptop (it also happens to be the first computer I installed Linux on. I had a special edition model, but it was very similar to this one). The laptop was replacing an old standard E-machines desktop, so I kept the spare monitor to use with the laptop. From that point on, I usually had two screens to look at, even if one was just my laptop’s screen at the far end of my desk, with an AIM window open in it. Before moving to college, I built a desktop computer with a new monitor (picture above), and utilized the old one as a secondary display.

My desk during my senior year of collage.

Throughout college, as I started to explore Linux and programming more, I began to experiment with using monitors in portrait orientations. Eventually, I purchased a second 23.6" 1080p monitor so they would match when paired. This gave me much more vertical real estate, which was useful when writing code, essays, or lab reports.

Why go from two, to one?

When using two monitors, I always had to have a primary and secondary.

While I like having the screen real estate of two monitors, I was never able to optimally use them because of the bezels between the monitors. While working, I would naturally shift over and use one monitor as a primary display, and the other as a secondary display. This was okay for some tasks, but it usually just meant that I just used the primary monitor, and half of the secondary one, because it was too hard to see the far end of the screen.

This issue was slightly better when using my displays in portrait mode, but that wasn’t always ideal when connecting to remote machines, or viewing wide content (videos, for example). It also didn’t solve the main issue. When looking at the screens in a natural position (head centered, looking strait forward)… I still saw two pieces of black plastic. I wanted to see my primary window directly in front of me, with some auxiliary space on either side for documentation and terminal windows. I wanted a wide screen display.

Ultra-wide or Large 16:9 UHD?

Luckily, the once extremely expensive ultra-wide and large UHD monitors have started to come down in price. After following the market for a few years, I narrowed my decision to my favorite choice in each category: The Dell u3415w and the LG ud4379b.

The Dell u3415w 34" 3440x1440p Ultra Wide Monitor.

The Dell u3415w is the slightly older (compared to the u3417w) 34" ultra-wide monitor offering from Dell. It has a 3440x1440, 60 Hz, slightly-curved, IPS panel sitting on a solid and adjustable monitor stand. I preferred it over other ultra-wides because it could be found for a reasonable price, and had a professional build quality.

Based on my reasoning explained in the section above, an ultra-wide (1440p ones… not 1080p) would fit my mostly programmer use-case very well. Being a single monitor, I could position it directly in front of me, with my main code/content centered. The 3440px wide resolution would also allow enough horizontal space to have a full web page of documentation, and several terminal windows open on either side of the main display. Lastly, the 1440px vertical resolution would be an adequate improvement from my “starting to feel tight”, 1080p monitors.

The LG ud4379b 42.5" 3440x2160p Monitor.

The massive LG ud4379b is a 42.5" 3840x2160px IPS display. At this size and resolution, it is roughly the equivalent of having four 21" 1920x1080px monitors aligned in a 2x2 grid… except with 0 bezel between them. Compared to other 40"+ 4K displays, I preferred this one because once again, it was reasonably priced, had great input options, and also seemed to have a half decent panel. Reviewers consistently said it had vibrant colors and minimal back light bleed.

Being single monitor, I could easily sit myself in front of it, and have my main content centered, just like with the ultra-wide. Being so large, I could still have auxiliary windows opened on either side. One advantage the ud4379 had over the ultra-wide however, was that being a massive 16:9 screen, it could easily be divided into 4 1080p screens. In fact, this model supports spitting the screen into four separate displays, each taking a different input (more on that later…). While the ultra-wides often support splitting and picture-in-picture, it seems more natural on the 43" monitor.

Deciding on the 43" UHD

The LG ud4379 can support multiple inputs at a time to the single display.

I went back and forth on which monitor I wanted several times. Eventually, I realized the long term versatility of the LG monitor was just something I could not ignore. Being able to use it as one massive display that I could fit several full VM windows in, or even as 1,2, or 4 discrete monitors makes it extremely useful for me. I tend to be playing with more than one computer or VM at a time, and it is VERY nice to have them all fit on a single display. I was worried it would be a bit too large and look stupid on my desk, especially compared to how slick the ultra-wide would look. However, in the end, practicality won out.

Getting the UD4379b

The Monitor arrived a few days later perfectly safe, despite the beat up box.

After deciding that we would likely order the monitor as my birthday gift, my wife and I started watching the price a few weeks before. One day, we noticed the price had dropped $200… so we ordered it. The price soon went back up, so I’m glad we snatched it when we did!

It came a few days latter in a massive box that I happily carted up to our apartment. The box was a bit beat up, but after opening it, all the components were fine.

What I like

I can have my main content front and center, with other windows on the sides.

Screen Resolution - The most obvious pro using a monitor at this size and resolution… is the screen real estate. As explained above, it’s twice the space of my old, dual 1080p monitor setup. With this space, I can easily have several source code files, documentation, a few terminals, a chat window, and even a video opened on my single screen as I work.
IPS Display - Many people might assume that the driving factor pushing me to finally update my monitors was my desire for more screen resolution. While that was part of my goal… what truly motivated me was a desire to switch to an IPS display on my main setup after almost a year using my T470’s IPS display (which by the way, is not a good IPS panel). As someone who spends a large percentage of their computing time in dark terminals and IDEs, an IPS panel really does help text and colors to pop, because the darks look so much better. It turns out, this panel is actually quite good for it’s price, and I am very happy with it.

When working, I can look straight at a single display and still have enough room.

Single Monitor, No (center) Bezels - As explained earlier in this post, I like that this is is a single monitor. This means that I no longer need to twist my head back and forth, or stare at bezels. I can looks straight ahead and see my main content, with “auxiliary” windows view-able in my periphery.
Flexibility, multiple inputs - Lastly, I enjoy the multiple inputs and excellent PIP system of this monitor. Combined with something like Synergy, I can easily use this one display with several devices at a time. This is great when working with more than one device, or potentially watching content on something like a chromecast stick, while still working on the computer. It really is the advantage of having 4 monitors, but in a single package.
Speakers - They aren’t amazing, and I don’t use them too often too often, but for “monitor” speakers, they really aren’t half bad. They’re what you’d expect from a TV. However, this means that if I were to use something like a media stick, the built-in speaker is acceptable enough for such a device.

What I don’t like

Viewed from the center, the monitor edges have shadowing.

Edge Shadowing - The one issue that myself and others have experienced with this monitor is a sort of “edge shadowing”. This is apparently something that is common on monitors this massive, and it’s really hard to engineer around. If I sit back far enough from the display, you can’t see it anymore. (But I don’t normally sit like that)
Input Switching Lag? - While this is more of a critique of monitors in general, one thing I wish was a bit better was the input switching. Having so many inputs, it can get confusing remembering which one is attached to what device, or trying to quickly configure a 2x2 multi-input layout. Having to wait for each device to connect when flipping through all the inputs makes it that much more frustrating (and disappointing when it’s the wrong input…).
Corners Out of View - Being such a large display, one issue I sometimes have is that the outside corners are out of my field of view. I mostly notice this when I occasionally miss notification pop-ups that I want to see. On the flip-side, notifications don’t seem to distract me as much now when I’m trying to focus :P.

Conclusion

New monitor on my desk.

In conclusion, I an very happy with my new monitor. It fits all of the criteria I had, and works for my use-case perfectly. While I wouldn’t recommend it for a pure gaming machine, it might be one of the best deal productivity monitors out there.

Creating a CI/CD 'Draft' Website with Jenkins (and Hugo)

2018-08-15T00:00:00Z

American Tobacco Campus, Durham NC

The last few months I have been working more with the open source automation server, Jenkins. While digging into it, I have been thinking of ways to improve my home build environment. One idea, was to utilize Jenkins to automatically build and deploy a “draft” website, so I can stage new posts/website change on my home network, before publishing it to the “production” website. Here is how that idea was Instantiated…

My Website

I have previously described how my website is currently generated, using the Hugo static website generator. To organize this system, I have two git repos: One that consists of all the hugo source files (where I write content), and one that contains the generated static website (that gets deployed to my web host).

When writing a post, I use hugo server -D -F to live view the page in my browser. However, I occasionally want to view the state of all the committed code in the repo, to see what the site would look like if I decided to publish a post. So, I created a “drafts” website, which shows the current state of my website’s source repo (including draft and future posts). If I want to check how a post looks on my phone, or any other device, I can just open up the draft website after pushing my changes.

Jenkins

I had previously created a dedicated Jenkins server on my home network (Mr. Mime), using a CentOS 7 VM hosted on my home server. However, any Jenkins setup should work for this project (including a docker container). To get started, checkout the Jenkins Website, and be sure to take advantage of the the documentation for help.

Note: just make sure hugo is installed on the Jenkins server, as we need it to generate the website.

GitHub Integration

Jenkins Service

My website repo is hosted on GitHub, so we need to configure it to work with our Jenkins server. To do that, go to the project’s GitHub page, and navigate through Settings -> Integrations & services. Click the Add service drop-down and select Jenkins (Git Plugin). Next, add the Jenkins server url (assuming the server is accessible from the internet. If not, hosting the Jenkins server on something like Digital Ocean might be an easy solution). Lastly, make sure the Activate box is selected, and click the Add Service button.

SSH Keys

While on the project’s GitHub page, make sure that the Jenkins server’s ssh keys are added to the project. To add them, navigate to the Deploy Keys page (under the project’s Settings tab). Then select Add deploy key, and add the public key.

A Nginx Server

The default nginx page.

With Jenkins ready, let’s quickly setup the web server before configuring the Jenkins project. Any web server will do (it just needs to serve the generated static website content). I used nginx in for setup. After installing, make sure it is running. To install and check the status of nginx on an Ubuntu System:

## Install
sudo apt install -y nginx

## Check it is running
sudo systemctl status nginx

## Optional: Ensure it is enabled to start up after reboots
sudo systemctl enable nginx

With the web server running, we need to know where the website files need to go. Nginx will by default serve content at /user/share/nginx/html/, so remember that location for later…

Note: Don’t forget to add the jenkins user’s ssh key from the jenkins server to the authorized_keys file of the nginx server. This will make file transfers easier when setting up the jenkins project.

Configuring a Project

Let’s configure our Jenkins project! Log in to the Jenkins server and click the New Item option on the left side bar. Enter a name for the project, select the Freestyle Project option, and hit OK.

The Project's General Configuration Section.

In the General section of the configuration screen, optionally write a description about the project. Next, select the “GitHub Project” check-box, and add the GitHub repo’s url into the Project url text box.

Source Control

Setting Credentials.

In the Source Code Management section of the configuration, select the Git option. Then, enter the repo’s url for the Repository URL box (I did the ssh url). For Credentials, select Add to configure a new credential. Select SSH Username with private key for Kind, use jenkins for the Username.

More source control options can be configured, but this should be the minimum setup required. Again, for this to work public keys for the jenkins user on the jenkins server must be generated, and added as a deployment key on GitHub.

Build Trigger

Under the Build Triggers section, select Poll SCM. Without adding any schedule parameters, it will trigger each time a new commit is detected. This is what we want.

Build Step

In the Build section, click Add build step, and select Execute shell. This is where we can add the shell commands to build the website with hugo. Add the following command to the box (don’t forget to change the url):

hugo -D -F -b "http://10.1.1.77" -d public

The -D flag tells hugo to include all draft posts, while the -F flag has it include all posts with a future date. The -b flag sets the url for the generated website. This should the be url or IP address of the nginx server setup previously. Lastly, the -d flag tells hugo to output the generated static website to the public directory. This will be useful to know when deploying the build.

Deploy to Webserver

For deployment, I used rsync to copy the build files to the nginx web server. This step will be another shell command, so I’ve actually added it as another “build” step. Add another Execute shell and paste the following command inside the text box (again, changing the url):

rsync -r "$WORKSPACE/public/" ryan@10.1.1.77:/usr/share/nginx/html/

I used the Jenkins $WORKSPACE variable to get the location of the build, and was able to append the public directory to that, since we defined it with the -d flag in the hugo build step above. This will copy the generated website, to the web server.

Hit Save, and test it out by clicking the Build Now link on the left. If the build is successful, check the nginx website to see if the website was deployed!

Note: If it doesn’t work, double check all permissions and credentials between accounts and servers.

Better Yet… Pipelines

What’s better than using Jenkins for automated “draft website” deployments? Using a Jenkins Pipeline. A Pipeline allows the jenkins project steps to be defined in a Jenkinsfile that, among other benefits, can be source controlled. In fact, by default a Jenkins pipeline searches for the Jenkinsfile right in the root directory of a project’s git repo.

While a pipeline and Jenkinsfile might be a bit more confusing to learn how to setup, it is well worth it. For example, the following Jenkinsfile can be used to do essentially what we setup in the previous steps:

pipeline {
    agent {
	label 'mr-mime'
    }
    stages {
	stage ('build') {
	    steps{
		sh 'hugo -D -F -b "http://10.1.1.77"'
	    }
	}
	stage ('deploy') {
	    steps{
		sh 'rsync -r "$WORKSPACE/public/" ryan@ponyta:/usr/share/nginx/html/'
	    }
	}

    }
}

I’m not going to cover pipelines in this post. However, I do encourage readers to check them out.

Conclusion

That’s it. While I currently host my website using GitHub pages, if I ever self-host it again, I will definitely automate publishing it using Jenkins as well. This has been a very basic example of what Jenkins can be used for, but I have found it rather useful when working on the content of this website. There is so much more it can do. Have fun!

Nginx as a Reverse Proxy to Forward Sub-Domains

2018-07-03T00:00:00Z

Farmers Market, Foster Street, Durham NC

Last month, Rebecca and I moved to Durham, North Carolina. As a result, I’ve had to re-setup our home network. In years past, I setup a virtual machine running Nginx as a reverse-proxy (Tangela), and I decided to do that again on the new network. While it is a simple process, it is one that other people often ask me about. So, this time… I’m taking notes!

Why?

The purpose of this reverse proxy is to direct outside traffic to the appropriate host internally, by looking at the sub-domain of the URL request. For example, I may have servers for both website.himmelwright.net and dashboard.himmelwright.net running internally on my network, but they will have the same public IP. Using nginx, I can point all of my web traffic to tangela, my reverse-proxy. If tangela sees that the incoming request is for website.himmelwright.net, it will forward that traffic to the website server. On the other hand, if the request is for dashboard.himmelwright.net, it will direct it to the dashboard server. A reverse-proxy expands what can be accomplished on a single network, and is a cleaner (and possibly safer) method than doing everything through port-forwarding.

Setup Server

Fitting my home network naming convention, my reverse-proxy VM is named 'Tangela'.

To get started, configure a server/container/droplet that will host nginx. I’m using a CentOS 7 minimal install VM on Nintales (my home server). I don’t have a bunch of traffic (well, I shouldn’t), so I’m just giving it 1 core and 512MB RAM.

Setup Nginx

Next, it’s time to setup and install Nginx.

Note: the rest of the post will be focused on using a CentOS 7 base, since that is what I am using. Adjust for your distro accordingly.

Add the nginx repo, and install it:

sudo rpm -Uvh http://nginx.org/packages/centos/7/noarch/RPMS/nginx-release-centos-7-0.el7.ngx.noarch.rpm
sudo yum -y install nginx

Tell the firewall to allow http traffic:

sudo firewall-cmd --permanent --add-service=http
sudo systemctl reload firewalld

Start and enable nginx:

sudo systemctl start nginx
sudo systemctl enable nginx

Configure Nginx

Examine the config file, just to check that everything looks alright.

vim /etc/nginx/nginx.conf

Specifically, we want to ensure that the following line exists before proceeding:

include /etc/nginx/conf.d/*.conf;

That line basically states that any *.conf file inside the /etc/nginx/conf.d/ directory will also be loaded and used by nginx. This allows us to add our own configuration file in the next step.

Configure Proxy

I created a reverse-proxies.config (it can be named anything with a .config extension) file in /etc/nginx/conf.d/ to contain all of the reverse proxy definitions. These are just server block entries. For example:

server {
        listen 80;
        server_name website.himmelwright.net;
        location / {
                proxy_pass      http://192.168.1.198:80;
        }
}

server {
        listen 80;
        server_name dashboard.himmelwright.net;
        location / {
                proxy_pass      http://192.168.1.200:8080;
        }
}

Restart nginx for the changes to take effect:

sudo systemctl restart nginx

Side Note:

For some applications, you may need to add the URL to the /etc/hosts file, and use that for nginx. I have experienced this in the past with Gitlab. For example:

/etc/hosts
---
192.168.1.201  git.himmelwright.net

and then in the config file:

/etc/nginx/config.d/reverse-proxies.config
---
server {
        listen 80;
        server_name git.himmelwright.net;
        location / {
                proxy_pass      http://git.himmelwright.net:80;
        }
}

SELinux Fixes

At this point, you may be done. However, I was having issues getting nginx to forward some of my ports… until I remembered that I was on CentOS and it may be an issue with SELinux. It was.

One “fix” is to just disable SELinux. A better solution is to use setools to allow the http connections:

sudo yum install -y setools
setsebool -P httpd_can_network_connect true

Conclusion

Besides having to configure your router to forward http traffic to the server, that is really it. This is a real basic configuration, but it has worked well for me over the years. If I start doing something more complex, I may provide an update. Until then, enjoy!

Organizing my Emacs config with Org-Babel

2018-06-27T00:00:00Z

Golden, CO

In my previous post, I completely redid my emacs configuration from scratch, building it around the use of evil mode and use-package. As I was wrapping up, I learned of yet another emacs package that wil forever change how I maintain my emacs configuration… org-babel.

Org Babel

Org-babel is an emacs package, that lets take an org-mode file, and evaluates all of the code blocks contained within it. This means I can write an annotated org file, filled with emacs-lisp code blocks, and export just the emacs-lisp code. After testing it out, I realized that the main emacs lisp file I use is… my .emacs file…

Setup

Installing babel couldn’t be simpler. If you are running Emacs version 24 or higher, and a current version of Org-mode, Babel is already available by default. In order to generate an .emacs config from an .org file, we need to first setup a config.org file, and then tell .emacs to load it with org-babel.

config.org

First, start by creating an .org file to use for all of the configuration content. This can be named anything, and exist anywhere, but I like to keep mine in the emacs section of my dotfiles repo, named config.org. This file functions like any other org file, so I added a small header section at the top… because why not?

#+TITLE: My Emacs Configuration
#+AUTHOR: Ryan Himmelwright
#+EMAIL: ryan@himmelwright.net
#+OPTIONS: num:nil

After that, I broke down my file by creating org headings for the different groups of the configuration. For example, I started with the headings Repos & Core Packages, Core Setup, Evil Mode, Ivy, Writing, Development… etc.

Note: In org-mode, use “*” to create headings. Each contiguous “*” corresponds to the equivalent HTML heading level. Ex: * == <h1>, ** == <h2>, and so on.

I started transferring the emacs-lisp code from my previous .emacs file, to under each heading in config.org. As I transferred text, I transformed the comments that described each code snippet into normal org text, and wrapped the emacs-lisp inside an org code block.

Example snippet of my org-babel config.org file.

To create a code block, use the #+BEGIN_SRC emacs-lisp and #+END_SRC org tags to encapsulate the emacs-lisp code. Continue to do this until all of the desired emacs-lisp code is contained inside org code blocks.

Side Note: Easy Org Code Blocks

Code block can be easily created with `

In org-mode, you can write <s, then hit TAB and it will expand into the BEGIN_SRC and END_SRC tags. Just don’t forget to add emacs-lisp to the BEGIN_SRC tag!

.emacs

For the contents of .emacs, call package-initialize and then have org-babel load the config.org file.

(package-initialize)
(org-babel-load-file "~/dotfiles/emacs/config.org")

That’s it. Assuming the config.org is complete, emacs should now initialize using the code snippets from it. While these are the only two required lines, do note that emacs will still write the #'custom-set-variables to the bottom of the .emacs file. That’s fine. If anything, it makes it easier to source control the actual configuration file, since emacs isn’t constantly adding to it, changing the diff.

What I like

Organized

Expanding org headers to easily find code.

Org mode’s hierarchical structure inherently organizes the content of files. Using it for my emacs configuration is no different. I can use org headers to easily break down my file into sections and sub-sections, which I can then expand and collapse as needed. For example, my config.org is currently over 500 lines long, but with all the headers collapsed, it displays in less than 20. From there I can just expand to the section I need.

Maintainable

Being so organized, the config.org file is very easy to maintain. If I want to edit a setting, I can just search through the headers for the section, and then edit the code block and/or text. If I want to add a new item, I can just insert a new header, add the code block I want, and maybe some explanatory text. Done. Most importantly, the structure helps prevent it from turning into an in-production “scratch code” file…

Easy to Read

It should go without saying that the organization and maintainability of org-mode configurations make them extremely readable. Not only is the source code easier to read, but org files can be exported to all sorts of outputs (HTML, LaTeX, OpenDocument, etc). Combined with a style-sheet, these outputs can look quite professional.

Github renders org files as a markdown.

Beyond normal org exporting, Github does something awesome… they render org files as a known markup language on their website! This means that if you click an *.org file on Gihub’s web interface, it will display a rendered version of the content, instead of defaulting to the raw org text.

Speaking of Github…

While I have always maintained my emacs configuration in my dotfiles repo, org-babel has helped me step up my game. My current dotfile system has all of my emacs files in a separate emacs folder. On Github, each directory in a repo can contain a README file (or… a symlink to another file…) to be displayed below the list of files.

My Maintained Config

Github renders org files as a markdown.

In the emacs section of my dotfiles, I have created a symlink, README.org, to my config.org file:

sudo ln -s config.org README.org

Github recognizes this, so my README file “contains” whatever the contents of config.org are. Now, when one visits the emacs section of my dotfiles repo, an organized, annotated, and always up-to-date version of my “.emacs” file is displayed as the README.

The Github support was the cherry on top of a new emacs configuration system I am already ecstatic about. I’ve been a fan of org-mode for years, and org-babel is one more feature to add to the growing list of reasons why I love it. I highly recommend checking it out.

Emacs Config Redo - Evil & Use-Package

2018-05-18T00:00:00Z

Roger Williams Park, Providence RI

After switching to Spacemacs for the last year or two, it’s about time to back and pull together my own emacs configuration again. However, spacemacs has shown me several packages that I want to incorporate into my new emacs setup. Rather than resurrect and Frankenstein the changes into my old .emacs file… I’m starting from scratch.

A window of my New Emacs configuration.

Evil Mode

History

The one emacs package that spacemacs really got me addicted to was Evil, which emulates the main features of the vim text editor. I started with emacs in college, but eventually switched to Vim which, became the first power editor that I really got into (I even bought a book to learn it better). I stuck with Vim until I became a professional LISP developer, and the switch back to Emacs was impossible to ignore, and obvious.

Even after switching to back Emacs, I have still enjoyed using command line applications like cmus, ranger, and w3m, many of which are influenced by Vim and often incorporate similar keyboard commands (at least the j, k, l, ; navigation). While it isn’t ideal for everything (I prefer to write, but not necessarily edit code with emacs-syle navigation), I never lost my love for the homerow-centric, and efficient vim-style movement commands.

Best of Both Worlds

For the last two years, Spacemacs’s default Vim configuration has provided the best of both worlds. It had all of the Evil packages pre-configured and optimized, but I could simply hit Ctrl-z, and jump back to Emacs-mode.

Spacemacs was easy to use and I enjoyed it, but it really started to have stability issues on my Windows 10 work computer. I thought that going back to a stock emacs configuration may tone down the complexity, and increase stability. I started to build a new configuration based around Evil. During setup, I was happily surprised to learn that the Ctrl-z option to switch back to Emacs-mode was a default Evil behavior… so I still have the best of both worlds!

Starting Config

Here is my starting Evil setup. I’ve nested other use-packages inside of it, so that if evil is configured, the packages that depend on it go ahead and configure themselves too. I’m sure this will grow over time as I add missing packages, but so far, it seems to provide all the core functionality I need.

;; Evil Mode
(use-package evil
  :ensure t
  :config

  (evil-mode 1)
  (use-package evil-leader
    :ensure t
    :config
    (global-evil-leader-mode t)
    (evil-leader/set-leader "<SPC>")
    (evil-leader/set-key
      "s s" 'swiper
      "d x w" 'delete-trailing-whitespace))

  (use-package evil-surround
    :ensure t
    :config (global-evil-surround-mode))

  (use-package evil-indent-textobject
    :ensure t)

  (use-package evil-org
    :ensure t
    :config
    (evil-org-set-key-theme
	  '(textobjects insert navigation additional shift todo heading))
    (add-hook 'org-mode-hook (lambda () (evil-org-mode))))

  (use-package powerline-evil
    :ensure t
    :config
    (powerline-evil-vim-color-theme)))

I think it needs more key bindings over time…

Use Package

After my setting up an initial install of the Evil mode parts, I discovered the amazing use-package. I have come across it before reading blog posts, but never actually tried it. After realizing how well it worked, I immediately combed through my .emacs file, converting all of my (require 'package-name) calls to use-package forms instead.

Setup

After ensuring the MELPA repos are added and initialized:

;; Add Melpa packages to Repos
(require 'package)
(add-to-list 'package-archives '("melpa" . "http://melpa.org/packages/"))
(package-initialize)

The use-package package can be installed next. I wrapped the install commands in an unless, so that when my emacs loads, it only installs use-package if it is not already installed.

Afterwards, make sure to (require 'use-package). It only needs to happen during compile, so I have mine in an (evail-when compile ...).

(unless (package-installed-p 'use-package)
  (package-refresh-contents)
  (package-install 'use-package))

(eval-when-compile
  (require 'use-package))

I keep all of this at the top of my configuration, which works well even when initializing a new computer.

Bye Bye emacs_init.el

When I last had my own custom emacs config, I had to also maintain an emacs_init.el file. This was an emacs-lisp script that used a mapcar to apply #'package-install across an ever-growing list of emacs packages my configuration required. My goal was to have a script, so that when configuring Emacs on a new system, I could just load and evaluate the contents of emacs_init.el, and everything required for .emacs would automatically install.

The reality was that it never fully worked. There were always a few packages that would error, or that I had forgotten to add the last time I updated my .emacs file. With use-package, this might be an issue of the past, as it allows me to combine my emacs init script with my configuration files.

Key Bindings

After converting all of my (require 'PACKAGE-NAME) calls and related expressions to filled use-package wrappers, I learned about the :bind parameter. Instead of manually setting key binds with a setq, :bind takes a list of dotted pairs and binds the function (defined in the second spot of the pair), to the key sequence stated in the first spot of the pair.

For example, to setup my preferred ispell key-bindings, I used the following :bind parameter in my ispell use-package call:

(use-package ispell
  :ensure t
  :bind
  (("C-c w" . 'ispell-word)
   ("C-c r" . 'ispell-region)))

This helps to keep the configuration a bit cleaner and organized. The syntax is also straight-forward and easy to remember.

System Specific Load Files

Finally, I moved all my work-specific emacs configuration (setting up Allegro Common Lisp, defining some helper functions… and anything Windows specific) into it’s own emacs-work.el file. With that in it’s own separate file, I just needed to load it from my main .emacs configuration. The only issue with that, is I only want it to load on my work computer. So, I wrapped it with a nice little handler:

(when (string-equal system-name "LAFAYETTE")
  (load "~/.emacs-work.el"))

This works because I don’t name my home computers with the same name as my work machine. This little tweak worked so well, that I decided to make another file, emacs-linux.el, that I could dump my Linux and/or home specific configuration into. I only load it when on a GNU/Linux machine:

(when (string-equal system-type "gnu/linux")
  (load "~/.emacs-linux.el"))

I’ve really enjoyed the ability to only load parts of my configuration when a certain condition is met. Breaking down my configuration into use-specific components seems like a good idea (like abstracting messy code to smaller functions). Right now, I try to keep my configuration file partitioned into sections, based on use-case (Writing, Development, Appearance, etc). However, as I continue to fine-tune my emacs setup, I think I might break those sections into actual files, and then load them from the main config.

The only issue I can see with that is that it can be confusing with overlapping use-cases. However, I already have to deal with that in a single configuration (which section to put it in), and my process of converting everything to use-package has actually started to minimize that issue. So it might work…

Next Steps

Now that I have an “Evil” Emacs setup configured, things should be returning to business as usually. As I work, I sure there’ll be a forgotten package here or there that needs to added. However, with use-package, that should be a piece of cake, and easy to maintain from here on out.

SQL Intro

2018-04-24T00:00:00Z

Ross School of Business - University of Michigan, Ann Arbor, MI

SQL is one of those technologies that which to be everywhere, but yet… I somehow haven’t had to use it for anything in school or at work. I know, I know… it’s quite a feat. Still, the pervasiveness of SQL-like databases argues that I really should learn it. So I am.

The main purpose of this website is to document some of the tech stuff I do in my free time. This serves three purposes: 1) Organizing my thoughts into a post enhances my learning, 2) I can easily refer back to the post to refresh my memory in the future, 3) It’s a good medium to share what I’ve learned with others.

This post is a prime example, as it didn’t start out as a post, but a file of the notes I took as I learned SQL basics. As I progressed, I realized all of the accumulating information should be cleaned up and posted. So here we are. I’ve redone the examples, and turned my shorthand blurbs into sentences, but if this post still seems a bit different than previous ones (no images, more code snippets than words)… that’s because it’s my learning notes with makeup heavily applied. Enjoy.

Install Setup

I first installed mysql, by going to the mySQL download page and getting the appropriate the version for my VM (centos 7). This may differ based on Distro/DB.

I’m not going to cover any of the installation steps, as this post is more about the SQL language, not setting up the DB. Besides, I did this in mysql, but in the future I’ll likely look at using MariaDB anyway…

I do however want to note this issue I encountered, so I don’t stumble over it in the future. At first, I couldn’t get the /usr/bin/mysql_secure_installation command to run due to a permission denied error…

Eventually, I learned that the first time mysql runs, it creates a temp password in the log, located at /var/log/mysqld.log. Using that password, I was able to login.

DBs and Commands:

Conventionally, SQL commands are typed in all CAPS, and statements end with a terminating ;. The rest of this post contains some SQL functions, with examples of how they are used.

Show databases

To display all of the available databases, use the SHOW DATABASES; command. Note, some of the DBs displayed are ones generated for the database system. For example, in the output below, I only created dbCustomerInfo and dbTest:

mysql> SHOW DATABASES;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| dbCustomerInfo     |
| dbTest             |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
6 rows in set (0.00 sec)

Create database

To create a new database, use the CREATE DATABASE command. Simple.

mysql> SHOW DATABASES;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| dbCustomerInfo     |
| dbTest             |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
6 rows in set (0.00 sec)

mysql> CREATE DATABASE shotLivedDB;
Query OK, 1 row affected (0.00 sec)

mysql> SHOW DATABASES;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| dbCustomerInfo     |
| dbTest             |
| mysql              |
| performance_schema |
| shotLivedDB        |
| sys                |
+--------------------+
7 rows in set (0.00 sec)

Drop a database (delete):

As easy as it was to create that database, deleting, or dropping it is just as effortless with the `DROP DATABASE command, so be careful!

mysql> DROP DATABASE shotLivedDB;
Query OK, 0 rows affected (0.00 sec)

mysql> SHOW DATABASES;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| dbCustomerInfo     |
| dbTest             |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
6 rows in set (0.00 sec)

Select DB

To select which db to use inside the mysql shell use… USE. For this tutorial, I created a database named dbTest and selected it with USE dbTest.

Tables

Note: make sure to first select a DB to work with: USE dbTest;…

Show Tables

To show all of the tables in a database, use SHOW TABLES;. For example:

mysql> SHOW TABLES;
+------------------+
| Tables_in_dbTest |
+------------------+
| tblUsers         |
+------------------+
1 row in set (0.00 sec)

Create Table

To create a new table, use CREATE TABLE. The CREATE TABLE function takes

the name of the new table, and 2) a list of the table fields (with their data types). For example, to create a user information table that contains a user’s first name, last name, age, and state, as well as an identification number, the following SQL command can be used:

mysql> CREATE TABLE tblUsers (id int PRIMARY KEY AUTO_INCREMENT, firstname varchar(50),lastname varchar(50), age INT,state varchar(2));
Query OK, 0 rows affected (0.01 sec)

The first name, last name, and state columns have a varchar data type, which are strings of various sizes (50 and 2 characters in this case). The id and age columns have an int data type. Notice that the id column has some other junk defined after the int identifier…

Constraints and Fields

In addition to specifying data type, other constraints can be imposed on columns when defining a new table. Constraints are used to limit the type of data that goes into the table, and can be implemented at the column or table level. To see the fields of a table, use the SHOW FIELDS FROM tablename command:

mysql> SHOW FIELDS FROM tblUsers;
+-----------|-------------|------|-----|---------|----------------+
| Field     | Type        | Null | Key | Default | Extra          |
+-----------|-------------|------|-----|---------|----------------+
| id        | int(11)     | NO   | PRI | NULL    | auto_increment |
| firstname | varchar(50) | YES  |     | NULL    |                |
| lastname  | varchar(50) | YES  |     | NULL    |                |
| age       | int(11)     | YES  |     | NULL    |                |
| state     | varchar(2)  | YES  |     | NULL    |                |
+-----------|-------------|------|-----|---------|----------------+
5 rows in set (0.00 sec)

The id column uses the PRIMARY KEY and AUTO_INCREMENT constraints. PRIMARY_KEY is a combination of the NOT NULL and UNIQUE constraints, meaning it ensures that all values in the column are unique and not NULL. The AUTO_INCREMENT field generates an unique number that is automatically incremented during each insert to the table.

INSERT INTO

To actually add data to the table, the INSERT INTO command is used. As an example, to add some users to the table created in the previous step:

mysql> INSERT INTO tblUsers (firstname,lastname,age,state) 
VALUES ('Joe','Fry',32,'RI');
Query OK, 1 row affected (0.01 sec)

mysql> INSERT INTO tblUsers (firstname,lastname,age,state) 
VALUES ('Emily','Flanders',22,'CA');
Query OK, 1 row affected (0.01 sec)

mysql> INSERT INTO tblUsers (firstname,lastname,age,state) 
VALUES ('Tina','Oak',42,'NC');
Query OK, 1 row affected (0.00 sec)

mysql> INSERT INTO tblUsers (firstname,lastname,age,state) 
VALUES ('Bob','Builder',51,'MO');
Query OK, 1 row affected (0.01 sec)

The tblUsers table should now contain the information of the 4 users added. To check this, use SELECT * FROM tblUsers; to select everything from the tblUsers table:

mysql> SELECT * FROM tblUsers;
+----|-----------|----------|------|-------+
| id | firstname | lastname | age  | state |
+----|-----------|----------|------|-------+
|  5 | Joe       | Fry      |   32 | RI    |
|  6 | Emily     | Flanders |   22 | CA    |
|  7 | Tina      | Oak      |   42 | NC    |
|  8 | Bob       | Builder  |   51 | MO    |
+----|-----------|----------|------|-------+
4 rows in set (0.00 sec)

INSERT INTO Another Table

The INSERT INTO command combined with SELECT FROM, can insert contents of one table into another. This technique can be quite useful, providing a simple way to create quick backups.

mysql> CREATE TABLE tblUsersBackup (id int PRIMARY KEY AUTO_INCREMENT, firstname varchar(50),lastname varchar(50), age INT,state varchar(2));
Query OK, 0 rows affected (0.02 sec)

mysql> SELECT * FROM tblUsersBackup;
Empty set (0.00 sec)

mysql> INSERT INTO tblUsersBackup SELECT * FROM tblUsers;
Query OK, 4 rows affected (0.00 sec)
Records: 4  Duplicates: 0  Warnings: 0

mysql> SELECT * FROM tblUsersBackup;
+----|-----------|----------|------|-------+
| id | firstname | lastname | age  | state |
+----|-----------|----------|------|-------+
|  5 | Joe       | Fry      |   32 | RI    |
|  6 | Emily     | Flanders |   22 | CA    |
|  7 | Tina      | Oak      |   42 | NC    |
|  9 | Bob       | Builder  |   51 | MO    |
+----|-----------|----------|------|-------+
4 rows in set (0.00 sec)

Drop (delete) Table

To delete a table, the DROP command is used. For example, to delete the backup table from above, issue DROP TABLE tblUsersBackup;:

mysql> DROP TABLE tblUsersBackup;
Query OK, 0 rows affected (0.01 sec)

mysql> SELECT * FROM tblUsersBackup;
ERROR 1146 (42S02): Table 'dbTest.tblUsersBackup' doesn't exist

NOTE: Using DROP commands will delete the entire structure (including the schema), even if there is data in it. Be Careful!

SELECT

In addition to selecting all of the data in a table using SELECT *, The SELECT command can be utilized to select all sorts of combinations of the data. Here are a few examples:

– Select specific columns from the table:

mysql> SELECT firstname,lastname FROM tblUsers;
+-----------|----------+
| firstname | lastname |
+-----------|----------+
| Joe       | Fry      |
| Emily     | Flanders |
| Tina      | Oak      |
| Bob       | Builder  |
+-----------|----------+
4 rows in set (0.00 sec)

– Grab column(s), after matching in another:

mysql> SELECT firstname,lastname FROM tblUsers WHERE state="RI";
+-----------|----------+
| firstname | lastname |
+-----------|----------+
| Joe       | Fry      |
+-----------|----------+
1 row in set (0.00 sec)

In addition to searching using = (which only grabs exact matches), other operators, such as >, <, <=, >=, <> (not equal), BETWEEN (between an inclusive range), and LIKE (search for pattern) can be used with the WHERE clause. For example:

mysql> SELECT firstname,lastname FROM tblUsers WHERE age>=25;
+-----------|----------+
| firstname | lastname |
+-----------|----------+
| Joe       | Fry      |
| Tina      | Oak      |
| Bob       | Builder  |
+-----------|----------+
3 rows in set (0.00 sec)

mysql> SELECT firstname,lastname FROM tblUsers WHERE age BETWEEN 30 AND 50;
+-----------|----------+
| firstname | lastname |
+-----------|----------+
| Joe       | Fry      |
| Tina      | Oak      |
+-----------|----------+
2 rows in set (0.00 sec)

ALTER

To change the table schema (add, drop or modify columns), the ALTER command is used along with ADD, DROP COLUMN, or MODIFY COLUMN (respectively) after the tablename. For example:

– Add a born column to the table

mysql> SELECT * FROM tblUsers;
+----|-----------|----------|------|-------+
| id | firstname | lastname | age  | state |
+----|-----------|----------|------|-------+
|  5 | Joe       | Fry      |   32 | RI    |
|  6 | Emily     | Flanders |   22 | CA    |
|  7 | Tina      | Oak      |   42 | NC    |
|  9 | Bob       | Builder  |   51 | MO    |
+----|-----------|----------|------|-------+
4 rows in set (0.00 sec)

mysql> ALTER TABLE tblUsers ADD born year;
Query OK, 0 rows affected (0.07 sec)
Records: 0  Duplicates: 0  Warnings: 0

mysql> SELECT * FROM tblUsers;
+----|-----------|----------|------|-------|------+
| id | firstname | lastname | age  | state | born |
+----|-----------|----------|------|-------|------+
|  5 | Joe       | Fry      |   32 | RI    | NULL |
|  6 | Emily     | Flanders |   22 | CA    | NULL |
|  7 | Tina      | Oak      |   42 | NC    | NULL |
|  9 | Bob       | Builder  |   51 | MO    | NULL |
+----|-----------|----------|------|-------|------+
4 rows in set (0.00 sec)

– Change born column from a year data type to a date

mysql> ALTER TABLE tblUsers MODIFY COLUMN born date;
Query OK, 4 rows affected (0.03 sec)
Records: 4  Duplicates: 0  Warnings: 0

– Drop the born column from the table

mysql> SELECT * FROM tblUsers;
+----|-----------|----------|------|-------|------+
| id | firstname | lastname | age  | state | born |
+----|-----------|----------|------|-------|------+
|  5 | Joe       | Fry      |   32 | RI    | NULL |
|  6 | Emily     | Flanders |   22 | CA    | NULL |
|  7 | Tina      | Oak      |   42 | NC    | NULL |
|  9 | Bob       | Builder  |   51 | MO    | NULL |
+----|-----------|----------|------|-------|------+
4 rows in set (0.00 sec)

mysql> ALTER TABLE tblUsers DROP COLUMN born;
Query OK, 0 rows affected (0.04 sec)
Records: 0  Duplicates: 0  Warnings: 0

mysql> SELECT * FROM tblUsers;
+----|-----------|----------|------|-------+
| id | firstname | lastname | age  | state |
+----|-----------|----------|------|-------+
|  5 | Joe       | Fry      |   32 | RI    |
|  6 | Emily     | Flanders |   22 | CA    |
|  7 | Tina      | Oak      |   42 | NC    |
|  9 | Bob       | Builder  |   51 | MO    |
+----|-----------|----------|------|-------+
4 rows in set (0.00 sec)

NOTE: Depending on the type of SQL database being used, the MODIFY command may be known as ALTER.

Delete

All the data in a table can be removed using DELETE * FROM tblName, without deleting the table schema. However, this is almost never used because there are better commands to do that.

Specific items from the table can be removed using DELETE FROM commands:

mysql> SELECT * FROM tblUsersBackup;                                                                    
+----|-----------|----------|------|-------+
| id | firstname | lastname | age  | state |
+----|-----------|----------|------|-------+
|  5 | Joe       | Fry      |   32 | RI    |
|  6 | Emily     | Flanders |   22 | CA    |
|  7 | Tina      | Oak      |   42 | NC    |
|  9 | Bob       | Builder  |   51 | MO    |
+----|-----------|----------|------|-------+
4 rows in set (0.00 sec)

mysql> DELETE FROM tblUsersBackup WHERE age>35;
Query OK, 2 rows affected (0.01 sec)

mysql> SELECT * FROM tblUsersBackup;
+----|-----------|----------|------|-------+
| id | firstname | lastname | age  | state |
+----|-----------|----------|------|-------+
|  5 | Joe       | Fry      |   32 | RI    |
|  6 | Emily     | Flanders |   22 | CA    |
+----|-----------|----------|------|-------+
2 rows in set (0.00 sec)

Multiple criteria can also be used to specify what to delete:

mysql> SELECT * FROM tblUsersBackup;                                                                    
+----|-----------|----------|------|-------+
| id | firstname | lastname | age  | state |
+----|-----------|----------|------|-------+
|  5 | Joe       | Fry      |   32 | RI    |
|  6 | Emily     | Flanders |   22 | CA    |
|  7 | Tina      | Oak      |   42 | NC    |
|  9 | Bob       | Builder  |   51 | MO    |
+----|-----------|----------|------|-------+

mysql> DELETE FROM tblUsersBackup WHERE age>50 OR state="RI";
Query OK, 2 rows affected (0.00 sec)

mysql> SELECT * FROM tblUsersBackup;
+----|-----------|----------|------|-------+
| id | firstname | lastname | age  | state |
+----|-----------|----------|------|-------+
|  6 | Emily     | Flanders |   22 | CA    |
|  7 | Tina      | Oak      |   42 | NC    |
+----|-----------|----------|------|-------+
2 rows in set (0.00 sec)

Indexes

An index speeds up the data retrieval time of a table. However, this comes at a speed cost for updating the table. So, it is usually a good idea to only index columns or tables that are frequently searched on.

To add an index, use CREATE INDEX. (Again, may differ depending on database technology being used)

mysql> SHOW FIELDS FROM tblUsers;
+-----------|-------------|------|-----|---------|----------------+
| Field     | Type        | Null | Key | Default | Extra          |
+-----------|-------------|------|-----|---------|----------------+
| id        | int(11)     | NO   | PRI | NULL    | auto_increment |
| firstname | varchar(50) | YES  |     | NULL    |                |
| lastname  | varchar(50) | YES  |     | NULL    |                |
| age       | int(11)     | YES  |     | NULL    |                |
| state     | varchar(2)  | YES  |     | NULL    |                |
+-----------|-------------|------|-----|---------|----------------+
5 rows in set (0.00 sec)

mysql> CREATE INDEX indexTblUsers ON tblUsers (id);
Query OK, 0 rows affected (0.01 sec)
Records: 0  Duplicates: 0  Warnings: 0

Drop Table

As we already know, you can drop a whole table with DROP. However, like many of the other commands, DROP can be used with other items, like an index:

mysql> ALTER TABLE tblUsers DROP INDEX indexTblUsers;
Query OK, 0 rows affected (0.01 sec)
Records: 0  Duplicates: 0  Warnings: 0

Truncate

The truncate command will delete the data, but leave table schema the same… again, still be careful because it will delete all the data in the table.

mysql> SELECT * FROM tblUsersBackup;
+----|-----------|----------|------|-------+
| id | firstname | lastname | age  | state |
+----|-----------|----------|------|-------+
|  6 | Emily     | Flanders |   22 | CA    |
|  7 | Tina      | Oak      |   42 | NC    |
+----|-----------|----------|------|-------+
2 rows in set (0.00 sec)

mysql> TRUNCATE TABLE tblUsersBackup;
Query OK, 0 rows affected (0.03 sec)

mysql> SELECT * FROM tblUsersBackup;
Empty set (0.00 sec)

Auto Increment

An Integer value, that once assigned, auto increments by 1 everytime the table is updated.

You can auto increment a key, which is very useful for IDs in a table (as we saw earlier).all the data

mysql> SHOW FIELDS FROM tblPeople;
+-----------|-------------|------|-----|---------|----------------+
| Field     | Type        | Null | Key | Default | Extra          |
+-----------|-------------|------|-----|---------|----------------+
| id        | int(11)     | NO   | PRI | NULL    | auto_increment |
| firstname | varchar(50) | YES  |     | NULL    |                |
| lastname  | varchar(50) | YES  |     | NULL    |                |
| age       | int(11)     | YES  |     | NULL    |                |
| state     | varchar(2)  | YES  |     | NULL    |                |
+-----------|-------------|------|-----|---------|----------------+
5 rows in set (0.00 sec)

mysql> INSERT INTO tblPeople (firstname, lastname, age, state) 
VALUES ('Josh', 'Rivers', 19, "PA");
Query OK, 1 row affected (0.00 sec)

mysql> INSERT INTO tblPeople (firstname, lastname, age, state) 
VALUES ('Kim', 'Medows', 32, "CO");
Query OK, 1 row affected (0.00 sec)

mysql> SELECT * FROM  tblPeople;
+----|-----------|----------|------|-------+
| id | firstname | lastname | age  | state |
+----|-----------|----------|------|-------+
|  1 | Josh      | Rivers   |   19 | PA    |
|  2 | Kim       | Medows   |   32 | CO    |
+----|-----------|----------|------|-------+
2 rows in set (0.00 sec)

Note: The auto increment value can be manually set.

mysql> INSERT INTO tblPeople (firstname, lastname, age, state) 
VALUES ('Dan', 'Valley', 40, "NH");
Query OK, 1 row affected (0.01 sec)

mysql> SELECT * FROM  tblPeople;
+------|-----------|----------|------|-------+
| id   | firstname | lastname | age  | state |
+------|-----------|----------|------|-------+
|    1 | Josh      | Rivers   |   19 | PA    |
|    2 | Kim       | Medows   |   32 | CO    |
| 1000 | Dan       | Valley   |   40 | NH    |
+------|-----------|----------|------|-------+
3 rows in set (0.00 sec)

When setting the auto increment value manually, just be careful that it isn’t set to one that will eventually overwrite another value in the table that must be unique, or it will error.

SQL Functions

SQL functions, are SQL statements that don’t directly manipulate data, but can be use to extract other useful information from the database and tables. They are often used in conjunction with SELECT. There are a bunch of base SQL functions to use. Here is a sampling of a few:

COUNT

Shows the number of matched results returned. In this example, the number of rows in tblUsers, and then the number of users over the age of 35:

mysql> SELECT * FROM tblUsers;
+----|-----------|----------|------|-------+
| id | firstname | lastname | age  | state |
+----|-----------|----------|------|-------+
|  5 | Joe       | Fry      |   32 | RI    |
|  6 | Emily     | Flanders |   22 | CA    |
|  7 | Tina      | Oak      |   42 | NC    |
|  9 | Bob       | Builder  |   51 | MO    |
+----|-----------|----------|------|-------+
4 rows in set (0.00 sec)

mysql> SELECT COUNT(*) FROM tblUsers;
+----------+
| COUNT(*) |
+----------+
|        4 |
+----------+
1 row in set (0.00 sec)

mysql> SELECT COUNT(*) FROM tblUsers WHERE age>35;
+----------+
| COUNT(*) |
+----------+
|        2 |
+----------+
1 row in set (0.00 sec)

Average and Sum Functions

To get an average of the number values in a column, use the AVG function:

mysql> SELECT AVG(age) from tblUsers;
+----------+
| AVG(age) |
+----------+
|  36.7500 |
+----------+
1 row in set (0.00 sec)

To get a total of a column’s number values, use the SUM function:

mysql> SELECT SUM(age) from tblUsers;
+----------+
| SUM(age) |
+----------+
|      147 |
+----------+
1 row in set (0.00 sec)

Note, that select statements using functions can still be combined:

mysql> SELECT COUNT(*),AVG(age) from tblUsers;
+----------|----------+
| COUNT(*) | AVG(age) |
+----------|----------+
|        4 |  36.7500 |
+----------|----------+
1 row in set (0.00 sec)

The Like Operator

The LIKE operator can be used for matching, with wildcards (%). For example in the following searches, %S and S% yield different results because the first looks for last names which end in an “s”, and the second grabs last names which start with “s”. The last example returns names which have “er” anywhere in the last name.

Note, LIKE uses higher CPU usage. With larger data sets, try to use it on columns which are indexed if possible.

mysql> SELECT * FROM tblUsers;
+----|-----------|----------|------|-------+
| id | firstname | lastname | age  | state |
+----|-----------|----------|------|-------+
|  5 | Joe       | Fry      |   32 | RI    |
|  6 | Emily     | Flanders |   22 | CA    |
|  7 | Tina      | Oak      |   42 | NC    |
|  9 | Bob       | Builder  |   51 | MO    |
| 10 | Zach      | Scout    |   27 | NV    |
+----|-----------|----------|------|-------+
5 rows in set (0.00 sec)

mysql> SELECT * FROM tblUsers WHERE lastname LIKE '%S';
+----|-----------|----------|------|-------+
| id | firstname | lastname | age  | state |
+----|-----------|----------|------|-------+
|  6 | Emily     | Flanders |   22 | CA    |
+----|-----------|----------|------|-------+
1 row in set (0.00 sec)

mysql> SELECT * FROM tblUsers WHERE lastname LIKE 'S%';
+----|-----------|----------|------|-------+
| id | firstname | lastname | age  | state |
+----|-----------|----------|------|-------+
| 10 | Zach      | Scout    |   27 | NV    |
+----|-----------|----------|------|-------+
1 row in set (0.00 sec)

mysql> SELECT * FROM tblUsers WHERE lastname LIKE '%er%';
+----|-----------|----------|------|-------+
| id | firstname | lastname | age  | state |
+----|-----------|----------|------|-------+
|  6 | Emily     | Flanders |   22 | CA    |
|  9 | Bob       | Builder  |   51 | MO    |
+----|-----------|----------|------|-------+
2 rows in set (0.00 sec)

Views

Views create a custom filter to display a set of the data. This can be useful when defining several use cases of an application. For example, a view can saved, and then just updated, instead of recalculated, when querying for dash boards, and/or reports.

mysql> SELECT * FROM tblUsers;
+----|-----------|----------|------|-------+
| id | firstname | lastname | age  | state |
+----|-----------|----------|------|-------+
|  5 | Joe       | Fry      |   32 | RI    |
|  6 | Emily     | Flanders |   22 | CA    |
|  7 | Tina      | Oak      |   42 | NC    |
|  9 | Bob       | Builder  |   51 | MO    |
| 10 | Zach      | Scout    |   27 | NV    |
+----|-----------|----------|------|-------+
5 rows in set (0.00 sec)

mysql> CREATE VIEW myView AS SELECT COUNT(*), AVG(age), SUM(age) FROM tblUsers;
Query OK, 0 rows affected (0.01 sec)

mysql> SELECT * FROM myView;
+----------|----------|----------+
| COUNT(*) | AVG(age) | SUM(age) |
+----------|----------|----------+
|        5 |  34.8000 |      174 |
+----------|----------|----------+
1 row in set (0.00 sec)

mysql> DELETE FROM tblUsers WHERE firstname='Bob';
Query OK, 1 row affected (0.01 sec)

mysql> SELECT * FROM myView;
+----------|----------|----------+
| COUNT(*) | AVG(age) | SUM(age) |
+----------|----------|----------+
|        4 |  30.7500 |      123 |
+----------|----------|----------+
1 row in set (0.00 sec)

Joins

Inner Join

An Inner join will return the selected rows from multiple tables, when there is at least one match in each table. For example:

mysql> SELECT * FROM tblUsers;
+----|-----------|----------|------|-------+
| id | firstname | lastname | age  | state |
+----|-----------|----------|------|-------+
|  5 | Joe       | Fry      |   32 | RI    |
|  6 | Emily     | Flanders |   22 | CA    |
|  7 | Tina      | Oak      |   42 | NC    |
| 10 | Zach      | Scout    |   27 | NV    |
+----|-----------|----------|------|-------+
4 rows in set (0.00 sec)

mysql> SELECT * FROM tblUsersPts;
+----|-----------|----------|-------|--------+
| id | firstname | lastname | team  | points |
+----|-----------|----------|-------|--------+
|  1 | Joe       | Fry      | Red   |  20000 |
|  2 | Emily     | Flanders | Blue  |  17000 |
|  3 | Tina      | Oak      | Red   |  32800 |
|  4 | Bob       | Builder  | Green |  40100 |
+----|-----------|----------|-------|--------+
4 rows in set (0.00 sec)

mysql> SELECT tblUsersPts.firstname, tblUsersPts.lastname, tblUsersPts.points
    -> FROM tblUsersPts INNER JOIN tblUsers
    -> ON tblUsers.lastname=tblUsersPts.lastname 
    -> AND tblUsers.firstname=tblUsersPts.firstname;
+-----------|----------|--------+
| firstname | lastname | points |
+-----------|----------|--------+
| Joe       | Fry      |  20000 |
| Emily     | Flanders |  17000 |
| Tina      | Oak      |  32800 |
+-----------|----------|--------+
3 rows in set (0.00 sec)

Right Join

A right join will return everything in the right table, and any existing/matched items on the left. Any non-matching data will display as null.

mysql> SELECT * FROM tblUsers;
+----|-----------|----------|------|-------+
| id | firstname | lastname | age  | state |
+----|-----------|----------|------|-------+
|  5 | Joe       | Fry      |   32 | RI    |
|  6 | Emily     | Flanders |   22 | CA    |
|  7 | Tina      | Oak      |   42 | NC    |
| 10 | Zach      | Scout    |   27 | NV    |
+----|-----------|----------|------|-------+
4 rows in set (0.00 sec)

mysql> SELECT * FROM tblUsersPts;
+----|-----------|----------|-------|--------+
| id | firstname | lastname | team  | points |
+----|-----------|----------|-------|--------+
|  1 | Joe       | Fry      | Red   |  20000 |
|  2 | Emily     | Flanders | Blue  |  17000 |
|  3 | Tina      | Oak      | Red   |  32800 |
|  4 | Bob       | Builder  | Green |  40100 |
+----|-----------|----------|-------|--------+
4 rows in set (0.00 sec)

mysql> SELECT firstname, lastname, points FROM tblUsersPts 
-> RIGHT JOIN tblUsers ON tblUsers.lastname=tblUsersPts.lastname 
-> AND tblUsers.firstname=tblUsersPts.firstname;
+-----------|----------|--------+
| firstname | lastname | points |
+-----------|----------|--------+
| Joe       | Fry      |  20000 |
| Emily     | Flanders |  17000 |
| Tina      | Oak      |  32800 |
| NULL      | NULL     |   NULL |
+-----------|----------|--------+
4 rows in set (0.00 sec)

Left Join

Basically the same as a RIGHT join, but with the left table items all joining.

Full Join

A full join shows all records from both the right and left table, regardless of matching the relation records of either.

NOTE: Full outer joins do not work on mysql, but would on postresql. It is recommended to use Unions to emulate them if needed.

Unions

Unions are used to combine and concatenate SELECT statements from multiple tables.

This example doesn’t work well because it doesn’t make much sense (age isn’t the same as a point), but at least it displays what is happening during the UNION.

mysql> SELECT firstname, lastname, points FROM tblUsersPts 
UNION SELECT firstname, lastname, age FROM tblUsers;
+-----------|----------|--------+
| firstname | lastname | points |
+-----------|----------|--------+
| Joe       | Fry      |  20000 |
| Emily     | Flanders |  17000 |
| Tina      | Oak      |  32800 |
| Bob       | Builder  |  40100 |
| Joe       | Fry      |     32 |
| Emily     | Flanders |     22 |
| Tina      | Oak      |     42 |
| Zach      | Scout    |     27 |
+-----------|----------|--------+
8 rows in set (0.00 sec)

Sorting Records

Record rows can be sorted in the ascending or descending order of a column by using the ODER command and either ASC for “ascending” or DESC for “descending”. The results can be limited or trimmed using another statement, like LIMIT 1.

mysql> SELECT * FROM tblUsersPts;
+----|-----------|----------|-------|--------+
| id | firstname | lastname | team  | points |
+----|-----------|----------|-------|--------+
|  1 | Joe       | Fry      | Red   |  20000 |
|  2 | Emily     | Flanders | Blue  |  17000 |
|  3 | Tina      | Oak      | Red   |  32800 |
|  4 | Bob       | Builder  | Green |  40100 |
+----|-----------|----------|-------|--------+
4 rows in set (0.00 sec)

mysql> SELECT * FROM tblUsersPts ORDER BY points DESC LIMIT 3;
+----|-----------|----------|-------|--------+
| id | firstname | lastname | team  | points |
+----|-----------|----------|-------|--------+
|  4 | Bob       | Builder  | Green |  40100 |
|  3 | Tina      | Oak      | Red   |  32800 |
|  1 | Joe       | Fry      | Red   |  20000 |
+----|-----------|----------|-------|--------+
3 rows in set (0.00 sec)

mysql> SELECT * FROM tblUsersPts ORDER BY team,lastname ASC;
+----|-----------|----------|-------|--------+
| id | firstname | lastname | team  | points |
+----|-----------|----------|-------|--------+
|  2 | Emily     | Flanders | Blue  |  17000 |
|  4 | Bob       | Builder  | Green |  40100 |
|  1 | Joe       | Fry      | Red   |  20000 |
|  3 | Tina      | Oak      | Red   |  32800 |
+----|-----------|----------|-------|--------+
4 rows in set (0.00 sec)

Minimum and Maximum Values

Similar to ORDER, the minimum and maximum values in a particular column of the table can be returned using the MIN and MAX commands:

mysql> SELECT * FROM tblUsersPts ORDER BY team,lastname ASC;
+----|-----------|----------|-------|--------+
| id | firstname | lastname | team  | points |
+----|-----------|----------|-------|--------+
|  2 | Emily     | Flanders | Blue  |  17000 |
|  4 | Bob       | Builder  | Green |  40100 |
|  1 | Joe       | Fry      | Red   |  20000 |
|  3 | Tina      | Oak      | Red   |  32800 |
+----|-----------|----------|-------|--------+
4 rows in set (0.00 sec)

mysql> SELECT MIN(points),MAX(points)  FROM tblUsersPts;
+-------------|-------------+
| MIN(points) | MAX(points) |
+-------------|-------------+
|       17000 |       40100 |
+-------------|-------------+
1 row in set (0.00 sec)

To return other fields with the min/max item, a sub-query (another SQL query inside parenthesis) may have to be used:

mysql> SELECT firstname,lastname,points FROM tblUsersPts 
-> WHERE points=(SELECT MAX(points) FROM tblUsersPts);
+-----------|----------|--------+
| firstname | lastname | points |
+-----------|----------|--------+
| Bob       | Builder  |  40100 |
+-----------|----------|--------+
1 row in set (0.00 sec)

Upper and Lower Case Conversions

Strings (such as names), and be easily altered using functions like UCASE and LCASE. These two functions change the displayed text (the data is not altered) to be upper or lower case.

mysql> SELECT * FROM tblUsersPts;
+----|-----------|----------|-------|--------+
| id | firstname | lastname | team  | points |
+----|-----------|----------|-------|--------+
|  1 | Joe       | Fry      | Red   |  20000 |
|  2 | Emily     | Flanders | Blue  |  17000 |
|  3 | Tina      | Oak      | Red   |  32800 |
|  4 | Bob       | Builder  | Green |  40100 |
+----|-----------|----------|-------|--------+
4 rows in set (0.01 sec)

mysql> SELECT UCASE(lastname),LCASE(firstname) FROM tblUsersPts;
+-----------------|------------------+
| UCASE(lastname) | LCASE(firstname) |
+-----------------|------------------+
| FRY             | joe              |
| FLANDERS        | emily            |
| OAK             | tina             |
| BUILDER         | bob              |
+-----------------|------------------+
4 rows in set (0.00 sec)

mysql> SELECT * FROM tblUsersPts;
+----|-----------|----------|-------|--------+
| id | firstname | lastname | team  | points |
+----|-----------|----------|-------|--------+
|  1 | Joe       | Fry      | Red   |  20000 |
|  2 | Emily     | Flanders | Blue  |  17000 |
|  3 | Tina      | Oak      | Red   |  32800 |
|  4 | Bob       | Builder  | Green |  40100 |
+----|-----------|----------|-------|--------+
4 rows in set (0.00 sec)

Now()

The Now*() function creates a new value, using the current date and time. This can be appended when creating a view, to mark when changes happen over time.

mysql> SELECT id,firstname,lastname,team,points, Now() AS updated
    -> FROM tblUsersPts;
+----|-----------|----------|-------|--------|---------------------+
| id | firstname | lastname | team  | points | updated             |
+----|-----------|----------|-------|--------|---------------------+
|  1 | Joe       | Fry      | Red   |  20000 | 2018-04-17 10:56:53 |
|  2 | Emily     | Flanders | Blue  |  17000 | 2018-04-17 10:56:53 |
|  3 | Tina      | Oak      | Red   |  32800 | 2018-04-17 10:56:53 |
|  4 | Bob       | Builder  | Green |  40100 | 2018-04-17 10:56:53 |
+----|-----------|----------|-------|--------|---------------------+
4 rows in set (0.00 sec)

Conclusion

I think that is enough to at least get started with SQL :) (it was for me anyway). There’s not much else to say other than hopefully this post server as a good SQL quick-reference down the road :). Enjoy!

Trying Out Seafile

2018-04-01T00:00:00Z

Outer Banks, NC USA

In college, I mostly used Dropbox to handle all of my file syncing needs. As I approached graduation, I setup an Owncloud droplet (which is now a Nextcloud instance) to supplant my Dropbox usage. While it has worked fairly well, I’ve been watching seafile from a distance the last few years, but haven’t taken the time to try it out. I have now.

Seafile

The Seafile Logo.

As stated on their website:

Seafile is an enterprise file hosting platform with high reliability and performance. Put files on your own server. Sync and share files across different devices, or access all the files as a virtual disk.

While self-defined as an “enterprise file hosting platform”, there is a Community Edition that is Free & Open Source… which is exactly how I like my software. So lets get started.

Setup

Similar to a Nextcloud configuration, to get seafile up and running, a server component is first installed. Afterwards, clients connect to it.

Server

The countless number of configurations seafile supports may make setting up the server component for the first time a bit intimidating. There are several database back-ends (SQLite, MySQL…), web servers (Nginx, Apache), and advanced options (Memcached, LDAP, etc) which can be selected. However, the seafile developers do provide an installation script to easily install both the Pro and Community editions under Linux. Which is what I used.

Download

The first step is to download the server application. A link for the install package can found partway down the download page. After Downloading, extract the contents of the package.

wget https://download.seadrive.org/seafile-server_6.2.5_x86-64.tar.gz
tar xf seafile-server*.tar.gz

Inside the extracted directory, there are several setup-seafile-*.sh* scripts. I just used the basic setup-seafile.sh one.

Dependencies

Depending on which setup script it used, and the state of the host machine, there may be some missing dependencies. While I am sure there is a nice list of the required dependencies posted somewhere… I didn’t look for it. If I am being completely honest… I just kept repeatedly running the script, and installed whichever package it yelled at me for each time. Eventually, the install worked. On my fresh install of Ubuntu 16.04 server, this meant installing the following dependency packages:

python-setuptools
python-imaging
sqlite3

This means that on an Ubuntu 16.04 server, a simple sudo apt install python-setuptools python-imaging sqlite3 should do the trick…

Install Script

With the dependencies installed, the script should run through without screaming for missing packages. However, it will ask for a few pieces of information including server-name, server-ip/domain, data-dir, and fileserver port to install the fileserver component. Answer accordingly.

The Seafile Web Client, Seahub.

After configuring the fileserver, the script will flow right into configuring Seahub, the web interface that sits on top of the fileserver. This part is less needy, and will only require an [Enter] press to continue.

At the end, the script should display useful information about how to start/stop/reset the servers, as well as what ports each part is running on.

Run & Start

Starting the servers is done by doing what the script says… run the following commands:

./seafile.sh start
./seahub.sh start

With the servers running, it may be necessary to configure the firewall to allow their ports. On Ubuntu 16.04:

sudo ufw allow 8000/tcp
sudo ufw allow 8082/tcp

That’s it. With any luck, clients should be able to connect to the server (assuming the server is reachable from the client computer… but that’s a lesson for another day).

Note: The first time seahub is run, it will need to setup an admin account. So while it was nice and considerate in the last step… it will insist on being supplied a username (email) and password for the new admin account. Again, just comply.

Client

The seafile desktop client was trivial to install. It was in the Solus repos, so I just needed to run sudo eopkg it seafile-client, and I was done. I also installed it on my work computers (which unfortunately runs Windows 10), and even that was simple. I just downloaded and installed the Windows “Desktop Syncing Client” at the top of the download page.

My Thoughts

Seafile Desktop Client

I have been using seafile as a syncing solution for the past few weeks. After the initial setup, it has been rather uneventful… which is a good thing. It just runs in the background, and I don’t really think about it. Exactly what you want with a syncing solution.

That being said, I haven’t fully dug into all of the details of the application yet. So, if I incorrectly critique something due to my own ignorance that simply isn’t true, I apologize in advance.

What I Like

Desktop Client - The desktop client is simple. It responds quickly, and is a nice little command center where I can view how seafile is configured on each individual system.
Select which Libraries to sync - Seafile libraries allow me to sync only what I need. Nextcloud had this feature, but it was implemented a little differently. I had to go to the Nextcloud folder in the client UI, and then check/uncheck all of the sub-folders I wanted to sync on my machine.
Pick where each libraries sync - Even better than being able to pick which libraries to sync, is having the power to choose where each one will reside. In seafile, the location of each library is completely independent. For example, I can sync my work library to ~/Documents/Work/, my Music to ~/Music, and emacs to ~/Documents/Programming/Editors/why/so/many/diectories/emacs/. On another computer, I may choose to only sync my work library to ~/Desktop/work (It doesn’t have to be the same across computers). I love this. It lets seafile integrate with my workflow, rather than forcing me to make it be my workflow.
Encrypted Libraries - When creating a new library, it is possible to make an encrypted one. These libraries use client-side, end-to-end encryption and require a password. The file contents (Note: not directory or file names) are encrypted on the client side, and not on the server. This means that even the server admin cannot access the file contents on the server. This is a feature I’d like to see in more applications.

When syncing an encrypted library, a password must be entered.

Fast Sync - I only have anecdotal evidence, but the syncing in seafile feels fast. Files seem to pull down very quickly, and setting up my libraries on a new device doesn’t take a very long time. I might have to actually measure if it’s any better than something like nextcloud, but if I’m happy… does it really matter?

What I Don’t Like

No Folders/Nested libraries - As far as I can tell, I don’t think it is possible to organize seafile libraries with folders, or a nested library structure. Admittedly, this is one of those areas that I haven’t had to look too much into yet. However, as I start to increase my seafile usage and the number of my libraries increases, I could see this being a feature I’d really enjoy.
Phone Sync Tricky - The phone sync (specifically auto photo upload) was tricky to get working at first. After setting it up, it didn’t seem to work, or at least not as promptly as nextcloud does. I took a few photos to test it out today, it seemed to work okay, but I don’t think it started syncing right away. Right now, I don’t 100% trust it to work without me thinking about it. Hopefully this improves with time.

Future Plans

To summarize, I have been loving seafile, and I think I have only scratched the surface. I haven’t even tried playing with features like History and Snapshots, Full Text File Search, or yet. My plan is to continue experimenting a bit more, and then switch to it as my main syncing system when I redo my main server setup in a couple of weeks. If you haven’t given seafile a try recently, I recommend it.

Docker Quick Start

2018-02-25T00:00:00Z

Inner Harbor, Baltimore, MD USA

Over the past few months (particularly over the holiday season), I started to explore and learn several technologies I’ve had my eye on the last few years. First on the list: Docker. These days, Docker has a massive ecosystem surrounding it, and can take years to truly master. This post on the other hand, will hopefully help you get up and playing with docker containers in just a few minutes. Lets get started.

Installing Docker

Installing docker on Solus was easy enough. I just had to install the package, and then enable the service:

sudo eopkg it docker
sudo systemctl enable docker
sudo systemctl start docker

On other distributions, it may not be in the package manager, or it might be under a different name. To be sure, check out the community edition installation documentation for your specific distro.

Adding User to Docker Group

By default, user accounts will not have permission to run docker commands without root access. So, after first installing docker, it might be tempting to just run everything using sudo, but that isn’t the best idea. To get around this, simply add the user account to the docker group:

sudo usermod -a -G docker ryan

If the docker group is not created for some reason, it can be added:

sudo groupadd docker

Afterwards, that user should be able to run docker commands without requiring sudo. They might have to log out and back in first.

Note: These commands may differ based on distro.

Some Useful Commands

Before getting too caught up in creating and using containers, lets first go over a few useful commands that make navigating docker a bit easier.

docker ps

Example of the docker ps and docker ps -a commands.

Similar to how Unix-based systems have the ps command to see running processes, docker has docker ps to see created and running containers. To view the currently running containers, use the basic docker ps command:

The base command however, doesn’t always tell the whole story. A system might have containers that exist but are not running. A container may have been stopped, or exited if it encountered an error. Containers that are stopped or exited will not show up in the default docker ps command. To see all of the current containers on the system, run ps with the -a flag.

docker inspect

Using the docker inspect command to get the container's status and IP address.

Another useful command when working with docker containers is docker inspect. The inspect command will dump the xml for all the low level information of the container/docker object. The output contains basically everything about the container: Full ID, time created, state, volumes, network information… everything. It can be useful to pipe the output of inspect to grep, in order to get specific information about the container quickly.

docker inspect container_name | grep IPAddress

For example, the above command will grab and return only the lines which contain “IPAddress”.

help

Using the docker help command flags.

Last but not least, don’t forget about the help command. To see all the available docker commands, run docker help.

Additionally, when using each of those specific commands, (inspect for example), a description and possible options can be shown using the --help flag (as in docker inspect --help).

Just as man pages can be extremely valuable when working on a Linux system, help is just as essential when using docker.

Images

A docker container is created from a base image. Images can be pulled down from Dockerhub. To search docker hub from command line, use the docker search command. For example, docker search nginx will search for images related to nginx.

The search output for nginx images.

The search will list all of the public images in order, from most to least stars. When a desired image is found, pull it down using the docker pull command. For example, docker pull nginx will pull down the nginx image.

Pulling the latest nginx image.

Docker will then download all the layers for the image. When the download completes, the image will be locally available to create docker containers from. To see all of the local images, use the docker images command:

To delete an image, use the docker images rm command, or my preferred, lazier command, docker rmi:

Viewing downloaded images with the docker images command, then deleting the 'test-ubuntu' image using the docker rmi command.

Lastly, it should be noted that custom images can be created/tailored using a Dockerfile. With a Dockerfile is defined, an image can be created from it using the docker build command. For example, the following Dockerfile would use an ubuntu image for the base, but also update & install several packages in the image:

# This is a custom ubuntu image with SSH already installed
FROM ubuntu:latest
MAINTAINER himmallright <ryan.himmelwright@gmail.com>

RUN apt-get update

RUN apt-get install -y vim stow git tmux fish htop emacs

To build the image, run the following command in the same directory as the Dockerfile (if defining a specific file, the -f flag can be used):

docker build -t ubuntu-base:v1 .

I like to use the -t flag, so that I can specify a name:tag for the image. This helps to make it easier to find it in the docker images list.

Creating Containers

Docker containers can be created (but not run) with the docker create command. When creating containers, it is useful to use flags to tailor the details of the container. For example, the -m flag can be used to create a memory limit, --name to name the container, and so on. To create a simple nginx container from the image we previously pulled:

docker create --name web-test nginx:latest

Starting & Running Containers

To start a container created with docker create, or one that has been previously stopped, use the docker start command. For example, to start the container created in the previous step:

docker start web-test

Similarly, a container can be stopped with docker stop, or restarted with docker restart.

Instead of using a docker create and docker start combination, docker run can be used to both instantiate and start a container. To create and start the “web-test” container from the previous examples:

docker run --name web-test -d nginx:latest

Cleaning containers

Removing all the older container images.

Over time, old containers may build up on the system. To remove an old (but not running) container, use docker rm with either the container name, or the id. Note, to easily delete all of the containers on the system, docker rm can be fed the output of docker ps -aq, where the -aq flag returns a list of all the container ids.

docker rm `docker ps -aq`

(Note: this doesn’t work in my Fish shell, but does in bash)

Attaching to Containers

After a container is running, it might occasionally be necessary to attach to it and poke around with a shell. There is the obvious way to do this, docker attach, and a somewhat work-around way, which I prefer to use: docker exec.

An example using the docker attach command.

The attach command works as one would expect. It allows a user to “Attach local standard input, output, and error streams to a running container”. This is all well an good, except for one issue: when exiting the attached container, the container also exits.

An example using docker exec with a bash shell, as an alternative to docker attach.

A way around this annoyance is to utilize the docker exec command, which allows a command to be executed inside a container. Executing a shell program inside the container, such as bash, mimics the attach command, but with the added benefit that when exiting, only the shell exits, and not the entire container.

Ports & Volumes

As mentioned earlier, docker containers can be built and run using many specific commands to tailor the container and how it interfaces with the host system. There are two concepts in particular that I want to briefly touch on in this quick start post. Those two items, are ports and volumes.

Ports

Container ports can be forwarded to the host's ports.

While it is nice to spin up a web server inside a docker container, it isn’t always very useful to only have it available to the host machine. By using the -p flag when running a container, the container’s ports can be forwarded to ports on the host system.

Creating an nginx container, forwarding it's port 80 to the host's port 8081.

Using -p with just a single number, as in -p 8080, will declare that port of the container to be exposed. To forward exposed ports to the host, use two port numbers, separated with a :. The first number is the host port to bind to, and the second is the container port to expose and forward.

docker run -itd --name webtest -p 8081:80 nginx:latest

The above example, a nginx container is started with port 80 forwarded to port 8081 on the host. As a result, any computer connecting to port 8081 of the host machine will be directed to the nginx web server inside the container.

Volumes

Containers can attach volumes from the host system for persistent data.

Lastly, by design docker containers are mean to be expendable. They are run, and then disposed. It should not be assumed that any data inside the container will be preserved by default. That is, unless volumes are used.

Creating an nginx container, forwarding it's port 80 to the host's port 8081.

Docker volumes are the preferred mechanism for preserving data across container runs, and are specified using the -v flag. Similar to setting ports, volumes can be created by providing either a single path, or two separated by a :. When a single path is provided, as in -v /Data, docker will create a volume and bind it to that location within the container. Two locations can be provided to bind a directory on the host system, to the volume inside to container.

docker run -d --name testsite -v /home/ryan/testsite/:/usr/share/nginx/html nginx:latest

In the above command, for example, -v /home/ryan/testsite/:/usr/share/nginx/html will use the /home/ryan/testsite/ directory of the host system, as a volume located at /usr/share/nginx/html inside the container. This means that the container will server the website files, which are located (and can be easily edited), on the host system.

In Conclusion

So, this has been… a rather long post. However, when it comes to Docker, this really is just the tip of the iceberg. With any luck though, you should now know the basic to at least get started. So… go ahead and have some fun!.

Living the Dock Life - My Thinkpad T470

2018-02-01T00:00:00Z

In my previous post, I discussed how I sold off my System76 Bonobo laptop, and planned to replace it with a new computer. Well, I did… awhile ago. For the last few months, I have been using a Lenovo T470 Thinkpad (with dock) as my new daily driver. Here are my thoughts.

The Choice

My T470 on a Sunday morning.

The T470 is the 14" model of Lenovo’s “business” laptop lineup. The T470 series actually has several variants itself, including the “high end” T470s, that has a slimmer chassis, made from “higher quality materials”, and a better screen compared to the T470. There is also the "high performance"T470p, which features a faster, quad-core CPU, but a slightly larger chassis. While these variants caught my attention, I ultimately decided on the normal T470. This was partially because I could max the T470 out at 32GBs of RAM (The T470s can only hit 24GB due to one of the sticks being soldered in), but mostly because I assumed that the combination of having a lower powered CPU (compared to the T70p), but a slightly larger case (compared to the T480s) would make it the optimal choice concerning my heat/noise concerns outlined in the previous post.

The Specs

The Guts of the T470.

So, the T470 then…

I got my T470 from a vendor on Ebay. It came configured with a 2.5GHz (3.1 GHz Turbo) i5-7200u cpu, 8 GBs of DDR4 RAM, a 14" 1920x1080 screen, and a 500GB Hard Drive with Windows 10 on it. After checking that it booted up fine, I immediately swapped the slow HD with the 250GB SSD that was in my x230, and installed Solus on it. In Solus, everything worked “out of the box”, and I would guess other Linux distributions would as well. For Christmas, I got a new 500GB Samsung EVO SSD to upgrade the 250GB one. In total, the T470 has less total storage than the Bonobo had, but all of it is fast SSD storage. I am getting better at using my server for network storage anyway, so it all works out :) .

Installing Solus the first night.

While I still plan to upgrade the T470 in the future, here are it’s specs at the time of writing this post:

T470 Thinkpad
Intel i5-7200u [2.5 Ghz (3.1 Ghz Turbo), 2 Cores, 4 Threads]
8 GB RAM
500 GB Samsung EVO SSD
14" 1920x1080 IPS Display
Solus

The hardware on the T470 is nice, and the technical specs are okay, but the real killer feature of switching to a Thinkpad for my main computer is that I also purchased the ThinkPad Ultra Dock (40A2). This means that I am able to easily take advantage of my periphery devices whenever I am working at my desk.

Thinkpad Ultra Dock 40A2
2 x 23.6" ASUS 1920x1080 Monitors
Happy Hacking Keyboard Pro2
Audioengine A2+ Speakers & Stands
Bose AE2 Headphones
Logitech HD Pro Webcam c920 Widescreen
Blue Yeti Microphone - Backout Edition
Inateck USB 3.0 to SATA Dual-Bay Hard Drive Docking Station

What I like

I've enjoyed the portability and great keyboard of the T470.

Now that I’ve provided some background information and a summary, I’ll quickly list what I like and do not like about my new T470.

Cool & Quiet - After realizing that I am picky about heat and noise in computers, one of my biggest goals when replacing my old computer was to find one that ran cooler and quieter. I think I did well here. This thing is silent. Using one of my new favorite command line applications, s-tui, I have observed that most of the time when I’m writing code, browsing the web, or listening to music, the fans in the T470 don’t even kick on. When I do push the machine a little by running VMs, or being on a video conference, the do kick up, but even then I have to put my ear right against the computer to hear anything. At it’s loudest, the T470 is still a very quiet device.
Size & Portability - I love the size and portability of this laptop. The 14" form factor is large enough that I don’t feel cramped when using it as a stand-alone device, but still small enough that I can easily throw it in my bag and bring it with me. While I wouldn’t mind the T470 shedding a little weight (which is mostly from the battery), I find it’s thickness perfectly acceptable for my uses (hear that Apple?). As far as build quality, it is a solid machine. When I rest my hands on it to type, they feel fully supported and there is zero flex. When holding the closed laptop, it feels as though it could take quite a beating and still be fine. I love that.
Keyboard & Track-point/pad - It’s a Thinkpad keyboard, what else do I have to say? The keys are adequately spaced, have travel (though not as much as my x230), and feel crisp when pressed, not mushy. It has a trackpoint with physical buttons, which at this point I don’t think I could do without (Crap. Unintentional pun. Oh well, I’m leaving it.). While I don’t use it often, the trackpad is also quite nice. I find it to be an ideal size, and has rubberized texture which prevents my fingers from feeling raw after long use. I find it hard these days for others laptops to compete with the Thinkpad’s keyboard/track-point setup.
Dock-able - I’ve already mentioned this above, but I love that the T470 is dock-able. This is becoming more common with the pervasiveness of USB-C, but Thinkpads have been in the dock game for years. It works well, and fits my current computing use-case perfectly.
Thunderbolt/USB C - The big difference between the T470 and the T460 essentially comes down to the Thunderbolt 3/USB-C port on the T470. I don’t use it much so far, but I hope the industry continues the move down the this route, and I will be glad to already have the port as they do.
Battery(s) - The battery configuration and life on this laptop is great. The T470 can get a 3-cell battery that doesn’t stick out, or an extended battery that comes in 6 and 9 cell variations (Note: the 6 and 9 cell are the same size). The T470 also supports an internal 3 cell battery, that can be used to keep the laptop powered on while the external battery is swapped. I currently have the least optimal configuration: A bulky 6-cell external, with no internal battery. Even so, it easily lasts me throughout the day, and I can always upgrade it down the road. I love the flexibility.

What I don’t like

While I overall love the T470, there are a few things I’m not thrilled about.

The Screen - When I first got my laptop… I thought the screen was defective. I had read in reviews that it wasn’t very bright, but the colors were terrible. My screen always looked as if I was using something like f.lux. After doing some reading, it turns out that IPS panels (which I had never owned before) can appear warmer in color. That combined with the low brightness and not great color accuracy of the T470 panel, made it appear particularly awful to me. I found and configured a ICC file which helped, but the biggest fix was time. As terrible as it was at first, I really don’t mind the screen. Yea, it could be brighter with better colors, but now I think it looks nice, and colors on top of black really pop.
Track-point - I stated in the section above that the track-point was one of the features I loved about this laptop. That is true… but, if I had to get picky, the track-point itself is not that great. It’s hard to move around with and it’s sensitivity differs depending on which direction it is traveling. At times it can be a real pain. By contrast, my x230 is very stable and precise. I wish the T470’s track-point was like the one on my x230’s.
Graphics - I can’t complain too much here because I knew what I was getting myself into, but… the graphics performance isn’t the best. I know not having dedicated graphics is what allows the T470 to run cool and sip power, but I wouldn’t mind a little more umph. I don’t expect to do heavy gaming on the T470. If I want to game, I am going to build a gaming computer. However, I do wish I could play basic games a little better in the meantime. (On the bright side…with the thunderbolt 3 port… an eGPU is a possibility…hmmm…)

Future Plans

The empty RAM slot I hope to add a 16GB stick to... if prices ever drop again...

Overall, I have been very happy with the T470 as my main computer. I’d still like to upgrade the RAM at some point, and I have entertained the idea of trying an eGPU setup… I might eventually build a new desktop, but even then, I would likely have it headless, running VMs and connect to it remotely from the T470 (except for occasional gaming). That being said, right now, the T470 is working great as is!

Sold My Bonobo

2018-01-14T00:00:00Z

Last fall, I did something I didn’t think I would do anytime soon… I sold my main workstation computer. Specifically, my System76 Bonobo Extreme laptop. Several events over the last year have led me to the realization that my actual computing needs are drastically different than what they were when I purchased the Bonobo in early 2015. This post will highlight some of those realizations, and why I ultimately decided to sell my Bonobo laptop.

Why I got the Bonobo

Bonobo Workstation Setup.

I bought my Bonobo laptop from System76 during my first post-college year. I needed to replace my old workstation: an aging desktop that I built before my High School graduation. While its i7-930 CPU, 6GB of RAM, NVIDIA GTX 260, and hodgepodge of hard drives (SSD and HHD) serverd my needs throughout college, I was starting to feel restricted. After some research, I came to the conclusion that it would be better to get a new system, instead of trying to upgrade my old desktop. I wanted a new, powerful computer that could easily run multiple VMs, play some video games, and compile software/run models fast.

After a few months at my new job, I noticed I didn’t use a portable computer as much. I went to work, used my work computer, and then came home to my desktop. So portability wasn’t the biggest concern when searching. That being said, I still lived out of state from my family, and would leave for a week or more at a time during hollidays or vacation. I wanted to still be able to lug my computer with me during family vists. I also liked to game a little bit when visiting with my brothers, or after work, so I added a dedicated GPU to the search criteria.

Portable Bonobo Setup.

All of this pointed me towards desktop-replacement laptops. I wanted to support a Linux-only vendoor, so I looked at System76. The Bonobo seemed comparable to other laptops on the market, but being a System76 device, I felt more confident all the hardware would play nicely with Linux. It was a big laptop, but my plan was to use it mostly as a desktop computer, with the option to still travel with it when I needed to go somewhere for more than just a day or two. Besides, I liked that the case was larger, as I hoped it would help increase airflow to help with heat management.

I purchased the Bonobo, and it worked out very well for several years. It was a very powerful computer (more than my old desktop), but still portable enough for when I wanted to take it somewhere.

The Modern Reality

Bonobo Side View.

As the years passed, my computing needs changed. On top of that, some of the initial assumptions about my computer use turned out to not be entirely true.

I’m Not a Big PC Gamer.

For example, when my brothers and I would pull out our laptops to play games together, the games we played were never highly demanding. Our top played games were Minecraft, Serious Sam 3, Dungeon Defenders, and maybe CS GO. Needless to say, the Bonobo was a little overkill. Unfortunately, with my brothers and I all living in different states now, we really don’t see each other in person enough to play computer games in the same room anymore. These days, when we want to remotely play video games together… we play on the Xbox ONE (which I bought mostly to keep in touch with my friends and family. I didn’t own one when I purchased the Bonobo).

Xbox One

I have enjoyed owning a computer that can handle most of games I throw at it, but if I’m being honest… I don’t play games too often. Even on the Xbox, I go in bursts of playing once or twice a week, and then don’t touch a game for months. While I wouldn’t mind having the ability to play more demanding games on my computer, it isn’t a requirement for me… expecially on a laptop.

I Travel More. Sometimes On Airplanes.

One change compared to when I originally bought my bonobo is that I travel more. When I first moved to Massachusetts, I figured most of my traveling would be from here back to Pennsylvania, or to vacation once a year. However, my wife and I have managed to keep in touch with some of our old friends… who live all over the country. Plus, now that my brothers have moved out of the house, they have started to spread out too (as already mentioned in the section above). So, I travel more often. In particular, I fly more frequently.

To say it bluntly, the Bonobo sucks to travel with. It takes up the majority of my bag, and most bags don’t even fit it. Besides the laptop size, TSA is often freaked out by the MASSIVE power supply, and has had to re-scan my bag several times, thinking I had a “bluetooth speaker” or something else I neglected to tell them about (despite my polite warnings that “my laptop’s power charger is massive”). Beyond security, It isn’t a computer I can pull out of my bag and easily use in an airport terminal. In fact, the last time I flew with it, I tried to work on it at table, but couldn’ because the power outlet didn’t supply enough juice to charge the laptop. So, I was forced to switch to my phone for entertainment while waiting several hours for my flight, despite having a massive gaming laptop in my bag.

It turns out I’m noise picky.

When building or purchasing a computer, something to consider is noise. With my previous machines, I didn’t worry about it. I’d joke to myself, “Noise? Why should I be concerned about that? I’m not building a recording studio”! After several years of focusing on how I actually use my computing devices, I’ve noticed that I am very picky with noise, and on a related note, heat. Every time I hear the fans kick up, I start to worry that I am over-taxing the system and start to monitor the biggest culprits. When running taxing processes, I watch the cpu temps and hate seeing them high.

Parallel Compression on the Bonobo Spiked the Temps.

I hoped the extra thickness of the Bonobo would help with any heating issues. While it might have a little bit, when you shove a decent GPU into a laptop… it’s going to give off heat. Additionally, when using a powerful CPU to it’s full potential, it is very difficult for a laptop to compete with a desktop-level cooling system. The Bonobo is a very powerful laptop, but the loud fans prevented me from actually using it to its full potential, because I would immediately start shutting down processes when the noise kicked up. I know the fans spin up to help with thermals, the Bonobo was just a tad to hot and loud for my taste.

A Desktop Replacement?

My issues with the Bonobo boiled down to the fact that based on my current computing use-case, the Bonobo functioned less as a desktop-replacement-laptop, and more of just a shitty, somewhat portable, desktop. Most of the time, I am just browsing or writing code on the machine in front of me. If I need more power, I connect remotely to a bigger, better, computer. I also never game “on the go”, so if I want to have a computer for gaming, it makes much more sense to eventually have a half-decent desktop (or invest in an eGPU), which will fit my needs much better that even high-end gaming laptops.

The power cord alone made it difficult to pick up the Bonobo and move it elsewhere.

The Bonobo is great if you sit it on a desk and use it as your full computer. I did not. I have a ton of periphery devices, and started to even use it in clam-shell mode, making the nice 17" screen a waste. Additionally, I hated having to unplug and re-plug all the devices when I wanted to move the computer. In theory, I could move the Bonobo from my desk into the other room and work with my wife at the kitchen table whenever we wanted, but I rarely did. Unplugging all my devices and fishing the massive power cable out from behind my desk (because the battery life is … yea, I don’t think I need to explain this), was usually enough to prompt me to treat the Bonobo as a desktop, rather than a laptop.

Recent Discoveries & My New Plan

A while back, I bought a used x230 Thinkpad. Last summer, I finally purchased a used docking station for it, so I could try out a docked setup. I loved it. It matched my needs exceptionally. I was able to have all of my periphery devices attached to the dock and thus, the x230, but with the push of a button I could detach the laptop be on my way. The setup worked so well in fact, that I started to use my x230 as my desk computer, and would remote to my Bonobo on the other side of the room when I needed it. At that point, I realized I might want to switch things up.

I want to conclude by again stating that my System76 Bonobo Extreme was not a bad machine. It is great when used as a standalone, semi-portable computer. I didn’t use it that way, and as a result, it just wasn’t for me anymore. I learned that a quite, cool, portable, but dockable, laptop is ideal for my main device right now, even if it isn’t as powerful. Unfortunately, System76 doesn’t offer such a machine at this time. So, my plan is to replace the Bonobo with newer thinkpad + dock combination. Hopefully that combination will give me a nice dock setup that I have grown to appreciate and love.

New Happy Hacking Keyboard Pro 2

2017-12-28T00:00:00Z

After years of using a Happy Hacking Keyboard Lite, I have finally decided to splurge on a Happy Hacking Professional 2 keyboard. And I love it.

Note: I have been slowly chipping away at this post for the last few months. I have been very busy, but I should be able to get back into the swing of writing again.

My history with the HHKB Lite

My Computer (HHKB lite included) during my senior year of college.

I started using Linux during my senior year of high school. In college, I started diving deeper down the rabbit hole, learning more about shells, window managers, and programming. By senior year, I found myself sitting in front of an Arch Linux install, configured with a tiling window manger to organize the scattering of VIM and VIM-like terminal applications I used every day. To be kind to my wrists, I wanted something that was optimized at the hardware level for my heavy CTRL key use. This is when I first learned about the HHKB. However, as a broke college student, I opted for the much cheaper lite version to test out. I enjoyed the layout so much, that years later when I needed to replace a keyboard at work, I had them order me a HHKB lite (again, the Pro was very expensive).

Time for a Mechanical Keyboard

After years of using the standard rubber dome HHKB lite, I wanted to finally switch to a true mechanical keyboard. I searched long and hard, looking at different layouts designs, and learning about the various switch types (mostly the commony cherry MX switches). After purchasing and playing with a switch tester, I confirmed that I would likely enjoy MX clear switches best. They are quiet enough I could still reasonably use them in an office, but still had the nice tactile bump that I love (more than what I experienced on the Browns). Additionally, I knew that I wanted to stick with a small, minimalist layout, so I exclusively looked at 60% keyboards, with the occasional 40% board catching my eye.

The Pok3r Keyboard.

Like many, I eventually landed at the Vortex Pok3r. It is beautiful, with a minimal, but sturdy appearance, and has a MX clear variant. I had made my decision, but at the time, new boards weren’t available. I decided to sit back and wait to see if it would drop on Massdrop any time soon. I had waited years to purchase a mechanical keyboard, and I could wait a little longer to get the one I wanted.

The Planck Drop

The Planck Keyboard.

While I was waiting for a Pok3r drop… something interesting caught my eye on Massdrop… the Planck keyboard… The planck is a keyboard kit that you solder together to create a programmable, ortholinear, 40% keyboard. When I say programmable, I don’t mean that you can just bind some keys. The Planck is fully open and the firmware C code is available on Github for modification. So, you can actually customize and hack together the firmware, which was very appealing to me. So I joined the drop.

After joining, I purchased a soldering kit so that I could construct the keyboard once it finally shipped. Unfortunately, the planck drop had several delays, but as I previously stated, I didn’t waiting. The planck got delayed so much, that they eventually allowed people to cancel their order and receive a refund. I thought the order cancellation was something you opted in to do, but it was actually an opt out situation… so my lack of response had my order canceled, pre-maturely ending my planck experience.

Deciding on the HHKB Pro

With my planck order canceled, and a refund imminent, I started to look at other options again. The main contender was still the Pok3r. I started researching it again and learning about how I could program the layers to have a layout I would like… and eventually realized I just kept re-creating the infamous HHKB layout.

So I started to consider the HHKB again. My main reasons for not getting it originally was that it is a good bit more expensive than the majority of mechanical keyboards on the market. However, after waiting months for the planck, and noticing I kept comparing everything to the HHKB, I realized I would eventually end up with an HHKB anyway, and decided to just go for it now. Decision made.

Getting the HHKB

My HHKB

Once I decided on the HHKB, (and it was approved by my wife :P ), I purchased it on Amazon. The one thing I did contemplate about was which color to get. I didn’t know if I wanted to get the black or white model. I knew I wanted printed key-caps, and that made the white board very appealing. The printed keys on it look so crisp. Eventually, I decided on the dark gray because it matches everything else I own, whereas the white board would drastically stick out.

I placed the order and within a few days, it was on my doorstep (well, the neighbor’s, but that’s an issue with our post…).

What I like

Layout

As already mentioned, I love they HHKB layout. It was a bit tricky to get accustomed to at first, but that was several years ago at this point. The HKKB Pro does have a few difference in layout when compared to the lite version (the pro does not have a bottom left fn key, or any dedicated arrow keys). However, I had prepared myself for this change over the last few years by only using the right fn key and the fn arrow keys. The hardest thing to adjust to was the slightly different sized alt and super keys on the left (due to the missing fn key).

I have learned to appreciate all the little HHKB layout differences. This includes the “delete” by default backspace key, with the ~ key above it, and the ESC key next to 1. Last but not least, the CTRL in the CAPS location (with CAPs still available as fn + Tab). I honestly think this should be the standard keyboard layout.

PBT key-caps

I didn’t expect the PBT key-caps to be such a big deal, but I absolutely love them. I could tell from images that they would look better (especially the white ones), but I didn’t anticipate how great they would feel. They are solid, but with a soft touch, and are a much better improvement over every keyboard I have previously owned/used.

The Switches

I wasn’t sure how I would like the topre switches after getting excited for Cherry MX clears. However, I seem to really like them… at least on the HHKB. I have heard that they feel different on the HHKB compared to other Topre keyboards due to the HHKBs plastic back. I’m not sure if this is true or not, but I like the result. Key presses are smooth but at the same time very crisp. Being a heavy typist, I often bottom out, but when I do it feels very soft, and not harsh at all. I figure that if I am going to bottom out on my keys regardless of what keyboard I have, it is probably best to handle it how the HHKB does.

… Can I say sound?

I love the sound of typing on this keyboard. While it is definitely an audible keyboard, it isn’t nearly as obnoxious as louder clicky switches, like MX Blues or Greens. It has a soft, muted, lower pitched sound as I type (compared to normal “clicky” switches). I really do enjoy hearing it, and find myself sometimes taking off my headphones to listen to it if I need motivation to get more characters on the screen.

What I don’t like

Cost

As I’ve mentioned earlier in the post, probably the biggest downside to the HHKB is the substantial cost. At >$230 USD, it is expensive, even for mechanical keyboards. The cost is the single problem that kept me from buying it all these years. Eventually, I realized it is what I wanted, and it would likely last me years, so it all works out. Additionally, if I didn’t like the PRO, or didn’t want it anymore, they do have an incredible resale value, so I could likely get a large chunk of my money back fairly easily.

Transporting it (related to cost)

While the cost issue is mitigated after just biting the bullet and hitting the “check out” button, there is another minor annoyance that is related to the high price. Ideally, I’d like to use my HKKB whenever I am at the computer. Being a software developer… I use a computer for 8+ hours a day at work. So, I like to use my keyboard at work… but I also like to bring it home. Ideally, I’d get two: one for home and one for at the office. At the high price point though… I might have to wait to make that purchase. As far as keyboards go, I’d say the HHKB is probably one of the best travel boards out there. It’s light, but sturdy, and there is even a [travel case designed specifically for it](https://www.amazon.com/Carrying-Happy-Hacking-Keyboard-PD-KB01SD/dp/B000FHSNFW/ref=sr_1_1?ie=UTF8&qid=1514499138&sr=8-1&keywords=HHKB+case go). So far I’ve been using the box to transport it, but am going to get the traveling case soon. That should hold me over until it is justifiable to get a second board. One day…

I actually was just gifted the carrying case for Christmas. I will check out how it works in the new year. It should at the very least be better than the cardboard box it came in…

“Mac” mode / Delete “fix”

One benefit of getting the HHKB Pro over the lite version, is that it has some media key functionality (volume, start/stop). However, these keys only work when the dip switches are configured to put the board in “Mac mode”. In addition to enabling the media keys, it changes a few other things, like changing the iconic HHKB “Delete” key to a more traditional “Backspace” default. Now, I understand that 99% of people probably welcome this change with open arms. However… after using the HHKB lite for so many years, I have really become accustomed to the “delete” key there, and actually prefer it now. With how I jump around my editor/terminals with emacs bindings, I have actually developed a workflow where I use delete more often than backspace.

I cannot figure out a way to have both the mac mode and “delete key default” functionality enabled at the same time. So… I have opted to just forgo the new media keys, which is unfortunate. It’s a nice upgrade from the lite version, and a shame I can’t take advantage of it just because I prefer the default layout of the keyboard.

Missing more “Modern” Features/Perks

The HHKB being a “classic”, seems to be lacking in some of the more “modern” features of high-end keyboards. There is no backlighting, no custom programming/macro setups, or wireless connectivity. I even believe that the two USB ports on the back are USB 2, not 3 (let alone C). While it would be nice to have some of these features in the HHKB, I’m not hurting too badly for them… yet. A modern refresh would be amazing though.

Conclusion

All in all, it is a great keyboard and I plan to use it for a long time. Typing on it feels and sounds so natural, and it has the same layout I have grown to love over the last few years. I am thrilled to finally upgrade to the pro, and wish I did so years ago.

Website Switched to Hugo

2017-09-02T00:00:00Z

Fort Rodney, St Lucia

In my last post, I stated that within a few days, I would officially generate and publish the website using Hugo. I then proceeded to publish that post using Hugo. Close enough. If I remember correctly, I ended that post listing off a few tasks that I wanted to complete before switching the site (oops). Well… they’re complete now. Here’s how.

Create Single Pages (About/Homelab)

This wasn’t so hard to do once setup, but it took a little while for me to get to that point. I was unaware that I needed to configure a layout for the pages, which was the source of most of my confusion.

After that realization, I created a new layout directory (in my theme directory) for each of the pages to add: /layouts/about/ and /layouts/homelab/. I then copied the /layouts/post/single.html file into the two directories to use as a template for the two new layouts (/layouts/about/about-page.html and /layouts/homelab/homelab-page.html). The pages only required a basic layout that would inject the .Content from the markdown files. Additionally, I tweaked the header slightly to display an “updated on” date, rather than a “posted on” date.

With the templates made, I constructed a new /content/pages/ category, and added about.md and homelab.md files to it. In both files, I defined the type and layout parameters, so that the new layouts would be used. Lastly, I used the menu parameter to declare that each page would be part of the main menu.

---
title: About
date: 2017-08-28T09:51:18-04:00
type: about
menu :
  main:
    weight: -150
layout: about-page
image: img/header-images/park-books.jpg
---

Setting up an RSS Feed

It turns out that Hugo ships with its own RSS 2.0 template by default. When I first saw this, I thought that I may still have to dash off a layout or markdown page for the feed, but even that was unnecessary. Each “content” section (ex: post or pages) has an RSS automatically generated at /section-name/index.rss. I don’t need a feed for my static pages, so I just found the feed for my posts. To make it easily accessible, I added a menu link. Without a defined markdown file for the rss feed page, I needed another way to add to the navigation menu. I accomplished this by adding the following code to the bottom of my config.toml file:

[[menu.main]]
  name = "RSS"
  url = "/post/index.xml"
  weight = 0

This is how both the Home and Archives links were already been added to the navigation bar by default in the theme. I just added another item for the RSS link, and adjusted the weight to have it to show up at the end of the menu. That was it.

Check how the posts display

One task I needed to complete was going through and editing each post. The main issue that requiring a fix, which I explained in the previous post, was that the image tags needed to be switched from markdown to html syntax. While I originally planned to set a width=100% parameter for the image tags, I learned that using a max-width: 100%; worked much better for my use-case. I know this can be handled in the overall css file, but I like explicitly defining how to handle each image when I write a post.

While editing the posts, I noticed that the post summaries weren’t displaying the content that I intended them to. I have hugo configured so that I manually cut off the summary location using a more tag in the markdown.But it didn’t appear to be doing that.

A: An over-extended post summary.
B: The extra content of the post that was included in the summary.
C: The corrected post summary.

Some of the summaries seemed to extend beyond the cutoff point, still including the next section header, and some of the section’s content. After further inspection, I noticed that I had a space on either side of the “more” in the tag. So, I had to go through and delete the extra spaces in each post.

Next/Prev Posts

The main “Small Tweak” that I wanted to figure out was setting up navigation links at the bottom of each post page. I added some code between the and tags of my theme’s /layout/post/single.html file. I first made a section using if statements to establish the “Next Post” and “Prev Post” header line. Then in a second block, I placed the actual links using on the next line using with statements. The if and when statements are required so that previous and next posts are only linked if they exist. So, the first and last post will only display one of the two links.

<!-- Next Post/ Previous Post Links -->

    <div style="float: left; text-decoration: underline; text-align: right;">Next Post:</div>


     <div style="float: right; text-decoration: underline; text-align: right;">Prev Post:</div>


<br>


    <a href="" style="text-align: left; float: left; max-width: 40%;"></a>


    <a href="" style="text-align: right; float: right; max-width: 40%"></a>

Conclusion

I have continued to enjoy using Hugo. The few issues I had with it, keep being quickly dismissed as I learn more about the system. It is simple to use, but at the same time provides an immense level power and control. I will continue tweak the site here and there, but I think for the most part, I can consider the transition complete. Which I am very happy about :).

Transitioning Website to Hugo

2017-08-27T00:00:00Z

Fort Rodney, St Lucia

While I have loved using Cryogen to create this website for over a year and a half now, I have started the transition to using another static website generator. Specifically, I have been experimenting with Hugo. This post will detail why I am switching, what I have ported over thus far, and what still needs to be completed before generating the official site with hugo.

From Cryogen to Hugo

My departure from Cryogen really has nothing to do with the project itself. It is a prime example of what clojure is capable of, and I feel that more people should give it a shot. Recently though, I’ve been itching to switch up my website’s theme a bit. While I fancy my Immutable Theme I created a couple months ago, it isn’t quite doing what I had hoped for. I love dark themes, but the type of posts I’ve been creating really don’t look good with them. I like to add diagrams, code snippets, and images to all of my posts. Diagram posts look wonderful with a white background, but are garbage in a dark theme. This difference was glaring last week as I switched between my markdown editor’s preview window (default white theme), and the website’s live preview view, while writing my reverse tunnels post.

Similar to the logic I employed earlier this month when switching back to Solus, I thought that if I was going to scrap my theme and start from scratch, I might as well check out different website generator. I had been keeping an eye on the various site generators, but the one I considered the most (and even dabbled with a bit last month), was Hugo.

What I’ve Done So Far

I started the process of transporting my website from cryogen to hugo. So far, my experience with Hugo has been great. Here is what I’ve done:

Installed & Setup a test Hugo site

Obtaining and installing Hugo on my computers was very simple, as it was in the reops (Solus). I just had to run the command:

sudo eopkg it hugo

After I had hugo installed, I experimented with creating new website projects for a few minutes before finally creating one to start my transition. To create a new website, I used the command:

hugo new site ryan-hugo-test

This created a new directory with all the project’s default core files, and adhering to the required hugo directory structure.

Like cryogen, hugo can spin up a website in a test server during development. To do this, use the command:

hugo serve -D

Note: I used the -D flag to additionally include any files marked as “drafts.”

Setup a Theme and started tweaking it

With the hugo site generated, I wanted to setup a proper theme. After sampling a handful of demo sites from hugo’s theme page, I decided on the startbootstrap-clean-theme. I’ve seen it used on other sites, and I think it is a commonly used theme with other site generators. However, it is clean and simple, with a white-background base like I wanted. Additionally, I love having header images for posts and pages. It lets me better personalize the website by exclusively using images I’ve photographed myself.

To get the theme, I cloned it from git:

git clone git@github.com:humboldtux/startbootstrap-clean-blog.git

Note: this particular theme has an abundant amount of features, so it is a good idea to copy the provided example config.toml and build off of it. This is one reason why I started with configuring the theme right off the bat.

Before I started ripping into the theme too much, I copied the theme directory to make my own version, and set the theme line in my config.toml to reflect the change. I also went through all the lines of the example config.toml and changed them accordingly.

With the theme setup, and the configuration edited, I started making some minor tweak to the site. The main tweak I made was to add the Summary contents to the post list on the home page. I currently use this feature in Cryogen, so all of my posts are written to support it. Summaries are baked into hugo, but I needed to edit the theme to include it on the posts page. To achieve this, I edited the /layouts/post/summary.html file of the theme slightly:

original:

<div class="post-preview">
  <a href="">
    <h2 class="post-title"></h2>
    <h3 class="post-subtitle"></h3>
  </a>
  <p class="post-meta">
      
 </p>
</div>
<hr>

with post summaries:

<div class="post-preview">
    <a href="">
        <h2 class="post-title"></h2>
        <h3 class="post-subtitle"></h3>
    </a>
    <p class="post-meta">
        
        <div class="summary">
            
        </div>
    </p>
    <p class="post-meta">
        <a href=""> Click to Read More --></a>
    </p>
</div>
<hr>

I also setup and fixed the code syntax highlighting. I first installed pygments, which again was very easy to do on my Solus computers:

sudo eopkg it pygments

After that, I just made sure that the language was specified at the start of each code snippet. I ran into an issue where the colors for the code were poorly selected, resulting in some invisable text. Apparently, this is a common issue, and I found some .css code to add to the pre section of my /static/css/clean-blog-min.css file that resolved the issue.

If all went well, the code above should have proper syntax highlighting.

Dumped my Backlog of Posts and altered the header content

To test out if the website worked properly, I wanted to import all of my posts. First… I needed to figure out where to put them. Hugo has a slightly different file structure, but I eventually figured out that I could create a /content/post/ directory, and dump them there.

Cryogen, written in clojure, uses a clojure map for the post’s meta data. Hugo on the other hand, uses a several font matter formats (toml, yaml, json) for meta data. So, I had to convert the post headers. I’m sure that there was a simple, programmatic way, or even a tool, created to accomplish this… but I just did it by hand. It wasn’t so bad. I used emacs.

After importing the markdown files for the posts, I needed to add all the images the posts contain. Again, this took a tiny bit of research to figure out the file structure, but I quickly learned that anything in the /static/ directory gets copied to the site’s root directory when the site is compiled. So, I was able to copy my cryogen /img/ folder directly to /static/img/ in hugo, and all my image paths worked out-of-the-box!

Unfortunately, I immediately noticed that some images in the posts were massive, and not constrained to the content width. I looked for a solution, and tried editing the css, but I eventually just started to convert the markdown syntax images, to use normal html <img> tags, with a width=100% parameter. Ultimately, this gives me a bit more power with how I set images anyway.

Made a bunch of header images

After fixing my in-post images, I started to play with post/page header images. I figured out where they were located in the theme, and added two of my own to replace the defaults. I read that a header image can be set with the image == "..." option in a post/page’s font matter… so I went a little crazy. I stayed up late browsing through some of my photos, and converting them to header images (I did this by shading them a just a tad, so the overlaying text is legible).

What Needs to be Done

This post has turned out to be a giant monster, so I’ll be brief here and turn the sub-sections into a quick list instead.

Create Single Pages (About/Homelab) - I still need to figure out and create my website’s single pages. These include the About Me and Homelab pages. The content of those pages has also been slipping out of date… but I won’t let that hold me up from switching the site over first.
Setting up an RSS Feed - Similar to the task above, I want to make sure I have an RSS feed for the blog configured, and accessible in the main menu.
Check how the posts display - I need to go through the posts and make sure they are displaying content correctly. I’ll check that links work, images aren’t massive, code syntax languages are set, etc. This actually shouldn’t be as bad as it sounds because I’ve done much of it already.
Small tweaks - There are bound to be a few tweaks here or there that I’ll notice and want to changed (example from above: I’ve already added the post summaries to the home page post list). One item that comes to mind is adding next and previous post markers at the bottom of each post. Again, I won’t let that hold me up though.

Well, that’s about it. I’ll work now on editing and publishing this post, and with any luck, the website should switch over to the Hugo generated one within a few days. I hope you enjoy it!

Simple Reverse SSH Tunnels

2017-08-26T00:00:00Z

Cliff Walk, Newport, RI USA

Reverse SSH tunnels are very useful and simple to setup, but can be a bit tricky to figure out at first. So, here’s a brief and simple guide on how to easily create reverse tunnels.

SSH Tunnels

A secured shell (SSH) tunnel is an encrypted tunnel, created with a connection using the ssh protocol. It can be thought of as a pipe between two computers that data travels through. Being encrypted, people outside the pipe can only see that data packages are traveling through it, but cannot read the actual contents of the package. SSH tunnels are used to securely connect between devices, as well as forward ports between devices. For example, if I am working on a website hosted on port 3000 of my internally networked laptop, I can use a ssh tunnel to forward that port to one on a public server so that friends can view the website.

Reverse Tunnels

Reverse tunnels are just like normal ssh tunnels except… well… in reverse. This means that I can connect to a remote computer, and have its port tunneled to me, which can be very handy. This is largely used for one particular usecase: providing easy, temporarily, ssh access to computers behind a network and/or firewall.

To make all of this (hopefully) easier to understand, I have drafted up a few diagrams. For the example, lets say I am away and my wife wants me to fix something on her laptop. Unless I have her properly configure the router to forward ssh traffic to her laptop, I normally cannot do this. Additionally, she might be at a friends house, office, or other public place where there is no access to the router controls (well, she shouldn’t). So, her laptop doesn’t have a direct public IP address, but our server does.

Two computers (at least one without a direct public IP), both with access to a cloud server with a public IP.

Because my wife can connect to the public cloud server, she can initiate a reverse tunnel from her laptop, with the server. The tunnel directly connects the server to her laptop.

The laptop computer creates a reverse ssh tunnel to the cloud server.

On my end, I first ssh to the server from my computer. Once the tunnel is started, I can then ssh again to a specified local port on the server, and it will tunnel me directly to her laptop. This will give me a command prompt as if I was sitting with an open terminal at her computer.

With the reverse tunnel setup, the first computer can ssh to the second via the cloud server and tunnel connection.

When I am done working, my wife can close the tunnel, and I will no longer be able to access her computer.

Creating The Reverse Tunnel

SSH tunnels can be initiated on linux easily from the command line (assuming ssh is setup and properly configured). To create a reverse tunnel, use the -R flag. After the flag, provide what I call the “path” of the tunnel. So, the server’s port where the tunnel will be found, the host of that port (I almost always use localhost), and the port to be tunneled. Lastly, make sure to specify the IP or hostname of the remote computer just like in a typical plain ssh hostname.

ssh -R remote-port:localhost:local-port host

Example:

ssh -R 19999:localhost:22 meowth

Connecting

After setting up the tunnel, the initializing computer can be accessed from the server by ssh’ing to the remote-port of localhost (defined above):

ssh -p 19999 localhost

This should start an ssh session to the initializing computer. In my example, this is the laptop.

Conclusion

SSH tunnels are a fascinating and useful piece of technology. The ssh protocol makes modern computing much more secure, and easier to manage multiple computers. That’s reverse tunnels in a nutshell. Have fun tunneling!

Back to Solus

2017-07-31T00:00:00Z

Well, that was faster than I expected. This past weekend, I wiped the Fedora installation on my main computer, and replaced it with Solus. My Fedora install broke, and I needed to use a Fedora Live CD to fix it. I thought that if I had to use a live CD to fix the issue, I might as well just do a clean install. With the idea of a clean install in my head… I thought (possibly influenced by my recent motivation to play with Go), that I might as well do the clean install with Solus…

Nvidia Issues

Basically, what I think happened was that my Nvidia drivers got messed up during an update. When I rebooted my computer, my screens were black. This happened no matter which kernel I booted into from Grub. I assumed it was a graphics problem and preceded to switch to another tty to login and fix the problem. I was able to get to a CLI login screen (with the normal user: prompt) and tried login in. However, after typing in my user name and hitting enter, instead of being promoted for a password, I received an “Error Logging In” message. So, I tried the same thing with the root user account. Same thing. After typing in a user name, it yelled at me. There was no way to log in.

I jumped into the IRC chat and started asking around for advice. As always, everyone was very helpful and tried to help me troubleshoot. They shared my bewilderment. We came to the conclusion that it must of been something with the Nvidia drivers. I was told that it is common to have issues when installing the drivers from Nvidia’s web site instead of the RPM Fusion Repos (I intended to install from the RPM fusion repos, but I went to the nvidia site to find out what version to should use… and then just downloaded it from there. My bad :P).

Anyway, I quickly realized that at the very least, I would have to boot up and log into a Live CD to fix the problem. As stated in the intro paragraph, while waiting for the ISO image to download I thought that if I have to load up the CD, I might as well just do a clean Fedora 26 install. A couple minutes later as I was booting up the Live CD, I through that as long as I was reinstalling my OS… l should consider installing… Solus.

Video Link – Linus Torvalds famously flipped off Nvidia during a Q&A

Installed Solus

With the Fedora live environment all booted and ready, I started downloading the latest Solus ISO (Budgie) on my other laptop. Once the ISO image was mounted to a drive, I booted it up and started installing Solus. After the install, I immediately updated because I knew there have been a TON of updates since the last ISO snapshot (mid-April). When I rebooted, I absolutely loved what I saw. It is amazing how much Solus and Budgie have improved, even during my short distro-hopping vacation.

Looking at my post history, it may appear as though I used Fedora for a few days, became fed up with it, and came running back to Solus. That is not entirely true. The reality is that I sat on writing my Fedora post for a long time. I used Fedora for about a month, and it was generally a great experience. If it didn’t break, I probably would have stayed for a bit longer. When considering a new install though, I realized that I had missed Solus.

As of now, my plan is to keep using Solus, at least on my main computer. I am also hoping to get more involved with the project, and start packing up some software again. Maybe I can even help with some debugging and development. We shall see…

Dabbling with GO

2017-07-30T00:00:00Z

Portland, ME USA

After spending most of this month’s dedicated learning time working on system, server, and network activities, I have been itching to start some home programming again. To motivate myself, I even considered dabbling with a new programming language… and with very little internal debate, I decided to just Gopher it (I’m so sorry).

What is GO?

Go (sometimes referred to as “golang”) is an open source programming language, developed in 2007 by a team at Google (Robert Griesemer, Rob Pike, and Ken Thompson). Distributed under a BSD-like license, Go is also maintained and developed by open source volunteers all over the world.

The GO mascot.

Like C, it is a compiled and statically typed language. Unlike C, Go includes garbage collection and memory safety features, as well as other design aspects that are reminiscent of modern dynamic languages like python (ex: type inference and package import statements). Lastly, Go has a concurrent programming implementation that utilizes what are known as ‘goroutines’. Goroutines are special light-weight “threads” that can process many concurrent tasks. All of these features work together to form an extremely relevant language for modern computing.

Like the programming language itself, the Go project summarizes all of this in a nice and concise statement on their documentation page:

The Go programming language is an open source project to make programmers more productive. Go is expressive, concise, clean, and efficient. Its concurrency mechanisms make it easy to write programs that get the most out of multicore and networked machines, while its novel type system enables flexible and modular program construction. Go compiles quickly to machine code yet has the convenience of garbage collection and the power of run-time reflection. It’s a fast, statically typed, compiled language that feels like a dynamically typed, interpreted language.

Motivation

I’ve wanted to try Go for a long time now. A couple of months ago, I was trying to decide if I should pursue learning Rust or Go. At the time, I ultimately ended up experimenting with Rust. I was toying with the idea of creating an experimental lisp, and Rust’s feature set makes it great language for writing compilers and interpreters. However, while Rust is a great language, it can be quite complicated to learn. Go on the other hand, is apparently simpler and quick to grasp, which is why it was created in the first place.

When the Solus team announced that they were declaring Go a first class citizen language of the project, my interest peaked. Solbuild, the Solus package build system was written in Go. In the announcement, the project stated that they intended to use Go for building tools. True to that statement, this past week, Ikey (the creator of Solus) published a patreon post detailing the new repo manager (ferryd) he’s been working on… again in Go. After reading that post, I decided it was time for me to give it a Go (again, very sorry).

Installation

I first went to the Go install page to figure out if there were any odd components to install. It didn’t appear so.

Note: Before progressing any further, I should point out that if you just want to try go, the project has a little embedded editor/compiler on the home page of the website. Beyond that, they have an amazing Go tour that also has an embedded programming environment, and can be completed entirely in a web browser.

After playing with the online editor online for a bit, I decided that I wanted to install Go on my system. I was on my NixOS laptop at the time, so I installed go with the command:

nix-env -i go

On Alakazam, it was

sudo dnf install golang

(… and then when I jumped back to Solus on Alakazam, it was sudo eopkg it golang… but more on that later…)

Next, I walked through the test your install steps, building a simple “hello world” app to make sure everything was working properly. This was especially important, given that I was on nixOS, which can sometimes be picky with system paths and environment variables. Luckily, everything worked fine. If I continue down the Go path, I might write a “Getting started” post to detail how to setup a proper Go environment on Linux.

First Steps

After confirming my install, I went back and continued A Tour of Go to better learn the language. I am interested to eventually read the Effective Go documentation. It will be interesting to simply read the correct style and conventions for the language, instead of trudging through a holly war to find answers.

After working on the tutorial for awhile, I started playing around with the language on my own. Below is a snippet of code I wrote while fooling around. It’s nothing fancy. I was impressed though with how easy it really was to pick up the basics of the language and get going. I am excited to learn more.

package main

import (
	"fmt"
	"runtime"
)

func main() {
	fmt.Printf("hello, world\n")
	getOS()
	fmt.Printf("\n") // I know this is dumb

	x, y := 74, 83
	sum := sumInts(x, y)
	fsum := factorial(sum)

	fmt.Printf("sum: %v\n", sum)
	fmt.Printf("Factorial of sum(%v): %v\n", sum, fsum)
}

func sumInts(x int, y int) int {
	sum := x + y
	return sum
}

func factorial(n int) int {
	if n <= 1 {
		return 1
	} else {
		return n + factorial(n-1)
	}
}

func getOS() {
	os := runtime.GOOS
	fmt.Printf("OS: %v\n", os) // impure
}

Migrating from Solus to Fedora for now

2017-07-25T00:00:00Z

For awhile, I have been debating the idea of switching back to Fedora (from Solus). At least on my main computer. First, let me state this right up front: I am still very satisfied with Solus. I think it is one of the best current Linux distros, and I want to still contribute to the project. However, there are a few reasons why Solus isn’t the best fit for my needs right now, and I will highlight them below.

Why Switch

The Solus (Left) and Fedora (right) Project Logos.

The first phrase stated on the Solus Project homepage is “Solus is an operating system that is designed for home computing.” I find this to be true, and Solus does a great job at it. The Linux community needs a few good, focused, distros. While I have been using Solus for my “home” computing, the computing tasks I’ve focused on recently do not fall into the category of standard home computing use. Recently, my main top computing activities and goals are:

Writing (okay… but still)
Running all sorts of VMs
Trying various Server Technologies
- ZFS
- KVM
- Containerization Technologies (LXC, Docker, …)
- Ansible and other automation tools
- Etc.
Learning about Clustering
- OpenMP
- High Availability
- Distributed File systems
Trying to get involved with some other Open Source Projects
- Fedora (Infrastructure, Dev)
- NixOS

As you can see, many of the above items are not desktop based, but really server based operations. Solus is not a server distribution, as it doesn’t try to be one. Which is a good thing. It is focused on its audience. I just happen to not be in that audience at the moment.

Additionally, one of my goals for the near future is to transform my long-time Proxmox server into a CentOS box. Using Fedora on my main workstation does help me get accustom to that environment. It also allows me to more accurately test out ideas before I plan the big move.

Lastly, I had been eyeing up the Plasma desktop, and wanted to try that out again. At the time of writing this post, Solus doesn’t fully support the Plasma desktop (yet). However, Fedora does have a KDE Plasma Desktop Spin.

The Switch

Screenfetch on my new Fedora Install.

I eventually (and somewhat sporadically), made the switch to Fedora. I switched the week before the 26 release, so I decided to start with 25, and then upgrade later (although I did test out 26 on my other laptop). This let me ensure that the 26 issues were ironed out before upgrading. I also got to test out the dnf system upgrade process. I recorded post-switch notes during the last few weeks to document how everything went.

“Well… I did it. I switched to Fedora on Alakazam yesterday. I went with the Fedora 25 KDE spin and did all of the hoops to get that up and going. Not only is it taking some time to get used to Fedora again, but I am needing to reacquaint myself with the KDE environment… it does seem different that other Plasma setups I’ve used in the past… But I like it.”

I got used to it in no time:

Alakazam is doing well on Fedora. I’ve been enjoying it and think I will stay on it.

Updates

I ran some updates that I thought might be problematic, based on previous issues I’ve encountered with Fedora. I made sure to note the results as well. The first notable update was from the first time I updated the kernel, and the second was upgrading from Fedora 25 to 26. Both updates went very smoothly without any issues:

Kernel Upgrade

So I am about to do my first Kernel update since being on Fedora again (on Alakazam). We’ll see how the video drivers respond… To note, I am still on 25 so it hopefully won’t be too bad…

…reboot…

No issues whatsoever :)

Upgrade to Fedora 26

I upgraded my Fedora 25 Plasma Install to Fedora 26.

Other than some odd issues with the GUI tool, the upgrade from 25 to 26 was smooth and uneventful.

I just upgraded Alakazam from Fedora 25 KDE, to Fedora 26 this morning. I couldn’t really get the graphical installer to start, but that could be because I have several desktop environments setup (Plasma & Gnome), so it may have been confused (I was using the Gnome Software App in Plasma…)

I just did the upgrade using the (command line) dnf upgrade tool, like I normally use, and it worked wonderfully. It even looks like my nvidia drivers stayed and my monitors were configured correctly after rebooting. The only difference is I don’t seem to have the same Plasma animations I had before, but that is fine, and likely part of the update.

So that it. Those are my reasons for switching (for now), and the results of my switch. I am still happy with Fedora, at least on Alakazam, and will likely remain on it until I have a convincing reason to leave.

Creating a git Repo Remote

2017-07-19T00:00:00Z

Boston, MA USA

For over a year or so, I have been using a self-hosted Gitlab to host all of my private repos. For a few months now, I have been meaning to migrate my Gitlab repos to bare, minimal ones, hosted directly on a server. The majority of my code/configs are hosted publicly on my Github page, and it really doesn’t make sense to maintain a full Gitlab instance for the few (like…2) private repositories that I keep. Moving the git repos to new ones right on the server is actually fairly simple. For such a simple process, all the guides I saw online went way above and beyond what I needed. So, here are the two steps I did to migrate my repos.

SSH Keys

The pre (and somewhat optional) step to is to setup ssh key authentication. If ssh keys are not configured, git will prompt for the password of the repo’s host user . When using a git service (ex: Github or Gitlab), this is usually unknown, so ssh keys are required. When rolling your own remote git repo, the password will likely be known. Still, setting up ssh authentication makes the process easier and more secure. If you do not know how to configure ssh keys, I included a small ssh key how-to here in a previous post. Many of the git guides out there call for creating a git user and setting up ssh keys with that user. This is a great idea if multiple people need access to the git repo. However, for my purposes I will use my username, as I will be the only one accessing it (which in my case is a good thing).

Creating Server Repo

Once ssh authentication is configured, ssh into the remote server that will host the git repository. Creating the remote repo is a simple process. First, make a directory for the repo (the normal convention is to use a .git ending: REPO-NAME.git). Next, jump into the created directory (cd) and run the command git init --bare.

mkdir REPO-NAME.git
cd REPO-NAME.git
git init --bare

This will initialize the repository inside that directory. The git init command is used to create a git repository. The --bare option flag tells git to treat it as a bare repository. Bare repositories do not contain a working or checked out copy of the source files. Thus, the plain git init command creates a working repo, while git init --bare is used to create a sharing(server) repo. This allows the working repositories of many developers to be synced with the server repo.

Cloning Repo

If the remote git repository is a totally new repository, it can be cloned down to a working directory on a developer machine fairly easily:

git clone user@hostname:REPONAME.git

Pointing Local Repo to Server

However, I already had an existing working repository that I wanted to sync with the new remote shared repo. With the remote repo initialized, I wanted to point my existing git repository on the local machine to it. To do this, enter the directory of the git repository, and edit the config the (.git/config). To redirect the repo to point to the new remote, edit the url line to the location of the repo:

username@hostname:reponame

cd Server-Node-Files
vim ./git/config

After pointing to the new remote, feel free to push the content to it: (Only push everything (*) if it is desired)

git add *
git commit -m "First push to new Remote"
git push origin master

And that’s it. Enjoy spinning up and using your own personal git repositories!

Encountered Issues Setting Up Ubiquiti Network

2017-06-29T00:00:00Z

WWI Memorial Park, North Attleboro, MA USA

This past weekend, I setup my new ubiquiti network. It actually took up a good portion of Sunday, because I ran into a few minor issues. Fortunately/Unfortunately, these issues were mostly because it my first time configuring this type of setup, and there was a lot of trial and error. The basic network is now all configured and has been running great. It was a good day and I learned a lot :). In fact, I am confident that if I had to start over from scratch, the process would take me about 10-15 minutes. Just to be sure, I’m going to quickly jot down the major pain points I experienced my first time around.

Trouble Connecting to the EdgeRouter-x for Initial Setup

During initial setup, I was connecting the router wrong

The edgerouter needed to be directly connected to a computer during its initial setup, to make the EdgeOS configuration wizard accessible. The instructions clearly stated to connect an ethernet cable from my laptop to the eth0/POE port on the edgerouter, but I guess I didn’t believe them.

Instead, I plugged the ethernet cable from my modem into eth0, and my computer to eth1. That didn’t work. However, once I properly connected the devices (and manually set a static IP on my laptop, 192.168.1.2 for example), I was able to access the configuration page in my browser via https://192.168.1.1 (don’t forget the s in https). Lesson Learned: manuals are (usually) not out to get you.

Setting up POE and Connecting the AP

This was not actually an issue I encountered, but rather a confusion. I was unsure what the best setup for the POE hardware was. At first, I had the POE adaptor connected between the edgerouter and the AP, because I wasn’t sure if it could optimally power both devices. I found an informative guide that indicated that the POE adaptor could indeed power both.

Modem  --
        |
         --> POE Adaptor --> (*eth0*)  edgerouter-x  (*eth4*) --> AP Lite
        |
Power  --

Digram describing the correct link up

Once I swapped the cables all around, I had to go into the router configuration and enable the POE for eth4. Afterwards, the AP lit up, indicating that it was connected and powered.

Issues linking/configuring the AP

This was the problem I spent the most time on. I had to install the configuration software for the Ubiquiti access point, but the “Linux binary” was a .deb, and I didn’t feel like extracting the contents of the package so that I could install it on Solus (yet). So instead, I spun up a few Ubuntu VMs to try it out, but I over-looked the fact that VMs on my laptop use a different subnet (192.168.122. *) for the virtual network. I had hoped that because the VM’s network was routed through my laptop, which was connected directly to the edgerouter, it would still be able to see the access point. Regardless… the AP couldn’t see the VM and vice versa. Finally, I admitted that the issues were most likely caused by the 192.168.122.* IP address that the VM was assigned.

I spun up a new Ubuntu VM (Venomoth) to host the Ubifi controller

By this point, I had also learned that the Linux software is more of a server service, and not a GUI desktop application. So, I concluded that spinning up a dedicated VM on my server to host the wifi controller was worth it. Virtual machines hosted on my server automatically get configured on the main subnet, so it also resolved my issue. I was able to detect and configure the access point immediately. This setup made more sense anyway, as I can always connect to the AP controller by going to the VM’s IP on my browser, just like I can with my router.

Well, that was all of my setup “issues”. There was nothing I would consider to be an actual issue, just some confusions of an Ubiquiti/POE first-timer. Like I stated earlier, I am sure I could redo the setup in about 15 minutes without any issues… 10 now that I recorded everything in the post!

Home Network Ubiquiti Upgrade

2017-06-26T00:00:00Z

A few weeks ago, I went back to Pennsylvania for a week to attend my college swim team reunion, and my brother’s high school graduation. While I was away, the wifi-router Rebecca and I were using decided to die (of course). When I returned, I setup our old router as a temporary fix. It was terrible. So, I began researching how I should upgrade our network. This time around, I am doing this correctly.

The router that died was a Linksys WRT 1900. When I got home, I setup our old router as a temporary fix. It’s a very basic Linksys E1200. After using it for just a few hours, I remembered why I hated it so much. It is slower than a snail, and seems to stop working each day or so, requiring me to unplug and re-plug it daily (I think it is something with DHCP. It keeps trying to reassign IPs to devices, and then doesn’t seem to understand how to accept their requests afterwards). Thus, the more temporary this solution was, the better.

If I am redoing our network setup, I want to do it properly this time, splitting out the router from the wireless access point using good hardware and software. My plan is to get an ubiquity edge-router-x, and pair it with an Ubiquiti wireless access point.

Originally I wanted to build a small pfsense box to use as a router, but after digging a little deeper and doing some research, I saw that the EdgeRouter would more than meet my needs and is a great start to a network upgrade. Additionally, at $50 (USD), it truly is a great deal. As for the wireless access point, I was always considering an UniFi device, and I thought the EdgeRouter should pair rather seamlessly, considering both products are made by Ubiquiti.

From what I read, the EdgeRouter is a great router with an okay firewall, while pfsense is an amazing firewall that can do routing. So, if I want to dig more into pfsense in the future, I can still set up a firewall box, and connect it in front of the router.

Modem --> pfsense --> EdgeRouter-x -> devices

So… I think I’ve finalized my decision and will purchase the items soon. I will update when I get them.

Update (06/24/2017): I have followed through with this purchase, and the items arrived from Amazon. Expect a post or two about my experience setting them up (and yes, I added this update before actually publishing this post)

Refreshing my i3 setup with i3blocks

2017-06-22T00:00:00Z

a rofi window

The last couple of weeks I have shifted back to using the i3 window mangers. When I fired it up, my fingers danced across the keyboard, remembering all of the personalized keybindings I have cultivated in my i3 configuration over the years. It is a simple, beautiful setup… well, beautiful minus one of the components. My i3status bar was looking rather bland and dated, especially compared some of the i3 setups posted by all the cool kids over at /r/unixporn. I decided it was time for a refresh.

i3status

One of my simple i3status setups.

When I first configured i3 several years ago, I used i3status because it was easy to use with i3 and did everything by default. Over time, I learned how to create and modify my own .i3status.conf so that I could get it to play nice with un-standerd configurations (ex: /Data partitions and such). While i3status served me well for many years, using the same-old setup has become boring. I started noticing several other nice looking status bar tools being used in i3 setups, and wanted to try them out.

Polybar

The example polybar.

The first bar I saw and tried was polybar. I started with it because some of the examples look awesome. It looks very modern and has an infinite number of features. I set it up and was able to use the example bar just fine. However, when I started to customize my own, I started to run into a few issues. The biggest issue was polybar not detecting my work-space names, along with other elements. Additionally, due to the support for several window managers, the example configuration file seemed cluttered, and I was never sure what I could edit, and what I should delete. After some frustration, I decided put it aside for now. I might come back to it one day when I’m bored.

i3blocks

Next, I learned of i3blocks. It appeared to have everything I wanted in a status bar, yet remained simple, respecting the i3bar protocol. So I gave it a whirl.

Downloading from the Repos

Just like installing any other package on Linux, I decided to first check to see it it was in the Solus Repos:

sudo eopkg sr i3block

It was. So, I installed it (sudo eopkg it i3blocks) and started learning how to setup my configuration.

Fonts

A *Very Small* Sampling of the Awesome Fonts

The first hurdle I came across when first launching i3blocks was that I did not have all the fonts used in the default configuration installed. I temporarily removed the special fonts from the config, just so I could check that everything was working. But what’s the fun in that? One of the biggest reasons I wanted to redo my bar was to have cool modern icon fonts! So, I found the font awesome package in the Solus repos and installed it.

sudo eopkg it font-awesome-ttf

With the awesome fonts at my disposal, I had a plethora of icons to use. So I went through and picked out icons for each of the work-space tabs and status markers.

Finding Git Repos

Example of Modules in new i3-block repo

While researching how to configure i3blocks, I started encountering several different GitHub repos and forks of the project. Some forks seemed to have additional modules that my repo version didn’t. I downloaded and built this one to try out. I then configured i3 to point to the new build instead, and got started setting up my own blocks.

Forking my own for Solus Tweaks

As I was configuring the individual blocks, I noticed that many of them didn’t work by default on my computers. I dug deeper by opening up and peeking at the actual bash scripts blocks refer to. I noticed that many of them were trying to query data from applications that are not used in Solus. For example, the battery block didn’t work for me because it relied on acpi, which, while often used in Arch Linux for battery information (I use it myself when on arch) is not packaged in Solus. Instead, Solus relies on upower. To fix my issues, I cobbled together my own battery bash script, that queried battery information using upower instead. Note, the BATTERY_ICON uses the battery fonts (which likely won’t show in the browser). Unless it is charging… then it uses a lightning bolt :) .

#!/bin/bash

BATTERY=0
BATTERY_STATE=$(echo "${BATTERY_INFO}" | upower -i $(upower -e | grep 'BAT') | grep -E "state|to\ full" | awk '{print $2}')
BATTERY_POWER=$(echo "${BATTERY_INFO}" | upower -i $(upower -e | grep 'BAT') | grep -E "percentage" | awk '{print $2}' | tr -d '%')
URGENT_VALUE=10

if [[ "${BATTERY_POWER}" -gt 87 ]]; then
    BATTERY_ICON=""
elif [[ "${BATTERY_POWER}" -gt 63 ]]; then
     BATTERY_ICON=""
elif [[ "${BATTERY_POWER}" -gt 38 ]]; then
     BATTERY_ICON=""
elif [[ "${BATTERY_POWER}" -gt 13 ]]; then
     BATTERY_ICON=""
elif [[ "${BATTERY_POWER}" -le 13 ]]; then
     BATTERY_ICON=""
else
    BATTERY_ICON=""
fi


if [[ "${BATTERY_STATE}" = "discharging" ]]; then
    echo "${BATTERY_ICON} ${BATTERY_POWER}%"
    echo "${BATTERY_ICON} ${BATTERY_POWER}%"
    echo ""
else
    echo " ${BATTERY_POWER}%"
    echo " ${BATTERY_POWER}%"
    echo ""
fi

if [[ "${BATTERY_POWER}" -le "${URGENT_VALUE}" ]]; then
  exit 33
fi

Over the past few weeks, it seems to work well enough. As I continued to tweak, or downright create blocks to work well in Solus, I eventually decided that it might be a good idea to just create my own fork of the repo. This way, I can have my own i3blocks repo that works well with Solus. I noticed that the implementation of i3blocks found in the Solus repos is also using non-Solus items for it’s scripts (ex: acpi for the battery). Maybe one of these days I’ll jump into irc and see how people feel about swapping in mine instead (or at least one that works better in Solus)…

Color Update for i3 & rofi

New color scheme for rofi launcher

When I started using i3 several years ago, I used dmenu as my launcher (it’s the launcher used in dwm, another tiling window manager I used to use). I eventually switched to rofi after seeing it used in some very nice window manager setups (sound familiar?). However, I always used the plain default theme and never bothered to improve it. With all the work I did making i3bars look nice, I thought I should at least make rofi match. Making the color theme wasn’t nearly as hard as I anticipated it to be. That’s mostly because I found and used the rofi theme generator.

Newly configured rofi window switcher

While touching up rofi, I learned it can also switch through already opened windows (Previously, I only used it to launch new programs). I really liked that in my new i3 setup, it also displayed the work-spaces, icon font and all. I immediately bound it to my Super+Tab key for easy use.

Conclusion

Click to watch a demo of my i3gaps setup with i3blocks

Thus far, I have been very satisfied with my current i3 setup. Replacing i3status with i3blocks, combined with adding small improvements, for example, the awesome fonts, really helped to enhance the look and feel of my i3 configuration. Additionally, ROFI much better now that it matches everything.

I am glad to be using i3 again. It is so efficient to use and I love flying around the work-spaces and windows. It is particularly useful on my x230’s 12" screen, as it utilizes all of the limited resolution. If you have never used a tiling window manager in Linux, I strongly suggest you give it a shot. It can take some time to get up and going the first time, but once a custom configuration is built, it always feels like home.

To show off my new i3 setup with i3bars, I’ve posted a small video at the top of this section. Enjoy!

Exporting Proxmox VMs

2017-06-16T00:00:00Z

Pigeon Island National Park, St Lucia

For a long time I have been thinking about replacing my server’s proxmox install with vanilla Debian or CentOS, mostly for learning purposes. I would first setup zfs on the new system and import my existing data pools. Then, I would use either a system like ovrit or just plain kvm/lxc to run my VMs and containers. In order to do this though… I have to first figure out how export my containers and VMs running in Proxmox. As it turns out… exporting the VMs wasn’t very hard…

Exporting Proxmox VM Disk

My Proxmox VMs are setup on an LVM virtual group, with each virtual drive being a lvm volume passed to the VM. So, I wanted to be able to extract these disks to something I could more easily transfer. I ended up converting the lvm volumes to qcow2 images because it was easy and I’ve actually experienced okay performance with qcow2 on my workstations. Additionally, qcow2 being a single file, is easy to move around and I can always convert them to something else on the final system. To export one of the VMs, I ran the command:

qemu-img convert -O qcow2 /dev/pve/vm-108-disk-1 /Data/freebsd-vm.qcow2

That was it. It took a few minutes to finish, but honestly that is all I had to do (I’m sure I should of taken a snapshot or something then copy that, but this seemed to work fine). When it completed, I copied the image down to my desktop and decided to test it out with virt-manager.

Importing the image to Virt-Manager

I opened up virt-manager and selected the button to create a new VM. At the first prompt, instead of selecting my usual “Local install media (ISO image or CDROM)” option, I choose to “Import an existing disk image”.

Import the image.

At the second menu, I opened up the browse menu to see my already configured locations. From there, I found where I had saved the converted qcow2 image, and selected it.

Next, I continued setting up the virtual specs (CPU, RAM, etc.) for the machine just as I would with any other VM setup. When I was done, I started the VM and FreeBSD booted right up. I logged in and compared the installed applications and files with the still running proxmox VM. They were identical.

That’s all I really have for this post. It was extremely simple to export the VMs. I know I’m not fully done yet and still have to import the VMs to the final system, but I’ll save that for a later post. Right now… I have quite a few VM images to convert, so I might as well get started.

Updating the Pi Cluster with Ansible

2017-05-25T00:00:00Z

With Ansible configured on the Pi cluster, it is time to have it do something useful. When working with a clustered system, even the simplest tasks become tedious and time consuming. For example, updating the system. While I could manually update each of the 3 pi nodes, it is not scalable to 10 or 30 nodes, let alone hundreds or thousands. Tools like Ansible, make doing tasks such a supdating clustered systems, trivial again. In this post, I will walk through setting up an Ansible playbook to update my Pi cluster.

Hosts File

The first task when using Ansible is to setup the hosts file. No, not the normal /etc/hosts file, but the other one just for Ansible, which can be found at /etc/ansible/hosts. Configuring the Ansible hosts file is fairly straightforward. Groups of computers are defined using [brackets], with computer ip/hostnames of the group are listed below. For example:

A nice feature of group definitions is that hierical structures can be constructed using the :child suffix in order to create groups of groups. For example, for my homelab, I like to make an ansible hosts file that splits out my servers based on their distribution, and then group those by their packaging type. This makes it easier for me to do generic updates, which is what I mostly use ansible for (at this point). So, for example:

[ubuntu]
mrmime
geodude

[debian]
ninetales

[fedora]
fedora-test

[centos]
tangels

[arch]
meowth
staryu
diglet

[deb:children]
ubuntu
debian

[rpm:children]
fedora
centos

For use with the cluster, I kept it simple, although I did opt to create rpi/bpi subgroups:

[cluster:children]
rpis
bpis

[rpis]
pi0
pi1

[bpis]
bpi

Ping Hosts

Once the hosts file is setup, it can be tested using the ping module. I tested my cluser group, as well as the rpis and bpis subgroups.

ansible rpis -m ping
ansible bpis -m ping
ansible cluster -m ping

Assuming the steps of the last post were done correctly, this should work. If not, double check that post and make sure everything looks correct.

Playbooks

After confirming that the hosts file is properly configured, I started to dig into playbooks. Playbooks are Ansible’s scripting system used to configure, deploy, and orcistrate systems. They can describe ways in which systems should be configured (ex: enable ssh), or outline a set of steps for an IT task (ex: running updates, restarting a server). As stated in the playbook documentation:

“If Ansible modules are the tools in your workshop, playbooks are your instruction manuals, and your inventory of hosts are your raw material.”

Playbook files are expressed using YAML syntax, which is easy to read, but still powerful. The first step when creating a new playbook, being a YAML file, is to set the header and footer. The header consists of three -'s at the top of the file, and the footer ends the file with three periods (.). This indicates the start and end of the document.

When writing a playbook to update the pi cluster, I first needed to declare what systems the playbook is used with. To do that, I used the hosts key, and provided it with the cluster group name, which is defined in my /etc/ansible/hosts file, as the value.

---
- hosts: cluster

...

After the hosts are defined, modules can be added to update the nodes. To list the tasks, I used the taks: key, with the same indentation as the hosts: keyword. Instead of using a single value, I provided the tasks: keyword with a list of things to do. The first task I want to do when updating the nodes is to check that they running and connected. This can be accomplised with the ping module that I used earlier in the post. The ping module will try to connect to each node, verify that a usable python is installed, and return pong upon success. To add the module, I added - ping: ~, indented, to the line below tasks::

---
- hosts: cluster

  tasks:
    - ping: ~

Apt Module

After defining the ping module, I started to get a bit fancier. Well… a little bit fancier. Each node in my pi cluster is running some verison of Ubuntu, which uses apt as it’s package manager. If I wanted to ssh into each node and update them manually, the steps I would follow would be to 1) run the command sudo apt-get update to update the repository cache, and 2) run sudo apt-get upgrade to actually install the updates. To recreate these commands in the playbook, I used the apt module. To start with updating the repository cache, I added the following lines to my playbook:

- name: Update APT package manager repositories cache
  become: true
  apt:
    update_cache: yes

The name: defines the name of the task, and is the text printed out to the console when executing this step of the playbook. Setting the become key to true tells Ansible to run the command with privilege escalation (sudo). Lastly, the remaining two lines run the update_cache: functionality of the apt module.

With the repositories updated on each node, I can have ansible run the updates by adding the following lines to the playbook (after the cache update ones):

- name: Upgrade installed packages
  become: true
  apt:
    upgrade: dist

This set of commands is very similar to the last group. The name: again provides a description of what the task is doing, and privilege escalation is used again via become: true. The only difference is that the apt module is using the upgrade: dist command instead. This will run the updates for any installed packages on the system.

Update Cluster Playbook

I then had a completed playbook to update the pi cluster:

---
- hosts: cluster

  tasks:
    - ping: ~

    - name: Update APT package manager repositories cache
      become: true
      apt:
        update_cache: yes

    - name: Upgrade installed packages
      become: true
      apt:
        upgrade: dist
...

The last step is to test it out! Playbooks can be executed using the ansible-playbook command:

ansible-playbook update-cluster.yml

When running the playbook, ansible will first attempt to gather facts about each node, and then begin to run each of the tasks defined in the playbook. At each step, it will print out the name of each task, followed by the status/result for each node. When it completes, all the nodes in the cluster should be updated. Now you can update three+ computers with a single command! Enjoy!

Configuring Ansible on the Pi Cluster

2017-05-21T00:00:00Z

In my previous post, I pieced together my pi cluster, and installed variations of Ubuntu 16.04 Server on each of its nodes. With the cluster built, I quickly needed an easy way to maintain and interact with the system as a whole. This, is where Ansible comes in. In this post, I will walk through the steps I took to setup Ansible on my Cluster.

Ansible

Ansible is an open source, configuration management and automation system. It is written in Python, and financially backed by Red Hat. It simplifies the management of groups of computers, through the use of modules (standalone units of work. for example, apt, ping, rpm, etc). Ansible is script-able using simple YAML files, known as playbooks, that define a set of orchestration tasks for one or many computers. These scripts can be edited and version controlled, creating a simple infrastructure as code setup.

Setting up the User Account

When Ansible executes commands on the PIs, it will do so from the user-account (ryan) that I setup in the last post. However, many of these commands will require Root privledges. While I previously setup sudo and added the ryan account to the sudo group … it required that I manually enter my password. Ansible did not like this, so I had to update the sudo configuration to allow the ryan account to run sudo commands with out a password. To do this, I opened the sudoers file:

sudo visudo

and added the following line to the end of the file:

ryan  ALL=(ALL:ALL) NOPASSWD: ALL

I repeated this on each of the nodes, and afterwards was no longer promted for a password when running sudo commands. This made Ansible happy.

Setup SSH Keys

Well… almost happy.

Ansible’s main method of communication is via ssh, which by default, prompts me for a password when connecting. Ansible really hates passwords. So, I had to configure ssh to use keys instead. Honestly, this is proabaly a good step to do regardless, now that the ryan account no longer uses a password when running sudo. To setup key-based logins, I appended the contents of my main computer’s ssh public key*, to each pi’s authorized_keys file. This can all be done using a magic one-line pipe command (x3, one for each pi):

cat ~/.ssh/id_rsa.pub | ssh pi0 "cat >> ~/.ssh/authorized_keys"
cat ~/.ssh/id_rsa.pub | ssh pi1 "cat >> ~/.ssh/authorized_keys"
cat ~/.ssh/id_rsa.pub | ssh bpi "cat >> ~/.ssh/authorized_keys"

*Note: If keys are not already generated, they can be created using the command:

ssh-keygen

Key Only Login

To help secure access to the PIs (and to get on Ansible’s good side), I configured sshd to disable password logins, and only allow connections from clients with approved keys. To disable password authentication, I opened the /etc/ssh/sshd_config file, found the line containing # PasswordAuthentication yes, changed the yes to a no, and unncommented it by removing the #.

While I was in the sshd_config file, I also set PermitRootLogin to no, for good measure.

Lastly, I reset the sshd service and repeated the steps for each pi:

sudo systemctl restart sshd

Afterwards, I was unable to login to the PIs from a computer with unauthorized ssh keys, but, still could from the authorized computer.

Install Python

The last issue Ansible complained about was that it needed python installed on the Pis. Like everything else, the Bananna Pi already had this setup, but I had to install it on the two Raspberry Pis. It was simple enough:

sudo apt-get install python

Install Ansible

I have a confession. So, you know how I have been fun and cheery by anthropomorphisizing Ansible, saying that it was “happy” or “frusterated” during the previous steps? That wasn’t true. I made it up. Ansible wasn’t actually installed yet. So… to install Ansible…

sudo eopkg it ansible

I used eopkg because I am currently running Solus. You might use sudo apt-get install ansible, sudo dnf install ansible, or pacaur -S ansible depending on whatever distro you are using.

That’s all for setting up Ansible. I’ll cut this post off here, but in the next post, I’ll walk through the steps on how to get Ansible to be useful.

Setting Up My Pi Cluster

2017-05-05T00:00:00Z

I have a Raspberry Pi 2, a Raspberry Pi 3, and a Banana Pi. A while ago, I constructed a small tower to house my pi devices. Since then, I have additionally acquired a power source, and some CAT6 cable to connect them all up to a switch. I hope to use the Pis as a mini clustered environment, where I can learn (and play) with some of the “Devops” technologies/techniques out there. This post will briefly explain the initial setup of my cluster.

Setting up the OS Images

Before doing anything with the hardware, I had to setup the pi “hard drives” (micro SD cards), so they could boot.

Ubuntu for Raspberry PI

After mucking around with Rapsbian and Hypriot one Sunday, I decided to just go with a plain Ubuntu image for the Raspberry Pis. I don’t have anything against these specific OSes, but I am mostly setting up this cluster to simulate what I would do on a “real” system. For me, that often means using a straight OS like Ubuntu.

Luckily, Canonical makes special Ubuntu ARM images, specifically for the Raspberry Pi. I download the 16.04 server version for both the raspberry pi 2 and raspberry pi 3. The process to write these images to the microSD card differs slightly from ones I’ve used in the past. It still uses dd, but the image is first piped through xzcat, as such:

xzcat ubuntu-16.04.2-preinstalled-server-armhf+raspi2.img.xz | sudo dd bs=4M of=/dev/mmcblk0
xzcat ubuntu-16.04-preinstalled-server-armhf+raspi3.img.xz   | sudo dd bs=4M of=/dev/mmcblk0

Armbian

The one issue that I have with the Banana Pi compared to the Raspberry Pi, is that it is commonly not supported. It can be hard to find a bananna pi specific image, and the raspberry pi ones usually do not work. For example, while Canonical linked to Raspberry Pi images, it did not mention the banana pie. This is where Armbiangcomes in.

Armbian is a lightweight Debian and Ubuntu based distribution, that provides builds for various ARM devices. Thus the name, ARM-bian. One of these many supported devices… is the banana pi. I downloaded the Ubuntu 16.04 Server flavor of Armbian for the Banana PI, and copied it to my micro SD card with the command:

sudo dd if=Armbian_5.25_Bananapi_Ubuntu_xenial_next_4.9.7.img  of=/dev/mmcblk0 bs=1M

Hardware Setup

The Pi Cluster

After the operating system images have been copied the SD cards, the hardware can be setup. I started by inserting the microSD cards into the PIs, being careful to use the correct card with each device. Unlike the Raspberry PIs, the Bananna PI uses a normal SD card instead of a microSD, so I just left it in the converter card I used to connect it to my computer when imaging.

After adding the “hard drives”, I connected each PI to my network switch, via Ethernet. In the future, I would like to put the cluster on it’s own mini (managed?) switch so that I can have the nodes on their own private network, but connected to my main network. For now, this works.

Lastly, plug in the power connectors. Pi devices can be very finicky when not properly powered, so it is a good idea to use an capable USB charging device. I have had trouble in the past with my devices not working correctly due to insufficient power (especially the bpi). I knew this problem would be an even more pronounced with the cluster because I planned to connect a HD to the bpi with a SATA connector. So, I picked up an Anker Power Port 5 and it has been working great.

Lastly, two nice features of the banana pi is that it has a 1 GB ethernet port, and a SATA connector with power. So, to utilize this functionality, and get the most out of the bpi, I ordered the appropriate SATA connector from amazon for a few bucks. When it arrived, I connected it to the SATA and SATA power ports on the pi, and then to an old 300GB laptop HD I had laying around (it was the drive that came with my x230). After the drive was connected, running lsblk on the bpi automatically showed a /dev/sda device, in addition to the typical mmcblk0 microSD device. I mounted the drive to a folder using the following command to test it out:

sudo mount /dev/sda1 Data

With the hard drive mounted, I was able to browse its contents and see of all the crap still on it. The combination of the GB network and large HD makes the bpi a great little storage node, which is how I intend to use it.

Bootup and Connecting via SSH

When the Pis are plugged in, they should automatically boot up. In order to connect to them, I found their IPs from my main computer using nmap:

sudo nmap -sP 192.168.1.0/24

This command may differ depending on your network setup

The Raspberry PIs have (Raspberry Pi Foundation) in the MAC address lines, and my Banana Pi had bananapi in the host name. Once I had the IP addresses, I could ssh in using the default usernames and passwords for the images (the user/pass should be listed on the sites). After logging in for the first time, each PI prompted me to change the password (as it should).

Adding A Sudo User

When connecting to remote devices, I don’t like to be logged in as root, so the first thing I did was setup my user account with sudo privileges, on the two Raspberry Pis (Armbian actually prompted me through these steps the first time I logged into the Banana pi. Kudos to them).

To add the user, set it’s password (important), and then add it to the sudo group, I used the following commands:

useradd -m -s /bin/bash ryan
passwd ryan
usermod -a -G sudo ryan

Yes, I know I can add the user to the sudo group in the useradd command, but I prefer to do it with usermod. Personal Preference.

That’s it. At this point, all of my the PIs are minimally set up accessible. The next steps include updating packages, setting up ssh keys, and configuring … Ansible. But that will all be in the next post. See you then!

ZFS Snapshot Backups to an External Drive with LUKS

2017-04-20T00:00:00Z

I have been using ZFS data pools to store data on my server for some time now. As great as that is, I am ashamed to admit that I have not had a true backup system in place (raid/mirrors are not backup). I have a backup solution that I have attempted in the past, but ran into an issue and let it drift to the side. That changes now. It’s time to revisit my solution, and complete it to the end.

Currently, my server is configured with two main zfs mirrored pools. The first one, Data, is built on 2 x 3TB hard drives, providing 2.72 TB of usable disk space. It contains all of my wife’s and my data, organized into sub-category pools (ex: Data/Music, Data/Pictures, Data/ryan, etc). The second, Backups, is built on the 2 x 1TB hard drives from my old desktop, creating a 928 GB usable pool . It stores the automatic backups of some of the VMs and LXC containers hosted on the server.

Before I had my 3TB drives, I bought a 2TB external hard drive to backup the 1TB drives to. While it isn’t as large as the total usable space on my server, it is enough to store my data backups to, for the time being.

My goal is to setup a zfs pool on the external drive, so I can use zfs’s send & receive functionality to send bi-weekly-ish incremental snapshots to it. When I am not running backups, I want to store the drive at an off-site location. With the drive being stored elsewhere, I want to ensure that the data is protected, so I will be encrypting the it using LUKS, the Linux disk encryption software.

Setting up LUKS

LUKS (Linux Unified Key Setup) is the standard for Linux disk encryption. I will use it to encrypt the external drive, and then present the LUKS mapper devices to ZFS as a block device. To do this, we need to first install cryptsetup with sudo apt-get install cryptsetup (Assuming you are on a Debian-based operating system). Once that is installed, we can use the cryptsetup command to configure LUKS on the drive.

The cryptsetup tool has a plethora of settings and options. After researching around, I decided to use options that the author of this post used, because they were doing something very similar to what I am trying. I configured LUKS on my external drive using the following command:

cryptsetup luksFormat --cipher aes-xts-plain64 --key-size 512 --iter-time 10000 --use-random -y /dev/sdf

--cipher aex-xts-plain64and --key-size 512 refer to the algorithm and key size used to encrypt the data. In general, the larger the key, the harder the encryption is to crack.

--iter-time 10000 and --use-random -y are additional precautions to make it more difficult to crack the encryption. The --iter-time 10000 means it will spend at least 10 seconds processing the passphrase each time the disk is unlocked. This makes it much harder to brute-force the passphrase.

Once the device is encrypted, we need to unlock it and map it as a device. This is done using the command:

sudo cryptsetup luksOpen /dev/sdf sdf-enc

/dev/sdf is the external disk, and sdf-enc is whatever you want to name the unlocked device. This is the name that what will be used when referring to the unlocked device. With the drive is encrypted and unlocked, it’s time for some ZFS magic.

Creating a ZFS Pool

I am creating a zpool using just my single external drive, so the setup is very basic. No mirrors, no zvols. A simple zpool is created with the simple command:

sudo zpool create externalBackup sdf-enc

That’s it.

Taking Base Snapshots

With a zpool initialized on the externalDrive, I can now send snapshots to it. To start, I created a base snapshot to send. Starting with the smaller pool, /Backups:

sudo zfs snapshot -r Backups@VM-LXC-BackupBase

This command creates a recursive snapshot of my Backups zpool, named VM-LXC-BackupBase. Making a base snapshot for my /Data zpool is nearly the same:

sudo zfs snapshot -r Data/DataBackupBase

Sending the Base Snapshots

After taking base snapshots of the zpools, I can transfer them to the external drive using the zfs send and recv commands. Again, starting with the VM-LXC-BackupBase:

sudo zfs send Backups@VM-LXC-BackupBase | sudo zfs recv externalBackup/VM-LXC-BackupBase

Looking back, I realized I named this poorly… The external zpool should be /externalBackup/VM-LXC-Backup, not BackupBase, that name is just for the first snapshot. Oh well.

Now for the slightly harder pool, /Data, with all of the sub zpools. The first time I attempted this, only the parent Data snapshot was copied, but none of the children were (Data/Music, Data/Pictures, etc). After some digging around the docs and online I realized I was missing the -R to my zfs send command. Also note, that when using the -R flag, the snapshot name for the destination pool are not specified (because it is copying multiple). It will use the same snapshot names from the source pool.

sudo zfs send -R Data@DataBackupBase | sudo zfs recv externalBackup/DataBackup

Incremental Backups

Taking a snapshot of my data and sending it to an external drive is nice, but I don’t want to send all of the data each time I backup. Transferring can take a very long time, especially as my data pools continue to grow. I don’t want to sit around all day, listening to hard drives hum as my data transfers.

A useful feature of the zfs send and recv commands is the ability to send incremental snapshots. This means when I want to update my backups, I can just send the changes between the two snapshots. This is similar to source code diffs, but for file systems.

To send incremental snapshots, the -i or -I flag is used. The difference between the two is that the -i flag will send the difference between the two snapshots listed, whereas -I will send a series of snapshots between the two listed. For example, if I’ve taken several snapshots of my data (A, B, C, and D), but have neglected to copy them to the external drive since snapshot A, I can use -I A D in my zfs send command, and all four of the snapshots will be sent to the external.

To send an incremental update to my backup, I first created new snapshot for my pools (this time with a date):

sudo zfs snapshot -r Backups@VM-LXC-Backup20170418
sudo zfs snapshot -r Data@DataBackup20170418

Next, I sent the incremental changes between the base snapshots, and new ones I just made:

sudo zfs send -R -i Backups@VM-LXC-BackupBase Backups@VM-LXC-Backup20170418 | sudo zfs recv externalBackup/VM-LXC-BackupBase
sudo zfs send -R -i Data@DataBackupBase Data@DataBackup20170418 | sudo zfs recv externalBackup/DataBackup

Notice that I specify two snapshots in the send command, to define the range of differences to send.

A Minor Issue

The first time I tried sending an incremental backup, I encountered a minor issue. ZFS gave me an error stating that my destination had been changed since last snapshot (meaning the base snapshot on the externalBackup pool). I looked this up online and it seems that sometimes, just looking around the pool can change files. Some people recommended setting the destination pool to read-only, so I did that to my backup pool with:

sudo zfs set readonly=on externalBackup

I am not sure if this will eliminate this problem in the future, but I guess I will find out.

I still had the error when sending, so I added the -F flag to the zfs recv command. I am not sure if this was the best solution, but it seemed to be okay. I also thought about rolling back to the snapshot, and then copying which is likely a safer method (if you don’t mind losing the “changes” on the destination pool).

Safely Closing and Removing the External Drive

When the incremental backups finishes transferring, the external drive can be removed. The sequence of steps to do this safely are 1) export the zpool 2) close the LUKS device, and 3) unplug the drive. To export the zpool and close the LUKS device I used the commands:

sudo zpool export externalBackup
sudo cryptSetup luksClose sdf-enc

After that, I was able to unplug the external drive, and store it in a safe location, until I need to backup data to it again.

Opening and Importing zpool for Recurring Backups

Lastly, there are a few steps to take when reconnecting the drive to run a daily/monthly/weekly (whatever) backup. First, the drive must be decrypted and mounted, using the same command as above:

sudo cryptSetup luksOpen /dev/sdf sdf-enc

Next, the zpool on the drive must be imported so that it can be used from the main system. Like exporting the pool, the command is simple:

sudo zpool import externalBackup

That’s it. Now the steps detailing taking snapshots and sending incremental backups can be repeated.

Summary

I am happy with this solution for now. It allows me to leverage ZFS a bit more, and become more familiar with it. The biggest issue I will likely face is space on the external drive. Luckily, ZFS makes it easy to delete old snapshots. In the future, I might also consider using an online backup solution like Tarsnap, but I need to find a cost-effective one first. I’ll be sure to update as I continue to expand my backup solution.

LIA 1.0 Beta Released

2017-04-17T00:00:00Z

World's End, Hingham, MA USA

I have released the 1.0 Beta version for a personal project of mine: the Ledger Import Assistant, or LIA. This post will talk briefly about the background of LIA, what it does, and explain the beta release.

What is LIA?

Example Cred Card Statement CSV File

A while ago, I discovered Ledger, the command line double-entry accounting application. Its powerful, yet simple design attracted me, and I wanted to try it out. To use it effectively however, I needed a method to import our bank and credit card statements into ledger journals. Then, I could use ledger to analyze the finances. However, there was an issue. While there are several great ledger convert/import options out there, many were more complicated than what I was looking for. So…I wrote my own.

Example Ledger Journal File

When I started writing LIA, I just wanted a python script that could help me convert the contents of a .csv file into a ledger-journal formatted file (without any of the fancy features. Just the basics). I thought it was a simple task and should only take me a few hours. It was, and it did. I wrote the the first basic implementation of LIA on a Sunday afternoon. While coding that bare-bones version, I realized that even though it technically worked, it would not be enjoyable to use, and therefore I would never use it. So I decided to expand it into a full project, something more than a simple script.

What does LIA do?

LIA executes the core functionality that originally prompted me to write it: converting bank/credit card statement csv files into ledger journal files. Beyond that basic functionality, LIA has a few nice features that help the user manually convert these files in an enjoyable way. By going through each transaction manually, the user has full control to make sure data is being input correctly. However, LIA helps make this otherwise dull process fast and efficient. Some of LIA’s features that help accomplish this are:

Data order is recognized by a header mechanism
Prompts the user to potentially edit the transaction information
Sets default transaction information from the values of the csv file
Manual transaction entries when needed (No input file)
Supports multiple destination accounts
Automatic placement system. The user can specify a file containing rules to automatically place transactions. (ex: anything with “Dunkin” in the description will default to Expenses:Food:Coffee)
Colored prompts

Click for demo animation

What does LIA 1.0 Beta mean?

LIA 1.0

My Todo list to release the 1.0 Version

When I expanded LIA to a personal project, I recorded several features that would make converting CSV statements easier, and got started. I worked on the project here and there, adding each feature over time. These features are what I determined were required in order for the application to be acceptably usable. When all of those requirements were met, I would release an official 1.0 Branch.

The main functionality of LIA has been implemented for a while now. Being a python application, it has been possible to run LIA by calling the files with python. However, I didn’t want to release the 1.0 version without first making an installer. I wanted LIA to be run like a normal linux application. I have now finished configuring the project and a setup.py file, so users can use python’s setuptools to install LIA as an application on their computer. Additionally, I have even packaged LIA as a Solus eopkg. It looks like I am ready for release.

Beta

Sort of. Until now, I have been developing, but not using LIA day to day. I want to spend some time actually using the application to see if there are any remaining issues. Also, I have not confirmed that it fully does what is needed for ledger. I want to get a few ledger users to quickly look at it and let me know if they see any issues. After testing it for a bit, I will release it as the official 1.0 release. This will mean it should be stable enough for people to use, if they so choose to.

More Information

To test out LIA, read some documentation, or find out more check it out on my github page. I plan to continue to develop it further in the future. If you have a suggestion, or even some code you’d like to contribute, feel free to let me know, either on github or my other contact methods. Enjoy!

New Website Theme - Immutable

2017-04-11T00:00:00Z

Blue Hills Reservation, Milton, MA USA

When I first started using Cryogen to generate this website, I wanted to create a personalized theme. This desire led me to hack a the default theme into a mutant, which I boringly named “ryan1” (I anticipated it to be temporary). It looked like a relic, designed from when I first learned how to make a web page… the early 2000’s. Like ~~many~~ all websites from that time, it was not mobile friendly. I hope to change all of that, by introducing my new website theme: Immutable.

Homepage with the old Theme

Homepage with the new Immutable Theme

While it may possibly still look dated (I wouldn’t know, I’m a backend dev), it addresses several of the issues I had with the ryan1 theme. Here are a few examples of these improvements:

Removed the Sidebar & Dispersed Its Contents

The sidebar components have been distributed to other parts of the site

I have to be honest here… I originally planned on keeping the side bar when I started to build the theme. I liked how it held all of my links (Github, LinkedIn, etc), recent posts, and tags in one, easy to find, location. I also liked having my avatar picture in the side bar. I felt it made the site more personable (or maybe I’m just egotistical). After working on the base of the theme however, I realized the site might be better off without it. I’ve relocated the items to other locations of the site. The majority of the links have been relocated to the drop-down menu (more on that below), and I at least added my picture to the About Page.

I might reinstate a sidebar in the future, but only if it looks good and I can add it correctly. The way I implemented the old sidebar caused many of the issues prompting this new theme, and I do not want to bring back those problems.

Removed the Bad Footer

Bad Footer Placements

One of the issues created by the sidebar and my poor css floats, was the footer. On pages with a large enough the main content, it looked fine, consisting of a dark gray bar along the bottom of the page . However, when viewing a page with a small content section, such as the About page, the footer would rise up behind the sidebar. It looked terrible.

New Footer

In the Immutable theme, I have fixed the footer and removed the gray bar. The background looks great, and the text is clearly visible over it, making the bar no longer required. I like minimal footers. I only need a small copyright statement, and I enjoy having a Cryogen shout-out here. So that’s all I have.

Mobile Support

Mobile viewing the homepage and a post, on both the old theme and Immutable

The biggest reason for making the new theme was that viewing the website on a mobile device was a poor experience… unusable even. Again, because I implemented the sidebar poorly, it always remained on the side. Even on a narrow mobile screen. There was not enough room for the actual main content section, and pages/posts consisted of a skinny line of text down the side of the phone. Images… well, don’t get me started on how well images were displayed. Immutable solves these issues. The main content window takes up the majority of the screen, allowing for easy content reading.

Mobile Navigation

Main navigation drop-down and the extended drop-down menu on mobile

In addition to better viewing, site navigation has also been improved on mobile. When I say improved, I of course mean added, as is was sort-of non-existent before. In ryan1 there was an drop-down menu icon, but when it was tapped… nothing happened. At all. Now, there is an icon that drops down a site navigation menu when tapped. At the bottom of the navigation items, there is a more tab. When clicked, it extends the menu to also include my contact links (Github, LinkedIn, etc.), recent posts, and the list of post tag links.

Continued Work

While its name is immutable, the actual theme itself is not (bad joke). There are a few things I plan on tweaking over time. For example, I don’t love the fonts and spacing and I will continue to improve them. I also want to edit the code colors. I did a quick edit of the colors so that they worked with the theme, but they not what I ultimately want. I use the code segments a lot on this site, so it is important that they look nice.

Well, that is it. I finally got around to making a new theme, and plan to make it even better over time. I hope you enjoy, even on a phone ;).

Linux from Scratch - SBUs and Binutils

2017-04-04T00:00:00Z

Well, after all of the preparation, we are ready to start compiling packages. This post cover compiling all the packages, but it will detail the first build of Binutils, which is arguably the most important package to compile. Why is Binutils so crucial? It determines the SBU time for your build system. What’s an SBU? Read on to find out!

SBUs

A table of package SBUs and their predicted build time]

When completing LFS, people commonly want to know how long it will take to compile each package. Unfortunately, build times are very much dependent on the power and configuration of the system the packages are being compiled on. Some packages may take a few minutes on a powerful workstation, but hours on an aged laptop. While it cannot be said how long a specific build will take on any device, we can normalize how long each package build takes comparatively to each other. This normalization is done using Stand Build Units, or SBUs.

A SBU is the unit of time measurement it takes to make and install a standard package. Each package in the LFS book has a SBU value, so that build times can be gauged. So, if The first package to be constructed in the book (and in this post), is Binutils, so that is the package which SBUs are normalized to. For example, if it took 10 minutes to build Binutils on your machine, then 1 SBU = 10 minutes for that machine. This means a 4.5 SBU package can be expected to take ~45 minutes to build.

SBU Accuracy

SBUs are not completely accurate, and should be used as an estimate at best. Due to the many factors that may differ between setups, SBUs can be off by dozens of minutes in worst-case scenarios. Certain make options might also throw the system off.

For example, systems with multiple cores can run “parallel make” using the -j make-flag, as in make -j4. This tell make to compiled the package using multiple cores. Parallel compilation has the potential to speed up the build process significantly. However, due to how compilation jobs are divided for parallel builds, SBUs are even harder to predict and may be even more sporadic. Just remember that and don’t expect too much SBU accuracy when using make -j. Also, if you ever run into a problem during a build step, it is a good idea to first retry with a single processor build. If this does not fix the issue itself, the error message can at least be more easily analyzed.

My Encountered Issues with tar…

The first time I attempted to make binutils, I encountered a few errors. The gist of it was that I was not able untar the package correctly, at least from the lfs user. Everything worked fine from the root or even ryan user accounts, but running tar on lfs returned the following error:

tar (child): bzip2: Cannot exec: Too many levels of symbolic links
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now

I searched around but much of the initial advice didn’t help my problem. It often indicated that my /usr/bin/bzip2 might be a symlink and should be altered. That wasn’t the case. Then tried something that illuminated the issue: I removed the /tools/bin from the begging of the path variable (defined in the lfs .bashrc file). That temporarily fixed the issues. So I knew the problem was related to the symlink I setup in the previous LFS post, specifically the ln -sv $LFS/tools / command. It must have failed and I wasn’t paying attention.

Now that I was knew what the problem was, I was able to fix it by running the following commands (some of them might need to be run from a root/sudo account):

rm -rf $LFS/tools
rm -rf /tools
mkdir -pv $LFS/tools
ln -sv $LFS/tools /

These commands remove and reset the tools symlinks. I then made sure to re-add /tools/bin to the begging of the path var in the lfs .bashrc and test it. Problem fixed!

Extracting BinUtils

It is important that Binutils is built first in the process. This is mostly because when Glibc and GCC are built, they perform various tests on the linker and assembler to figure out which of their own features to enable.

To start building BinUtils, move to the sources directory ($LFS/sources) and extract the package with (If you encounter issues, see section above):

tar xfjv binutils-2.27.tar.bz2

Note: You’re version might be different

The Binutils documentation recommends building it in a dedicated build directory, so lets go ahead and make, then enter, that directory:

mkdir build
cd build

Making & Executing a Build Script

Now it is time to build. Normally, this would best be done by sequentially performing a series of configure, make, and make install commands, but for the first binutils compilation, we want to get an accurate reading on how long it takes (to determine our SBU time). To accomplish this easily, I put all of the commands into a bash script. This way, I could execute the script, and easily time the whole process using the time utility. To create the script, I wrote the following commands into a file (build-binutils.sh):

#!/bin/bash

../configure --prefix=/tools            \
             --with-sysroot=$LFS        \
             --with-lib-path=/tools/lib \
             --target=$LFS_TGT          \
             --disable-nls              \
             --disable-werror
make

case $(uname -m) in
  x86_64) mkdir -v /tools/lib && ln -sv lib /tools/lib64 ;;
esac

make install

The various options of the configure command mean the following:

--prefix=/tools: Configures the build to install the Binutils programs to the /tools directory
--with-sysroot=$LFS: For cross compilation, tells the build system to look in our $LFS directory for the target system libraries, as needed.
--with-lib-path=/tools/lib: configures the library path that the linker should be configured to use.
--target=$LFS_TGT: the machine description in the LFS_TGT variable is slightly different than the value returned by the config.guess script, so this option will tell the configure script to adjust Binutil’s build system for building the cross linker.
disable-nls: disables the internationalization, as it is not needed for the temporary tools.
disable-werror: Prevents the build from stopping in the event that there are warnings from the host’s compilier.

Also, the case statement creates a symlink to ensure the sanity of the tool chain, if building on a x86_64 architecture.

To runs the script, first make it executable,

chmod +x build-binutils.sh

Finally, time and execute the script:

time ./build-binutils.sh

When the script completes, the time it took to run it will be printed out. Congratulations, this is the SBU for your system!

Once I had my standard single-thread SBU value, I wanted to do a run with the -j4 make flag for comparison. To “reset”, I deleted the contents of the binutils/build/ directory, as well as $LFS/tools/. I then edited my build-binutils.sh script to do a parallel compile by changing make line to make -j4. I then re-ran the script with time. Note: This is not described anywhere in the official documentation, but is just my best guess at what to do for a re-run of binutils. I very well may be missing steps. However, remember LFS is a learning experience, so we will find out!

Checking the Build

After the build is complete, it is a good idea to run the tests, especially for binutils. In this case, use the make command:

make -k check

Building the Chapter 5 Tools From Here on Out…

For the sake of time, I will not be writing posts detailing the build process for each of the remaining packages in Chapter 5. I might keep a little log about how each of the builds went and post it, but I am unsure. If I encounter any major snags along the way, I will be sure to write a post detailing them. Otherwise, with any luck, my plan is to keep compiling and see you in Chapter 6!

Linux from Scratch - Final Preparation Steps

2017-03-21T00:00:00Z

Now that the repeated setup steps have been defined in my previous LFS post, there are a few more preparation steps to complete in order to start building the LFS system. I promise… we will start compiling soon. If all goes well, this should be the last preparation post.

Downloading Sources

When it comes down to it, Linux from scratch is just a bunch of packages, and the Linux kernel, all compiled from source and linked together. To build all of this, we need to download the source code… for all of those packages. Luckily, LFS keeps a list of what is needed, and downloading it is trivial. (Note: These commands should be run as root)

First, lets make a new directory to put all of the source code. To make the directory:

mkdir -v $LFS/sources

The LFS book suggests for this directory to be writable and sticky. A “Sticky” directory allows only the file owner to delete a file in it. To make the directory both writable and sticky, use the command:

chmod -v a+wt $LFS/sources

(Click to open example animation)

To download all of the source packages at once, download the LFS wget list:

wget http://www.linuxfromscratch.org/lfs/view/stable-systemd/wget-list

After downloading the wget list, it can be used as the input-file for wget. This will download all the sources with one command:

wget --input-file=wget-list --continue --directory-prefix=$LFS/sources

It should take a few minutes to download everything (or longer if on a poor connection).

(Click to open example animation)

Ever since LFS-7.0, a md5sums file is provided, which can be downloaded and used to verify the integrity of downloaded packages. Download this file, again with wget:

wget http://www.linuxfromscratch.org/lfs/view/stable-systemd/md5sums

Then, compare the hashes in the list to the md5sum for each of the source packages. This is done with the commands:

pushd $LFS/sources
md5sum -c md5sums
popd

The results should all say OK. If not, try re-downloading the sources and verifying again.

Creating the $LFS/tools Directory

LFS is built in two main steps. The first step builds a set of temporary tools to build the system, but not be included as part of the Final LFS system itself. To help prevent these tools from accidentally being included in the final system, they are kept in a separate directory that can be deleted after they have served their purpose. Make this directory as root:

mkdir -v $LFS/tools

Next, we will create a /tools symlink to the host system.

ln -sv $LFS/tools /

This enables the tool-chain to be compiled so that it always refers to /tools, which ensures that the compiler, assembler, and linker will work in both the first, and second steps of the LFS build.

Update: I did this step wrong the first time (I think it failed), and encountered errors later when trying to run tar. If you encounter issues as well, jump to my next post to see how I resolved these issues

Adding the LFS User

Running a system as root is a dangerous. Running the wrong command can completely obliterate a system, and having a typo bork the LFS build, or even the host system, would be horrific. To prevent this, the book recommends creating an unprivileged user to build the packages from. To do so, first create an lfs group and then create + add a lfs user to it using the commands (as root, ironic for this section…):

groupadd lfs
useradd -s /bin/bash -g lfs -m -k /dev/null lfs

If you are not familiar with the useradd command, then RTFM by typing man useradd. I’m just kidding (although reading the man pages is never a bad idea). Here is a quick summary of what all of the flags mean.

-s /bin/bash sets our lfs user’s default shell to bash
-g lfs adds the user to the lfs group that was created in the previous command
-m creates the user’s home directory (/home/lfs)
-k /dev/null changes the input direction to the special null device to prevent the copying of files from a skeleton directory
lastly, lfs is the new user’s name.

Before logging into the user, the password must be set.

passwd lfs

We also want to grant the lfs user full access to the tools directory we made ($LFS/tools), so lets make lfs the owner of that directory:

chown -v lfs $LFS/tools

Do the same for the sources directory we made:

chown -v lfs $LFS/sources

Lastly, login as lfs!

su - lfs

Note: the “-” tells su to start a login shell, rather than a non-login shell. This mostly ensures that various files are read at login to setup environment variable and other profiles.

Setting up the Build Environment

Now with the lfs user created, we need to setup a proper working environment for that user. To do this, we will create the .bash_profile and .bashrc files.

Creating .bash_profile

(Click to open example animation)

When logging in as the lfs user, the shell first reads the /etc/profile of the host, followed by the .bash_profile. So, lets start with the .bash_profile. Create/open .bash_profile and add the following line to it:

exec env -i HOME=$HOME TERM=$TERM PS1='\u:\w\$ ' /bin/bash

This line replaces the running shell with a new one that contains a completely empty environment, except the HOME, TERM, PS1 variables. This ensures that there are no stray environment variables, that may interfere with the build environment.

Creating .bashrc

(Click to open example animation)

The new instance of this the shell is a non-login shell, so it does not read the /etc/profile or .bash_profile files. However, it does read the .bashrc, so lets go ahead and create that. Open ~/.bashrc` and add the following lines:

set +h
umask 022
LFS=/mnt/lfs
LC_ALL=POSIX
LFS_TGT=$(uname -m)-lfs-linux-gnu
PATH=/tools/bin:/bin:/usr/bin
export LFS LC_ALL LFS_TGT PATH

The set +h line turns off bash’s hash function. This is normally a usefully feature, as it essentially caches the path-names of executables. Removing this will ensure that the newly compiled tools will always be found by the shell once they are available, because the shell will have to re-search the PATH each time. Similarly, placing /tools/bin ahead of the standard /bin:/usr/bin PATH (line 6), also helps force the shell to immediately locate up all the programs in chapter 5 after installation. These two techniques will hopefully prevent the risk of using old programs from the host instead of the newly compiled ones.

The umask 022 line defines the umask to 022, which sets up the system so that created files and directories are only writable by their owner, but are readable and executable by anyone.

The LFS=/mnt/lfs line should look familiar, as it sets the LFS variable to our LFS mount point.

Lastly, setting the LC_ALL variable to POSIX or C ensures that everything will work as expected in the chroot environments (regarding localization settings).

To enable this new environment we’ve setup, source the user-profile:

source ~/.bash_profile

Congratulations! We are now ready to start compiling some code in the next post!

Linux from Scratch - Repeated Setup Steps

2017-03-13T00:00:00Z

During the Linux From Scratch process, there may be times when the build environment (computer, VM, chroot, whatever) must be restarted. If so, there are a few steps from the setup phase that have to be re-initialized. This post maps out those steps.

Setting The $LFS Variable

After setting up the virtual disk for my LFS build, I needed to define where I wanted to eventually mount it. This location is important, because it is the path that the $LFS variable is set to. The $LFS variable is used throughout the book, to easily point to where the LFS system is being built.

To set the #LFS variable, I ran the following command: *

export LFS=/mnt/lfs

To check that the variable set correctly, just print it out using echo (if successful, the path that was specified should print out).

echo $LFS

Ensuring the $LFS Variable is Always Set

There are several ways to ensure that the $LFS variable is always loaded during login. One method the book recommends is to edit the .bash-profile found in both ~ and */root, by appending the export command defined above to them. This way every time the build machine resets, simply logging into the system (which loads bash, assuming it is the default), will export the $LFS variable.

Mounting the LFS Partition(s)

After setting the $LFS variable, I could finally mount my LFS drive/partition to that location. First, I ensured that the directory existed by running:

mkdir -pv $LFS

Note: In this command, the -v again means verbose, so a message will be printed for each directory created. The -p flag is for --parents, and will instruct “mkdir” to also make parent directories, as needed. So, if /mnt/ does not already exist, will be created along with /mnt/lfs.

After creating the directories, I mounted them with the command:

sudo mount -v -t ext4 /dev/sdb $LFS

If multiple partitions are being used for the LFS build (such as a separate /home partition), they should also be mounted at this time.

After mounting my partition, the LFS book recommended that I check that the partition was not mounted with restrictive permissions. To do this, I ran the mount command again, but this time without any parameters. From the output, I was able to see and confirm that the partition was not mounted with restrictive permissions, such as nosuid or nodev. If either of these options are set, the partition should be remounted.

Lastly, if a swap partition is being used, do not forget to enable it using swapon:

swapon -v /dev/xxx (with xxx the name of the swap partition)

Conclusion

Remember, if the LFS host system is restarted for any reason, these steps must be completed upon logging into the rebooted system. Even if measures were taking to always complete these steps (such as adding the $LFS variable to the bash profile, or mounting the partitions via the fstab file), it is still a good idea to check and make sure that they actually initialized as intended. This can prevent several headaches down the road.

My New Dotfiles Management - Using GNU Stow

2017-03-07T00:00:00Z

Rodney Bay, St Lucia

I have maintained a “dotfiles” repository since I made my github account in 2013. However, overtime it became more of a post-apocalyptic wasteland, cluttered with remnants of obsolete configurations and scraps of scripts. It was no longer the pristine, culled, collection that I desired. I also did not have an efficient method of easily linking the files on a new system. I had to manually make symlinks for each dotfile. I knew there were much better dotfiles setups out there, but I never got around to it. Until now.

One day, after reading this post, I finally decided to sit down and clean up my dotfiles directory. I wanted to re-organize it so that I could use GNU Stow to initialize my dotfiles.

After setting it all up, I decided to just start from scratch with a new repository.

Using Stow and dotfiles

If you haven’t seen it before, I highly suggest reading the post I have linked above. But in the meantime, I can provide a quick summary of how my dotfiles are setup.

Each application has an associated sub-directory (ex: dotfiles/emacs), which contains all of the dotfiles/folders associated with that application. Structurally, I treat the items in each application directory as if they were in my ~. For example, the vim sub-directory has my .vimrc, as well as the .vim/colors/ directory. This is so that when I use stow, it will properly link them in ~.

When I setup my dotfiles on a new system, or install an application for which I already have dotfiles saved for, setting them up is as easy as typing:

stow application-dir (ex: stow vim).

GNU Stow then links the files under my home directory. In my vim example, this means symlinks are created for ~/.vimrc and ~/.vim/colors/*, pointing to their respective locations in ~/dotfiles/vim/.

I think this setup is brilliant. Initializing an application’s directory is so simple, and I can choose to only initialize specific sub-directories.

In the future, I might make multiple branches of the repository, one for each of my computers, so I can maintain specific configurations. In theory, I could also just make different folders (ex vim-laptop and vim-server), but I like the branch idea better because it’s a little easier for me to merge changes. We shall see.

Anyway, that’s the new setup. Enjoy :-D

My New, Used x230

2016-11-29T00:00:00Z

As I start to do more with a computer on the go, the netbooks I previously used just are not cutting it anymore. They were great for minimal use, but couldn’t run VMs, over-heated, and didn’t have great battery life. Now that I am going to libraries and other locations to work on personal and open source projects, I really needed a better mobile pc setup.

Other Considerations

I had been keeping an eye open for a new mobile computer for well over a year to find out what is out there. I mostly looked sub $300 ubuntu laptops like the Asus 11" celeron (x201e) I got in college. I don’t need much HD space on my portable computer, because I usually use a minimal OS, and only have my projects (which are git controlled), and sometimes a little bit of music. Even with dropping my requirements to computers with very tiny HDs (often < 60GB), none of the options seemed all that impressive. They usually had Celeron processors and little ram. Even worse, they were not easily expandable. I also briefly entertained the idea of putting Linux on a chrome book, but they had the same issues that all the other sub $300 options had.

Used Thinkpads

When I intensified my search, I started to consider other options. There is one alternative that is very popular among Linux users, but I always seem to forget about it when looking for a new device. That option of course, is buying a used Thinkpad and installing Linux on it. This appeared to be an ideal time to try it out. I don’t mind if there is a scratch or two on a cheap device. This is my portable computer after all, so I’m sure a shinny new device wouldn’t stay pristine for long anyway. So, I started researching used Thinkpads and loved how much you could get for the money. There were i5 devices with an acceptable bit of ram and “okay” hard drives for under my $300 price point. Better yet, the Thinkpads seemed easy to upgrade . So, even if the device had a bad battery, or needed a RAM/HD upgrade, I could could do it.

I eventually narrowed my search down to the X230 or T430s. That year’s model seemed like the perfect mix of still being relatively new, while also being old enough that many business had started to replace them, flooding the market and driving the cost down. They also didn’t have the terrible track pad that the x240 and T440s had (without buttons). I eventually decided on the x230 because the main use of this laptop would be portability.

This Particular Thinkpad x230 …

I found my used x230 on Amazon, for about $185. The vendor an acceptable rating, and the specs were good for the price:

Intel i5-3320M
4 GB RAM
12.5" 1366x768 HD LED Display
320 GB HHD

The listing also said that it came with a “used 6-cell battery”. So I planned from the beginning to have it replaced. Sure enough, when the laptop arrived, the battery was completely dead. If I unplugged the laptop, it would shutdown. So, I ordered a 9-cell battery online to replace it. I added a 4GB stick of RAM that I took from an older laptop, and replaced the slow HHD (with Windows pre-installed), with a cheap 120GB SSD I had lying around. After installing Korora 24 on the new SSD, I had a nice, portable computer that only cost me about $200 (plus some spare parts). As advertised, swapping the battery, adding RAM, and replacing the HD were super simple and accessible.

What I like about it

Track point - I love track points. Using other laptops with just track-pads is really frustrating now. I completely understand why people are so crazy about them.
Keyboard - The keyboard really is great. I feels solid and not like other laptop keyboards. It gives good feedback and is very nice to type on. I know many people hate this model because keyboard layout changed from the older models like the x220, but I still think the x230 has a better keyboard than most laptops.
Physical buttons - The x230 has a physical mute, volume control, and mic buttons. This is a feature I enjoy much more than I thought I would.
Upgradability - As I have already mentioned, it is very easy to upgrade the x230. Right when I got it, I replaced the HHD with an SSD and added another stick of RAM in under five minutes. In fact, I have swapped the SSD out temporarily with a second drive several times now to try out different distros on the hardware. It is just so easy to do.
Feel - I really like how this laptop feels. I’ve already mentioned that I enjoy the solidness of the keyboard, but I also love the exterior texture of the case. It has a slight rubbery feel to it that makes it easy to hold.
Bang for your buck - Getting a used thinkpad really is a great deal. Even counting the minor upgrades I installed, I was able to get an actual computer for around the same price as the netbook alternatives I was looking at.
Size - This is my mobile computer, so I need it to be small. While it isn’t as thin as my wife’s macbook air, it is still very portable. I actually don’t mind it being a little bit thicker, as I find it to be the perfect size to hold and carry around. It is a similar size to a few of my programming books and slides into my backpack very easily.

What I don’t like

Has a little bit of flex - While the laptop overall feels pretty solid, the screen does have a bit of flex when pressed.
Resolution - I really hate 1366x768 on laptops. I don’t need 1920x1080, which would be very tiny on a 12.5" screen. 1600x900 however…
Track pad (but honestly who cares) - The track-pad isn’t great. In fact, it can be downright annoying. The bottom of the track-pad curves around the edge of computer, which looks nice, but rubs against my stomach when I am working with the laptop on my lap. This causes the cursor to fly around and do everything but what I want it to do. This would be a much bigger issue on other laptops, but with how much I use the track point, I really don’t care. I usually just disable the track-pad all together.
Speakers aren’t great (but again, not a bit deal) - The speakers are very quiet and don’t sound very good. This isn’t a huge deal for me because being my mobile computer, I am using headphones 99% of the time. Still, it would be nice if when I occasionally share a video with someone they could actually hear it.

Conclusions

I have had my x230 for several months now and still love it. I use it at work daily (to listen to music, podcasts, and to have a linux computer available to me at all times in a Windows office). It is a perfect mix of being portable while still having enough power to accomplish whatever I want to work on. I still prefer to vote with my wallet by purchasing from a vendors like System76 when possible, but buying used business hardware, such as thinkpads, is a very affordable alternative. I have no regrets buying my used x230. It is infinitely better than my other options at that price point.

Update

I have taken so long to publish this post, I am actually now using Arch Linux on the x230. It also runs great :)

Trying out snapd on Arch Linux

2016-07-17T00:00:00Z

Boston, MA USA

The past few weeks I have been listening to all of the buzz about Ubuntu “snaps”, but I just recently decided to try this new technology out.

Snapd Install and Setup

I decided to install snapd on my small netbook (Abra), which is running Arch Linux. So, installing snapd was as easy as running:

pacaur -S snapd

Now, I use pacaur so that I can install packages from Arch Linux’s main repos, as well as the AUR. However, because snapd has actually been moved to the community repos, you can install it with vanilla pacman, without aur support.

sudo pacman -S snapd

After snapd is installed, it needs to be started. To start the snapd service, just use systemd:

sudo systemctl start snapd

Additionally, If you want to start the snapd service automatically after a reboot run the command:

sudo systemctl enable snapd

Using snap

After snapd was installed and running, I started playing with the snap application. The first thing I should note is that to the extent of my knowledge, snap has to be run as root (at least for now). So my commands all start with sudo.

First, I wanted to find a snap package. This can be done using the snap find command as follows:

sudo snap find package-name

(where package-name is the name of the package to search)

Finding Snaps.

Just like in any standard package manager search, the potential matches were returned, along with a version number, the developer name, and a summary.

Snap has several other commands. To see a full list of available snap commands and a description for each one, just type:

sudo snap help

Snap Help.

Testing a snap

After learning about the basic snap commands, I wanted to install a snap to see how well it works. I decided to see if there was a Telegram snap (I heard there was, so it was a good bet). I ran sudo snap find telegram, and a list of my options appeared.

I choose the last one, and installed it using the command sudo snap install telegram-serguisens. A progress bar tracked the download of the snap, and then the installation. That was it. To my delight, within a few seconds the Ubuntu snap application had downloaded the telegram snap, and installed it on my arch linux box, without any issue.

My Only Issue

Well… almost without any issues. The one problem I did have was after installing the telegram snap, the executable wasn’t in my path so I couldn’t start it from my launcher or even a terminal. I think this was actually because snapd wasn’t setup fully. I rebooted my computer and telegram now launches just fine. The snap is even identified with its full name (telegram-serguisens), so I can be sure it is the actual snap, and not another Telegram install.

Thoughts

Telegram Snap

While the fact that snaps are easy to package and install for developers and users alike, I think my favorite feature was something that I didn’t even think about prior to trying it out, and I might not of realized had I installed snapd on another computer…

This little computer is low power, and tends to heat up and go crazy when I install certain packages (particularly ones from the AUR). Sometimes it is so bad that I have to kill the installation and forego that application for awhile. Telegram is one of them. I think this is because the aur build pulls down the telegram .deb package and then has to extract and compile it in order to install it on Arch. This little computer doesn’t like that. The snap however, seems to of just download and installed itself. It hardly pegged my CPU at all. If this is true, snaps might have an additional use-case on weaker hardware, like my netbook. (yes, I know snaps might have a bigger disk footprint, but honestly that trade-off is fine with me)

I think the universal application frameworks are a great idea that is very much needed on Linux. Snapd is very promising and I look forward to seeing what it and other solutions have in store for the future of Linux.

Trying Out the Plasma 5 Desktop Environment

2016-06-26T00:00:00Z

After watching the continued development of the Plasma 5 desktop environment from a distance, I determined I needed to give it another try. It was just before version 5.6 was released, and the project appeared to be finding it’s stride. Right up front, I want to come clean and admit that I have never really liked the KDE/Plasma desktop environment (more on that below). However, with that said… I am still using the plasma desktop on my main computer (months later, and haven’t aborted yet**…). Here are my thoughts.

My History with KDE/Plasma

Now, as I said above, I have never really liked KDE (now Plasma). In fact, sometimes I down right hated it. It is a great project, and I wanted to like it. It has so many features and is amazing for power users. I have tried it again and again, desperately wanting to be swept off my feet, but it’s design always kept me bolted to the ground. The theming made me feel like I was using a Fisher-Price operating system (which is ironic, because like I said… the plasma desktop is probably one of the best power user environments out there). The bulky features always made it feel childish. So, with that said… I have been using using the Plasma desktop on all of my personal computers for several months. Here are my thoughts.

Pros

The theming has come a long way. It no longer looks like a child’s toy. Even the default theme is very appealing. Also, it has dark themes, which I am always a fan of.

I Love the customization. I love the level of customization the plasma desktop allows. I can set transparency, add/remove panels or widgets, set custom keybindings (very important to me), and a bunch of other operations very easily.

Lets me hide window bars One customization I love is the ability to hide the window bars. I am not a fan of window bars, because I think they look clunky and just take up space. One feature I like about many tiling window managers is it is usually very easy to hide the window bars. However, that isn’t always the case in Desktop Environments, but I was pleased with how easy it was to do it in Plasma.
It Lets me create an awesome-wm environment, but as a full DE Similar to above, I am able to implement some of the features I love about tiling window managers like awesome and i3, but with the full desktop environment. In addition to removing window bars, I like to easily move and resize windows by hitting the meta or alt key and left/right clicking the windows. Again, in Plasma this was simple.
I love krunner. A lot. I like using application launchers, and krunner looks nice, is lightning fast, and gets the job done. I didn’t really know much about krunner during previous attempts at using KDE. This was definitely contributor to why I liked it much more this time around.

The ability to use old-school animations is fun There is not much to say here. While not very practical, sometimes spinning a desktop cube is just straight up fun.

Fast. Even for a full DE

Cons

Multi-monitor support. WTF. This was by far the biggest issue I had. My main computer is a 17" laptop that I connect to two 24" displays. However, because it is a indeed a laptop, I sometimes move it around and be semi-mobile with it. This requires disconnecting the monitors and going from 1 screen to 3 or vice-versa. Well it turns out that this is usually a pain in the ass for me in Plasma. First, I have to configure the monitors one-by-one, hitting “apply” between each configuration (it doesn’t like when I try to move and place all 3 displays in one swoop). Second, once I setup the screen placement, the panels often go crazy and can’t be found, or I have to play with them to set them up correctly. Lastly, sometimes the displays overlap each other oddly, even though they appear normal in the display settings. The wallpapers almost never set correctly and my right-click menu is disabled on some screens but not others. Basically, it’s just a hot mess and the most infuriating problem I faced when using Plasma. If I used a laptop without monitors, or even a permanent desktop setup, I would be fine. However, that is not my use case.

Randomly just stops working Sometimes, KDE applications just crash and I can’t seem to get them back. Widgets seemed to beak things more frequently, so I just stopped using them. (Again, not a good solution)
A full DE While I said above that it was nice having an environment that resembled a window manager, but had the benefit of the full suite of applications that come with a desktop environment, sometimes it was also a pain. There’s just a lot of crap that I don’t use. The obvious reminder of this was when I would run pacman -Syu, and see that the massive KDE stack would need updates.
Sometimes all the settings can be very confusing Having all these options is great, but sometimes it was overwhelming. Additionally, the settings felt disjointed and I didn’t know where the settings for some things where.

Final Thoughts

I can finally use Plasma
Very fast and lightweight for all that it is
Still has a bit of stability/polish work to go

Update: I took forever to get this post out and I have actually moved off of Plasma. I just couldn’t handle the multi-monitor issues I had. However, I will definitely be trying out the Desktop Environment again in the future (I moved to i3-gaps for now. I love my tiling window managers :) )

Using Cryogen for Website

2016-04-13T00:00:00Z

Ever since resurrecting my personal website, I have experimented with several static website generators. Thus far, I have tried org-page, Jekyll (several times), and even (almost) made my own emacs org-page solution. Now that I have started using the clojure programming language, I have come across cryogen. As you may have already guessed, this site is now being generated using cryogen.

Emacs/org-mode Solutions

About a year ago, obsession with emacs (and more importantly, org-mode) eventually led me to try out org-page while revitalizing my personal website. The idea of writing webpage content in .org files was very appealing and I wanted to gain a more experience writing emacs lisp, so it seemed like a good choice. Overall, org-page was a good learning experience, but I found the documentation and support to be lacking. It seemed to be more of a personal project, rather than a fully supported framework. while I know other people were able to get it working, org-page simply wasn’t working for me.

For over a year now, I have used org-mode at work each week to track my hours and to take notes (including code snippets). I then export the org files to html, creating a full index of my work notes/logs. After digging deeper into org-page, realized it was just a fancy wrapper around the org-project functions I used at work. So, I decided to implment my own emacs org-page solution (well, half implement… I guess I never fully finished it). Creating my own solution helped me learn even more about org-mode (which helped with my work notes), but I ultimately abandoned using this method for my personal website. Managing my own solution quickly became a pain because I had to setup emacs, install the dependencies, and get a .emacs file working just right on any compuer I wanted to generate the site from.

Jekyll

My first experience with Jekyll was about two years ago, right after I graduated college. It was the first true static website generator that I tried and I absolutely loved the concept. You see, I taught myself how to build websites in the early to mid 2000’s when I was in middle school, and apparently never developed beyond that point. During college, I maintained a personal site by hand. It was super simplistic html that used tables for the layout, and css for coloring. That’s about it. Using a static website generator like Jekyll for the first time was amazing, since it automatically produced static webpages that looked much better than anything I could do by hand. All I had to worry about was the content. It was what 12-year-old-me longed for (and tried to do using iframes and other messiness). I eventually stopped using Jekyll because I was unable to get the theming quite right, and … well … I started really getting into emacs. But you’ve already heard that story.

After taking a breif hiatus from Jekyll to adventure deeper into emacs land, I returned. This time, I was able to find and configure a enticing theme that fit my needs. My personal website probably looked the best is ever has, and I really enjoyed it. However, I have recently become frusterated using Jekyll (again). It is a great static website generator, but because I don’t often develop in Ruby (right now), setting up the proper ruby gems environment and dependencies on my computers makes me want to bang my head against the wall. Additionally, I have been learning clojure and while digging through different clojure projects, I found cryogen.

Cryogen

Cryogen is a static website generator written in Clojure. If you already have Leiningen installed (which if using Clojure, you should), starting a new Cryogen application is as easy as entering lein new cryogen project-name into a terminal. Once the project is created, you can cd into the directory and run lein ring server. Clojure will then fire up a local webserver of the compiled project (by default on port 3000). Whenever a change to a project file is saved, the cryogen server re-compiles the project and updates the webserver. As a result, it easy to edit and see the changes live. Cryogen has a rather large, but simple directory structure that is used to organize the project. This structure is slightly different from Jekyll, and takes a bit of getting to, but I think it does a better job at keeping everything organized once you learn it.

The one thing I really like about Cryogen is the fact that… well… it’s clojure. This means that things can often feel more “lisp-y”. For example, in Jekyll, the preferences and configuration of the website are kept inside a yaml configuration file. Similarly, the meta-data and information for a page or blog post are defined in a very specific yaml header at the top of the markdown files. Cryogen follows a similar concept, but instead of yaml headers, it simply uses a p-list for the configuration setup.

To change a post’s information (ex: title, author, date), one just has to change the keywords in the list at the top of the post’s markdown file. This flexability means that figuring out how to setup use my own configuration was a breeze. For programmers that have previously used Clojure or another LISP, the configuration in Cryogen is very intuitive and natural.

The only downside (sort of… it’s my fault) I have experienced while using Cryogen is that being a smaller community, there isn’t much out there in terms of templetes and themes (at least that I was able to find during a lazy search). So, while I was able to easily setup an amazing looking website using Jekyll (by using someone else’s hard work), I am forced to be a bit more hands-on using cryogen. Initially, I thought this was a negative, but after spending some time hacking away at the default theme and cleaning some rust off css/html skills, I think I have the site looking good enough for now. As a bonus, I am starting to re-learn web design. However, I am slowly catching up, so “modern” design features like mobil support might not happen right away. It’s not the best looking site, but I has personal touch, which I guess is good in a personal website.

That’s about it. That’s why my site suddenly looked different one weekend. If I look forward to using both Clojure and Cryogen more for not only this website, but other personal projects as well.

* PS: Re-reading this post I realize trying Cryogen after getting excited about Clojure is VERY much like when I started using org-page after getting excited about emacs. Hopefully this setup is a keeper that I stick with. ;)

Back On Arch, After Frantic Distro-hopping

2016-03-13T00:00:00Z

In the last few weeks, I have started my CS masters classes, and needed to setup a productive work environment on my portable laptop (abra). Also, while I love Fedora, the Nvidia stuff was being a pain on my main computer (alakazam), and I was looking for a replacement there as well. So, after a long, drawn out battle trying nearly all of my favorite Linux distros, I have found myself once again using Arch Linux on both of my personal computers.

When I started my journey, I had Fedora on both of my computers. I really like the Fedora Project and I think Fedora is a great distribution with a lot of innovative new features. I seems that the Fedora Project is aiming to be the Linux Distro for developers, and I feel that they are really starting to hit that target.

There is one remaining big issue I had while running Fedora on my main computer. Nvidia drivers. They are just a pain to maintain, and kept breaking my system during updates. I know the RPM Fusion Repos are supposed to help, but they just aren’t there yet. With my classes starting, I needed something that was a bit more stable (and would hopefully support installing VMWare Workstation 9).

The ‘Easy Stability’ of Ubuntu-based?

Guided by desires to have a “stable” distro, I decided that the obvious answer would be to use an Ubuntu-based distro. So, I went with Ubuntu Mate, based on the awesome work the team has done recently. I’ve used Ubuntu Mate in the past on several different computers, and in fact, still have it running on (charmander). I had some issues installing Ubuntu Mate on alakazam for some reason, but using a simple nomodeset at boot seemed work.

After I got it installed, I was able to install the nvidia drivers, but I had other random issues here and there. I also tried installing VMWare, but had issues at every step. To be completely honest, I really wasn’t feeling using an Ubuntu distro on alakazam. If I was going to be replacing Fedora, I wanted to make up for it by using something that gave me more power (I’ve started to get the itch to try out Gentoo again. Spoiler, I am not using Gentoo. At least until my class is over…). While Ubuntu could easily support my needs being the well supported and advanced distro that it is … I craved something else.

Leaping to Suse?

During my last few months on Fedora, I took notice of another rpm distro that was making noise in the Linux community: OpenSuse. Specifically, the new OpenSuse Leap piqued my interest. I had played with it in a VM a few days earlier and liked what I saw. The OpenSuse installer along with the YaST configuration tool makes it stupid easy to setup powerful and advanced features in Linux. Want to setup a KVM virtualization environment or a samba share? You basically just have to click a button. Testing out Snapper was also slick. I wish more distributions would start to include tools like snapper, built in.

Leap was a solid and extensive distribution. Ultimately though, that was the biggest qualm I had with it: there was a lot going on. I did not know what many of the pre-installed configuration tools were. Normally, I am fine with learning new tools, but I wanted a distribution that I could set up and get to work. OpenSuse had too many “toys” already in it. I would not be able to resist spending hours learning everything. Ironically, my other complaint was that some of the applications I did want, were not easily available. Often, they were not included in the default repositories, and while many application websites will provide a *.deb download, *.rpms are a bit more scarce. I know there is the wonderful openSuse Build Service, but like Fedora’s copr build system, I often found it to be hit or miss. Both do a good job at addressing a major issue in their respective distributions, but neither isn’t quite to the level of … well, the AUR.

Antegeros

Which brings me to Antergos. By this point, I was no longer able to hide the fact that I was starting to severely miss many features a distribution like Arch Linux provides. I love that you can easily customize just about anything. I also appreciate that if I want to try something new, 99% of the time the latest version is already in the official repos or the AUR. Additionally, when I am trying a new application, or customizing my setup, the Arch Wiki has some of the best documentation in the Linux ecosystem. However, I wanted something that I could just install and have working. I didn’t feel like going through a full vanilla Arch install. So I installed Antergos. Antergos has a simple but thorough installer that pulls down all the latest packages during the installation. This leaves the user with an up-to-date and gorgeous setup after installation.

Unfortunately, the “simplicity” of Antergos (which is sort of the antithesis to the definition of simplicity in The Arch Way) always seems to bite me in the end. Eventually, something breaks, and I have a hard time figuring out what. While it might be a pain to install everything in vanilla Arch, the user "has a much fuller understanding of his or her system" as a result. A few years ago, when I first started using Arch, I thought this argument just stupid gray beard nonsense. Now, I appreciate the benefits of this sentiment, and even agree with it (I’ve also learned to script my post install process, so it is less of a pain :P).

After a few days of fighting with Antegeros, I decided to just bite the bullet and install Vanilla Arch. I still think Antegros is a great distribution that works for a bunch of people. It is great for users that want to experience the power of arch Linux, without having to go through all the overhead. Antegeros is actually how I started to learn Arch Linux. Nowadays though, I seem to prefer simplicity of vanilla Arch.

All Roads Lead to Arch

I have been running Arch on both of my personal laptops for several weeks now. Arch doesn’t magically solve all of my issues, but it makes fixing them much easier. If something isn’t working, the combination of the wiki and the community make the problem much more approachable. Additionally, the rolling release model appears to mitigate bug-related issues, instead of cause them. For example, I installed Arch on alakazam first, but left Ubuntu Mate on abra (my portable laptop). I did this because my school has a weird network setup that is often difficult to connect to. After searching for hours, I determined that my issues under Ubuntu were caused by a known, but unfixed, bug in my version of network manager. I did not feel like dealing with this, so I installed arch. My internet at school works works fine now. I was able to use the wiki to properly configure my settings, and I was good to go. (Side-note: I never fully got VMWare 9 to work properly on any of my distros, but I came closest with Arch.)

Abra running Arch Linux with the [Awesome](http://awesome.naquadah.org/) window manager

Being back on arch has been great. I love using tiling window managers like i3, dwm, xmonad, stumpwm, and awesome. While this this is possible on all distributions, it is much easier on arch. Starting with a blank slate, and (again) having the wiki makes setting up alternative desktop environments and window managers a breeze. I now have a very productive work environment that is tailored to my needs. While Arch Linux might not be a traditional “stable” distribution, I have found it to be much more durable for my use-case.

I am planning to stay on Arch for awhile. That being said, I really did enjoy all the other distributions I tried out. Each one seems to be carving out a niche in the Linux environment:

Fedora is becoming a great distribution for Linux developers. In fact, I still have Fedora 23 (Server Edition) running on my home server. Assuming that the upgrade to 24 goes smoothly (my upgrade from 22 -> 23 went great, thanks to dnf), I see no reason to switch.
OpenSuse Leap is a solid distribution that is great for power users that want a turn-key distribution with all the bells and whistles. I think it is especially geared toward people with system administration backgrounds.
Ubuntu Mate is really the best distribution out there for people that want a classic feeling Linux environment, with a modern flair. It is what Ubuntu would be if it never left gnome 2. It is also great for low resource hardware, particularly boards like the raspberry pi.
Antegros is fantastic for Arch users that hate the Arch install. It is also great for people that want to move to Arch, but aren’t yet comfortable with tools like pacman.

It is great to see all the distributions finally finding their groove. This isn’t something I could say about Linux 2 years ago. As I have already stated, I plan to stay on Arch Linux for the time being, but I am very excited to continue trying out the new releases of the all distributions listed above. With so many great options, it really is a great time to be a Linux user :) .

How I asked my Groomsmen

2016-01-06T00:00:00Z

Over the last few months Rebecca (my Fiancee) and I have continued making progress on all the little tasks that we have to complete before our wedding. One of these tasks is asking our intended bridesmaids/groomsmen to be in the wedding.

Rebecca approached this by asking each of the girls in a way that was somewhat unique to each person, over time. Even though each person was asked differently, all the methods were executed in a very “Rebecca-like” manner.

So, I was told that I couldn’t simply ask the guys. I had to figure out a “fun” way to ask them, and if possible, to do it in a way that was very “Ryan-like”.

Asking the Groomsmen

While Rebecca was slowly chipping away at asking her bridesmaids, we kept failing at finding a way to ask the guys. Finally, one night while driving back north to Massachusetts, I said out of frustration:

“I wish I could just write a stupid piece of code that asks them, compile it, and give it to them… wait a minute…”

At that moment, I started to figure out how I could make it work. The results of that brainstorming session are described below:

Snail Mail Part

I first had to find a way to give the intended groomsmen a link to get any code that I wrote. So, Rebecca and I designed the tri-fold letter pictured below (Sorry for the photo quality).

Figure 1: The Front of Letter

The front of the envelope had a simple black boarder and said “She said that I had to make it official.”

Figure 2: The First Flap of the Letter

When the reader flipped up that flap, another message read “So we’re doing this my way”.

Figure 3: The QR Code posted on the Inside of the Letter

Lastly, when the reader folded that flap down, they got a taste of what “my way” might entail when the only thing that was printed on the page was a single QR code.

When the reader scanned the QR code, it opened up a list of instructions on their phone that said:

Find your computer.

Go to /message/ and follow the instructions.

Text Me

*Note: Link no longer active.

Website

When the intended groomsmen went to the link provided, they saw a webpage that I made, containing our wedding logo on the bottom right, and a welcome message. The welcome message basically stated that the webpage was not the end of the scavenger hunt, and that they would have to download some code that I compiled, that contained the message.

Below the welcome message, I had separate instructions that guided Window, Mac, and Linux users, as well as users that wished to compile from source (sort of).

Below that I had links to the executable downloads for each operating system mentioned. Lastly, I posted the source code for the code they were downloading, because … you know… open source. Executable(s)

When the intended groomsmen ran the compiled code (source code posted below), it showed them that it was “decrypting” their message, and printed the percent complete. After about 20% or so, I confessed that there was no “decrypting” and that each message is decoded when printed. I then proceeded to rant in these “decrypting” messages about how I originally wrote a nice GUI python app, but it was a pain to cross compile, and how I eventually just scrapped it and wrote it in LISP. I also mentioned that if they all ran Linux, it wouldn’t of been an issue. When it finally reached 100%, I decided to finally give them the message.

Source Code for the executable I wrote containing the message. Nothing fancy, but everything hidden.


;; Used to build the executable
(defun build-exe (exe-src)
  (sb-ext:save-lisp-and-die exe-src :toplevel #'main :executable t))

(defun main ()
  (intro-message)
  (decrypt-loading)
  (sleep 1.7)
  (secret-message))

(defun intro-message ()
  (format t "~a~%~%" (decode-message '(72 101 108 108 111 33 32 80 108 101 97 115 
                                       101 32 119 97 105 116 32 97 32 109 105 110 
                                       117 116 101 32 119 104 105 108 101 32 121 
                                       111 117 114 32 115 101 99 114 101 116 32 
                                       109 101 115 115 97 103 101 32 105 115 32 
                                       100 101 99 114 121 112 116 101 100 33)))
  (sleep 2)
  (format t "~a~%" (decode-message '(83 116 97 114 116 105 110 103 32 100 101 99 114 
                                     121 112 116 105 111 110 46 46 46)))
  (sleep 1))

(defun decrypt-loading ()
  (let* ((messages '((#\Enq 89 111 117 32 107 110 111 119 44 32 105 116 32 114 101 97 108 108 121
                            32 100 111 101 115 110 39 116 32 116 97 107 101 32 116 104 105 115 32 108 111
                            110 103 32 116 111 32 100 101 99 114 121 112 116 46 46)
                     (#\Newline 73 110 32 102 97 99 116 44 32 105 116 32 116 101 99 104 110 105 99
                      97 108 108 121 32 105 115 110 39 116 32 39 100 101 99 114 121 112 116 105 110
                      103 39 32 97 110 121 116 104 105 110 103 46)
                     (#\Si 69 118 101 114 121 32 109 101 115 115 97 103 101 32 105 115 32 39 100
                      101 99 114 121 112 116 101 100 39 32 119 104 101 110 32 100 105 115 112 108
                      97 121 101 100 46)
                     (#\Dc4 73 110 99 108 117 100 105 110 103 32 116 104 105 115 32 111 110 101 46
                      46 46)
                     (#\Nak 65 110 100 32 84 104 105 115 32 111 110 101 46 46 46)
                     (#\Syn 65 110 100 32 84 104 105 115 32 111 110 101 46 46 46)
                     (#\Etb 65 110 100 32 84 104 105 115 32 111 110 101 46 46 46)
                     (#\Can 65 110 100 32 84 104 105 115 32 111 110 101 46 46 46)
                     (#\Em 65 110 100 32 69 118 101 110 32 84 104 105 115 32 111 110 101 46 46 46)
                     (#\Sub 73 32 116 104 105 110 107 32 73 39 118 101 32 109 97 100 101 32 109 121
                      32 112 111 105 110 116 46)
                     (#\Rs 65 108 115 111 32 115 111 114 114 121 32 102 111 114 32 116 104 101 32
                      99 111 109 109 97 110 100 32 108 105 110 101 32 105 110 116 101 114 102 97 99
                      101 46)
                     (#\! 73 32 119 114 111 116 101 32 97 32 114 101 97 108 108 121 32 110 105 99
                      101 32 71 85 73 32 117 115 105 110 103 32 81 116 53 32 97 110 100 32 80 121
                      116 104 111 110 46)
                     (#\$ 66 85 84 44 32 98 101 99 97 117 115 101 32 121 111 117 32 97 115 115 104
                      111 108 101 115 32 117 115 101 32 101 118 101 114 121 32 111 116 104 101 114
                      32 115 104 105 116 116 121 32 79 112 101 114 97 116 105 110 103 32 83 121 115
                      116 101 109 32 107 110 111 119 110 32 116 111 32 109 97 110 46 46 46)
                     (#\' 73 32 104 97 118 101 32 116 111 32 99 114 111 115 115 45 99 111 109 112
                      105 108 101 32 116 104 105 115 32 97 99 114 111 115 115 32 87 105 110 100 111
                      119 115 44 32 77 97 99 44 32 97 110 100 32 76 105 110 117 120 46)
                     (#\* 65 110 100 32 105 116 32 116 117 114 110 115 32 111 117 116 32 115 101
                      116 116 105 110 103 32 117 112 32 112 121 116 104 111 110 32 97 110 100 32 81
                      116 53 32 111 110 32 97 108 108 32 116 104 111 115 101 32 112 108 97 116 102
                      111 114 109 115 46 46 46)
                     (#\- 73 115 32 97 32 103 105 97 110 116 32 112 97 105 110 32 105 110 32 116
                      104 101 32 97 115 115 46)
                     (#\0 46 46 46 69 115 112 101 99 105 97 108 108 121 32 111 110 32 87 105 110
                      100 111 119 115 46 46 46 40 84 104 97 110 107 115 32 77 105 99 114 111 115
                      111 102 116 41)
                     (#\2 40 83 99 114 101 119 32 121 111 117 32 80 101 116 101 114 32 58 80 41)
                     (#\7 83 111 44 32 73 32 100 101 99 105 100 101 100 32 116 111 32 115 99 114 97
                      112 32 116 104 101 32 71 85 73 32 97 110 100 32 106 117 115 116 32 117 115
                      101 32 109 121 32 112 121 116 104 111 110 32 99 111 100 101 46)
                     (#\: 72 111 119 101 118 101 114 44 32 99 111 109 112 105 108 105 110 103 32
                      105 116 32 119 105 116 104 32 116 104 101 32 114 105 103 104 116 32 118 101
                      114 115 105 111 110 32 111 102 32 112 121 116 104 111 110 32 111 110 32 97
                      108 108 32 116 104 101 32 112 108 97 116 102 111 114 109 115 46 46 46)
                     (#\= 87 97 115 32 97 108 115 111 32 97 32 112 97 105 110 32 105 110 32 116 104
                      101 32 97 115 115 46)
                     (#\@ 65 105 110 39 116 32 110 111 98 111 100 121 32 103 111 116 32 116 105 109
                      101 32 102 111 114 32 116 104 97 116 46)
                     (#\C 83 111 44 32 73 32 104 97 100 32 116 111 32 114 101 45 119 114 105 116
                      101 32 109 121 32 99 111 100 101 32 105 110 32 67 111 109 109 111 110 32 76
                      105 115 112 46)
                     (#\F 76 117 99 107 105 108 121 44 32 73 32 97 109 32 97 32 112 114 111 102 101
                      115 115 105 111 110 97 108 32 76 105 115 112 32 80 114 111 103 114 97 109 109
                      101 114 46)
                     (#\I 66 117 116 32 115 116 105 108 108 46 46 46 32 87 84 70 46)
                     (#\M 66 97 115 105 99 97 108 108 121 44 32 116 104 101 32 109 111 114 97 108
                      32 111 102 32 116 104 105 115 32 115 116 111 114 121 32 105 115 58)
                     (#\P 73 102 32 121 111 117 32 106 101 114 107 115 32 97 108 108 32 117 115 101
                      100 32 76 105 110 117 120 44 32 73 32 119 111 117 108 100 110 39 116 32 111
                      102 32 104 97 100 32 116 104 101 115 101 32 105 115 115 117 101 115 46)
                     (#\S 83 111 32 115 111 114 114 121 32 105 102 32 121 111 117 32 119 111 117
                      108 100 32 112 114 101 102 101 114 32 97 32 110 105 99 101 32 71 85 73 32 97
                      112 112 32 105 110 115 116 101 97 100 32 111 102 32 97 32 116 104 105 115 32
                      99 111 109 109 97 110 100 32 108 105 110 101 32 111 110 101 46)
                     (#\V 66 117 116 32 105 116 39 115 32 114 101 97 108 108 121 32 121 111 117 114
                      32 100 97 109 110 32 102 97 117 108 116 46 32 40 58 80 41)
                     (#\Y 65 110 100 32 98 101 115 105 100 101 115 44 32 116 104 101 114 101 32 105
                      115 32 110 111 116 104 105 110 103 32 119 114 111 110 103 32 119 105 116 104
                      32 116 104 101 32 99 111 109 109 97 110 100 32 108 105 110 101 46)
                     (#\\ 66 117 116 32 73 32 100 105 103 114 101 115 115 46)
                     (#\_ 73 32 110 111 116 105 99 101 32 119 101 32 97 114 101 32 110 101 97 114
                      108 121 32 97 116 32 49 48 48 37 46 46 46)
                     (#\b 83 111 32 73 32 103 117 101 115 115 32 73 32 115 104 111 117 108 100 32
                      115 116 111 112 32 114 97 110 116 105 110 103 32 97 110 100 32 115 104 111
                      119 32 121 111 117 32 116 104 101 32 109 101 115 115 97 103 101 46 46 46)
                     (#\d 70 105 110 101 32 58 41 46)))
         (decrypted-messages (decode-pair-list messages)))

    (format t "~a~%" (decode-message '(68 101 99 114 121 112 116 105 110 103 32 77 101 115 115 97 103 101 58)))

    (dotimes (i 101) 
      (cond ((equal i (caar decrypted-messages))
             (format t "~d% ~a   ~a ~%" i (decode-message '(67 111 109 112 108 101 116 101 46))
                     (cdar decrypted-messages))
             (setf decrypted-messages (cdr decrypted-messages)))
            (t
             (format t "~d% ~a~%" i  (decode-message '(67 111 109 112 108 101 116 101 46)))))
      (sleep 1))))

(defun secret-message ()
  (format t "~%~a~%~%~%- ~a~%~%" 
          (decode-message '(83 111 44 32 112 117 116 116 105 110 103 32 97 108 108 32 106 111 107 101 115
                            32 97 110 100 32 110 101 114 100 121 45 110 101 115 115 32 97 115 105 100 101
                            44 32 73 32 100 111 32 104 97 118 101 32 97 110 32 105 109 112 111 114 116 97
                            110 116 32 113 117 101 115 116 105 111 110 32 116 111 32 97 115 107 46 32 65
                            115 32 121 111 117 32 97 114 101 32 112 114 111 98 97 98 108 121 32 97 119 97
                            114 101 44 32 82 101 98 101 99 99 97 32 97 110 100 32 73 32 97 114 101 32 102
                            105 110 97 108 108 121 32 103 101 116 116 105 110 103 32 109 97 114 114 105
                            101 100 32 105 110 32 83 101 112 116 101 109 98 101 114 46 32 84 104 105 115
                            32 105 115 32 119 105 108 108 32 98 101 32 97 32 118 101 114 121 32 115 112
                            101 99 105 97 108 32 42 68 65 89 42 32 105 110 32 109 121 32 108 105 102 101
                            44 32 97 110 100 32 97 115 32 97 32 118 101 114 121 32 115 112 101 99 105 97
                            108 32 42 80 69 82 83 79 78 42 32 105 110 32 109 121 32 108 105 102 101 44 32
                            73 32 119 111 117 108 100 32 108 105 107 101 32 116 111 32 97 115 107 58 32 32
                            87 105 108 108 32 121 111 117 32 98 101 32 111 110 101 32 111 102 32 109 121
                            32 103 114 111 111 109 115 109 101 110 63))
          (decode-message '(82 121 97 110))))

(defun encode-message (message)
  (mapcar #'char-code (concatenate 'list message)))

(defun decode-message (num-list)
  (map 'string #'code-char num-list))

(defun encode-pair-list (pair-list)
  (mapcar (lambda (pair) (cons (code-char (car pair)) (encode-message (cdr pair)))) pair-list))

;;(load "decryptMessage.cl")

(defun decode-pair-list (pair-list)
 (mapcar (lambda (pair) (cons (char-code (car pair)) (decode-message (cdr pair)))) pair-list))

Message

After putting up with my ranting, I finally posted the message asking the guys to be my groomsmen. It read:

So, putting all jokes and nerdy-ness aside, I do have an important question to ask. As you are probably aware, Rebecca and I are finally getting married in September. This is will be a very special DAY in my life, and as a very special PERSON in my life, I would like to ask: Will you be one of my groomsmen?

Ryan

About 2 days after we mailed out the letters, all of then intended groomsmen had messaged me back, confirming that they would be my groomsmen. I also think they enjoyed my methods of asking.

I guess I was successful in asking everyone in a very “Ryan-like” fashion.

UPDATE: The source code repo for this little project is now publically available on my Github Page.

25 Days of C

2015-12-24T00:00:00Z

As I continue to learn more and more programming languages, I am somewhat ashamed to admit that I have yet to learn C. However, it is not because I am trying to avoid it.

The Reason

C is one of the top languages I wish to know, and know well. C is the backbone for so much code, especially in the Linux ecosphere that it is almost imperative for one to at least understand it. I keep trying to poke my head into open source projects, and so many of them are built or based on C (especially when looking at projects more at the distro and distro applications level).

Also, this spring I will start working on my CS MS degree, and while I think the first course I take will be theory based, I know several other courses that will require it (ex: computer architecture and compilers). Being comfortable with C will allow me to focus on the topics, and not on the syntax.

I have some experience with C++ from my software engineering course I took during undergrad, but that is about it. My goal however is to understand just plain C, which will hopefully make implementing other C-based languages such as C++ a bit easier. My ultimate goal is to eventually have C be one of my top 3 languages (I’m thinking LISP, C, and maybe python or ruby. I know. I’m old school.).

The Plan

In the last few weeks, I have set up and started looking at Learn C the Hard Way. I read the first section or two, but haven’t been consistent with working through it.

Now that it is December, I thought I should maybe start a new monthly challenge, or even a “12/25 days till something”. I was thinking about things I’ve been trying to constantly work on the last few months, and C popped into my head. Learn C the Hard Way fits particularly well because it is broken down nicely into small segments that I can maybe commit to being able to finish 1 per day.

I have to look ahead a bit to see if this challenge is doable, but i think that as of now, the plan is to try to complete at least 1 (I can work ahead if I want to) segment of the book a day for the month of December. I am also going to allow myself the ability to “pre-cache” segments ahead of time so that I don’t have to pull away and work on it during the holidays if I don’t want to.

Overall, I think this is a good plan that should allow me to start getting some C experience under my belt. I’ll be sure to update in the future with how it goes.

Update

So, now that I have made it a good bit through the month, I must admit that I have not adhered to this challenge very strictly. I have done many of the exercises, but I have fallen somewhat behind. Basically, I accomplished a bunch of other things and let this challenge slide. I will continue to work on it, but I haven’t been able to keep with with the one exercise per day rate. Oh well.

ryan.himmelwright.net

2026 Yearly Theme

Deciding the Theme

Year of Sovereign Inquiry

So Far

Reliance on Apple

Shifting Local-first and Home-labbing again

Notes/Journal

Time Tracking

Task Management

Limiting Mental Inputs

Reading

Exercise

Target Second Language

Going Forward

Tech Got Expensive - Revisiting My Order History

The Data

Conclusion

BTW, My Chuwi Minibook X Died...

Intro

Fixing Attempts

Conclusion

Now Update - 2026-02-24

Indie Dev

Hardware Experiments

Home

Books

Clojure Script for Project Git Logs

Week Post #1797

Projects

Music

Links

I Asked Gemini Why My Linux Computer Feels Peaceful

Week Post #1795

Projects

Music

Links

Re-enabled Contrast Accessibility Settings on my Apple Devices

Background

Why Switch Back

The Switch

Conclusion

Timewarrior Trial - My iOS Shortcut

Desiring Timewarrior

The Setup

Terminal Computers

My Phone

Conclusion

2025 Recap

Home & Family

Career

Website Changes/Improvements

Notable Tech Hardware Changes

Notable Tech Software Changes

Concerts/Shows

Books Read/Listened to

Podcasts

Conclusion

Now Update - 2025-12-20

Learning & Projects

Home

Books

Added New Pages to Website

Explore Page

Mumbles

/POSSE

Script-Generated Pages

Reading List

Blogroll

Various Other Slash Pages

/defaults

/guy

/save

/chipotle

Conclusion

Chuwi Minibook X - Initial Thoughts

The good

Build/value

Performance

Size