By George!

npins + colmena personal configuration

2025-03-12T00:00:00-04:00

Inspired by various blog posts, amongst which Jade’s Pinning NixOS with npins, or how to kill channels forever without flakes, I decided to migrate my Flake-based configuration, which included nixosConfigurations, Home Manager homeConfigurations, system-manager systemConfigs, and a devShell, away from Flakes and to npins. I don’t claim that this is inherently better than Flakes, but it was an interesting process and I think I learned some things working on it.

One npins feature I was particularly interested in is the ability to follow the tags of a Git repository. A Flake can only follow a ref (which can be a tag, but doesn’t automatically update). It’s also interesting to be able to follow a nixpkgs channel instead of the corresponding Git branch, though that doesn’t seem to have any observable effects.

If you want to follow along at home, the last Flake-based commit is last-flake.

Rather than the wrapper-script approach described by Jade, I decided to use colmena to deploy NixOS systems. While I’m gaining experience with tools, I might as well gain experience with two tools at once.

System Configurations

Before I started the conversion, I did some general cleanup. For example, I switched from the Lix overlay module to using the Lix included in nixpkgs so I could take advantage of Hydra builds.

I started by migrating the nixosConfigurations to colmena. This was pretty straightforward. The only thing that tripped me up is that colmena checks for the presence of flake.nix to decide if it should use Flakes or not, so I renamed my flake.nix so I could have it for reference during the conversion and have colmena read from hive.nix.

In the initial conversion, I lost only nix-index-database since I couldn’t immediately figure out how to reference its home-manager module without going through the flakes interface.

Initially, I tried to configure cross compilation for my Raspberry Pi. Unfortunately, this basically requires building everything since I can’t take advantage of Hydra builds, so I switched to using buildOnHost = true. That was faster even though I end up building some software on the Pi since most of the software is available from the cache. The ideal thing would be if I had a more powerful aarch64-linux machine, I could configure it as a remote builder and do the builds there, but I don’t have one.

While writing this blog post, I tried building the configuration for the Pi under emulation but not cross compiling. This worked, and did substitute most of the configuration, and it’s slightly faster than building directly on the Pi, so I’m going to leave that in place for now.

Home Manager

Home manager was a little harder. When not running in Flake mode, the home-manager command uses <nixpkgs> and <home-manager> from NIX_PATH. I used the wrapper script approach described by Jade to have these reference the proper pinned values.

system-manager

On my work laptop, which runs Linux but not NixOS, I was using system-manager to set up GPU drivers for Nix-built software. When I first did this conversion, there was no documented way to use system-manager without Flakes, and I didn’t look into it. When I came back a few weeks later, system-manager had had modifications that simplified using the Nix components without Flakes, though still no documentation. The system-manager command still assumes Flakes, but I was able to write a shell script that uses raw Nix commands to build, register, and activate the system-manager configuration.

nix-index-database

nix-index-database is also designed to be consumed as a Flake. It has a default.nix that provides the wrapped packages, that works fine, but the Home Manager module is provided as a functor that expects the Flake (self) as the argument. Even so, we have all the tools available to make this work. I wrote this adapter module:

{ pkgs, ... }:
let
  sources = import ../../npins;
  packages = (import sources.nix-index-database) { inherit pkgs; };
  dummyFlake = {
    packages.${pkgs.stdenv.system} = packages;
  };
  hmModule = (import "${sources.nix-index-database}/home-manager-module.nix") dummyFlake;
in
{
  imports = [ hmModule ];
}

I read the home manager module functor to figure out what attributes were needed, which turns out just to be the packages for the current system. It re-imports the npins sources since using the sources that I provided through extraSpecialArgs caused an infinite recursion error.

And there you have it: a Flake-free pinned configuration for all the things.

Why to prefer Option, in one table

2024-10-14T00:00:00-04:00

Some time back, I came across Logan Smith’s Choose the Right Option video, and the other day I was trying to summarize it for a colleague.

I think it boils down to this: If we assume T: ?Clone, then

to convert…	to `&Option<T>`…	to `Option<&T>`…
`x: T`	`&Some(x)` (moves `x`)	`Some(&x)`
`x: &T`	not possible	`Some(x)`
`x: Option<T>`	`&x`	`x.as_ref()`
`x: Option<Box<T>>`	not possible	`x.as_deref()`
`x: &Option<T>`	`x`	`x.as_ref()`

plus niche optimization.

Four ways to get or insert in a Rust map

2024-09-07T00:00:00-04:00

The Rust standard library contains two map types: HashMap and BTreeMap. Both of them implement the entry API to allow efficient insert-or-update behavior. I will give examples using HashMap, but BTreeMap implements the same API, too.

The Simplest Case: Inserting the default

This one is pretty trivial. If the value type implements Default with the right behavior, you can just use Entry::or_default.

Here’s a snippet that consumes an iterator and returns a map containing the counts of each element:

let mut counts = HashMap::new();
for x in iter {
    *counts.entry(x.to_owned()).or_default() += 1;
}

For example, if iter is "abracadabra".chars(), then counts ends up being {'b': 2, 'a': 5, 'd': 1, 'c': 1, 'r': 2}.

Still Simple Case: A very cheap constructor

If the value you want to insert isn’t the default, but has a literal representation or is otherwise cheap, you can use Entry::or_insert.

The example is a little contrived (you could easily use Entry::or_default and increment before printing instead of after) but it shows how it works:

let mut numbers = HashMap::new();
for x in "abracadabra".chars() {
    let number = numbers.entry(x).or_insert(1);
    println!("{x}: {number}");
    *number += 1;
}

Output:

a: 1
b: 1
r: 1
a: 2
c: 1
a: 3
d: 1
a: 4
b: 2
r: 2
a: 5

Moderately Complicated: An expensive constructor

Suppose now that the value type has a constructor, but you want to avoid running it when the value already exists. In this case, use Entry::or_insert_with.

This example is again contrived (Entry::or_default would also work here) but it shows how it works:

let mut words_by_length = HashMap::new();
for word in "Four score and seven years ago".split_ascii_whitespace() {
    words_by_length
        .entry(word.len())
        .or_insert_with(|| Vec::with_capacity(4))
        .push(word.to_owned());
}
println!("{words_by_length:?}");

Output:

{5: ["score", "seven", "years"], 3: ["and", "ago"], 4: ["Four"]}

Hard case: A fallible or async constructor

Here’s the reason for this blog post’s existence: if the constructor isn’t a synchronous, infallible function, you have to reach for the full generality of the entry API to get the job done.

This writes lines into many files, where (file, line) pairs arrive in random order, but the files are kept open to avoid the overhead of opening each one multiple times. I’ll show this one in its full glory, including imports and function boilerplate:

use std::collections::hash_map::Entry;
use std::collections::HashMap;
use std::fs::File;
use std::io::prelude::*;

fn main() -> std::io::Result<()> {
    let mut files = HashMap::new();

    for (path, line) in [
        ("a.txt", "Hello"),
        ("b.txt", "Hello"),
        ("b.txt", "This is file B"),
        ("a.txt", "This is file A"),
        ("c.txt", "This is file C"),
    ] {
        let file = match files.entry(path.to_owned()) {
            Entry::Occupied(o) => o.into_mut(),
            Entry::Vacant(v) => v.insert(File::create(path)?),
        };
        writeln!(file, "{line}")?;
    }

    Ok(())
}

The thing that tripped me up (twice) about this example is that it feels like OccupiedEntry::get_mut is the right thing to use in the Entry::Occupied branch because it has the same return type as VacantEntry::insert, but that actually results in the borrow not living long enough:

error[E0597]: `o` does not live long enough
  --> examples/fallible.rs:17:39
   |
16 |         let file = match files.entry(path.to_owned()) {
   |             ---- borrow later stored here
17 |             Entry::Occupied(mut o) => o.get_mut(),
   |                             -----     ^         - `o` dropped here while still borrowed
   |                             |         |
   |                             |         borrowed value does not live long enough
   |                             binding `o` declared here

For more information about this error, try `rustc --explain E0597`.

Final Thoughts

I feel like this may be pointing to a gap in the documentation somewhere, but I’m undecided about where I think the documentation needs to go. For the moment, I’m happy having written this down where I can find it again, and I hope it’s helpful for you, too.

I’ve posted fully-runnable examples if you want to see all the detail.

Update History

2024-09-08

Updated “expensive” example from Vec::with_capacity(1) to Vec::with_capacity(4), as suggested by @mo8it.

Generating secure passwords

2022-05-12T00:00:00-04:00

I’ve used password managers for many years now, and they have many security benefits, including secure password generation. One pain point is, and always has been, that websites have random rules for what constitutes an “acceptable” password. I’ve ended up with an extremely basic solution: a shell:

tr -cd '!-~' </dev/urandom | head -c 16

That example gives a random password consisting of all printable ASCII characters uniformly. Here’s an example I used recently for a site that only accepted a subset of special characters:

tr -cd 'a-zA-Z0-9!@#$%^&*' </dev/urandom | head -c 16

This doesn’t help with other rules, like “you must include one character of each type” or “you can’t include more than three characters of the same type in a row”, but I can usually just draw a couple times until the conditions are all satisfied.

2020 Stewardship Testimonial

2020-09-20T00:00:00-04:00

This is the text of the Stewardship Testimonial I gave during the service on September 20, 2020.

Good morning. My name is George Macon, and I am one of the Stewardship Co-chairs for this year.

I’d like to unpack my title a little bit here. I’m a “co-chair” with Barbara Begner. Together, we run the Stewardship Committee, that is, in parliamentary language, we “chair” it. That’s pretty straightforward, so that brings us to “stewardship”.

This is where it gets deep.

A “steward” is a person who manages or maintains something on behalf of, and in the interest of, another, usually absent, person. This has two parts: what thing are we, as a committee, managing, and on whose behalf are we managing it?

The obvious thing that the committee does is raise money to fund the congregation’s operational budget. That doesn’t really fit in with the “maintain” part of the definition, though. So you look around for something that’s being maintained.

There’s no obvious “property” in the traditional sense. While UUCA does have an endowment, it’s stewarded by the UUCA Endowment Fund, a separate legal entity, so that can’t be the answer. The Stewardship Committee is stewarding something intangible: the congregation itself, or “church” in its original meaning of a group of people. Our goal is to ensure that this community, this congregation, continues to exist into the future. That takes money, certainly, but the money is not the point.

So, the stewardship committee is maintaining the congregation on behalf of … itself. But, importantly, on behalf of the future congregation’s selves. We all maintain this community, this loving community, so that it will exist for those people who will find it in the future, not only those of us who stay.

It’s important for this community not to stagnate, but to continue to grow, not just in size, but also in our positive impact on Atlanta, on Georgia, on the United States, and on the world at large.

This community is built by every member giving generously of their time, talent, and treasure. If you have the time to volunteer to help with something, I encourage you to do so. If you have the talent to enrich this community, I encourage you to use it. If you have the treasure to enable the amazing things happening here, I encourage you to give generously.

We cannot do our work in the world without your financial support, whether it takes the form of a pledge to the Capital Campaign, a stewardship pledge to our operating budget, or a virtual dollar bill placed in our virtual offering basket at uuca.org/give. We thank you very much for your gifts to all of these efforts.

The offering will now be given, and very gratefully received.

A limerick which must be seen

2015-11-26T00:00:00-05:00

You can’t call the British Queen Ms.
Tain’t as nice as Elizabeth is.
But I think that the Queen
Would be even less keen
To have herself mentioned as Ls.

Asimov, Isaac. “Out of Sight”. Ellery Queen’s Mystery Magazine Dec. 1973.

A Tool to Handle /etc/passwd.pacnew and Friends

2014-07-11T00:00:00-04:00

The last several times I’ve updated filesystem, .pacnew files were created for /etc/passwd, /etc/shadow, /etc/group, and /etc/gshadow. My default approach for merging .pacnew files is to use vimdiff to edit the original and new files simultaneously. Unfortunately, this doesn’t work very well for /etc/passwd and friends because they are databases, not configuration files.

My first thought was to make the files friendlier to vimdiff by sorting all the lines. That works well enough, but I decided that it would be better to use the tools provided for updating the password database to update the password database, so I wrote a Python script that reads /etc/passwd, /etc/passwd.pacnew, /etc/group, and /etc/group.pacnew and suggests useradd, usermod, groupadd, and groupmod commands to run to bring the live versions up to date.

The script looks at each user and group present in both files and users and groups present only in the .pacnew files. Users and groups existing only in the live files are ignored; these are the users that I have added or that were added by packages. If either (or both) of the .pacnew files are missing, the live copy is used instead. This allows the script to be run if only one or the other of the files was updated.

Group membership is evaluated as part of the user for two reasons. First, because usermod is used to modify group memberships, not groupmod. Second, because this avoids printing suggestions to remove all my users from the predefined groups (like systemd-journal, for instance).

You can download the script and use it yourself. It’s licensed under the 2-clause BSD license.

Liars and Outliers Review

2013-10-30T00:00:00-04:00

I’ve just finished reading Liars and Outliers by Bruce Schneier. I received a signed copy thanks to Schneier’s discounted signed book offer of $11 plus a review. So here’s a review:

In this book, Schneier takes on all of security: What is it, and why does it work? The answer flows through diverse areas of study, from evolutionary psychology to game theory. He begins (appropriately enough) with history; a discussion of predators and prey. From microbiology, we move rapidly forward through time to modern society.

After taking a look at history, Schneier moves into a discussion of the four societal pressures: moral, reputational, institutional, and security. Each kind of pressure is built off of the previous ones, with security being the most advanced.

Once these basics are established, he moves into discussion of the real world. In this part, he examines conflicting interests, organizations as actors, corporations specifically, and institutions covering all the ways that the theory breaks down in practice.

One of the most important things covered is the “security gap”: defectors are faster to pick up on new technologies than defenders. This means that security cannot solve all the problems. There are many examples of this, the classic one being the arms race: attackers use bows and arrows, so defenders wear armor. Then firearms are developed, and the armor is no longer effective, so the defenders lose the armor and hide in ditches. Defenders don’t invest in new technology without a reason, and better attacks are the best reason.

This is another book I’ve read recently that is long on a description of the problem and short on specific solutions. Near the end of the book, Schneier gives the following list of principles:

Understand the societal dilemma.
Consider all four social pressures.
Pay attention to scale.
Foster empathy and community, increasing moral and reputational pressures.
Use security systems to scale moral and reputational pressures.
Harmonize institutional pressures across related technologies.
Ensure that financial penalties account for the likelihood that a defection will be detected.
Choose general and reactive security systems.
Reduce concentrations of power.
Require transparency—especially in corporations and government institutions.

These principles provide an excellent foundation on top of which solutions can be built, and, given the scope of what he’s chosen to tackle, it seems reasonable not to propose solutions to narrow issues like airline security. However, I would have liked to see a specific proposal for getting society’s “agent”, i.e. the government, to follow these principles when implementing policy.