Homemade system provisioning and dotfiles management: SetMeUp!

jjpk.me

Whenever I get a new laptop to set up for my own use, I always end up asking myself: alright, how am I going to bring all of my desktop setup here? I'm a little bit OCD for those things, and if there's one thing I hate, it's having slightly different polybars on my home and work laptop because I've made that one little change and haven't taken the time to apply it everywhere.

Suuure, I could just use git. I've seen a lot of people with a dotfiles repository, and what they typically do is clone that somewhere and then make a bunch of symlinks. That's central enough but with stuff under ~/.config and those old apps who haven't got the memo yet (hi bash), you end up doing a lot of it manually (I mean, you could script it, but who's got the time).

Now, there are a lot of nice little tools out there for just that purpose (dotfiles management). Unfortunately, I did not get that satisfying feeling from any of them, so I started working on a little idea. Also, I don't want my homedir crowded with symlinks, I don't find that... subtle. What if all I had to do what SSH into a remote server, and have it do the work for me? I work with Ansible all day, surely something can be done here.

Introducing SetMeUp!

SetMeUp! is the name I ended up giving to that little project. In a nutshell, it is an Ansible-based provisoning tool. You install it on a publicly-available server, and all the clients have to do is SSH there and run it. With a little bit of key setup and reverse port forwarding, you end up with very little to do on the provisioned client.

Get it

SetMeUp! is available on my GitLab (I'm keeping it synced on GitHub as well). If you trust GitLab CI/CD, then you can keep it "simple" by downloading a compiled release from there. The pipeline provides both a dynamic and a static executable.

Otherwise, getting it isn't as easy as apt install curl (yet...) but it's pretty straightforward to build with a Rust toolchain.

$ git clone https://gitlab.com/julienjpk/setmeup
$ cd setmeup
$ cargo test
$ cargo build --release
$ sudo install -m 755 target/release/setmeup /usr/local/bin

If you can't build on your server directly, I suggest compiling with MUSL and a statically-linked OpenSSL.

Set it up

Now, there are many ways to set up how SetMeUp! is used. Here, I'll describe how I use it myself. My goal is to simplify the command I will be using from the client. To that end, I first create an smu user on the server:

# useradd -md /var/lib/setmeup smu

Then, I force it to always use SetMeUp! when it logs in (no shell access). This is adjusted in /etc/ssh/sshd_config :

Match User smu
    ForceCommand /usr/local/bin/setmeup

Finally, I add my SSH public key to the user's ~/.ssh/authorized_keys file so that I can actually use that user. In my case, I always carry my SSH key (part of my PGP key) on a little encrypted USB flashdrive that sits well on my keychain. This way, all I need to do is plug it into the new laptop, open the key, and login.

Use it

Now, the one thing I did not want to bother with here is NAT. I don't want anything to be an issue between SetMeUp! and my client. Fortunately, SSH has a very neat feature that can help me bypass all of that: reverse forwarding.

ssh -R 0:localhost:22 smu@smu-server

What this does is bind a random port number on the SetMeUp! server to the client's port 22 via the SSH connection we're establishing here. Because this happens over the SSH stream created by the client, the connection is already established and nothing will prevent us from getting through.

SetMeUp! will therefore start by asking you about that port:

$ ssh -R 0:localhost:22 smu-server
Allocated port 40251 for remote forward to localhost:22
Welcome to Set Me Up!
Which port did ssh bind to for remote forwarding?

Then, you need to provide a username. That user should exist on the client. Note that if you intend to use Ansible's become feature to do privileged things, that user needs to be a sudoer (eg. you want to install packages).

Which username should SetMeUp use to reach you over SSH?

Finally, SetMeUp! needs a way to authenticate to the client. It does not support password login and will instead provide you with a public key:

SetMeUp will be using an ECDSA keypair to authenticate with your machine.
Please make sure user bob has the following public key in their ~/.ssh/authorized_keys file:

ecdsa-sha2-nistp256 ...

Press the Enter key where you are done:

Before you press Enter, add that key to the aforementioned user's ~/.ssh/authorized_keys file. SetMeUp! will then perform a quick authentication test. If all goes well, it should print a list of sources :

Here are the available provisioning sources:

[1] dotfiles

Select by index (1-1) :

Sources are basically directories in which your Ansible playbooks are available. We will see how those are configured on the server later on. Once you've chosen a source, you get to chose the playbook you want to run:

Preparing the source...
Here are the available playbooks for source dotfiles:

[1] user_session.yml
[2] desktop.yml

Select by index (1-2) :

In my case, user_session.yml sets up my shell and my Emacs (nice for servers). desktop.yml goes further and also sets up my desktop environment. Once you've made that choice, SetMeUp! hands everything over to Ansible and the playbook is run!

Set Me Up run

Configuring SetMeUp!

Which playbooks are available to use depends on the server-side configuration. SetMeUp! looks for its configuration file in the usual places, and in my case I put it at /var/lib/setmeup/.config/setmeup/setmeup.yml.

Basic configuration

For the basics, here's an example:

sources:
  dotfiles:
    path: "/var/lib/setmeup/dotfiles"

The configuration file is YAML and contains a single sources map. Each item is keyed by the source name (eg. dotfiles earlier) and is associated with various parameters. The only mandatory parameter is path. It should point to a directory which contains YAML playbooks. By default, SetMeUp! will only include playbooks right under that directory (it does not recurse) based on a simple \.ya?ml$ regex.

Going further

For more complex setups, SetMeUp! actually supports a handful of source options. Here's a more complete example:

sources:
  some_local_source:
    path: "/etc/setmeup/playbooks"
    playbook_match: "^public/.+\.ya?ml$"

  some_git_repository:
    path: "~/some_git_repository"
    recurse: yes
    pre_provision: "git pull"
    ansible_playbook:
      path: "/usr/local/bin/ansible-playbook"
      env:
        - name: "ANSIBLE_CONFIG"
          value: "ansible.cfg"
        - name: "ANSIBLE_ROLES_PATH"
          value: "roles"

You can find the parameter descriptions on GitLab but the main elements should be self-explanatory:

  • You can write a custom REGEX to match playbooks, and search resursively.
  • You can have SetMeUp! run a local command before provisioning. This is useful if your directory is a git repository and you want to make sure it's pulled.
  • You can customise the environment for ansible-playbook : path and variables. Note that ansible-playbook runs from the source's root directory, so you can use relative paths (eg. for ANSIBLE_ROLES_PATH).

And there you have it! Obviously I haven't (yet) had the opportunity to use it to its full potential, but what I ended up with on some test VMs is quite pleasant. A quick re-login and a startx later, I had a neat little replica of my home setup.

Now of course I have to round up all those little changes and add them to my playbooks...