Timeshift and grub-btrfs in Linux Arch

UPDATED 02/Jan/2023, ChangeLog:

  • 02/Jan/2023: documented that the new version of grub-btrfs is now an official package (you still have to install another package: inotify-tools);
  • 02/Dec/2022: documented the new version of grub-btrfs and its new grub-btrfsd daemon; the configuration for Timeshift is much simpler, but you have to install another package: inotify-tools.

After looking at the very nice videos of Stephen’s Tech Talks, in particular, this one https://www.youtube.com/watch?v=6wUtRkEWBwE, I decided to try to set up Timeshift, Timeshift-autosnap, and grub-btrfs in my Linux Arch installation, where I’m using BTRFS as the filesystem. These three packages allow a timeshift snapshot to be automatically created each time you update your system; moreover, a new grub entry is automatically generated to boot into a specific snapshot.

The video mentioned above is handy, but unfortunately, some recent changes in Timeshift itself broke the behavior of the two other packages. In this post, I’ll try to show how to fix the problem and go back to a working behavior. I’ll also show an experiment using the snapshots so that, hopefully, it’s clear what’s going on in the presence of such snapshots and how to use them in case you want to revert your system.

Install timeshift and timeshift-autosnap

First of all, let’s install timeshift and timeshift-autosnap (the latter depends on the former, and they are both available from AUR; I’m using the yay AUR helper here):

The programs will be installed from sources; thus, they will be compiled (it might take some time).

Let’s create a new Timeshift snapshot to make sure it works (the first time, you will have to configure Timeshift; of course, it is crucial that you choose “BTRFS”).

You can configure timeshift-autosnap with the number of snapshots to keep (in this example, I specify 10):

Install grub-btrfs (new version, installation from the official repository, 02/Jan/2023)

The new version of grub-btrfs is now available as an official package (Please remove the old AUR version if you still have it installed):

So you can simply install it with pacman:

Now, let’s make sure grub-btrfs can find Timeshift’s snapshots (remember, we’ve just created one). So let’s update the grub configuration, and we should see in the end something like the following output:

The last lines prove that grub-btrfs can detect snapshots.

We now need to configure that to monitor the Timeshift snapshot directory instead of the default one (/.snapshots).

Automatically update the grub menu upon snapshot creation or deletion (2 December 2022)

What follows is based on the new version of grub-btrfs. At the bottom of the post, there are still the old instructions, which are to be considered stale and left there only for “historical reasons”.

Grub-btrfs provides a daemon watching the snapshot directory and updates the grub menu automatically every time a snapshot is created or deleted.

Important: This daemon requires an additional package:

By default, this daemon watches the directory “/.snapshots” (the default directory for Snapper). Since Timeshift uses a different directory, we have to tweak the configuration for the daemon.

Let’s run:

We must change the line

into

This is required for Timeshift version 22.06 and later because Timeshift creates a new directory named after their process ID in /run/timeshift every time they are started. Since the PID will be different every time, also the directory will be different. Grub-btrfs provides the command line argument –timeshift-auto to correctly detect the current snapshot directory (In previous versions of grub-btrfs, we had to tweak /etc/fstab to deal with that, as shown later in the old section).

Let’s start the daemon:

In the journalctl log, we should see something like (where the date and time have been stripped off):

Let’s start Timeshift. In the journalctl log, we should see something like this:

Let’s verify that if we create a new snapshot, grub-btrfs automatically updates the GRUB menu: in a terminal window, run “journalctl -f” to look at the log, then create a new snapshot in Timeshift. In the log, you should see something like the following lines:

Similarly, if we delete an existing snapshot, we should see something similar in the log.

Remember that it takes a few seconds for grub-btrfs to recreate the grub menu.

Once we’re sure everything works, we can enable the daemon to always start at boot:

The next time we boot, our grub menu will also show a submenu to boot snapshots.

Concerning doing some experiments booting a snapshot and restoring it, please look at the next section.

IMPORTANT: If you have several Linux distributions on your computer and you use a multiboot system like the one I blogged about, and this distribution is not the main one, you will have to manually tweak the entry in your main distribution’s GRUB menu. See the linked blog post near the end.

Some experiments

Let’s do some experiments with this configuration.

Here’s the kernel I’m currently running:

I’m updating the system (I’m skipping some output below, and you can ignore the “stale mount” errors):

So it created a snapshot before updating the system (in particular, it installed a new kernel version). Let’s reboot and verify we are running the new kernel (5.18.8 instead of 5.18.7):

Let’s reboot and select from GRUB the latest snapshot (remember, the one before applying the upgrade), so timeshift-btrfs/snapshots/2022-07-02_15-35-53 (snapshots are presented in the grub submenu from the most recent to the oldest one). We do that by pretending that the update broke the system (it’s not the case), and we want to get back to a working system before the update we have just performed.

You see that the “Authentication Required” dialog greets us, and in the background, you can see the notification that we “booted into Timeshift Snapshot, please restore the snapshot”:

The password is required because it’s trying to run Timeshift:

In the screenshot, you can see that we are now using the older kernel since we booted in that snapshot, where the update has not yet been performed. We have to restore the snapshot manually; otherwise, on the next boot, we’ll get back to the updated system version and not in the snapshot anymore.

So, let’s restore the snapshot:

You see, Timeshift has created another snapshot ([LIVE]). We now reboot normally (that is, using the main grub entry, NOT the snapshot entries).

Once rebooted normally, we can verify again that we are running the old kernel:

Let’s have a look at Timeshift, and we can see the last snapshot is an effective one, not a LIVE one:

Yes, we are now in a system where the update above has never been applied.

Let’s try to rerun the update command (we don’t effectively execute the update, it’s just an experiment):

Why? Because the snapshot had been created automatically by timeshift-autosnap before applying the updates while the package manager was running, its lock is still there.

Let’s remove the lock and try to rerun the update:

The output is similar to the one shown above (unless there are even more new updates in the meantime, which might happen in a rolling release), but something is missing:

Why? Because the downloaded packages in the cache are NOT part of the saved snapshot, they are still present in the current system, even though we restored the snapshot. Why are the cached packages still there, but the lock has been restored with the snapshot? That’s due to the way subvolumes are specified in the /etc/fstab:

You see, the cache of downloaded packages and the logs are NOT part of the snapshots, while /var/lib (including the pacman lock) is part of the snapshots.

Let’s now revert the snapshot: we select the one with “Before restoring…”.

Again, we are now in a LIVE situation, and Timeshift tells us again to reboot to make it effective.

Let’s reboot (by using the main grub entry).

We’re back to the updated system, and there’s nothing to update (again, unless new updates have been made available in the meantime):

If we’re happy with the updated system, we can also remove those two snapshots (remember that grub-btrfs monitors the snapshots so that it will update its grub submenu entries):

I hope you find this blog post helpful, and I hope it complements the beautiful video of Stephen’s Tech Talks mentioned above.

Old version (with old release 4.11 of grub-btrfs)

UPDATE 02/Dec/2022: These are the older instructions for the previous version of grub-btrfs, where there was no “grub-btrfsd.service” and there was another systemd program (“grub-btrfs.path”).

I leave these instructions here just for “historical reasons”.

The first problem is that timeshift has recently changed the strategy for creating snapshots. Instead of creating them in /run/timeshift/backup/timeshift-btrfs/snapshots, it now creates them in /run/timeshift/<PID>/backup/timeshift-btrfs/snapshots, where <PID> is the PID of the Timeshift process. Each time you run Timeshift, the directory will be different, breaking grub-btrfs (which expects to find the snapshots always in the same directory).

Fortunately, there’s a workaround: we add an entry to /etc/fstab in order to mount explicitly the path /run/timeshift/backup/timeshift-btrfs/snapshots:

where, of course, <UUID> has to be replaced with the same UUID of the physical disk partition.

Reboot, and then Timeshift will also put the snapshot in that directory (besides the one with the PID, as mentioned above). You can try to create a snapshot to verify that (this also allows us to use the Timeshift wizard so that we specify to create BTRFS snapshots).

Let’s make sure the mount point is active (and note the unit name)

Let’s now install grub-btrfs

We need to configure that to monitor the Timeshift snapshot directory instead of the default one (/.snapshots).

The file contents

should be replaced with

Let’s reload and re-enable the monitoring service:

If we have already created a few snapshots, we can run update-grub (or, if you have not installed the package update-grub, use the command “grub-mkconfig -o /boot/grub/grub.cfg”) and verify that new grub entries are created for the found snapshots:

We can also restart the system and prove that we can access the GRUB submenu with the generated entries for the snapshots.

9 thoughts on “Timeshift and grub-btrfs in Linux Arch

  1. Pingback: Linux EndeavourOS Cassini Review | Lorenzo Bettini

  2. David

    Hello Lorenzo, What a great blog. Thank you for posting these detailed instructions, I thought I was missing something until you pointed to the fact that grub-btrfs had been updated and required a different configuration. How you managed to work it out, I have no idea but I am glad you did.

    Reply
  3. TomX

    Hello Lorenzo, I’m using Endeavour. After a recent update when I run systemctl –failed grub-btrfsd.service loaded failed failed Regenerate grub-btrfs.cfg I have tried look on the internet but with on luck. Can help by pointing the right place to look.

    Reply
    1. Lorenzo Bettini Post author

      Without further error details, it’s hard to understand what’s going on.
      Some time ago, IIRC, an update to btrfs-progs broke grub-btrfs. At that time, it was enough to downgrade btrfs-progs and block the upgrade til they fixed the problem in that package.
      Maybe that’s the same.
      It’s better to ask the EndeavourOS forum or, even better, create an issue in grub-btrfs.
      For the time being, it’s better to restore a working snapshot.

      Reply

Leave a Reply to Lorenzo BettiniCancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.