Nicolas LIMARE / pro / notes / 2013 / Disposable Filesystem Deployment Time

Brief: Copying a basic standard Debian filesystem takes 0.3s. Extracting from an uncompressed tar archive takes 0.28s. By optimizing the filesystem content we can get 0.12s and probably even better.

Exploring solutions for on-demand sandboxing, I observed last week that KVM/qemu takes 5 seconds to initialize, too much for quick-launch setups. So, instead of full virtualization, I am now investigating tools to simply isolate system resources.

These tools need a minimal filesystem to live in. But this filesystem must be pristine at every launch and any change discarded after execution. So I start by looking at disposable filesystem solutions.

copy-on-write

One option is Copy-On-Write (COW): instead of working directly on the filesystem, one uses a working overlay on top of a read-only view of the base filesystem. As the main COW solutions are implemented as filesystems (aufs, lvm) and can not be used without administrator privilege, so I prefer to avoid them for the moment.

cowdancer, could be simple userland alternative, but it only works via its cow-shell interface. If we use the qemu emulator/virtualizer, then its qcow2 disk image has options to preserve a pristine base image and store changes in an external overlay; this seems to be a robust userland tool, but restricted to solutions based on qemu.

But let's start with the simplest solution. Can we quickly create a basic filesystem by copy and simply delete it after use?

building a basic filesystem

The basic filesystem is created for a Debian distribution system with debootstrap. Instead of the standard options

sudo debootstrap wheezy http://http.debian.net/debian

we use fakeroot/fakechroot to proceed without admin privileges:

fakeroot fakechroot debootstrap wheezy --variant=fakechroot http://http.debian.net/debian

As of 2013-01-29, this basic filesystem uses 247M for the amd64 architecture.

the actual timings

I did the following timings on my laptop.

  • Intel i5 M520 CPU at 2.4GHz, no hyperthreading, frequency scaling set to maximum performance, no other demanding task running
  • Debian Wheezy amd64 system, kernel 3.2.0-4-amd64
  • every file copied/extracted from/to a ramfs

A simple copy takes 0.33s:

$ time cp -a chrootfs tmpchrootfs

real    0m0.326s
user    0m0.016s
sys     0m0.304s

Extracting from a tar archive is faster and takes 0.28s, probably because it required less fopen():

$ time tar xf chrootfs.tar

real    0m0.277s
user    0m0.028s
sys     0m0.244s

But using a gzipped tarball (125M instead of 235M) is counter-productive and requires ten times more CPU resources to uncompress the archive:

$ time tar xzf chrootfs.tgz

real    0m2.185s
user    0m2.108s
sys     0m0.376s

and with a smaller filesystem

I quickly downsized the basic filesystem, removing the following folders not needed to execute a userland program:

rm -rf usr/share/{doc,locale,man} var/cache var/lib/{dpkg,apt}

This reduces the filesystem from 241M to 91M, the tar archive from 235M to 85M, and the deployment time from 0.28s to 0.12s:

$ time tar xf chrootfs.tar

real    0m0.120s
user    0m0.028s
sys     0m0.088s

Finally, it's gettng good. We need a deployment time significantly shorter than 1s, and the benefits from this quick-and-dirty trimming suggests we can achieve very good results by carefully choosing the list of packages install by debootstrap.