More Space For Cargo

17 October 2018

I love Rust and so much enjoy writing it that I will often do so voluntarily.

However, some things make it harder for me than needed to write Rust. See, I have precious little time, and this time is also often interrupted. During this time, I may or may not be connected to the internet, and my Chromebook’s main persistent storage is 32GB, so I usually have less than 5GB of free space.

Needless to say, this can present problems. Cargo, while otherwise very good, will often not be very thrifty with hard drive space. The target/ directory of a project will contain all compiled dependencies and various other files taking up space. I am unsure why this is necessary – I often have many projects which have similar dependencies (for example serde), so wouldn’t it suffice to store them in one place, perhaps distinguished by selected features, target triple etc. so the compiled libs can be reused without recompiling?

Apart from that, cargo keeps a per-user directory where it stores the registry and downloaded sources for each version of each crate ever used. So in practice this sources fill up to a few GB of my precious hard drive space.

I haven’t found a reliable procedure to clean up outdated crate sources, so I just delete that directory every now and then. Except that means I’ll have to re-download all of them whenever I’ll try to recompile a crate that uses them, over a possibly spotty internet connection.

So what to do?

I imagine two utilities. One would be applicable to any .cargo source directory, walk all sources and remove all but the newest and their recursive dependencies. The other would get a number of projects directories and search their Cargo.tomls for dependencies, removing every source download that isn’t used in those projects afterwards.

Actually, those utilities are very similar – the sole difference is the first step, where the former scans just the registry source directory and keeping a map of crate name to latest version, discarding older versions on the way, to recreate a set of directories that we’ll use as a base. The latter must walk all subdirectories and seek out Cargo.tomls.

There are a few bumps on that road. The first is finding the cargo source directory on all platforms reliably. For now, I’ll use $HOME/.cargo. Then walking the directory, selecting the newest versions and the maximum by semver. Luckily the semver crate has got this one covered. Finally tracking the dependencies and walking the dir again, removing all unmarked directories and dealing with possible errors on each step.

Also we may want to report outdated dependencies for crates, so we can send update PRs to further reduce disk space usage by dependencies.