Llogiq on stuff

Measuring SmallVec Footprint with Smallvectune

Rust is all about paying only for what you use, and gives us plenty tools to eliminate unneeded allocation. One of the tools that is used in a lot of crates (crates.io shows 98 dependent crates) is SmallVec. It is also used in the Rust compiler. I recently got around to speed up the operation of getting a SmallVec from a slice of copyable data. In short, they’re awesome.

SmallVecs use a trick to put data inline if it fits, only spilling into an internal Vec when their inner capacity is exceeded. For maximal flexibility, they get a type parameter that is an array of 1..36, 64, 128, 256, .. up to 220 of the component type. But how to decide on the right array size, especially if you use a lot of different SmallVecs?

First, the size in memory of a SmallVec<[Item; N]> is mem::sizeof<usize>() + max(2 * mem::sizeof<usize>(), mem::sizeof<Item> * N) (there is also four bytes for a discriminant unless you activate the union feature which gets rid of it). Also the spilled variant will incur an allocation of mem::sizeof<Item>() * capacity.

Note that size in memory is not the only thing to optimize for – having a slightly larger SmallVec on stack in exchange for less allocs might be a good thing – but let’s first look at memory, as that’s usually an easy win, and we cannot in general distinguish SmallVecs on stack from those on the heap.

Even so, in a larger application, it’s not trivial to even keep track of the actual capacity distribution of all SmallVecs, and I think having that data would be valuable. So I wrote a small wrapper crate around smallvec to keep track of all capacity changes, logging them to a file so we can summarize them later.

This is the smallvectune crate.


Let’s take your typical smallvec-using crate. The Cargo.toml has a

# comment this out
# smallvec = "0.6.5"

# use this instead
smallvectune = "0.0.1"

In your lib.rs (if it’s a library crate), change

// don't use directly
// extern crate smallvec;

// use this instead
extern crate smallvectune as smallvec;

Now set the environment variable SMALLVECTUNE_OUT to somewhere writeable and run the workload you want to analyze. This will get you a CSV file with

item size in bytes;internal array size;+ or -;capacity

The + entries show that a SmallVec was created or resized to the new capacity, the - entries show a SmallVec that was deallocated or resized from the old capacity. This should be enough to keep a running count of the smallvecs at any point in time, and also estimate their memory footprint. We can differentiate use sites by using different internal array sizes.

I intend to use this with the Rust compiler. Stay tuned. Also if you have one of the 98 SmallVec-dependent crates, try it out and tell me what you think!

PS.: Please someone help me with summarizing this data.