Measuring SmallVec Footprint with Smallvectune
Rust is all about paying only for what you use, and gives us plenty tools to
eliminate unneeded allocation. One of the tools that is used in a lot of crates
(crates.io shows 98 dependent crates) is SmallVec. It is also used in the
Rust compiler. I recently got around to speed up the operation of getting a
SmallVec from a slice of copyable data. In short, they’re awesome.
SmallVecs use a trick to put data inline if it fits, only spilling into an
internal Vec when their inner capacity is exceeded. For maximal flexibility,
they get a type parameter that is an array of 1..36, 64, 128, 256, .. up to
220 of the component type. But how to decide on the right array
size, especially if you use a lot of different SmallVecs?
First, the size in memory of a SmallVec<[Item; N]> is
mem::sizeof<usize>() + max(2 * mem::sizeof<usize>(), mem::sizeof<Item> * N)
(there is also four bytes for a discriminant unless you activate the union
feature which gets rid of it). Also the spilled variant will incur an
allocation of mem::sizeof<Item>() * capacity.
Note that size in memory is not the only thing to optimize for – having a
slightly larger SmallVec on stack in exchange for less allocs might be a good
thing – but let’s first look at memory, as that’s usually an easy win, and we
cannot in general distinguish SmallVecs on stack from those on the heap.
Even so, in a larger application, it’s not trivial to even keep track of the
actual capacity distribution of all SmallVecs, and I think having that data
would be valuable. So I wrote a small wrapper crate around smallvec to keep
track of all capacity changes, logging them to a file so we can summarize them
later.
This is the smallvectune crate.
Usage
Let’s take your typical smallvec-using crate. The Cargo.toml has a
[dependencies]
# comment this out
# smallvec = "0.6.5"
# use this instead
smallvectune = "0.0.1"
In your lib.rs (if it’s a library crate), change
// don't use directly
// extern crate smallvec;
// use this instead
extern crate smallvectune as smallvec;
Now set the environment variable SMALLVECTUNE_OUT to somewhere writeable and
run the workload you want to analyze. This will get you a CSV file with
item size in bytes;internal array size;+ or -;capacity
The + entries show that a SmallVec was created or resized to the new
capacity, the - entries show a SmallVec that was deallocated or resized
from the old capacity. This should be enough to keep a running count of the
smallvecs at any point in time, and also estimate their memory footprint. We
can differentiate use sites by using different internal array sizes.
I intend to use this with the Rust compiler. Stay tuned. Also if you have one of the 98 SmallVec-dependent crates, try it out and tell me what you think!
PS.: Please someone help me with summarizing this data.