|
The next natural experiment is to ask: what if we don't care about waiting
for the data to be committed to nonvolatile disk? To simulate this, we
wrote the image files to /tmp, which is a swap-backed filesystem in
Solaris.
On the left hand side of the graph, we see that latency is again dominated by a bandwidth issue; this time it's the memory bus bandwidth. We measured the memory bandwidth of our machines at 231MB/sec, and 67MB/sec is about a quarter of that. We suspect here that the /tmp filesystem makes a second copy of the data, for a total of two reads and two writes for each byte of data moved. On the right side of the graph, the performance goes down the tubes once the image file is over 32MB (one a quarter of the installed RAM). We suspect that once /tmp begins swapping to disk, the pages aren't written as orderly and efficiently as they are when written to a real ufs filesystem. The real conclusion, of course, was that the /tmp experiment didn't really tell us much about asynchronous checkpointing. |