Ex. 39: Warm-up period?

@zedshaw In exercise 39 you talk about a “warm-up period” that the algorithms might need until they work at full speed.

I wonder, why would a computer need to warm up? That sounds like a curiously human trait for a machine.

I experienced this for real today. I was comparing the performance of dictionary implementations in a real world application (counting unique words in a long text file) and for the life of me I couldn’t figure out why my balanced tree was reported to be much slower than the primitive binary search tree when it had been faster or way faster in previous tests. Then I changed the order of the tests and that pretty much gave the reverse result. When I added a few seconds of garbage allocations before, things seemed to “normalize”.

Is this because the CPU speed is not a constant factor but changes in reaction to workload, temperature etc?

And what can I do to be sure I get accurate results? Can I infer that performance tests are always likely to be way off unless the computer has been working steadily for a few seconds before?

Investigating further, it seems that I get results that match my expectations if I execute the exact processes I want to measure for a few seconds before I actually clock them.

Even performing a few seconds of random garbage allocations yields funny results that are totally not consistent on identical input.

How can I ever be sure my results are accurate?