Drunkonunleaded's list is pretty solid. I'll add a few observations as someone who has been monitoring C5Z prices closely as of late:
1) '01s are considerable cheaper than similar condition 02-04s.
2) Mechanically, the cars tend to be solid regardless of mileage, with the exceptions listed before. I'll also add suspension bushings to the list. They're practically a wear item—especially on an autocross of track car. Luckily they're cheap and relatively easy to replace (well-being, tricky to replace the bushings, but super easy to just replace the whole control arm).
3) Valve springs should be considered for replacement, but are also cheap and easy.
4) Cheap Z06s will be high-mileage rats. They'll still be fast as hell, but the pain will be faded and there's about a 99% chance that the interior will need serious TLC. If you're building a track rat, this might be a nice place to start. If you want a street car, the right price for an 02-04 is probably still $20k+, but not + much. Values seem to be pretty linear (i.e. a $19,000 car + $3000 of refurb = a $22,000 car.) So you can kind of buy at the level you want to play at and not fear diminishing returns too much.
5) That said, mileage and condition are pretty big in the Corvette community. Once the cars drop below about 40,000 miles, the prices start to creep up dramatically—especially among dealers.
6) '04 carbon hood Z16s command a slight premium, but not as much as you'd think. If you want the "ultimate" Z06, that's obviously the one. Prices tend to run a little more then other '04s of similar mileage, but that almost tends to be more because they tend to be well-cared for examples, being "commemorative editions" and such.
PS: You didn't hear this here, but I know a certain magazine gearing up for a C5Z project when a certain Mustang project has completed its cycle.
(EDIT: And, yes, everyone definitely needs to STFU about how these cars are "hot" and everyone should "buy now because prices will skyrocket!" Keep that E36 M3 to yourself until about October. Thanks.)