Better Aero Field Testing

My last try at aero field testing was plagued with a number of problems,
making the results quite unsatisfactory and untrustworthy.  Here were
a few of the known problems:

• The power data can be bad if you stop pedaling.  In particular, the interaction between a crank based power meter like my quarq and the 705 is bad when you stop pedaling because the quarq stops sending updates and the 705 interprets this as the same as the last reported power (possibly with a timeout and some interpolation).  In practice, this means getting files that can have repeats of a large power with a cadence of 0.  In my previous tests, I hadn't been aware of this and probably spent some time where I wasn't at least soft-pedaling.
• The quarq is sensitive to which chain ring you are in.  I did a simple comparison of my quarq against my powertap while running two head units and found that the two chain rings report a power difference of about 2-4 watts at 200watts.  This is the expected behavior and is considered pretty good.  The moral for testing with a crank based power meter is to stay in one chain-ring.
• I tried to experiment with multiple magnets after the previously reported test and got some weird results with speed dropouts, etc.  I eventually settled on getting a new battery for the GSC-10 and also got some nice rare-earth magnets from Walmart's hobby department and taped three to my H3Cs.  This then requires telling the head unit that my wheel is 1/3 the rollout.  Here's what that looks like:

• I was concerned about the repeatability between runs, so I started using the "calibrate" function on the 705 head unit and recording the zero-offset that the quarq reported back.  Perhaps a bit paranoid, but seems nice to have the data around.
• I decided I needed to confirm that I could get repeatable results with the same configuration, including stoping, reseting the head unit, etc.  In other words, do A/A/A/... tests till they matched.  Then when I wanted to test a new setup (assuming I could get A/A/A/... working), do A/B/A/B/... type tests.
• In some of my previous runs, I found that making the turnaround could be dicey.  I moved my turnaround points slightly further uphill to make it easy (at the cost of a slightly longer course).
• I started doing just two laps per setup.  That has the advantage of testing my cross test issues that I was concerned about, and also makes it less painful to bail on a trial when I get passed by cars and its clear that the run will not be clean.

So today looked like a good day to do some more testing.  I was scheduled to do a long ride in my training plan, but didn't want to go too far from home because the weather forecast was calling for rain.  The weather was pretty mild, although I did get hit with a little drizzle during the testing.  My initial goal was to see whether I could do some A/A/A/... runs with my new knowledge and protocol.  I then threw in some A/B/A/B tests.  When I got home, those test results looked pretty good, so I decided to do some more tests with A'/B'/...

So what were the configurations I tested.  Nothing too complex.  I tested my basic bike setup with a road helmet for the initial trial.  My initial A/B/A/B tests were just with A=baseline, B=A+no gloves.  My second set of runs had the same setup, except I switched to my aero helmet as the baseline.  I then tested that second baseline with no gloves and then with no bento box.  So we have A'=aero helmet, B'=A'+no gloves, C'=A'+no bento.  Here's a shot of my baseline bike position (taken afterward on a trainer) and the equipment that I changed:

I also tried a hill-climb repeat with different speed to try to pry apart CdA from Crr.  The protocol I used there was Fast/Slow/Fast/Slow.  For the hill climb, I just stopped the head unit at the start point and then
braked, returned to the start point, stopped the bike and then restarted the head unit.  The transitions are a little iffy, but the results looked pretty decent to me:

This gave me a Crr estimate of 0.0042, which seems to match what I might have guessed for my nice fast tires.  For the Rho, I just used a density calculator with conditions from a nearby weather station. My weight was from a household scale.

Here are the details of the runs.  I put a "*" next to the CdA of the trials that looked "good" in post-processing.

Trial   Time    Config  Offset  AvgSpeed        CdA     Notes
1       8:19    A       -178    19.3            .256*   GPS altitude worked!
2       8:28    A       -177    20.8            .254*           
3       8:36    A       -175    20.4            .252*
4       8:45    A       -182    20.2            .254*
5       8:53    B       -175    20.4            --      Bad car interference
6       9:02    B       -179    20.3            .248*
7       9:11    A       -181    19.9            .252*
8       9:20    B       -178    20.7            .244*
9       9:33    A       -166    20.0            .252*   

Trial   Time    Config  Offset  AvgSpeed        CdA     Notes
1       10:55   A'      -182    20.4            .236    Car interference
2       11:08   B'      -177    20.4            .242*   GPS altitude worked!
3       11:21   A'      -177    20.5            .232*   GPS altitude worked!
4       11:29   B'      -177    20.2            .238    Jagged looking VE
5       11:38   A'      -175    19.3            .230    Jagged looking VE
6       11:47   C'      -178    19.7            .240    Jagged looking VE
7       11:58   A'      -174    19.6            .246    Jagged looking VE
8       12:07   C'      -174    19.3            .242    Jagged looking VE
9       12:18   A'      -182    19.0            .229*
10      12:27   C'      -181    19.6            .244*
11      12:37   A'      -182    21.8            .240*

Generally, this attempt looks much better.  The A/A/A/A stuff is very close.  Config "B" with no gloves seems to test slightly faster, which is the expected result.  Comparing A to A' is a little dicey since they weren't done quite back to back in alternating fashion, but we are only talking about an hours difference and I didn't change anything else.   The second set of runs were less clean in both the results and the plots.  I'm not sure what was up between 11:29 and 12:18, I think it may have actually started raining more then.

I'll attach all the plots at the end of this post so you can get an idea of what I mean by good looking vs. bad looking Virtual Elevation plots.

So what's my conclusion.  It looks like I can actually get some reasonable repeatability.  I don't think I can really make any claims yet about being able to distinguish the configurations I tried.  There were not really enough "clean" runs to feel confidant about the results.  I'd guess that no gloves is indeed better by .004-.008, aero helmet is buying at least .010 and that the bento may be better, but needs some more clean runs to be sure.

For anyone interested in the raw data, I uploaded it into a Google docs folder.  The filenames should be pretty clear (data/timestamp embedded in the name).

EDIT 09/14/2011: I think I can explain the jagged plots.  The next ride I took, I noticed my speed was a little flaky.  It looks like my magnets were slowly sliding outward (centripetal force I guess) and the tape wasn't quite keeping them aligned.  Not surprisingly, if the speed gets flaky, the VE plots get flaky.  Guess I'll have to find a better way to attach them to the wheels.  I would prefer something that is removable later, so I don't want to do epoxy...

EDIT 09/17/2011: Yet another caveat.  I discovered that my Garmin 705 reset itself to "smart" recording when I upgraded the firmware.  The smart recording feature sucks for this type of thing.  It is a wonder that my data looks at all reasonable. 

Here was the second set of trials.