Hi all,
I was just looking through the endurance lap time results from the recent Michigan Comp.
Very interesting data, thank you to the organisers for providing it in its entirety.
Excel format would have been nice, but it didn't take too long to get everything from the PDF into excel to be able to run some numbers.
Like most engineers with big and interesting data sets I started playing around, taking averages and doing comparisons etc.
One idea that I have been thinking about lately is to develop a quantitative metric to assess or rank "Driver Team Performance" for the endurance event.
The big challenge is trying to make this metric independent of the vehicle performance.
Something like this would also help teams focus on, and track the development of driver team "depth" (or consistency), realising that the endurance event is scored on the lap times generated by both drivers and not just your best one.
It would also highlight the teams with lower performance car, but excellent driver training and preparation, and those with better cars but have room to improve on the driving front.
For the moment, I would like to put aside the issue of the importance of drivers in FSAE, and whether that is a good thing or not.
I hope everyone accepts that drivers (drawn from your own team) are a simple fact of life in this competition.
Given this reality I think we can all accept that it is obvious for teams to try and maximise their performance, as you would any highly sensitive variable in any other engineering problem.
I have always felt that well trained and experienced drivers highlight the quality of the cars we produce and show them in their best light.
There is nothing worse than seeing a high performance car driven badly.
And I think we all know that the best driver in the world will never take a car from the back of the pack to the front.
The days of bringing in "ringer" drivers from other formulas and dropping them in a FSAE car and expecting them to blow everyone away (in an otherwise average car) are well and truly past.
So back to my question... how could you quantitatively generate a measure or ranking of Enduro Driver Team Performance that is independent of Car Performance?
1: What data would you need?
Individual Endurance Lap times, Cones Hit, and Off Courses (both as a total per car, not per driver). Given that Fuel is not measured for individual drivers I wouldn't use the Fuel Numbers at all.
2: What data would I exclude?
I would only use data from teams that successfully complete the endurance event. Non-finishers with low lap counts can skew the data set quite a bit so I would keep them out.
How would I use this data to compare or rank the Driver Teams?
This is the interesting bit and I am interested in your feedback and thoughts.
My current idea is to:
1: Develop a number of different metrics to measure driver performance and consistency
2: Rank all finishing teams on each of these metrics
3: Add up the numerical rankings for each team
4: Re-rank all teams based on the sum of these categories (lowest being the best, highest the worst)
You could also non-dimensionalise each category, and/or have different scalings for each depending on perceived importance of the different metrics but I thought I would keep it pretty simple to start with.
My ideas for the different metrics are:
A: Total Penalties
So the sum of each team's total penalty seconds from Cones DOO and Off-Courses. That a pretty obvious one. Good drivers don't hit cones or run off course. This metric should be very independent or car performance. Even with bad ergo or fatigue setting in a good drivers should drive within their limits and that of the car and not hit cones or run off course. I have not included Other Penalties listed as I wasn't sure what they were (Driver change too long?).
B: Coefficient of Variation of All Lap Times
This is Standard Deviation divided by the Mean and reads as a percent. The most consistent teams get down as low as 2% which is impressive. As much as 20-30% at the other end of the spectrum.
C: Difference Between the Average lap times for each Driver
The assumption we make here is that if both your drivers return the same or very similar average lap times, then they are both driving to the limits of the car.
If these average times are different you have one good driver and one not quite so good.
Yes, it is possible to have two drivers that are equally bad (or average) but those teams will rate poorly elsewhere.
I played around with some other options (ie Difference First to Fastest Lap) but didn't think the data that came out added any more clarity or insight.
So once I had rankings for each team on these metrics I added the numerical rankings together (this assumes equal weight or importance between all metrics which is debatable), and then re-ranked them.
The final ordered list might be useful in providing some insight into the best prepared and performing driver teams and those that could gain a lot of points from further driver development.
It is also interesting to see where the biggest deltas are between Enduro event placings and the Driver Team Performance Rankings.
There are some teams that significantly outperformed and some that under performed according to the data.
Not having been at the event I will leave it to others and these teams to comment and decide if these indicators point to any truths, or are totally spurious!
Obviously there are a huge number of factors which will influence these rankings so take them all with a big grain of salt (weather, lap traffic, breakdowns etc).
Interested in you suggestions on how this analysis might be improved through the addition, subtraction or modification of the metrics I have proposed.
You can download my Excel results from the Monash Motorsport Alumni Facebook page here:
https://www.facebook.com/groups/1525.../?notif_t=like
Summary results from the recent Michagan comp should be viewable in the image attached.
Apologies if there are any mistakes in there.
Interested in everyone's thoughts
Scott
 |
|
|
Reply With Quote
Originally Posted by Jay Lawrence
