FSAE World Ranking: Who is Number 1 worldwide?

[TMichaels]

I think this is a good discussion going on here!
Don't you think that teams on a two year cycle would feel that they are treated unfair, if we for example say that a 0 points event will be recorded, if they did not compete within a season (season as defined by our current formula)?

@Kev:
What defines a World Champion in FS/FSAE? Certainly not the fact that the Fisita World Championship was rewarded. I know that the teams that won it claim to be World Champions and it is their right as they won this award. However, all of us know: It is just an award and must not necessarily reflect the real world. It has nevertheless always been awarded to very strong teams.

Regarding the comp factor:
This basically results from isolated events and can hardly be changed. Japan and Australasia are quite isolated with respect to teams taking part in other events internationally. Of course they are facing higher borders as they are also geographically isolated.
I also compared the average points spread of FSAE-A(180), FSAE-MI(69), FSG(90) and FSUK(103) from 2011 to 2009 (as there is no data for the FSAE-A2012 event yet) between the 1st and 3rd placed team. As you can see, this is quite different.
Now what does that tell us? Either the winner of FSAE_A is usually way over the top or the rest of the field is not as competitive as at the other events.
In other words, if more teams at FSAE-A score more points then the entire competition will climb up with respect to the competitiveness factor. That seems right to me as more teams scoring more points results in a more competitive event.

[Kevin Hayward]

Tobias,

I would suggest that a world champion is either a team that one Detroit prior to the rise of the European and FSAE West comps. At this time it was well accepted that to win there was to be the best in that year. From then on any team that has taken number one in the world rankings should be considered the current world champion.

It is under this basis that I would say there have been 3 Australian world champions (maybe a poor choice of words). Certainly not on the basis of winning the Fisita Award.

As for the competitiveness factor I don't think that Australia should be at 0.96, but it should probably still be higher than it is. I think it is a bit crazy to think that the 10th place team in Australia will have an effect on competitiveness. The gap being large supports this notion. At any given Australian comps there are maybe 5-6 teams in it for the win. Out of these teams it is difficult for an international team to come in and score high. Stuttgart's win in 2008 was probably a low point for Australian team competitiveness. UWA was on a decline, Monash was starting its rise, and previously very competitive teams such as Woolongong and RMIT were in a funk.

But by the time you get to the 10th team in Australia you are looking at teams that didn't finish endurance (for most comps) I would suggest for smaller comps that the average of the comps top 5 or 6 cars should determine the competitiveness. This might even extend to the larger comps. Only the top teams in each event will have an impact on how many points are on offer. Alternatively maybe the competitiveness factor should be based on the top x% of teams. Compare say the average of the top 25% of teams in the comp versus the average of the current world top 10.

An interesting question would be how far do you have to go down a list before you find that the teams no longer affect dynamic event results for other teams?

The competitiveness factor is a difficult one to manage given the isolated competitions, and is largely not an issue I worry about too much. It is better to have a world ranking that may cause a few minor arguments than not having one at all.

Kev

[TMichaels]

Kev,
ok, sorry, I misunderstood your comment regarding the World Championship. I don't think that we really need that term. Even if we would use the WRL to determine this, at which point will it be valid? Given that the season plans/schedule of teams is different based on what they determine to be their first event, we are not able to set a hard limit and thus we would have a new World Champion after each event.

I agree that maybe only 5-6 teams are able to set Tmin for a dynamic event, but we are currently only talking about dynamics. For Presentation or Cost basically every team is in for the win and may therefore set the standard. So they will influence the overall points to be scored. It may be of minor influence, but it has an effect.

Regarding taking only a percentage into account:
I think this becomes difficult. Think of an event were maybe only 3 teams determine the competitiveness, because only 15 teams take part. Now all these three teams fail to finish Endurance, which is not impossible. Then you could get an artifact where a comparable weak team gains many points which will not be corrected for the WRL, because C=1. The chance of that happening decreases with the number of teams taken into account to calculate competitiveness.

That makes me think whether the C-factor should be corrected by only taking teams into account that finish Endurance. Hmmm, difficult...on average it should not make a difference, it will just scale down.

I still like the idea of assigning 0 points for a season in which a team did not participate in any event. It will solve the biggest problem.

[Kevin Hayward]

Tobias,

I deliberately left out the cost and presentation events as you cannot be sure you catch the influential teams in any comp just taking the top 10.

I completely agree with the numbers getting too small for the small comps with a percentage basis. Assuming a 30% finish rate for all teams the likelihood of the top x teams all failing to finish is:

x - %
1 - 70%
2 - 49%
3 - 34%
4 - 24%
5 - 17%
6 - 12%
7 - 8%
8 - 6%
9 - 4%
10 - 3%

It is a bit of a simplification as the better ranked teams are partly better ranked due to reliability. I would suggest that the top ranked teams have a finishing rate far in excess of 50%. At 50% reliability you have less than a 2% chance that all of the top 6 teams will not finish. Furthermore a top team that fails to finish usually fails in endurance and will still have a marked effect on the point scores for the other dynamic events. This would support the idea that for small comps less than 10 teams could be considered while still providing a reasonably accurate C scale.

One of the best indicators for the balance for the competitiveness of Australia in the last few years has been Monash. Looking at the last international comp with the same car

Oz 2011 - 864 points (C = 0.87) P*C = 752
UK 2012 - 841 points (C = 0.95) P*C = 799
Germany 2012 - 841 points (C = 0.98) P*C = 824

I'm all behind the idea that a place at Germany is likely to be harder than a win in Australia, but 70 points seems pretty extreme. We see what happens to the regular Japanese visitors to Australia, yet the Japanese comp has a high C value and the Australian one has a low one.

Maybe it should be something like:

Number of teams considered = max(25% comp, 10)

That way the Oz (and small comps) C factor stays the same, but other bigger comps will reduce a little. Larger comps still have an advantage.

Alternatively maybe the C value should be much simpler and based on numbers alone. A comp of over 100 teams has a C of 1. Comps of 50-100 have 0.975. Comps with less than 50 have 0.95 Comps with less than 20 have 0.9. Or something similar.

I definitely back a zero points for a non-competed season. While it is harsh in a few cases I think the world ranking should favour regular competitors. What seems a little odd is that we looked at the world rankings for ECU this year. Our best strategy to improve the teams world ranking is not to compete this year (or next). Given that the team has had continuous improvements in score the WRP will rise due to non-competition. The team would move into the top 20 by the start of 2014.

I think everyone would agree that no team should be able to improve their world ranking by not competing.

Kev

[Bemo]

The Japanese competition is definitely rated too high at the moment due to its isolation.

The problem with lists like yours is (in my opinion) that it is hard to tell how much Monash improved their car from Australia 11 to FSG 12. There was quite some time for testing and uprading the car.

Unfortunately at the moment the Australian comp isn't very competitive. Monash seems to be the only team able to reach a Top10 place at the major events. UWA, RMIT and Wollongong are for whatever reason way behind their past performances.

[TMichaels]

We are looking into the isolation problem for a long time now, but have not figured out a fair way to address it.

The perfect solution would be to have the Top10 finishers of each event attend one single event and then you would have a) a real world championship and b) a good way to determine competitiveness of the events.
Sadly this will probably not happen in the near future.

Something which I tend to forget is that competitiveness discussion is often based on feelings rather than numbers. Maybe, because it is hard to acquire numbers which can be trusted.

I am truly against basing the comp-factor on number of teams. Otherwise an Indian, Chinese or Brazilian event will probably be ranked way outside their league. No offense , it is just a fact. It would also be prone to many artefacts, such as a 50 teams event with the entire Top50 of the WRL getting the same comp factor as a 50 teams event with the Last50 of the WRL. That does not work in my opinion.

I fully agree that a team should not be rewarded for not competing, as it currently is the case.
Daniel (who is putting all of the work into collecting the data and implementing the calculations etc.) and I are currently discussing to implement the 0 score to be able to see how it practically affects the scores.
Daniel also acted on a feature request of mine and the result of the respective event will be shown in the future next to the WRL. That means that all results since the start of FSAE will be available for everyone in a single place.

I know that judgement should not be based on single events, but Swinburne reached the second place at FSAE-A2010 and came in 4th at FSAE-JP2011 with the same car/team as far as I know. Thus it seems to be right that FSAE-JP has a higher comp factor than FSAE-A for example.

[TMichaels]

Daniel just told/showed me that the event results are now online as well. They are displayed on the right side of the WRL. Please note the Mouse-Over function which shows you the scores and whether the team is electric or combustion.

[JulianH]

It's really difficult to define "very good" at Formula Student.

Different teams have different approaches. GFR is running to win. Basically, they have won a lot of events when they finished in a row (FSA 2012, Michigan 2012, FSG 2011, FSA 2011, Michigan 2011...)

Ann Arbor for example didn't win big competitions in the last years but always scored high.


I think it is impossible to say, which team is "better".

On the one hand, the GFR combustion car (and also their static performance!) is one of the best AutoX/Endurance car of all time but it fails also quiet often. Ann Arbor is always fast but never "the" fastest car...


Another point to discuss is the difference between pure Electric / pure combustion events and mixed events.
With an eCar you lose in Endurance and in Cost against Combustion cars, but they lose at Accel and probably AutoX.

This should be weighted in the scoring.


Another point about the competitiveness (especially for the eCars).

In Spain 2012, we had Karlsruhe, Stuttgart and Zurich, 3 of the 5 fastest eCars of the year. And still the event got only 0.85 competitive ness because the 10th placed team was Hannover with 142 points!

Italy, that only got Freiberg as one of the Top 7 of FSG (which is basically the Electric World Championship...) had the same scoring. (Of course, Zwickau should be placed way higher but they had a rough German event...).

I think for the small "Top-List" of eCars, it would be better only to look at the Top5 cars or so.


But all in all, I don't think it is possible to assign a World Ranking for Formula Student. With this said, I don't think that it is a good idea to promote these times with early registrations slots or so.


Regards,

Julian

[TMichaels]

Julian,
it is a different discussion whether the WRL reflects the actual performance. It is, as said before in this thread by me, a ranking by definition. As is the ATP ranking list or the Formula1 Championship. Just tweak the point distribution a bit in F1 and you will have different World Champions. Thus the F1 World Champion does also not say, who is the best driver or team. It only tells you who won the World Championship by the definition of the FIA.

This is the same that we do. We create a ranking by definition.

Now we are discussing how to make this ranking more accurate, though we all know that it can never be perfect/suit everyones preferences.

[JulianH]

Tobias,

I think we all agree on this point, that it should be more accurate and so on. My point simply was, that in contrast to Tennis, you can't point the finger on the good and on the bad player/team.

In my example, GFR would kinda represent Rafael Nadal (won a looot of major tournaments but is out for injuries for a long time). In the ranking he is still ranked Fourth or so without a "running car/running legs".

Therefore I would argue that a "one time DNF" shouldn't be weighted that extreme.


Additionally, my input was to acknowledge that the Top Class for the Electrics is way smaller than in the Combustion Class.

We had 16 Combustion cars in a reach of 5 seconds at FSG in Autocross and only 4 in the Electric class.

Therefore a change in the "number of teams to decide how competitive the event was" would be a good starting point.