2014 FSAE-Australasia

[Pat Clarke]

Geoff, be fair!

The UWA car got a whole lot more than 3 points from the judges initially.
However, their car was far from complete and so suffered a significant point loss from that.

Then they were penalised for late/non submission of paperwork.

I admit that 3 points, when viewed in isolation looks bad, but there was more to the situation than has been expressed here in several posts over the last couple of years!

When a team bring an unfinished car to the event and have failed to meet the document requirements, the result will be a low score, regardless of how 'innovative' the car may be. UWA in Australia in 2011 is not the first time we have seen that happen.

Cheers

Pat

[Z]

I'd be keen to see the Design Event occupy a higher proportion of overall score.

GTS,

As Pete and Geoff have hinted at, the last time that happened the car with undoubtedly the most "innovative" chassis/suspension design in the history of FS/FSAE got 3 points out of 200! IMO that single act of Design Judging did more to discourage young Engineers from innovating than any amount of test-track failures.

If you truly want to help produce a better next generation of Engineers, then you must give them a better Education. Simply "incentivising" them with hollow promises of more Design points, when they all know they may well end up with 3/200 (or 1/200?), will only have them doing the same-old same-old, albeit out of carbonfibre-skinned titanium-honeywhatsit.

Education comes from explaining the millions of little details of what works really well, what less so, and what is a guaranteed disaster. All these little details must be repeated over and over again. "Theory" and "practice" must be shown to be constantly affirming each other.

For example, "Now look at this broken wheel-centre for a beautiful practical example of all this fatigue failure theory. See how the stresses from these spokes have...".

Only when good Education is in place will good Innovation follow.

So, once again...

"... maybe the DJs can provide a short written review of all the cars here? Please?
Or maybe just a list of outstandingly good features on some given cars, and another list of shocking design errors ..."

Z

(PS: (Pat just posted.) So, Pat, why have the DJs never made any of that public? How many DE points did UWA-12 actually score, before all those penalties were applied? Do YOU consider their design a good one, and worthy of praise? Would you, or do you, encourage other students to go down that route?)

[GTS]

Geoff;

How about starting with teams needing to present documentation and drawings for last year's car in addition to the current, with evidence of a proper document control system that dates both correctly? (Or just maybe an end-of-year drawing submission under an NDA etc) to be used to judge contribution against the next?

The whole 'did you run last year's car' issue is not rigorously implemented. I agree that we need to incentivize innovation, to do so we need a presentable baseline. I don't want to see a token "Bosch Award" or similar - I want to see points!

Teams of 60+ running broadly similar cars is... a bit ridiculous and unfairly disadvantages those who try something genuinely new with sound engineering reason. It's also contrary to the spirit of the competition. Have mega teams if the resources can be corralled, sure, but have a mega contribution to show for it. The cry (for those that heard it) as the cost event was called out from the winning team of "so what did we win now?" was unsportsmanlike, arrogant, and most importantly (and relative to what the competition's about) totally missed the point.

This recent notion of weighting dynamic events to a greater degree at the expense of design points is a bit silly unless we're talking teams that can reasonably afford to compete with a consistent effort at more than one competition in a calendar year with regularity. F1 teams get 18-20 races to strut their stuff and still blow it a few times a year, the probability of it happening to a FSAE team is considerably higher.

[GTS]

GTS,

As Pete and Geoff have hinted at, the last time that happened the car with undoubtedly the most "innovative" chassis/suspension design in the history of FS/FSAE got 3 points out of 200! IMO that single act of Design Judging did more to discourage young Engineers from innovating than any amount of test-track failures.

If you truly want to help produce a better next generation of Engineers, then you must give them a better Education. Simply "incentivising" them with hollow promises of more Design points, when they all know they may well end up with 3/200 (or 1/200?), will only have them doing the same-old same-old, albeit out of carbonfibre-skinned titanium-honeywhatsit.

Education comes from explaining the millions of little details of what works really well, what less so, and what is a guaranteed disaster. All these little details must be repeated over and over again. "Theory" and "practice" must be shown constantly affirming each other.

For example, "Now look at this broken wheel-centre for a beautiful practical example of all this fatigue failure theory. See how the stresses from these spokes have...".

Only when good Education is in place will good Innovation follow.

I think PC has beaten me to it, however the three points was genuinely on a number of other issues also. The last beam UWA car I judged was poorly executed despite a good concept... and the student representing the relevant area shared highest marks at event. The particular student - had either of the aero judges had a job available - would have earned an interview then and there. There are 8 areas of design and the criteria are known.

It is very difficult to judge any magnitude of innovation without a baseline. The rules are specific about carryover, but it's very hard to enforce unless sufficient documentation or equivalent measure is employed to substantiate a baseline. It is without question an area lacking in the current rule set.

Formula SAE does not provide baseline education, Z. As a former university educator I do share your concerns, and there's much that can be done for it, though the competition doesn't reach into university and dictate policy (as much as we might like it to). Education is neither anecdotal nor trial and error. We can provide lists of what worked and didn't within a limited context as observed one weekend a year, under specific conditions and as applied on specific vehicles. A university I was involved with 04-08 + '10 developed a culture of priding itself on working on what worked, putting aside what didn't. First principles didn't make the grade. The resulting culture turned the FSAE team into a broadly trade experience. A meeting is scheduled next week at that particular university to discuss this very issue, which has reached a crisis point.

Education shouldn't reward good features, it should reward good thinking. FSAE is a an avenue to put it in context, which is a long and complex road full of noise, pitfalls, reality checks, compromises, learning and experiences therin. If it doesn't come back to good thinking, there's no loop to close with why a university should support or involve itself in the competition. Particularly given the significant opportunity cost of investing in FSAE against other project-based learning initiatives. It costs a (generally) six-figure sum of money to run FSAE at any given university, and most university departments have about as much in capital expenditure allocation for an entire year - not just the 20 or so students at the core of the competition. Some of the chats at competition with faculty simply - and very seriously - concerned whether or not the competition would feature in their plans in ten years. Five years. Even two years.

We cannot simply reward and compare outcomes. The challenge is considerably bigger, and the competition is designed intentionally such that "million of little details" cannot be run through. The breadth and depth of the scope is intentionally significant to ensure that smart decisions need to be made early on, and executed practically with good technical rigour.

I've no doubt that research for good information shouldn't include assessment of the best in field - there's no reason to go Marco Polo on everything - however this needs to be weighted against a need to be critical about it. The discussion I had with a faculty advisor - concerning why his team absorbed another university's design wholesale in 2013 with little critical scientific rigour beyond it having the most points on the day the year prior - should never happen.

It is too easy to circumvent by incrementally iterating last year's car. This is probably a good and realizable place to start.

So, once again...

"... maybe the DJs can provide a short written review of all the cars here? Please?

Or maybe just a list of outstandingly good features on some given cars, and another list of shocking design errors ..."

Z

For those not having the head judge's details, I would be happy to forwards any messages (just PM me) supporting a motion towards complete transparency of the Design Event findings (I really don't understand why it's not done for all static events).

The head judge for this year's comp is considering organising an online feedback session (this unofficial medium seems only to attract me of the 16 judges...) what we can do to enhance and moderate the robustness of the feedback process is probably the subject of a separate thread, which should certainly be put together.

[Big Bird]

I think I was being fair. My whole argument was to do with the fact that the competition does not reward innovation, or at least not in a way that provides any real benefit for risk taken. I wasn’t pointing blame at anyone. I was simply calling a fact that a genuinely creative and interesting interpretation of the rules, albeit unfinished and with late paperwork, scored only three points, while I have seen some – dare I say – bloody awful cars over the years, scoring much much higher than that. Yes, the judges might have being doing everything to the letter of the law. But an instance like the UWA instance sends off alarm bells in my head that the letter of the law might not be achieving all that we want either.

Now it actually says in the judges marking sheet that “strictly speaking, innovation is extremely rare in FSAE”, which is then followed up by various marking guidelines to give a score out of 10. So, at most, a truly creative design, perfectly understood, gets you 10 points out of 1000. And what does a garden variety cookie cutter car get? 2?? 5?? So you might pull up to, say 8 points on your most pedestrian rivals and less than that on your more top-end rivals. To get those few points, you risk having maybe a new concept that needs a lot of validation, testing, etc. It seems barely worthwhile. When you then consider what you are exposing yourself to in terms of potential losses, it becomes a lot easier just to rebuild last years car –driver training will save a few points from cone hits, we will get our reports in on time since they are all the same as last year’s ones…

Now I of all people am probably most responsible for raising awareness of the points economy in this comp. And at times it makes my stomach turn when I hear teams cop out of stuff because of low reward points wise. But if we want more innovation and creativity, then we simply need more points for innovation and creativity…

[luxsosis]

Pat/GTS/others, having been involved in both the 2011 and 2012 UWA teams (which presented the "Wheel Pods" and the "Aerobeam"), it's not just the official scores and results that Pete is talking about. Being penalised for an unfinished car and late paperwork is fair enough (although in 2013 three teams didn't compete in dynamics and the lowest design score was 90).

The issue in my mind is that the reception and attitude we received from some circles was extremely unpleasant, unwelcoming and unappreciative of the effort and innovation that was in the concepts. Some people we talked to were very interested, and basically all the students I talked to were keen to learn more and enjoyed seeing how the 2012 suspension worked. Others appeared to approach it with pre-conceived thoughts which made it difficult to discuss what we did, why we did it and how we went about it. After our cars, which were a result of a genuine attempt to innovate, came close to being called outright legal and malicious attempts to circumvent the rules, it suddenly became a whole lot less fun.

Geoff said it right, if you want to do something completely different it can turn out amazing, but be prepared to sacrifice everything to get there.

[GTS]

Pat/GTS/others, having been involved in both the 2011 and 2012 UWA teams (which presented the "Wheel Pods" and the "Aerobeam"), it's not just the official scores and results that Pete is talking about. Being penalised for an unfinished car and late paperwork is fair enough (although in 2013 three teams didn't compete in dynamics and the lowest design score was 90).

The issue in my mind is that the reception and attitude we received from some circles was extremely unpleasant, unwelcoming and unappreciative of the effort and innovation that was in the concepts. Some people we talked to were very interested, and basically all the students I talked to were keen to learn more and enjoyed seeing how the 2012 suspension worked. Others appeared to approach it with pre-conceived thoughts which made it difficult to discuss what we did, why we did it and how we went about it. After our cars, which were a result of a genuine attempt to innovate, came close to being called outright legal and malicious attempts to circumvent the rules, it suddenly became a whole lot less fun.

Geoff said it right, if you want to do something completely different it can turn out amazing, but be prepared to sacrifice everything to get there.

If this is true, this must change. Time for a reflection paper, Geoff?

I'd add that the 2013 scores were rescaled (long discussion). The lowest raw would have been 77, on the 2014 150-point scale this would be 57. No late papers this year.

[luxsosis]

If this is true, this must change. Time for a reflection paper, Geoff?

For what it's worth, although I'm no longer directly involved, from what I see talking to people involved in the latest competitions and viewing all the feeds/updates etc, things do seem to be improving. That we have yourself GTS here now providing feedback is a sign of that!

[MCoach]

I have finals tomorrow, but I think we have sparked a new debate (or struck on old wound...) and have quite a bit to contribute as to the posts from the last 24 hours or so as soon as my dedication to classes blows over.

I'll drop some words soon on my thoughts about this on Saturday...Having stared the chopping block in the face for our entire SAE program for several years now, I can tell you quite about innovation and the complications of keeping a low budget program alive and how that's changed some of the way we do things around our parts. Part of the topics brought up that may be more interesting to explore are the teams converging on each others designs as visual reach has stretched globally making it really easy to say "huh, that's a cool wing, it was part of this year's winning car. I should make my wing look just like that because that's what the winner did." and converging on their own solutions, developing a competitive, but very repetitive formula.

Innovation keeps engineering alive. If it weren't for the creativity and shear genius (madness???) of the human brain, then we'd all have been replaced with computers decades ago.

That's it for now...

[Kevin Hayward]

I think one of the keys to understanding the design event is to move past the idea of it being an event that rewards the best designed car at the competition. While that might be what we want it to be the reality is that it is the best presentation of the design process that wins.

2014 Oz made that clearer to us than in any other competition. Compared to the eventual winners we arrived with a car that was nearly 20kg lighter (with a larger engine), more powerful, with some truly unique designs, excellent build quality, lower cost (by the cost report), was as fast, more reliable, and put together by a much smaller team (i.e simpler design). We were surprised not to win design.

However Monash clearly trumped us in design presentation. They were professional, and had plenty of people ready to speak to the judges. They had prepared for what the event was rather than what they wanted it to be, and we have learnt a valuable lesson from the experience. Monash knew that it was one of their alternate years (i.e. low development year on a 2 year cycle) and had obviously burnt the candle at both ends preparing a good presentation.

Innovative design has never been well recieved in Australia. Some of the best conceptual approaches in the country did not recieve good marks early on. Early Monash aero cars, RMIT 2003, UWA 2003, RMIT 2004, UWA 2004, Monash 2011, UWA 2012, UQ 2014 (sorry for the missing years and cars). Please note that in 2003 UWA did win design although by the marks rubric it was placed in 6th or 7th before direct intervention by a couple of judges causing what should have been an obvious decision in the first place).

I don't think this is a good situation. I would rate innovation much higher and rely a lot less on a rubric. Many truly great engineering products would fail miserably on a rubric against their competitors. However it is difficult to deny there is a benefit in assessing the students ability to present their design process, insight, and ability to field tricky questions. The event currently targets this rather than the actual design of the vehicle. On that basis the placings were accurate.

Kev