2015 FSAE Michigan

[Z]

Julian,

Thanks for the update. I guess there is an interesting story in there, somewhere.

I really wish there was more "inside" feedback from these big comps. No "secrets" or anything, just all the good gossipy stuff about who fell flat on their face, and why, and so on.

Also a lot more DJ feedback on what Design Event is supposed to be about, etc., would be really helpful. For example, some DJ opinions on why some Design Finalists failed to finish Enduro (maybe too "optimised"?), and how some non-DFs managed to finish on top Overall (because they...?), would be very educational!

Z

[JulianH]

Z,

I only judged in Austria last year. But still an important competition (probably the 2nd best field after Germany).

When we selected the Design Finalists, we could not assess if the car will finish Endurance or not.
How could you ever do that?
Yes of course, you see some "going to fail" designs on the suspension but I think about 90% of the DNFs of "Design Event teams" could not be spotted from the outside.
I think Graz had a fuel leak and Munich E had an electrical problem.

Therefore I think this discussion is worthless.

When it comes to "bad Design teams with a good performance", that's sometimes the fault of the teams.

I judged a Top 5 Overall team in Aerodynamics. Their main Aero guy was not at the competition and his replacement just did not know anything. So his score was really bad.
But that does not make the car any slower...

Additionally you can make fast cars without knowing a lot / without being well prepared for Design.
So don't blame the judges all the way...

Cheers
Julian

[bob.paasch]

Here's one of the biggest problems with FSAE/FS design judging: http://www.fsaeonline.com/content/FS...et%20150pt.pdf

Formula SAE Design Judging Scoresheet

Suspension 25
Frame/Body/Aero 25
Powertrain 25
Cockpit/Controls/Brakes/Safety 25
Systems Management/Integration 20
Manufacturability/Serviceability 15
Aesthetics/Style 5
Creativity 10

This points breakdown was perhaps appropriate 10-15 years ago, when Cornell was chewing up the track with their 95 HP turbo 600s, few top teams ran aero, and fuel economy (not efficiency) was worth 50 points. This point breakdown is the reason some design finalists don't do well in the dynamic events (design emphasis on things the judges like/score high, but don't matter on the track), and a reason some very fast cars don't do well in design.

It's ludicrous aero gets lumped in with frame, and is worth 0-10 points in current judging. If I redid this to reflect today's FSAE/FS competition:

Suspension 20
Frame/Body 15
Aero 40
Powertrain 10 (with a focus on reliability and efficiency, not power)
Cockpit/Controls/Brakes/Safety 15
Systems Management/Integration 40 (including project management, system engineering, physical testing, driver training)
Manufacturability/Serviceability 10
Aesthetics/Style 0 (can't be judged consistently. Good craftsmanship can be included above)
Creativity 0 (can't be judged consistently)

This competition is not about designing cars. The design goal is to engineer a system for scoring dynamic points at competition. In my humble opinion.

[JayNoon]

Julian,

Thanks for the update. I guess there is an interesting story in there, somewhere.

I really wish there was more "inside" feedback from these big comps. No "secrets" or anything, just all the good gossipy stuff about who fell flat on their face, and why, and so on.

Also a lot more DJ feedback on what Design Event is supposed to be about, etc., would be really helpful. For example, some DJ opinions on why some Design Finalists failed to finish Enduro (maybe too "optimised"?), and how some non-DFs managed to finish on top Overall (because they...?), would be very educational!

Z

If only they could televise the event or hold some sort of live web feed for endurance where people could watch the triumphs and failures of FSAE racing. I was in the dynamic area for the second half of endurance and kept track of the classically well performing teams. Here are my notes of the excitement:

Saginaw Valley - DNF - running quick until heavy smoke poured out.
Maryland - DNF- was smoking slightly at first but it worsened in the second stint.
San Jose State - DNF - Major smoke after one lap.
RIT - Car was limping the entire run. Unsure of the issue.
Akron - Car was limping very slowly from lap 1, apparently an engine map issue. Held up the top 5 teams during endurance several times.
Kansas - DNF - Control Arm Failure in second stint.
ETS - DNS - Starter issue.
Wisconsin - DNS - fuel issue.
UTA - Hit cones damaging front wing and DRS sytem, but finished well.
Cornell - DNF - Oil drip.

Again, this is not meant to put down any team, just provide a little more transparency for those who weren't there.

Cheers,

[Z]

Jay,

Ahh... just what I wanted. Thanks!

And congratulations on very good result!

Z

[Z]

Here's one of the biggest problems with FSAE/FS design judging: http://www.fsaeonline.com/content/FS...et%20150pt.pdf

Bob,

Having now finished reading all 8 pages of that Checklist, I agree that it is most likely the source of the Design Event problems.

This current approach to Design Scoring seems, to me, to be an incredibly narrow-minded, finicky, detail-focussed, only-look-at-the-small-picture, way of assessing a Team's Design effort!

There is not a single check-box item there that asks how the Team addressed the BIG-PICTURE questions inherent in this competition.

To spell this out, not even in the "Systems Management/Integration" section, seemingly about the BIG issues (?) but only worth 20/150 points, is there any focus on how the Team decided between, say, a high-power-turbo-car-with-NO-aero, and a low-power-mega-wing-car. Or any of the many other paths to "scoring maximum points". Or even if "maximum points" was, in fact, the Team's goal?
~o0o~

To explain this by way of analogy, way back in the olden-days when young Z actually worked as an Engineer, the Drawing Office hierarchy was something like this:

1. Chief Engineer - Meditates in a darkened room for ages, until ... he eventually emerges with a vague vision of the way forward. He then frantically produces many rough sketches of said vision, which are passed to...

2. Design Engineers - Draw more accurate sketches (eg. "FBD"s) of the CE's work and do many calculations to assess feasibility. Then either report back to CE that "It'll never work!!!", which sends CE back into his darkened room, or pass their more detailed sketches down to...

3. Design Draughtsmen - Turn the DE's sketches into accurate "Assembly" and "Sub-Assembly" drawings, which includes doing many routine calculations for things such as the required bolt-, bearing-, rod-end-, sizes, etc. Also give directions to next two...

4. Detail Draughtsmen - Produce accurate "Part" drawings, with properly written dimensions, tolerances, material specs, standard-notes, etc.

5. Tracer - Reproduces old "master" drawings that have passed through too many hands and are starting to fall apart. (Sexist note: This was usually the only female in an office full of hundreds of males!)

And there was also a Checker, usually a very experienced Draughtsman who took copies of "first-draft" drawings and covered them with red ink to indicated ERRORS!!! (after which they were sent back for correction, often many times...), and Document-Control who, if you asked very nicely, gave you working "blueprints" because the master-drawings were zealously held in a big fireproof safe.

Anyway, the Design Score-Sheet/Checklist looks very much to me as if it is assessing MOSTLY the work of numbers 3, 4, and 5 above. Yep, think about how many parts on this year's car are mere "Tracings" of previous work, yet are covered extensively in the Checklist.

There is VERY LITTLE assessment of a Design Engineers' work (#2 above).

I see absolutely NO ASSESSMENT of the sort of work that a Chief Engineer (#1) would do.
~o0o~

Bottom line, the Design Scoring process makes no attempt to assess the sort of HIGH-LEVEL Engineering skills that Univesities should be teaching, and these competitions should be testing.

Can this be good for the future of "Engineering"?

NO WAY!

Is there a better way to assess these high-level skills?

I think so, but I would like to hear other's views first.

Z

[JT A.]

Bob,

Having now finished reading all 8 pages of that Checklist, I agree that it is most likely the source of the Design Event problems.

This current approach to Design Scoring seems, to me, to be an incredibly narrow-minded, finicky, detail-focussed, only-look-at-the-small-picture, way of assessing a Team's Design effort!

There is not a single check-box item there that asks how the Team addressed the BIG-PICTURE questions inherent in this competition.

To spell this out, not even in the "Systems Management/Integration" section, seemingly about the BIG issues (?) but only worth 20/150 points, is there any focus on how the Team decided between, say, a high-power-turbo-car-with-NO-aero, and a low-power-mega-wing-car. Or any of the many other paths to "scoring maximum points". Or even if "maximum points" was, in fact, the Team's goal?
~o0o~

To explain this by way of analogy, way back in the olden-days when young Z actually worked as an Engineer, the Drawing Office hierarchy was something like this:

1. Chief Engineer - Meditates in a darkened room for ages, until ... he eventually emerges with a vague vision of the way forward. He then frantically produces many rough sketches of said vision, which are passed to...

2. Design Engineers - Draw more accurate sketches (eg. "FBD"s) of the CE's work and do many calculations to assess feasibility. Then either report back to CE that "It'll never work!!!", which sends CE back into his darkened room, or pass their more detailed sketches down to...

3. Design Draughtsmen - Turn the DE's sketches into accurate "Assembly" and "Sub-Assembly" drawings, which includes doing many routine calculations for things such as the required bolt-, bearing-, rod-end-, sizes, etc. Also give directions to next two...

4. Detail Draughtsmen - Produce accurate "Part" drawings, with properly written dimensions, tolerances, material specs, standard-notes, etc.

5. Tracer - Reproduces old "master" drawings that have passed through too many hands and are starting to fall apart. (Sexist note: This was usually the only female in an office full of hundreds of males!)

And there was also a Checker, usually a very experienced Draughtsman who took copies of "first-draft" drawings and covered them with red ink to indicated ERRORS!!! (after which they were sent back for correction, often many times...), and Document-Control who, if you asked very nicely, gave you working "blueprints" because the master-drawings were zealously held in a big fireproof safe.

Anyway, the Design Score-Sheet/Checklist looks very much to me as if it is assessing MOSTLY the work of numbers 3, 4, and 5 above. Yep, think about how many parts on this year's car are mere "Tracings" of previous work, yet are covered extensively in the Checklist.

There is VERY LITTLE assessment of a Design Engineers' work (#2 above).

I see absolutely NO ASSESSMENT of the sort of work that a Chief Engineer (#1) would do.
~o0o~

Bottom line, the Design Scoring process makes no attempt to assess the sort of HIGH-LEVEL Engineering skills that Univesities should be teaching, and these competitions should be testing.

Can this be good for the future of "Engineering"?

NO WAY!

Is there a better way to assess these high-level skills?

I think so, but I would like to hear other's views first.

Z

Z, I think the best way to evaluate the #1 & #2's work is objectively with a stopwatch rather than subjectively with a design judge. But it is possible to improve the design event (less subjectivity, more focus on the big picture design).

But currently there is a problem with relying on the stopwatch to evaluate design choices. As someone mentioned earlier, it is possible to build a really fast car without a good understanding of why it is fast. This is because the rules have been stagnant for way too long! The competition needs a shake up.

I think the rules committee has tried to shake things up by changing the aero rules every couple years, but it isn't making students re-think any of their concepts. Regardless of what the aero rules are, the winning philosophy is "shove as much wing into whatever sized box they give us". If that box is 1ft x 1ft, cram it full of wing. If the box is 3ft x 3ft, cram it full of wing. If there is an area where they didn't specify a box size, guess what - cram it full of wing. It isn't making students think. They can keep making the box more and more restrictive all they want, but it doesn't change physics - downforce is still good. And it takes freedom & creativity away from the students in design, which was one of the best things about this competition is how open the rules are compared to any professional racing series.

So instead of trying to shake up the competition by being more & more restrictive with the design of the car (which only makes the students think even less), shake up the competition by changing the events of the competition. And then change them the next year, and the next year. The team that knows how to engineer a system for a given set of goals (as Dr. Paasch correctly stated should be the intent of the competition) will be successful year after year, the teams that only know how to build 1 type of car won't.

Some example ideas:

-Could be as simple as drastically redistributing the point values for each event from year to year.

-Skidpad is driven only 1 direction instead of a figure 8. And you won't be told which direction until you enter the dynamic area. You have 5 minutes after entering to adjust your setup and then you must enter the line for the event. This makes students design for quick & easy adjustment and expand their VD understanding to include asymmetric setups.

-Mandatory engine removal/installation at some point in the competition. This one would be a logistics struggle to implement fairly (ie if you do it between autocross and endurance, the teams with a late endurance start order have an advantage). But the idea is to reward teams that actually design for serviceability.

-Acceleration is now 1.5x or 2x as long (then the next year, it's half as long). As always, no sprocket ratio changes are allowed during competition. So how does a team try to maximize overall points between accel & autocross/endurance.

-Your cost report has to be under $12000 to be allowed to compete. If the cost judging team finds discrepancies that put you over $12000 you must remove features from your car, or face harsh point deductions.

-Skidpad diameter changes from 25ft, to 40 ft, to 55ft on a 3 year rotating cycle.

-One year, the value of fuel efficiency event is doubled. The next year, there are no points for fuel efficiency but you are only given 1 gallon of fuel to run endurance.

-Skidpad, acceleration, and autocross are eliminated and instead a day is allowed for open testing on the endurance course. Who can make the best use of data to optimize their car for the track?

-Introduce a 400lb minimum weight rule. Do teams with 350lb cars keep building the same car and just add ballast? Do they swap to a bigger engine? Or do they distribute that extra weight into increasing durability/rigidity of critical components? Do teams with 450 lb cars keep building the same car or change their concept? Then of course, the next year go back to no minimum weight.

The important thing is to keep students on their toes. No legacy cars, no copy & paste designs. Now I only mentioned ways to shake up the dynamic events, so how does this improve the quality of the design event? Well, I think a by-product of the competition's format being stagnant for so long is that the judges have also gotten somewhat stagnant in the questions they ask & what they expect to see. A shake up is good for them too! The competition is now completely different from when they were in school, and the ideal vehicle concept that they knew from their era is no longer the case. This sparks big-picture design discussions with the students and less focus on what Z refers to as the #3, #4, and #5 level details.

[JT A.]

And Dr Paasch, I agree with you that the points breakdown for design is a bit outdated and doesn't reflect the current state of the competition. But ultimately it doesn't matter much because I don't think they follow it anyways. I've heard from a few different judges/event officials that they more or less get together over some beers, sort out the field into "Group A", "Group B", "Group C" etc...and then go back later and figure out how to fudge the points to make the teams fall into the correct category. So the first step in the right direction would be to get them to follow any kind of of formalized scoring system. Then worry about whether it's proportioned correctly.

[Westly]

Suspension 20
Frame/Body 15
Aero 40
Powertrain 10 (with a focus on reliability and efficiency, not power)
Cockpit/Controls/Brakes/Safety 15
Systems Management/Integration 40 (including project management, system engineering, physical testing, driver training)
Manufacturability/Serviceability 10
Aesthetics/Style 0 (can't be judged consistently. Good craftsmanship can be included above)
Creativity 0 (can't be judged consistently)

This competition is not about designing cars. The design goal is to engineer a system for scoring dynamic points at competition. In my humble opinion.

While I agree your weighting is a much more accurate representation of what is important in FSAE for many teams, i think the teams should be the ones to specifiy how much weight they place on each of those catagories and then justify why. I believe it would be better to give the Project Managers/Technical Directors a list, such as above, but with blank boxes and for them to assign their own weighting and then explain through their reasoning.
I believe there is very little within the current structure currently to assess these trade offs a team has to make.

For example: A smaller team may better direct their resources to building a reliable/well tested vehicle which is completed early to allow drivers testing time. I think its fair for the team to say- yes we believe aero is impoartant but with the resources available we believed this was a better solution to achieve success at competition - if they have justification to support this. This team in my opinion should not loses a proportion of points purely bacause aero is mandated to recieve a X% weighting in the design event.
If the team also had an Electric Throttle / Carbon wheels, etc then I would argue they had not followed their own brief and should lose points though.

Regards,

[Kevin Hayward]

JT A,

I'm not sure I agree that the rules have been stagnant. They have become larger, more complex, and more prescriptive since I was a student. There also have been small but significant changes to weightings that have seen the ideal concept move a fair bit.

I would also propose that without some stability in the rules (as well as freedom) there is little opportunity for real innovation. To do so takes a lot of time if you want innovative solutions that are performance improving. It is a common mistake to assume that improvement happens gradually at all times. Unexpected rapid changes is what we should be expecting. Spaceframes for an eternity, and then all of a sudden lots of carbon tubs. 13" wheels as standard, then everyone on 10"s. Almost no aero, then nearly all the top teams (yes a rule change helped this, but current regs allow about as much downforce as what they were pre big change). 600cc 4s then singles. The list goes on, but the pattern is the same. The difficult part is trying to predict the next big change, and be ahead of the curve.

If there is stagnation it is, and always has been, in the teams. Hopefully no-one enlightened enough on these forums think that GFR has produced the ultimate incarnation of a FSAE vehicle in the last few years. There are so many good ideas running around some of the lesser known teams that with a bit of development could turn into some great performance. At the moment I think we are on the verge of entering a new style of car build in the next few years, one that will be a little easier for smaller teams to complete when compared to the mini F3 cars we have had at the top since about 2006. I guess time will tell though.

I am always curious as to which team is currently on a five year plan that will produce the new "ultimate" FSAE car. We don't need F1 style shotgun style rule changes derailing these efforts.

Kev