As this topic seems to be overtaking a number of other threads I thought it would be best give it its own one.
So how would you improve the design, and other static, events?
Lets try and keep it civil, we are apparently grown ups.
Go....
Printable View
As this topic seems to be overtaking a number of other threads I thought it would be best give it its own one.
So how would you improve the design, and other static, events?
Lets try and keep it civil, we are apparently grown ups.
Go....
My 2c in no particular order. Can explain rationale if anyone's interested.
DE scores released with breakdown and scores to teams at the same time.
Run the teams through DE in sequence.
Provide a means for non-attending members contribute
Drop the arbitrary scaling (whether to max points or otherwise).
Require a change itinerary from last year's car.
An amended form for the DJ's (make commenting and transcribing easier, mail merge the car/pit/uni details - save time).
A feedback session (preferably team-to-DJ's rather than 'town hall).
Let the students edit their video submissions, and incorporate them in the static score (these need to be taken more seriously at both ends).
Provide a disclaimer at events that teams+faculty can sign to have footage, results, scores and comments released under an open license.
(On that) Beg the top three in any static event category to share their work.
(Wishing here, however) Record design event (at least) and produce a "best of".
(I'm going to be super contentious here) Consider adding other criteria for, or from FSAE-A in DE (the competition needs to get relevant to local industry).
Consider a code review for the EV teams (super industry relevant, has other competition implications).
(I'm going to be ultra mega contentious here) Allow static-only entrants.
Couldn't agree more, but could you please define what you mean by 'local' and 'industry'Quote:
Originally Posted by GTS
So far the focus in Australia has been the Victorian Automotive Industry. I would suggest that only a small minority of graduates end up there. I am still very confused as to why EA have not been included in the Australian event. They should be partners to encourage better outcomes for students. That way we can then see judges with backgrounds in mining, processing, heavy vehicles etc.
Having to justify some of these vehicles from first principles to people in the industries that students will get jobs would be a great experience. I don't think we need every judge to have a background in vehicle dynamics to be valuable.
Kev
GTS,
I also agree with most of your points. Some comments:
The videos should be edited. I know our students ended up putting a massive amount of time trying to get a perfect take. Much more time than would have been needed with editing for a worse end result. It was supposed to be a time efficient replacement for a design introduction. The restrictions make it very difficult to do, and the effort is not focused on delivering an introduction, but instead on the mechanics of trying to do a one take shot. However opening this up would end up with yet another big production / report. I say get rid of it altogether and assume the design report is the introduction to the design. (FSAE-A problem only)
FSAE-A needs formal design feedback delivered timely. There is currently none. A one person design review at trophy time would be a good start.
I wouldn't beg the top three to share. I would mandate sharing of the top three submissions for each static event. Ultimately we want to lift the bar everywhere. Protecting competitive advantages runs counter to the educational obvjectives. This can be done in Australia very easily. I have heard the complaint that this will disadvantage us compared to the world, but I think that is a weak argument to protect a couple of Oz teams at the detriment of the event goals.
I would also like to see means for contribution of non-attending members. In Australia with 4 judging teams each after around 6 students we would have needed to bring 24 people (assuming that some do not cover multiple areas). We had about a third to a half of that ready for the design event and suffered as a regard. We had no choice as we cannot force students to cough up the $1500 required to attend, as well as being from a small university. For example we had a junior member who was involved with testing the single (having done some great work), having to answer questions on a system he had nothing to do with, because the judges didn't want to wait for the system designer to finish his current discussion. I always liked the idea of a design finals for this reason. Choose your best small group and go up against the rest. Otherwise the big local teams get a big advantage.
Love the idea of a change itinerary or something similar. Teams are skirting very close to the minimal allowable changes and not suffering any consequences in design as a result. Photos of main systems such as chassis, uprights, wheels, intake, exhaust, engine, suspension etc that are kept each year (electronically) and compared to the new would be pretty easy to accomplish. Most teams take these sorts of photos every year with just a couple of juniors running around.
Kev
For such a contentious topic there has been little activity here.
I think in 10 years there have only been 1 or 2 grads that have ended up in the Victorian Automotive Industry, with only maybe a doz that could be classed as working in the automotive field at all.Quote:
Originally Posted by Kevin Hayward
GTS in reply to your comments on the UWA apology thread and design feedback - I don't think every team would want to question every design judge, well some would but most would only be after a few clarifications. Some options could be limit the number of "requests" or scale it so top teams get less time than lower scoring teams. The idea being that the middle to bottom guys would benefit more from the feedback.
I hear you on the extra time requirements placed on the judges. Maybe those that can't stay 2 days could do a limited number Friday night or make more detailed notes and skype/email later. It might not be perfect to start with but it would be a start towards improvement and the process could then be re-assessed for the next year. I think it would be key to stress to the teams that and review would be a review of the scoring not a review of their designs and not a debate on the scoring. If a team doesn't want to respect the "rules" it's review over and worst case a return to no reviews.
I always used to laugh at the teams that would cover their cars up and close their pit doors so no one could see there "secrets". Top 3 should do a high level design presentation cover off their goals, team management, design processes etc rather than full on here are our numbers, laminate, drawings. If we were worried about copycat cars now then full disclosure would make matters much worse and they wouldn't learn a thing. There are some cases where a team might have commercially sensitive information, so maybe an exemption from disclosure could be sought before hand.
Excellent point. The auto industry is much like a car on a brick life support system; there, but not gathering speed anytime soon. I'd actually consider a bit of a reflection exercise here to understand exactly where grads are going from the program, and frankly to understand where else university investment dollars are headed for project-based learnings with any touchpoints in automotive (I know of a few that aren't FSAE).Quote:
Originally Posted by Kevin Hayward
I'd personally love to see FSAE combined with a careers fair down here.
This be The Truth. All videos bar one looked like Engineers Being Made To Play Nice For Camera. Excruciating viewing. The one that wasn't broke rules and did multiple takes. Even showed and explained car bits - very useful.Quote:
Originally Posted by Kevin Hayward
Agreed.Quote:
Originally Posted by Kevin Hayward
Agreed. Of interest - are there any information sharing licenses that you're aware of that'd let this happen if signed to? I know UniMelb has a department that (in part) deals with this sort of stuff, but it's not currently affiliated with FSAE.Quote:
Originally Posted by Kevin Hayward
If we could have the SAE-A approach my day job about this, we might even have solutions... this is something that absolutely has to change.Quote:
Originally Posted by Kevin Hayward
Agreed. It's FSAE! Fortune should favour the brave :)Quote:
Originally Posted by Kevin Hayward
I'd make three but I was over 10 years ago as a grad and it took more than 10 years to get me into local automotive industry... in a corporate role no less :DQuote:
Originally Posted by Moke
Agreed - we can't end up running DE twice over in terms of timing, but an extra four hours for one judge from each area (8 total) could well work.Quote:
Originally Posted by Moke
If we couple this with a disclosed scoring breakdown prior, then those wanting to protest, can.
(Contentious suggestions number... whatever)Quote:
Originally Posted by Moke
How about the top three get to present their design in front of the entire competition and judging panel, and are marked competitively as a result, with the marks presented openly at the time?
Isn't this how the business presentation event is done? I know it is here in the States.Quote:
How about the top three get to present their design in front of the entire competition and judging panel, and are marked competitively as a result, with the marks presented openly at the time?
I think at Michigan the top 3 business presentations will present to a crowd, but their placing is decided before they ever get on the stage. At FSG and FSA the top 3 get up and are scored against each other for the podiums.Quote:
Originally Posted by MCoach
Contribute, people, for the Aus competition has a potentially shorter future than many realise...
I believe the main improvement for this event lies in the judges/organisers hands. SAE-A has the potential and people to hold the best DE in the world.
In order of importance:
Every judge goes on a 2 day training camp ~1 month before the event. At this camp the intentions of the event, the expectations of the judges and teams are clearly laid out and explained. 1 day is dedicated to going through every teams design report as a group. Ideally this weekend would be run by someone who is truly interested in improving the educational value of the competition.
Judge sourcing needs to be reviewed. While I don't doubt the ability of the judges some do not seem to understand the point/challenges of FSAE. There are many interested and dedicated alumni around - the SAE-A needs to put effort into finding them and giving them design judge priority. An example of this is a VD judge reviewing our mechanically mode separated beam axle car (which required explaining, even though it was explained in the design report AND video) and the only question I recall was "how do you adjust camber?"
Judges need to bring annotated design reports with them to the event. They should note questions they plan to ask, including topics potentially overlooked in the report. Once an answer is received (or not) they need to move on to the next topic. 15 minutes is not enough to try and think of questions on the spot. Things that are explained in the design report should not need to be explained again and again, this is a waste of time.
The judges need to be on time. If they start late, they need to finish late. Penalising teams because the event is not running to schedule is poor, this is a project management competition - what is going on SAE-A?
Teams should be able to allocate weighting to design event sections with justification. If a team believes investing engineering effort into elaborate engine tuning is a waste of time and can justify why, they should be able to reduce the weighting on their engine/driveline scoring. Obviously a minimum weighting would need to apply. Maybe 25 points for all four sections and an extra 50 that teams can allocate - but they must justify it.
Getting to crazy ideas now; potentially have a "peer review" section where each team is allocated a number (3 or 4?) of teams that they must go and question and give a peer score to. Don't release the information until the day, for example: at 12pm a volunteer comes and collects 2 team members from team A. They take them for 15 minutes each to team B, C and D. This idea needs development but I just wanted to put it out there.
Finally feedback for the event needs to be released with the points. There is a 1 hour protest period (although whether this rule would be enforced is questionable) and yet no feedback til weeks later? how is a team meant to lodge a formal protest when they do not even know where they lost points? On the topic of feedback it needs to be clearer. Comments like "long pushrod" when nobody on our team was asked anything about pushrods suggests that the judges are just having a glance and finding a way to give everyone 70-85% much like a disinterested tutor/lecturer marking boring reports.
Good! More...
Come on everyone...
Maybe the thread needs to be on a somewhat unrelated topic? This thread is will now be known as "Why skid pan should be worth more points than accel"
I think the inactivity is due the location within the forum, people only really read the open discussion.
Soon, soon... :)
Much to write on this topic, but too much to do elsewhere...
Z
I'm asking myself if this is a FSAE Australasia forum right now... It is impressive how you guys from down under are able to discuss here.
I try to share my views of the "European DE".
My background: I was part of the most successful team in the Design Event in the last years. Zurich made all Design Finals since 2010 that we could get into and we won a lot of them (Sweep in 2013: UK, Germany, Austria, Italy; Won Austria and Spain 2012, 2nd in Germany, 4th in the UK and so on...).
Do I think we were the "smartest" or the "most educated"? Not really. I think we figured out how to tackle a Design Event, we knew the right stuff, we had a lot of training and maybe after some time a reputation to be good in design. We actually wouldn't have made the UK Finals in 2013 but the Chief Judges "corrected" us in the Finals, where we won.
I started doing Design Judging in 2014 in Austria, I judged Aero/Cooling and Main Concept/Project Management.
To be honest, I was shocked.
In Zurich I was responsible for writing and "proof reading" the Design Report for a long time, so I had a feeling what is considered a good report by the judges and what isn't.
I read all design reports of the FSA competition. I think FSA has one of the best field world wide but what I read was sometimes really horrible. A lot of the reports were just rubbish. Z would have tackled all of them ;) (Nearly all students took an example in the rules on how to write a bad engineering report. "Business words", not at all precise and so on).
Even teams that built their 5th or 6th car weren't able to stick to the page limit or used a table of contents on the first page...
So the first thing to improve for DE is to ensure that students hand in a good design report. If the judges get a bad report, they are not willing to put a lot of effort in it. They stick to their "basic line of questioning" without getting into the report. I think that is not the right way.
I tried to give feedback to all the teams that I judged the day after Design. A lot of them were not interessted in what I said about their report. I'm looking forward to see the 2015 reports if they used my comments to make their reports a bit better. I don't say that there is one perfect way, but just putting a 3D drawing of your car on the extra page is just useless.
The second thing: Judges should read the design report. I mean we had a really nice and friendly group of judges in Austria but I had several co-judges that didn't read a single report, didn't look at one drawing or didn't even prepare questions. I think that is terribly wrong. Not all judges are a GTS or a Stefan Liechti that prepare really well for that but a basic knowledge of the teams should be necessary.
Maybe we could reduce the length of the Design Report to one page. Like an Executive Summary in the Business presentation. Just justify your top-level desicions so that it is a basis of your design judging but not more.
I think it is not feasibly to have judges do an extra 2 days of preparation. It is difficult enough to get judges. Don't give them more work load.
Next:
In the judging, I have to give points from 0 to 20 in my category. I have maybe 10 teams. In my opinion it is not possible to rank all 10 of them "fairly". I have like 5-6 minutes after each judging to make notes (because the team should get feedback..) and make a judgement call on how many points that performance is worth.
Well that's just one point. The other point is, all other judges have the same problem. At some competition this subjective is offset by some "chief judges" that look if some groups give higher points or not. At Austria this was done be making different judging groups for all teams (random selection of all judges so that every area of the car should be checked but it is not the same group twice).
It is still difficult to look if you are "too nice" or "too strict" with your points. And nobody says what "a perfect score" would look like...
To be honest I have no solution for this problem. It would be best to look at all teams and make a ranking as good as possible. But that is not possible for like 110 teams.
One more point: The students should always have a back-up. I judged a really good design team in Austria in Aero. Their Aero guy wasn't at the competition, so I had to interview his understudy or something. I didn't know a thing. Not one thing. So I had to score them really low. That is frustrating.
Last but not least: The scoring categories. I want to have a real "Main Concept" category where I can give points on how the team choose the concept and what they have done with it. So far this is not well judged in the DE.
And yes, throw out stuff like "creativity" or "style". It is stupid.
Z, I think "killing" the DE is not a good thing.
Students "justifying" their solution is an important part of the job in my opinion. Some teams that don't have the resources to build an overall competition winner can still perform really good in Design and can show "the world" (aka their sponsers) that they are a good team.
I think nearly all competitions are decided on the track. So yes, the fastest car probably is going to win the competition if they don't make an error. In 2013 GFR placed 13th in DE and won Germany... by 80 points.
And yes, we all now know about the horrible 3/200 score. Geoff apologized. I think such a farce will not happen again. Please do not use it as an example for "killing DE" again. It is played out :)
Last but not least: I think about 5-6 team members of us got an internship at really nice companies (LMP/F1) offered during design judging because the students impressed their judge. I think that is one good point.
Cheers,
Julian
Some great feedback here.
Just a quick point relating to Mitch's excellent idea about a design judges retreat. I don't think every judge needs to attend. Rather, a core group of Design Event staff could meet two weeks in advance and identify key points that should be discussed for each team, and prepare briefing notes for each design group for each car. This would include potential questions for the judges to ask on the day. I think such a retreat could help make the event more portable, since the structure of the design event questioning could be prepared in advance, and given to judges on the day - no matter where the event is held
Head judges from each category plus the head judge?Quote:
Originally Posted by Big Bird
That's 5, or 9 if it's a rep from each functional area. Manageable.
Yeah, I reckon that would be good GTS. The team leaders could then prepare questions for their individual team members, based on known area of expertise.
Can't see why we couldn't teleconference the findings of the retreat to the known judges on the Sunday night, or some other reasonable time
Cheers,
Geoff
I think the first part of making the design event better is a consensus amongst all competitions on what design event is actually scored on. I know that FSUK has far more focus on manufacturability, performance v cost, etc. than FSG for example (not saying that that's a bad thing per se). So are we discussing design event in general or just design event for one competition?
p.s. I was the nong who suggested and designed the Design Video. I think the motive was explained at the time - we didn't want to waste valuable time in the 15 minute design presentations doing top level overviews.
I never saw the design video as compulsory, rather as an additional opportunity to impress and communicate with the judges. The one-take restriction, as for the restrictions on props, were imposed so as to limit the time and expense of the video shoot. I was hoping that the teams might give a quick overview of what they wanted to achieve with their designs - particularly what areas they considered to be important, and what areas were less important.
As for dropping it because it has been poorly done - well a lot of teams do poorly at the Endurance Event. Should we drop that too?? :)
If it helps delineate between teams - then why not leave it in??
I'd not be unhappy to stick to FSAE-A - the competition is in a very difficult place right now - though if the discussion generates learning that can be applied elsewhere, so be it.
Don't drop it, it's a good thing. Some of them are hysterically bad though, as in the students look like deer in headlights. For 2+ minutes in one glorious take.Quote:
Originally Posted by Big Bird
It might need some criteria around it though I'd suggest multiple takes just so they can put more effort into talking about the project and less into ensuring the manage one (glorious) take. And if they can run through some key bits of the design as it's there in front of them it'd be great too.
I'd think if it's something done capably it should be able to serve students for more than just the competition: sponsors, interested parties etc...
Good call GTS. It did need more explanation. And maybe no more than 5 different takes?? Agreed the car would be a good prop.
I like Mitch's idea of a variable points allocations. We dont want teams dismissing areas outright, but it would be good to have some flexibility to suit different teams' design interpretations.
My thoughts.
* Break the car into a number of subsections (e.g. chassis, suspension, aero, ergo, etc)
* Prior to the event, the teams nominate the number of points each subsection is worth, within a set range. e.g ergo might be between 5 and 10 marks. This will reflect their team's design prioritizing. Their justification is assessed in the design management section
* judges score each section in terms of percentages (0-100%), final points per section is points weighting x percentage
Geoff,
I like the idea of prioritizing but in my experience you need to "fix" a couple of things in order to make that work.
We developed our own motors since 2011, so naturally we were really strong in that area of design. We always had the "motor guy" in our design team. At about 2/3 of all DE, no judge was able to ask the motor guy questions about motor design because it is a very unique area. Sometimes it was "what power/torque does it have". That was about it.
At -I think it was FSG 2012- the "powertrain judge" just asked "What is the critical temperature of part A,B,C,D...." that was powertrain judging.
So if you have to make a desicion to weight your design areas, you have to know what kind of judges do you get. Sometimes you are lucky and get the "one" expert in the whole DE judging team in your cue. But most of the time you have a problem in a specific area.
At an event like FSAE-A it could probably work because all teams get the same judges.
The next "problem" is, that most judges that I had as a participant didn't really care about the "area of design" they should focus on. Most of the time the judge which introduced itself as Aero judge started with 2-3 Aero questions and afterwards asked about basically every detail of the car, from tires over torsional stiffness to powertrain data. If this judges is only grading the "Aero section" but takes the whole "experience" into account, it probably gets complicated.
I don't want to take the freedom away from the judges though, to ask something from other areas which could be interessting to get the overall impression of the team.
How is that handled at FSAE A?
If this were to work, it'd be amusing to see teams nominate their own weightings (because I've a fair idea what'd happen next :D)Quote:
Originally Posted by Big Bird
No such aero problems at FSAE-A :) We guarantee 15 minutes of aero, which some of you are quite good at answering.Quote:
Originally Posted by JulianH
I agree Julian, it's nice to see some introspection on the part of the FSAE-A design judging organizers. I hope the other major FSAE/FS design organizers will join in.Quote:
Originally Posted by JulianH
Not necessarily stupid, but necessarily inconsistent per the peer-reviewed research I've referenced previously.Quote:
Last but not least: The scoring categories. I want to have a real "Main Concept" category where I can give points on how the team choose the concept and what they have done with it. So far this is not well judged in the DE.
And yes, throw out stuff like "creativity" or "style". It is stupid.
It is possible to win the competition without scoring well in design, but it certainly makes it more difficult! Julian, your example shows something even more important and germane to this topic, and that is the inconsistencies in design judging and design evaluation criteria between competitions. With the same car, design team and design report, GFR combustion's 2013 design placings were:Quote:
I think nearly all competitions are decided on the track. So yes, the fastest car probably is going to win the competition if they don't make an error. In 2013 GFR placed 13th in DE and won Germany... by 80 points.
1st at FSAE Michigan
13th at FS Germany
2nd (1st combustion) at FS Austria
This pattern repeated in 2014. With the same car, design team and report, GFR combustion:
4th at FSAE Michigan
20th at FS Germany
1st at FS Austria
3rd at FS Spain
The design judges at FSG are obviously looking for something different from the car and team than the other competitions. Also, the design priorities of the FSG design judges do appear to not match those of GFR combustion.
Another example of this problem, the 2014 GFR c-car had one of the highest levels of aerodynamic downforce per kilogram vehicle mass of any car in FSAE/FS history. Our design judging queue at FSG14 had no aerodynamics judge.Quote:
Originally Posted by JulianH
I'd like to expand on my post from the FSAE-A 2013 design judging thread:
Who are the customers/stakeholders for the FSAE/FS Design Event? The organizers themselves, certainly, but also the student design teams that participate. Also, the companies that supply the design judges, as Julian mentioned many of them are looking for students to hire. The universities that support the teams, and the faculty advisors that mentor the students.Quote:
Originally Posted by bob.paasch
What do the stakeholders want? The organizers want a score for each team, with manageable resource (especially time and money) requirements.
What do the students DE participants want? They want a fair and transparent process. What is "fair"? They want to know the evaluation criteria, they want a fair chance to perform and prove their knowledge, and they want feedback on their performance in order they can improve. They almost always want to know why and how the teams that placed above them did so, and that the design judges had good logical reasons for placing the teams the way they did. To me, that means the DE and Design Judges must employ a consistent set of criteria for judging the "goodness" of the vehicle and the knowledge of the team. Those criteria should come from the Design Objective, as stated in the rules:
The design criteria for the vehicle can be summarized as performance (as measured by the FSAE/FS events), ergo, aesthetics, cost, maintainability, manufacturability, and reliability. I personally think aesthetics should be thrown out as inconsistent and unmeasurable. Whilst other criteria could be added by the design judges and/or organizers, such addition is contrary to the rules of the competition and inconsistent with proper design process. The goal of the student designer is to design a product/system/vehicle that meets the published rules and with "goodness" measured against the published criteria. Sorry students, but adding a feature, creative or not, that doesn't make the car faster, cheaper, more ergonomic, etc. is a waste of resources and should be penalized as poor design process.Quote:
A1.2 Vehicle Design Objectives
For the purpose of the Formula SAE competition, teams are to assume that they work for a design firm that is designing, fabricating, testing and demonstrating a prototype vehicle for the non- professional, weekend, competition market.
A1.2.1 The vehicle should have very high performance in terms of acceleration, braking and handling and be sufficiently durable to successfully complete all the events described in the Formula SAE Rules and held at the Formula SAE competitions.
A1.2.2 The vehicle must accommodate drivers whose stature ranges from 5th percentile female to 95th percentile male and must satisfy the requirements of the Formula SAE Rules.
A1.2.3 Additional design factors to be considered include: aesthetics, cost, ergonomics, maintainability, manufacturability, and reliability.
A1.2.4 Once the vehicle has been completed and tested, your design firm will attempt to “sell” the design to a “corporation” that is considering the production of a competition vehicle. The challenge to the design team is to develop a prototype car that best meets the FSAE vehicle design goals and which can be profitably marketed.
A1.2.5 Each design will be judged and evaluated against other competing designs to determine the best overall car.
A fair and transparent design evaluation process evaluates the design process as presented by the student team, with the car as final proof-of-process. Given the stated Vehicle Design Objective above, the most important criterion is performance, with ergo, maintainability, etc. as additional design factors. Performance and cost are explicitly measured in the competition, and one design can be directly compared to another using the common currency of competition points. A fair and transparent design evaluation process would include specific criteria for ergo, maintainability, manufacturability and reliability, and a method for comparing design tradeoffs in these areas with performance and cost.
As always, I welcome comments.
The thing that I find disappointing about this event is that the teams don't really progress in their understanding of engineering from year to year. Each new team comes in and the same projects are handed out. So the FSAE project becomes a series of component-driven tasks - some design chassis, some design wheels and hubs, some design pedal trays etc. The marks breakdown in the rules drives this too. We think of this firstly and foremostly as a parts design event.
I was looking at the Curtin Uni car at FSAE-OZ 2014, and it was one of the best examples of quality control and race preparation that I had ever seen. Fasteners all correctly fitted and marked with paint, beautiful lockwiring, and from what I saw their raceday management seemed spot on. It made me wonder why we can't accommodate such excellence in the Design Event.
What I am thinking is with the abovementioned flexible points allocations for design, each team could nominate their own "feature" category, where they pick an area of engineering and present a special presentation on it. Maybe a two to three page report on it, plus a five minute presentation on, say, the saturday night to an open audience. You could even steal 25 points off the presentation event to facilitate it.
We then have an enticement to think more broadly (maybe presentations on pit management, OHS breakthroughs, vehicle dynamics, or whatever. We all see something new, and there is some enticement to break out of the same old "we have redesigned our pedal tray to be lighter and stiffer than last year" mindset.
)
Forgot to close bracket in previous...
We came to the same conclusion; however in our case we *think* we know what they require. We are building an e-car, and for some reason lack of major self-developed e-components, albeit justified, costs us lots of points, at least based on feedback...Quote:
Originally Posted by bob.paasch
Could not agree more! Maybe (in order of preference): Performance, manufacturability, a method for comparing design tradeoffs in these areas with performance and cost, ergo, maintainability, reliability...Quote:
Originally Posted by bob.paasch
Thanks Julian,
In regard to the comment about needing the right design judges if teams could self-prioritize, it would only work I think if we were to do the abovementioned Design
Event retreat two weeks before the event. Then we could make sure that we knew in advance what judging expertise we needed in advance and recruited accordingly.
I've been thinking more about it and self-prioritizing could be a really interesting exercise and could inspire some innovative designs. Of course, we would set minimum values o the allowed range, and would put a high minimum value on engine systems to encourage work in this area. But the variable weightings in general would make the teams a little more proactive in driving their designs and understanding and owning their decisions.
Cheers!
Bob,
I agree that it is a problem that different competitions are looking for different things in DE.
For us, it was always FS UK with a stronger focus on "simple manufacturing" and the rest of the European events looking for more "oh yeah, that's awesome".
But you experienced the "special statics" in the UK 2011, if I remember correctly...
Do you know if the strong difference in design scoring (especially in Germany) is a "GFR thing"?
We -and our competition- always placed very comparable throughout a year, so I am not an expert how this "problem" is in the combustion world.
I talked to a lot of judges last year (in order to prepare myself for my turn in judging..) and I think there is one "GFR specific" problem out there: (From an outsider perspective, because all judges are basically outsiders!) You dominate the performance part of the competition with a perfectly figured out car that is evolved since 2010-2011 with nothing really changing besides a bigger aero every year. A lot of judges do really have a problem with that ("They don't know why their car is fast, the "Chris Patton"-generation made the car fast and they are just sticking to it" and so on). I also witnessed a judge in the FSA finals that placed GFR last "because they show up with the same car over and over, that is not the spirit of the competition"... So maybe that explains why you especially have a "fluctating" DE placing.
What is your desgin feedback in Germany the last three years? You won Design in 2011 and afterwards dropped significantly but I'm sure that your preparation is roughly the same.
As I was never at FSAE Michigan, I have to ask from the outside: I heard -especially from the German GFR guys...- that the US teams are significantly worse prepared for DE compared to the "big" German teams. Is that true? E.g. Michigan 2012 only GFR and ETS (with a reputation of very good design) made it in the Top 6. All German teams that made the trip basically placed in the Top10.
In 2014 at FSG, we had Washington, Akron and OSU in a good position.
Would you say that DE in the states is different compared to say FS Austria (where you place comparable...)?
Bob, I was wrong in our offline discussions. I was adamant we don't score innovation in the Australian competition. I was wrong, we do. I advocate throwing it out as an explicit criteria. If something is innovative for good, demonstrable reason, then it'll show up in the other criteria, period. I'd delete the aesthetic requirement whilst we're at it as that much is subjective. Any car that wins its competition is beautiful.
We have some critical problems in communicating what the competition is about and in establishing what a good DE looks like, there are three key reasons (my observations):
1
Many students (and some faculty) don't actually appear to read the rules. They're not actually aware that FSAE is not a competition to build a quicker car than last year at all costs. FSAE is not motorsport. The number of people getting to competition after a year's investment to discover this much at DE... is frankly staggering. Many have zero idea of the Vehicle Design Objectives. Zero.
2
Good DE are not communicated openly and transparently. Part of it is the feedback against the rubric, part is lacking communication, part is simply not recording the DE for review. If you asked the 2nd placed team what separated them from the 1st in DE, there's literally nothing to point to beyond the headline number.
3
Having judges pull a car out and point out what's wonderful about it doesn't actually reflect the design event (it's not a replacement for (2) above) - the point of which is to "to evaluate the engineering effort that went into the design of the car and how the engineering meets the intent of the market" - this is very different to judging a design in lieu of its designers. To this end, cars that are little-modified can be a real problem - we're supposed to test the design process applied, and often students have no idea of as much with a broadly-inherited design. We end up with something that runs very much contrary to the intent of the competition.
Which is why I'm less interested in the cars getting faster year-on-year - it's not a metric that suggests an improved quality of graduate. If they do get faster, great. If not, not the end of the world.
We also need to be aware that whilst much can be improved on the event side, providing complete closure on a design loop in a year-to-year manner that encompasses prior learning is not the organizer's responsibility. If students can't be stuffed with first principles, evaluating prior efforts or both, that's not the organizer's problem. It simply becomes our job to judge the effort that ends in a delivered design. I'd like to see a criteria added for incorporating prior learnings into design delivery.
I don't agree with the reranking of what's left with performance first, because against the design brief they're all important. A car with a phenomenal performance envelope is, guess what, not likely to have that much explored if it's got crap ergo, and won't last a weekend of whatever it's intended if it can't be readily serviced. Even in F1 these factors are all considered broadly to contribute to having and extracting performance. We can have a more qualitative go at how it's all assessed. FSAE-A 2014 was a case in point: students (perpetual or otherwise) in the competition can say what they want, however not one of the top 3 cars appeared particularly easy to drive. Each had some severe performance deficiencies (yes, even the winner) that simply ranking the entrants against a stopwatch doesn't easily reveal - all that gives is a relative indication of performance on the day. The competition exists to judge a relative performance potential of the student groups involved - not the car on it's own.
I'm a little concerned about giving points for a car that's completed to a high standard. We already automatically take away points for cars not completed to a sufficient standard to get them through the weekend - either they fail scrutineering or something goes terminal over the weekend. Taking Curtain 2014... yes, a very well-finished entrant however I'd be more interested in picking apart their beautifully-finished-but-not-equally-beautifully-executed aerodynamic package as opposed to any review of their fit and finish...
...however were BB's observations made transparent and shared, it could well become a standard.
We need more transparency.
Hi GTS,
I probably didn't explain myself too well. My fingers are like bratwurst at the moment, and i tend to cut my explanations a bit short.
i wasnt so much advocating build quality as a criteria. I just like the idea of a seperate floating feature category that the teams can use to highlight something special they have done. In Curtin's case, i put the build quality under the banner of raceday preparation.
This points allocation could be used to feature, for example:
- component lifing systems
- service manual preparation
- raceday / pit management
- test rig development e.g. tyre testing
- material development
- design for sustainability
- design for manufacture
- team developed manufacturing techniques
- human resource management
- team developed OHS procedures and tracking
Run the presentations as an open session and we all learn something new.
Quote:
Originally Posted by GTS
GTS and Julian - I think it can be very difficult to convey the importance of small design improvements when a judge enters a DE with the mindset of "It must be all new!" This preconception is very difficult to get around when our priorities are not aligned with whichever judge happens to have the disagreeable preconception.Quote:
Originally Posted by JulianH
Why must it be all new? There isn't anything in the rules which says it must be all new (except for the 2nd year rules for FSAE-A and Brazil S6.15?). And even in that case, my interpretation from reading that rule was the intent is that the student understands the design process and fundamentals of the previous design if they choose to keep it. I know last year's GFR DE team had a very good understanding of anything that was left 'untouched' from a previous year which ended up on the new car. It is not within our team philosophy to let that happen without good reason. Nearly every component of last year's car was rederived to see if it was still a good idea. If we settled on a very similar concept to a previous year, why should we be faulted?
I bet more than 80% of last year's car was different than the 2013 car. The fact that it remained unseen to the casual observer is not something worth penalizing. Engineering Peacocking is an issue in the DE and it would be nice to see if there is a reasonable way to manage its influence.
GTS - I know you are not interested in cars getting faster year-on-year but it was a major factor in the 5 years from 2009 to 2014. Every year the top and midfield have gotten faster. We can track how much they get faster beyond the accel and skidpad events because we can reference the relative speeds of our new car against the old one. (All of the GFR combustion cars chassis are still in running condition, requiring an engine and ECU to be replaced in the car). If we know our newest car is faster than the old car by some %, and at competition our closest competitor is nearer than that %, bringing back the previous year's car would not result in a win at the competition. The previous year's car would have been slower than the new field of cars.
Some design development had to have been done to move the goalposts, no?
I am a lover of really odd factbits. Ask anyone on our team and they will tell you I know some of the most useless information in the world and am proud of it!
One of my favorites in the automotive world is the development of laminated windshields. Nowadays if a pebble jumps up from under a truck and hits your windshield, it'll crack only one side of the glass and make only a very localized crack. If X-Man Hugh Jackman gets thrown into your windshield at a high speed while in a confrontation with Magneto, the entire sheet of glass may break, but I'll bet it stays in one piece. It'll look like a wonderful blanket of mosaic glass, all shattered and cloth-like.
Windshields didn't start out being so wonderful. They were made of plate glass and shattered at the first sign of trouble. You could get a windshield in 1905, it just was going to break soon after rolling off the showroom floor. By the 20s they were laminating plate glass with a urethane layer in between the panes. This helped keep the entire window together if a pebble were to impact the outside. A larger impact would still shatter the backside and send large shards of shattered glass into the driver's compartment though. By the late 30s they had innovated with tempered glass (useful in WWII planes since it was so much stronger for its weight than plate glass). Tempered glass was the bee knees for 30 years. Then the US Government decided flying glass shards were making a mess of accidents. Culminating with the 'big bumper' rules, a part of the safety push was the mandatory implementation of seat belts and quickly 3 point seat belts (I guess an engineer point out that a body isn't fully constrained by a line, and doing so simply creates a very nice pivot point about a person's waist). They also pushed for better safety glass to be used in windshields. The strength and shatter resistance of the windshields increased greatly in a very small number of years because of this demand.
They have been adding UV protection and glare prevention to windshields since then. No more sun-dried dashboards of the 60s-70s-80s. And I don't know who at GM thought up the blue line at the top of the windshield, but I guess that could be considered a user improvement if not very aesthetically pleasing.
(TLDR) Sorry for the long example, but the point is that windshields have existed for more than 100 years. Nearly as long as cars have been going fast enough to bring tears to the driver's eyes. Is the innovation or engineering design development easy to see or obvious? No. Was there a sweeping change in the implementation of the windshield? In the first 50 years of car development for sure! 2-part windshield and folding windshields and vertical windshields and wrap-around windshields and extremely raked windshields and tinted windshields. These implementation design changes were easy to see and easy to comment on. Eventually everything settled down and the second half of the windshield's story is that of design improvements that go unseen to the casual observer. Better glass, better lamination, additional protections like UV ray protection, incorporation into the aerodynamic design of the vehicle, different mounting techniques. These later design developments are just as magnificent as figuring out that you should simply have a windshield! They just aren't as 'obvious'.
When a design matures, much of the 'fundamentals' become obvious to the design improver and the less 'obvious' advancements they make are what they become proud of. If an observer were to ask the windshield design improver why windshield, then the improver is going to be driven to get through the 'obvious' stuff and take the observer to the non-obvious improvements. The 'obvious' fundamentals are still well known to the improver, but in a time crunch it may be impossible to get past the fundamentals and into the improvements within the allotted time. After this, the observer will walk away from the conversation saying 'Its the same windshield as last year! Psh.'
We did not make it to design finals at FSG. Our car concept looked like the same windshield from last year. However, because of the addition of the UV protection and a better incorporation into the aerodynamic design of the vehicle, our latest windshield was not only the most reliable on track, but it also was the fastest on track.
Jay,
yes, the problem is, that design judges are not able to see "your" new windshield.
I only judged GFR in the Austrian Design finals... in aero.. so the area where the most obvious changes happened from 2013 to 2014. Phil was quite good, not the best but yeah, it was good.
So I'm cannot say if it is true that you are basically running your 2011 car with freakin' huge wings.
When I had to guess, then what would help is a new monocoque... You can't tell me that your 2011 solution cannot be topped. There are a lot of monocoques out there that are better than the GFR solution and you are still sticking to it. I understand how much work goes into building new moulds: Zurich is building a brand new monocoque every year. But just this "school of thought" is maybe a point where the judges jump to conclusion: "Ah, they don't change their monocoque of the dominating car of 2011, and the engine is still the same, and the tires are the same... soo what is it?"
And you "lost" the FSA Endurance against an electric car by 1.8seconds, maybe a "GFR wake-up call"? ;) To be honest, I am sure that at the moment your concept is the fastest for a combustion car. TU Munich showed their performance -sadly only in the last quarter of the season- with their brand new concept. So I can understand that you are not going to change your concept "for the sake of changing". The 4WD electric cars are in the same situation right now. They have basically done everything, so the top level concept stays the same.
I don't know how you can't win over "those" judges that go into Desgin thinking "ah those bloody GFR guys". That's the subjective human problem, that Z is always talking about... Maybe Geoff or GTS have an idea...
I think I saw some pictures of a mould a few days ago ;)Quote:
Originally Posted by JulianH
While it isn't easy to define, there is also a need for "integration" -- I've seen a few cars over the years that had reasonable components and yet were obviously crap overall. I gave one team a design review the day after judging and told them their car looked like a lot of science projects bolted together. As we discussed their poor team showing in design, it came out that they lacked a strong/experienced chief designer or chief engineer to coordinate all their individual efforts.Quote:
Originally Posted by Big Bird
Yep ;)Quote:
Originally Posted by DMuusers
https://www.facebook.com/TeamGFR/pho...type=1&theater
Good call regarding integration Doug. I hope within all the other garbage i have scattered all over these forum boards, I have made my views on the need for whole vehicle integration and holistic design pretty clear.
In Oz, we presently have four judging teams, for design management, vehicle dynamics, structural design, and powertrain / electronics. These are broad categories, and the Design Management group's directive is to assess whole vehicle integration and top level design. At least, that is the principle. In review, the event probably looked like a lot of science project assessments bolted together. We need a strong/experienced chief designer or chief engineer to coordinate all the individual efforts. :)