2020 Virtual Design Judging

[DougMilliken]

As one of the last(?) posters here, I thought I'd open a topic to discuss the 2020 design event. For anyone that doesn't know, due to the virus, this year's event will be online. SAE and many volunteers have organized a large WebEx including sessions that simulate traditional design judging Bays/Queues and smaller breakout sessions for students and design judges to discuss their work.

I was not part of the organizing process and I've just been through the training to use this version of the WebEx software in the last few days.

[DougMilliken]

I'm interested in hearing any other comments about this event, from students and/or volunteers.

Here are some initial comments after WebEx judging with 10 teams over the last two days:

+ In the previous post I mentioned advance practice with this special "WebEx Training" software. I logged in for several hours over a few days, had several problems and eventually felt like I had a passing familiarity with the capabilities and user interface. Different versions (download-software, web-no-download, Mac-vs-PC, FireFox/Chrome) all had different screen layouts and UI and the trainers were only on one of the versions--they told me to click on a button...but it wasn't there in my version. Moving the "Presenter Ball" from one user to another is a drag-drop in some versions, in other versions it is highlight then click on an icon in another part of the screen.

+ The organizers tried their best to duplicate the flow of traditional design judging
* New--each team first heard a short intro from the "virtual design assistant" (VDA), a new team member with IT background and additional training in the "WebEx Training" software
* The queue lead design judge (DJ) gave a short intro to announce timing
* Each team had 10 minutes for a general presentation on their work, enforced by the VDA (I thought this worked out well--a hard ending!)
* After that the intent was to go to separate "breakout" sessions (sub-WebEx) for various disciplines--engine, suspension, etc.--for 30 minutes.
* DJs caucus in a separate/private "Judges" breakout for 5-10 minutes
* DJs give feedback to the team as a whole for the remainder of the time (this often went over, using up the scheduled break between teams)

+ Differences from normal include
* No scored judging--nearly all teams were forced to stop work when campus closed for the virus
* Immediate feedback/discussion/design review--this worked well, but 30 minutes is much too short (historically the design review is the next day after design judging and is open ended, could run an hour)

+ Problems from my perspective (only--I have not discussed very much with other DJs)
* Lack of hands-on severely limited suspension/steering judging since I normally push on the car/tires to look for rates, deflections/compliance and friction in different areas, and the results often lead into a line of questioning
* Single line of communication only, normally I can be listening to team members while looking at display boards or the parts of the car I want to see (not the photos that were presented), this alone probably cut my effectiveness in half
* The breakout rooms had no video, only "show your screen" capability--facial expression, hand-waving and other non-verbal communication was completely lacking
* In our queue we had several different kinds of problems with the software, limiting the time to talk with four of our teams--out of 10 teams total that means the "WebEx Training" software worked as advertised only 60% of the time, which is a failing grade where I come from
* There were also problems with audio quality, often someone failing to mute (I'm as guilty as anyone, important to mute when not talking), also clipped/missing speech and weird distortions on occasion
* Our VDA was excellent (and I believe necessary for this to work at all), he was able to rescue things each time, in one case a student-DJ conversation was moved out of the WebEx to a private cell phone conversation, in other cases only one breakout session failed and that sub-group held a (shortened) discussion in the main conference, at least once we all got summarily bounced out of the WebEx completely and had to restart from login

+ As usual, the students were universally enthusiastic and appreciative which made the experience bearable (for me). If it was some sort of routine training I would not have lasted the two days.

As noted above, I'm interested in comments by others that were involved. It's too early for me to give any sort of rating or conclusions, I may get to that in a future post.

[js10coastr]

Similar thoughts from my end. Not enough time and not enough depth. Webex issues were present and annoying, but not overbearingly so. Kudos to the VDAs!

- Having a "back channel" text group with the other design judges helped to communicate our thoughts/concerns quickly and efficiently. IE comparing CG heights across subteams
- Some teams could be great in one section and terrible in another. I think this made it a bit more challenging for the judges as we can't see the whole car and can't glance at the other subsections. So a bit more trust/reliance on what the other judges were reporting. That being said, there were no "rookie judges" this year.
- Some teams seemed more prepared for the virtual judging... but then again, some teams are more prepared for normal judging anyways.
- There was some impressive work going on that I'm bummed we won't be able to see this year.
- There was an instance or two when a student had to hop from one room to another (ie, aero and composites, suspension and chassis...), this was super difficult with the time constraints.
- Overall, I felt my group was still able to accurately "judge" a car/team.