This is one that, I admit, never occurred to me, either. The fact that it did not is profoundly embarrassing. Ready for a little F-35 bashing?
In Popular Mechanics [pm] there is some discussion with an F-35 pilot, regarding the cockpit. And, since I was just neeping about strategic system design and component layering [stderr] it’s strangely relevant. Did you know that the F-35 has a “software-defined cockpit”? I didn’t. In principle, that seems like a pretty neat idea: you can drop bombs with it and you can play rogue. What might go wrong?
In an interview with the excellent aviation magazine Hush-Kit, an anonymous Panther pilot describes the F-35’s cockpit and interface system. The pilot says the cockpit itself is “beautiful,” full of screens that allow you to bring up an incredible amount of information about the fighter with just a few finger swipes, and customize the data to tailor it for the particular mission.
The customization was, at first, the part that worried me. I was thinking “oh god it’s just another thing for pilots to mess with.” Distractions in the cockpit of a very expensive combat jet (even if it’s relatively short-range and slow-moving) have to be a problem. But, no:
But the problem with touchscreens, the pilot explains, is a lack of tactile feedback. Switches have a nice, satisfying click that instantaneously lets the user know they were successfully flipped. Almost everyone with a smartphone has touched a virtual button on a touchscreen, expected a result … and then nothing happens. The anonymous pilot reports failing to get a result from a touchscreen about 20 percent of the time:
At present I am pressing the wrong part of the screen about 20 [percent] of the time in flight due to either mis-identification, or more commonly by my finger getting jostled around in turbulence or under G. One of the biggest drawbacks is that you can’t brace your hand against anything whilst typing – think how much easier it is to type on a smartphone with your thumbs versus trying to stab at a virtual keyboard on a large tablet with just your index finger.
That’s another problem with a software-defined control plane: it will not degrade gracefully due to load or damage. It sounds like, if the cockpit of an F-35 takes damage, it’ll just turn into a pretty array of black glass. Not to worry, the plane’s not survivable enough that it’ll matter for long: the damn thing only has one engine. But, when you combine the fact that the F-35’s ground link software uses vulnerable and unreliable WiFi protocols and contractor-built software, with the idea that a hostile power might want to incapacitate an aircraft, a “remote reboot” means that the plane is a flying brick. A briefly flying brick.
I wonder how well it all responds to system load. What happens if something is eating up more than its share of CPU power and the entire display panel system starts to get jumpy and laggy? You can’t reboot. Speaking of reboots, I sure hope they turned Microsoft patch updates off in flight mode; “System needs to reboot to install patch, in 15…” as you’re coming in hot on a warzone.
The bumpiness factor is a serious issue. I remember one time I tried to write a blog posting while I was on a bus. Couldn’t do it. It was better on a big commercial jet at high altitude, but I was holding my laptop practically on my chest because of the reclining seat in front of me. But that’s another story.
I swear I am not going to make any derisive comments about the 1980s graphics on that multi-zillion-dollar display, except to say that my friends in the Fuel Rats, who have made their own cockpits for Elite: Dangerous using iPads and button-blocks, would look down their snouts at the F-35’s displays. The state of the art in Elite is to use programmable USB button blocks with tactile buttons. The fellow whose cockpit is pictured above is using the Saitek controllers and a laptop and a couple of iPads and some button blocks. I used to do everything with an old Thrustmaster “Warthog” HOTAS – I think it has a total of 20 buttons and switches and that’s enough. Not enough for an F-35, though. Meanwhile, the Air Force fighter jocks will cry as they deal with the F-35’s “fly by fumble” interface.
Hahahahaha I hope the “Eject” control is a good old-fashioned pull-lever.
I battled for years with the automotive OEMs about changing the well-established tactile buttons and switches with touch-screens. To the point that we made prototype HVAC controls using capacitive touch sensors behind knobs, buttons, and sliders which had no moving parts but provided tactile feedback for where on the damn controls your finger is resting. Some of the sliders and knob designs were really clever and reasonably intuitive. We could have done the same thing with radio controls if there was any interest. Nope, touch screens were flashy, never mind that they are distracting and will destroy your night vision.
The idea that the driver would need to remove their eyes from the road to look at a touchscreen, locate the button they want, move their finger onto the button, and get visual feedback that their button press was doing something, all taking 3-10 seconds with their eyes not watching the road, just in order to adjust the HVAC controls, finally convinced me that autonomous driving wasn’t all that bad of an idea.
@flex – right! Novation and Roland and every music/synth manufacturer has implemented software-driven tactile controls. Its not rocket salad.
I see at least 3 actual toggles, a lever, and a sho-nuff turn knob on the outer edges of that control console.
But where’s the joystick?
Pierce R. Butler@#3:
But where’s the joystick?
They’re on the sides. If you imagine a car with bucket seats, the stick and throttle are where the seat arms would be, out of the way from the rest of the controls. They’re super nifty force-sensitive resistor-based controllers, too; very $$$ive.
A Novation Circuit grid-pad synth. It’s all software-controlled under the buttons. By the way, these things are hella fun to play with.
What I found funny in the article was voice control. Apparently the F35 has voice control but combat maneuvers messes with the pilots voices so badly that it’s useless.
As for rebooting I know that at least parts of the system can be rebooted while the plane is in flight. In early revisions of the plane the sensor system would degrade over time and had to be rebooted in the air. I expect the plane is designed around a network of computers and everything except the flight and engine controls can be rebooted in flight.
I think touchscreens are a bad idea in this context. Extremely bad idea for exactly the reasons mentioned.
I have experienced another bad use of touchscreen at my previous work. One testing device was equipped with a touchscreen instead of a mouse and keyboard and this was being sold as some incredible feature (it was new at that time). We did buy the device since it did do what we needed, but I have said to the manufacturer that of all the features it has, the touchscreen is actually a drawback and not a selling point. Unfortunately, many other testing equipment manufacturers went that route since then and this bad thing is now probably a state of the art.
Programming of the test was done by dragging building blocks in UI interface in place, similarly to how a flow-chart might be drawn. So far so good, that is not uncommon in testing devices. But its implementation on the touchscreen was daft.
Problem one: the touchscreen had thick protective glass on it, so depending on the viewing angle finger was actually touching a few mm off where you saw it to touch, which made hitting small icons impossible.
Problem two: the touchscreen was fixed at shoulder height. Writing a functioning program from scratch was not a task for just a few minutes and it was not easy.
Problem three: dragging elements with your finger across a huge 40 cm screen takes a lot longer than with a mouse where you can adjust the sensitivity (and rest your hand)
Problem four: writing variables, program names and descriptions, etc. was extremely slow on a touchscreen.
Problem five: during work I often had oily fingers and it was not possible to wash my hands every time I need to start/pause the program. So the touchscreen looked really iffy really quick, not to mention that it was sometimes difficult to get it to work at all.
So, in the end, I have fitted the machine with a USB mouse and keyboard.
Touchscreens are a fad. They are currently being overused.
@Charly:
I know an industrial process engineer who used to maintain some semi-robotic systems. One thing they had trouble with was keyboards kept breaking. The old system which was replaced had a big “go” button and it turned out that the production guys couldn’t reach it well, so they just tapped it (gently) with a hammer. The new system, with the keyboard, was cramped so the production guys used a small ball peen hammer. Keyboards lasted about a month. That part of the system was replaced with a big steel button that would withstand a sledgehammer.
I hate touchscreens in cars, to be honest. And the trendiness has spread. The new Ferrari Roma is controlled by touch screen controls that seem to be completely random in their placement. And are semi-hidden. It seems they deliberately made the controls as obscure and counterintuitive as possible.
The touch controls suck on my phone, they suck on my microwave oven, and they even suck on my washing machine (which has one nice physical knob and five sucky capacitive buttons). Anyone who has ever mounted a phone on the handlebars of a bike and tried to play Pokémon Go would know touchscreens and vehicles don’t mix. What kind of idiot would want such a thing in a car, let alone in a fighter jet?
And this comes from an industry where, very early on, even and when there were only a few controls in an airplane, the throttle, fuel mixture, and propeller pitch controls, whether they were levers or pull knobs, were all large and had different colors and shapes for easy visual and tactile identification. Where did we go so wrong?
I hate touch screens, full stop.
Do the pilots wear gloves?
What happens if your skin in dry and the screen doesn’t recognize the touch?
What happens if a nearby explosion jostles the plane at a critical moment?
Taking your hands off the joystick or throttle at a critical moment can be disastrous.
I predict that this will not end well.
Touch screens and voice control are short term kludges. I am holding out for the brain implant.
The Korean solution is to always carry sausage with you.
South Korean iPhone users turn to sausages as a cold weather ‘meat stylus’
cvoinescu wrote:
I fully agree!
The best control layout I’ve experienced in a car was my cheap ’98 or ’99 Honda Civic hatchback. The commonly accessed controls (vent selection, fan speed, audio volume) were all a short move of my right hand from the steering wheel. They were so perfectly placed that after a short while I could find each control and make adjustments without looking. Try doing that with a touchscreen!
Every car since then (including another Honda) has been a step backwards…
Even before touchscreens, things were going the wrong way. They used to use rocker switches, or push-on-push-off buttons that stayed pushed in when on, and popped out when off. And actual radio buttons! The buttons were spaced apart. You could tell whether something was on or off by feeling the button (and feeling it would not be the same as pressing it, unlike a touch screen, or the capacitive pads on my washing machine). My current car (of late 2010 vintage) has four buttons (AC, recirculation, front and rear windshield heaters) which are momentary switches and indicate their state with built-in LEDs. Turning something on feels exactly the same as turning it off. Two of them are easy to find because they’re slightly raised, but the other two are flat pads that are part of a continuous panel. Some of the radio buttons are segments of a flat, smooth surface too. They’re illuminated, sure, but not distinct enough to the touch.
A slightly different albeit not unrelated problem I am having with my new laptop is it happens to have an illuminated keyboard. At boot, it runs through all(?) the available colours, and then stops at blue at (I think) 100% power — which is fecking bright. I don’t need to turn on any lights at night. Its very very distracting.
I run Linux. As it happens, in the distribution (“Linux version”, broadly speaking) I am using, there is no ability to control the keyboard’s illumination. At first, the only way I found to control was to put the laptop to sleep and then wake it up — that turned off the disco lights completely, so now there was the opposite problem…
Some searching located a module which controlled (most of) the settings, which I now use. So, problem solved… The keyboard is now a (soft (dim)) green, which is not distracting yet also helpful (even though I am a touch-typist (a nice thing about this laptop / keyboard is you can really pound it, useful for people who like myself learned to type on manual typewriters)).
I hear BYOD is all the rage: they should have just given it a REST API accessible over the internal WiFi and let the pilots control it via an app running on their own smartphones.
Touchscreens. Combat aircraft. Wow. However waved that idea through hasn’t watched any of the later Star Trek (TNG, Voyager…) with their LCARS input thing: Basically every control and monitor was a prototypical touch screen with loads of coloured lights as buttons. Those also turned into black glass walls whenever the power went off. Only, this being TV, the shows got to cheat: Whenever it was convenient to the plot, people could still press the unpowered controls and they’d briefly come to life and do (or not do) something.* F-35 pilots won’t have that luxury. By the way, I’d be willing t bet the ejection lever is probably classic in style but connects via computer and electrnics to trigger the whole thing. Maybe there’s a hand crank to charge up the ejector like old-style military radios – or maybe only the deluxe version for an extra billions gets that sort of kit.
Oh yes, whenever the spaceships on Star Trek would be tossed about to the point where crewmen went flying out of their seats and had to laborious crawl and claw their way back they would never struggle to hit the right buttons on their flat, smooth consoles. Maybe the designers did watch Star Trek and thought, if they can do it, our pilots can do it, too.
As for the total customisability, there had better be a big and very prominent reset button for that. I’ve joked before about F-35s coming back from maintenance in Turkey set to Turkish because the technician forgot to switch it back. Now, to “being unable to find the language setting because you can’t even read the alphabet”, add “they rearranged the GUI so you don’t know where anything is.” Pre-flight checklists should probably include point 0.: Find controls.
Incidentally, this whole customisability thing and the simple graphics (nothing wrong with that) immediately made me think of Dwarf Fortress, which I think at least Marcus is familiar with: Incredibly detailed and great fun if you can figure it out. Dwarf Fortress doesn’t have a learning curve but a cliff with an overhang, which I’m sure is true for jet fighters, too.
—
“”But the problem with touchscreens, the pilot explains, is a lack of tactile feedback. Switches have a nice, satisfying click that instantaneously lets the user know they were successfully flipped.””
If only we had known…
I heartily secnd any and all detractors of touchscreen and -button. The lacking feedback, variable responsiveness and tendency – or even just the ability – to completely misunderstand their user should immediately disqualify them from any critical applications. Personally I think, it should also disqualify them from any applications that care about the user experience.
Apart from all the other problems already mentioned, they are irreperable. They work or they don’t and if they don’t there isn’t much you can do about it except hope that it starts working again. Ideally before the fuel runs dry and you really need that landing gear**
*Let’s face it: If you lost power on a starship that lugs around lots of antimatter and fusion reactors, you’d be dead, dead, dead. Therefore, no need to worry about unpowered control backups.
**I just checked and apparently it’s “iniş takımı” in Turkish. Just in case, you see. Also googling this the prime result included articles about a Turkish 737’s landing gear collapsing. Good omens all around.
That should be “whoever”, not “however”, right at the start and I should be in bed. However (not whoever), I’m always up for some F-35 bashing – how exciting! – no matter the hour. I’ll just issue a blanket apology (and pardon) for any other goofs I didn’t spot.
Another fine example of ignoring 75 or 80 years of research and experience, at the very least. A decent ergonomics/human factors analysis would probably have caught most or all of those problems in no time.
I think I remember Donald Norman describing how airline pilots used to board their aircraft armed with styrofoam cups, tape, and yellow sticky notes to make the plane safer.
Elon Musk claims Neuralink created a brain implant that a monkey is using ‘to play video games with his mind’
I, for one, welcome our simian cyborg overlords.