Writer Dorothy (D.C.) Fontana passed away this week at the age of 80. Fontana was a pioneering female screenwriter in the (then) male-dominated field of television. She started writing scripts for Star Trek, but then crafted great narratives for dozens of genre series for decades.
In celebration of her life, I decided to watch one of her Star Trek episodes. Many say he best episode she wrote is Journey to Babel, and I can’t disagree. But, my personal favorite was The Ultimate Computer. So that was what I revisited.
It was better than I remembered.
Not for all the reasons it’s acclaimed by Trek fans: the character work by Shatner and Kelley, the great guest turn by William Marshall as Richard Daystrom, the prescient notion of man being replaced by automation. No, what surprised me was how it was a how-to guide.
A how to guide on how NOT to apply AI to workplaces and workflows.
Yes, the premise is extreme – A scientist devises a computer that can better run a starship than a human captain, to allow humans to release their “full potential” and not risk their lives exploring strange new worlds. But the heart of the plot is about command, control, choice and humanity… How we should never lose ourselves to the machines we let be in charge.
As someone who has designed experiences that leverage artificial intelligence to enhance human decision making, the idea that a 1968 episode of Star Trek can contain such ideas is… Fascinating.
Here’s my thoughts on the “best practices” in AI design reflected in The Ultimate Computer.
Support human decision, don’t replace it
At one point the M-5 computer starts shutting down power to multiple decks, in order to more “logically” reroute that power to more effective means. While that may be the “right” decision in a near-abandoned Enterprise, the fact that this action was done without human agreement or consensus is an overreach.
As a technologist or a user experience professional, appreciate and live by one of the key principles of experience design: User control and freedom.
Ask for consent
A direct follow-up to the above. While an AI-powered Ultimate Computer may decide that certain decisions make the most logical sense, the decision is not their own. Ask the “user” (in this case, the captain of the Enterprise) if they agree. If not, accede to their wishes. The case for your decision may be rock solid, but be ready to take no for an answer.
Never take destructive action
The M-5 decided erroneously that an unmanned ore freighter was actually a threat, and blew said freighter up with photon torpedoes.
Oopsie.
Any AI needs to “decide” that when the only course of action is to destroy the threat that presents itself is to destroy said threat… then they shouldn’t do it.
(See also, Asimov’s Law of Robotics.)
When in doubt, do nothing
Future AI solution: Err on the side of caution, and when there is any calculated doubt, inaction is the best action. Being bold in the face of adversity is Captain Kirk’s job, not yours.
Communicate the “why”
Explain yourself, AI. This is one thing that M5 does quite well, when questions by its creator Daystrom. Providing context to the decision allows the people who engage with the technology to have context to interpret and understand the actions.
Except, don’t try and explain killing scores of Starfleet officers. Totes awkward.
Remove “Fight or Flight” from the programming
The big problem with the M-5 was that Daystrom had used his own brain “engrams” to program the M5 to be more like a human brain, To be more intelligent, responsive, and adapt.
Except… there’s one key aspect of the human experience that should never have been transferred. Fight or Flight.
An instinctive part of being human, Fight or Flight is something that we have deep in our psyche, and evolutionary left-over for when earlier humans were struggling for survival. This is something we sometimes still need to call upon, since even in the “utopia” future of Star Trek there is no perfect universe of peace and brotherhood.
AI can reset and reboot – People can’t.
Enhance people, don’t replace them
At one point, the M5 Computer is doing a superb job in the war games that Starfleet Command set up – so much so that a colleague of Kirk’s calls him “Captain Dunsel” – Dunsel being an old nautical term for a part that serves no useful purpose.
Which is the exact wrong way technology should be used.
When you look at stories regarding technology replacing jobs, I get frustrated. The first job I ever had was as a cashier and a stock boy at a drug store. I enjoyed interacting with customers and putting out seasonal stock. I still remember the moment of absolute joy when I opened up the case and built the display selling the VHS tapes of 1989’s Batman… And the excitement that customers showed when they discovered they could get the biggest movie of the year for only $25.
I’m happy that young people today still have the opportunity to do such jobs today, 30 years later. Though some companies have started using technology to push more and more ‘self-service” in retail. Me? I’m kind of a hypocrite on this, as I have taken full advantage of self-checkout more than a few times. That admission aside, I still think that the human, personal touch is always important.
Ironically, Daystrom defended the idea of replacing people with processors in the Star Trek episode.
Men no longer need die in space or on some alien world. Men can live and go on to achieve greater things than fact-finding and dying for galactic space, which is neither ours to give or to take. They can’t understand. We don’t want to destroy life, we want to save it.
Richard Daystrom, played by William Marshall
Living to our full potential is why I do user experience design, and when you build any AI-powered solutions, it should always be focused on unlocking people’s potentials by helping users make better decisions based on intelligent recommendations and insights. While Daystrom failed, at least his heart was in the right place.
Closing
Revisiting this Star Trek episode reminded me just how talented a writer D.C. Fontana was. While not every episode she wrote was gold, this one certainly was. All the actors have great character moments to play, and the plot dealt with issues that we are still dealing with today.
Thanks, Dorothy. You will be missed.