Why a New Golden Age for UI Design Is Around the Corner

Designers are working on their most ambitious challenge yet: weaving the digital world into our everyday lives so seamlessly that we don’t even notice.
ffexperienceDesign1025
Andrew B. Myers

Over the past four years, the Walt Disney Company has been engaged in a secretive effort to redesign the Disney World experience. It’ll go like this: You buy your ticket online and plan all the details of your visit. Then you’ll get a wristband in the mail, which will be a passport to the experience that you’ve curated. Snug around your wrist, the so-called MagicBand will use radio frequency to communicate with sensors around the park, all orchestrated by software that effectively turns Disney World into a computer interface. You can enter the park by holding your hand up to a kiosk; you can arrive at shows with 30 seconds to spare, having already reserved your seats; you can jump onto rides you’ve selected at preselected times without waiting in long lines; you can buy anything you want with a wave. An It’s a Small World character could call you by name and wish you happy birthday. So could Mickey, who can also greet you at a preselected meeting time. This is all in the service of fun, of course, but it is also a glimpse of the future: an integrated experience, a smooth hybrid of real-world and digital interactions.

This represents a new frontier for design. Over the past 30 years, as every facet of our lives, from our shopping to our schooling, has migrated onto computer screens, designers have focused on perfecting user interfaces—placing a button in just the right place for a camera trigger or collapsing the entire payment process into a series of swipes and taps. But in the coming era of ubiquitous sensors and miniaturized mobile computing, our digital interactions won’t take place simply on screens. As the new Disney World suggests, they will happen all around us, constantly, as we go about our day. Designers will be creating not products or interfaces but experiences, a million invisible transactions.

We are constantly adding new gadgets. Even as they have gotten simpler individually, the cumulative complexity of all of them together is increasing.

Already we’re seeing a groundswell of new products that insinuate themselves seamlessly into users’ lives. These include personal sensors like Jawbone’s Up, the voice- and gesture-controlled Xbox One, hyper-intelligent apps like Highlight that alert you to interesting people in your immediate vicinity, and Automatic, a gadget that communicates with your smartphone to tell you when you are driving inefficiently. But this is just the beginning. Within the next five years we will be surrounded by embedded devices and services. Just as the rise of the screen challenged designers to create software interfaces, the rise of screenless digital interactions will challenge them anew. After all, it’s one thing to invent a unique kind of digital experience in Disney World, a controlled space where people expect magic. It’s altogether trickier to do the same thing in people’s houses, offices, and bedrooms—the most intimate areas of their lives—in a way that feels both natural and inevitable.

Bill Buxton bears a striking resemblance to Doc Brown from Back to the Future—more strapping than you’d expect for a mad scientist, his bald head rimmed with a snowy hedge of hair. In conversation, he can be piercingly intense. And just like Doc Brown, in 1985 he unleashed a breakthrough. Buxton, a lifelong musician who has also worked for Xerox PARC and Silicon Graphics, created one of the world’s first multitouch interfaces when he turned an electronic drumhead into a tactile synthesizer control. That drum was a progenitor of every touchscreen in use today.

ff_experienceDesign3_1000 Andrew B. Myers In the mid-aughts, Buxton wrote a journal article that helped define a new discipline called experience design—a focus not on products or devices themselves but on the impact they have on people’s lives.As an example, he wrote about two orange juice presses—an electric model and a manual lever press called the OrangeX. The electric juicer had flimsy plastic buttons, and the motor screeched to life with an annoying whir. The OrangeX required a bit more effort but also sported an inverted rocker crank that gradually transmitted more force as you pressed down. Buxton’s point was that the OrangeX created a feeling of tangible mastery that helped him enjoy the juice more. Designers didn’t shape just gadgets but behaviors and visceral responses around those gadgets.

Microsoft Research’s Bill Buxton, holding the GRiDPad 1910, an early tablet computer. Eric Ray Davidson

Matt Webb, CEO of London design firm Berg, is inventing a system that will allow multiple users to share smart connected devices. Levon Biss

Today, Buxton, who is principal researcher at Microsoft Research, says that the next challenge for experience design is to create a constellation of devices, including wearable gadgets, tablets, phones, and smart appliances, that can coordinate with one another and adapt to users’ changing needs. This focus on the totality of our devices stands in contrast to where we find ourselves today: constantly adding new gadgets and functions without much thought as to how they fit together. (For instance, anyone lugging around a laptop, iPad, and iPhone is also carrying the equivalent of three video cameras, three email devices, three media players, and probably three different photo albums.) Even as our devices have individually gotten simpler, the cumulative complexity of all of them is increasing. Buxton has said that the solution is to “stop focusing on the individual objects as islands.” He has come up with a simple standard for whether a gadget should even exist: Each new device should reduce the complexity of the system and increase the value of everything else in the ecosystem.

To see what he means by increasing the value of the ecosystem, consider the phone syncing built into many cars. After you link your phone, the vehicle boots up its own voice-recognition technology so you can make hands-free calls. When you leave the car, you simply grab your phone and it blinks to life again. The car and the phone engage in a quiet dialog geared toward providing only the capabilities you actually need at any moment.

If all our devices interacted so cooperatively, whole new possibilities would begin to emerge. For example, Frog, the company best known for the Apple IIc’s industrial design of the early 1980s, has been building a prototype lightbulb that will sense where people are in a room and project touchscreens onto walls or tables. Now imagine if a device like that could communicate with your mobile gadgets—if the lightbulb, sensing your presence in the kitchen and knowing the apps on your phone, projected your cooking apps onto the refrigerator when you began preparing dinner.

ff_experienceDesign4_1000 Andrew B. Myers To deal with such complexity, our devices will have to become smarter. Dave Morin, CEO of mobile social-networking company Path, has a maxim to explain how to think about the coming age of experience design: “AI is the new UI.” That is, the effort and attention that designers once poured into interfaces should be extended to code that doesn’t just react to the push of a button but anticipates your actions. For instance, Path will automatically update your location when it senses that you’ve settled somewhere new. But that’s really just a proof of concept. Morin’s maxim hints at the silent conversations that our phones and wearable devices will have with the world around us and each other. For example, Apple’s new mobile operating system, which uses Bluetooth Smart to share data with devices in your vicinity, could power a number of these kinds of intelligent background features.

Innovations like this present great challenges for designers. Today’s app and software designers already have a deep understanding of how customers interact with their products. They know down to the pixel where to place a button, how fast a screen should scroll, and how to make an app simple without making it simplistic. But as designers move off of screens and into the larger world, they’ll need to consider every nuance of our everyday activity and understand human behavior every bit as well as novelists or filmmakers. (Otherwise they may engender the same kind of backlash as Google Glass, a potentially cool product that has unleashed a torrent of privacy concerns.)

That will require a shift in how tech designers view the world. Matt Webb, CEO of Berg, a London design firm that has created forward-looking prototypes for clients such as the BBC, Google, and Nokia, says it will demand thinking beyond today’s standard scenario of a person working on a computer. For example, he says, “our technology can’t understand what it means to be in a room of two to six people. I find it totally nuts that when you sign in to something, no one else can use it. Imagine having to sign in to a lightbulb before turning it on!”

Berg is trying to solve that problem by inventing a system that will allow multiple users to share smart, connected devices that can adjust to their individual tastes and preferences. “That’s the world we actually live in,” says Jack Schulze, Berg’s cofounder. “But it’s a massive challenge for software.”

Without the proper design, any new technology can be terrifying. The task of making it can’t be left to engineers alone.

Just consider how these challenges apply to Netflix: If your spouse watches something on your account, it probably renders the company’s super-sophisticated recommendation engine worthless. Netflix is trying to address this problem by creating a feature allowing multiple user profiles on one account. “But even that’s the wrong solution,” Webb says. “When we watch TV together, that group isn’t just multiple people added together. That group is something more.” It’s easy to imagine a smarter, future version of Netflix—one that uses, say, an Xbox Kinect camera to recognize who’s in the room and can determine everyone’s overlapping interests.

The true potential of experience design comes as that kind of sophistication gets applied to all of our interactions. “We have all of these incredible benefits to our online life,” says Jake Barton, founder of Local Projects, a media design firm, “and they’re suddenly being applied to physical space.” Take the example of Warby Parker, the online eyewear retailer that has begun opening retail storefronts. Imagine if the store automatically brought up your online profile—which you’ve already filled with your favorite styles—allowing you to focus only on the glasses that the company’s computers know you already like. Or look at Nespresso, whose physical stores offer its customers an RFID card that allows automatic billing and personalized service based on purchase history. Barton believes that the next step will be to create a universal, portable electronic identity that allows all of our experiences to be that customized. Companies like digital-payment pioneer Square are already working to build just that.

In the wrong hands, this is a dystopian prospect—technology’s unwanted intrusion into our every waking moment. But without the proper design, without considering how new products and services fit into people’s day-to-day lives, any new technology can be terrifying. That’s where the challenge comes in. The task of making this new world can’t be left up to engineers and technologists alone—otherwise we will find ourselves overrun with amazing capabilities that people refuse to take advantage of. Designers, who’ve always been adept at watching and responding to our needs, must bring to bear a better understanding of how people actually live. It’s up to them to make this new world feel like something we’ve always wanted and a natural extension of what we already have.

Constant Connections

Designers are adding invisible digital interactions that will optimize our real-world experiences. Here are some early examples. —C.K.

Automatic

A device that plugs into a car’s dataport and chimes to warn users when they are driving inefficiently.

Bluetooth Smart

A feature of iOS 7 that allows Bluetooth devices to update without user input.

GlowCap

A wireless pill-bottle cap that tracks medication consumption and sends updates to caregivers.

Highlight

An app that connects people near one another who share tastes and interests.

London City Airport

A sensor network, currently in development, that will present personalized travel information as users move through the terminal.

MagicBand

A radio-equipped bracelet that communicates with sensors at Disney World, which will let visitors skip long lines and have personalized meetings with characters.

Nest

A thermostat that uses motion sensors to automatically adjust temperatures according to users’ behavior.

Square Wallet

A mobile app that allows customers to pay by giving their name.

Tagstand

Uses programmable wireless patches to launch an app or turn on a Kindle when a user draws near.

Xbox One

Sensors allow users to control their TVs—and soon other devices—through gestures and voice commands.