Edward Tufte and Interaction Design

Went to a full day course today here in NYC given by Edward Tufte; a legend in the field of information visualization. I've always enjoyed reading Tufte's work. His historical odysseys, writing style, thoughtfulness, and carefully crafted books should be required reading for everyone from business schools MBA-ers to CS to the arts to social sciences to journalists to design educations. His books are that good and that important. For me, the central theme in Tufte's work has been the integration of text and visuals, content and form, which has been an important source of inspiration for my own work.

I'm not sure what I had in mind for a one day course and I did enjoy most of it, but I left feeling a little short changed. Perhaps the course is mostly meant as a basic introduction to his work and not a deepening of it? His presentation style is also slow, quite likely knowingly so—deeply rooted in the 'old american male professor' genre—but still slow. I like that he's not using Powerpoint slides but instead builds up a story (slowly) around a couple of key images that he zooms in and out of (although to be honest some of his pictures looked a lot like slides!) I also really liked that he just jumped right into it, on the hour, without even the shortest "hello and welcome." Refreshing!

Tufte's work is hence utterly relevant to so many different areas and has impact and implications for even more fields. Yet, one of the areas I feel his work is relevant to but he hasn't quite grasped is my field—interaction design. He's hovering over a host of relevant topics here, for sure, but doesn't quite get the details right, which unfortunately for him is exactly what he keeps calling out other people on, so... 

First of all, his thinking in this space seems a bit old. For instance, Tufte kept referring to some Dell laptop where the scrollbars apparently covered 11% of the screen real estate. That's a relevant anecdote if the year is 2003, but not really in 2016. In a world of smartphone apps, retina displays, tablets, hamburger menus, world wild west, notifications, creative online typography, etc. etc., it would seem that there are so many other more recent examples of the same idea (i.e. badly designed interface elements that hide rather than promote content) that this rather archaic anecdote more serves to confuse than enlighten. There were a few others like this as well, including web designers confusing the short term memory magic number 7 +/-2 theory. I'm sure this has happened, but probably not in the last decade and certainly not a common occurrence anymore, if it ever was.

There are so many examples of Tufte's work that are still so relevant for today's web and app designers, so why not talk about these instead? He mentioned one in passing: that many sites today are conceived and designed to be responsive, i.e. aiming at providing an 'optimal reading experience' regardless of what device/resolution you use—in effect separating content from layout and function. I think it's fair to say that Tufte's collected works can be seen as a critique of this very relevant and timely design idea. So why not spend time on massacring this? I would.

Tufte also mentioned that he was one of maybe 10 people in the world who thinks theoretically about these issues. Again, he's been at it for a while and this was maybe, even probably, true at some point, but I also think it's a bit negligent towards what has happened in the field in the last 20-30 years. Folks from all kinds of (academic) disciplines are doing it now: such as Human-Computer Interaction (HCI), interaction design, philosophy of technology, and design research, as well as a host of non-academic thinkers utilizing blogs, internet-based magazines, professional conferences and workshops, etc.

Third, Tufte's only substantial idea (at least in the way it was framed at this talk) about interaction design echoed the Heideggerian notion of "not letting the interface get in the front of the content". This idea was popularized in HCI and interaction design by Winograd & Flores back in 1987 and even earlier than that in the philosophical field called philosophy of technology. This idea is one that I've drawn on heavily in my own work, among several other researchers and practitioners in interaction design.

Yet, the problem here seems to be the distinction between form and content. In all his work, Tufte shows that these go best together if they are considered hand in hand. 

Let's look at this in a bit more detail. One of Tufte's recurring rhetorical refrains is that you now have better tools at your disposal in your smartphone than those that you use at work and that we should all rise up and demand at least equally good tools for 'work'. That's fair, but what Tufte misses here is that this also means that there is a substantial overlap between "work" and "leisure"—or whatever we want to call it, i.e. when we're not actually 'working'. I think one of the fundamental shifts in interaction design over the last 15 years or so is that the computer now just isn't something we think about as a machine for work. It's so much more than that. We use our PCs, laptops, and smartphones to mindlessly scroll through Facebook, play games, pass time, buy stock, watch movies, find plumbers, stalk coworkers on LinkedIn, read a text, write novels, keep swiping left and only occasionally right, anonymously rant on online fora just because you can, check out new music, do some work stuff (mostly emails), create graphs for our kid's soccer team, pass time before I can get out of here, plan the holiday in Cape Cod, and so on and on and on. All using the same magical machine.

With this in mind, I think Tufte's explicit notion that the primary role of interaction design is to make the interface disappear in favor of content is still a relevant perspective in many ways. However, it also implicitly suggests a rather old-school, work oriented perspective lurking underneath—which is that the user's only goal of using a computer or a smartphone is to get to the 'content' that their interfaces are hiding. It often is, but not always. Such a view is not enough to understand what interaction design is today. In what I have called the 'third wave of HCI', we see for instance web sites and apps where the interface is knowingly designed to be unclear and fuzzy and it is the user's task (or fun, to be more precise) to figure it out. Here, the interface and the content blend into one—they become the same thing. The interaction itself becomes valuable and meaningful, not just the so-called 'content' that it is supposed to hide or show. Computer games have always had an element of this. What makes Flappy Birds irresistible is the interaction, not the content.

At the end of the day, literally, I left Tufte's talk not bedazzled but yet hopeful. Tufte's thinking is still relevant for interaction design, it might just require someone with more detailed knowledge about the area to be able to interpret, see, and further develop its significance. One potential path is the current interest in digital assistants such as Siri, Alexa, Cortana, and 'Ok Google'. Applying Tufte's information visualization principles to these would probably reveal quite a few design obstacles to overcome in the next couple of years.

That said, I ended up taking a lot of notes and did get to doodle a bit too. This one, for instance, I call "A Bear with Many Faces" (yes, of course I name my doodles!)

Unapologetic Interfaces

A while back, I wrote a short piece about something I called 'apologetic interfaces', suggesting a new class of interfaces that pay attention to what their users are up to, what they're there to achieve, and seek ways of minimizing the hassle of dealing with unnecessary application maintenance, inclusive of updates, new feature tutorials, notifications, invitations to rate the app, etc.—you know all that stuff that drives us mad when all we're trying to do is to get stuff done.

I firmly believe that apologetic interfaces are the future. We need interfaces that realize that most of them are just that, interfaces. They are conceptually, factually, and by definition, in between us and our work. We need interfaces that realize that when I open up Microsoft Word I do that because I have a sudden need to write something down. Unless there's an earthquake, tsunami, major conflict, or a sudden outbreak of ebola in my area—I don't want to be be bothered with whatever-it-is. Just open the &$%&@# document so I can start to type. Please.

The state of the art, unfortunately, is still quite the opposite—the unapologetic interfaces rule, across platforms and devices. Notifications, update requests, badge icons, embedded tutorials, rating invitations, 'did you know?', and so on indefinitely, are still doing all they can do divert their users' attention from whatever they were trying to get done and paying absolutely no attention what so ever to what the user is doing at the time.

Here's a very telling example. Yes, it's the Wild West. Yes, it needs to change.

The Fishtank: An Agitational Artifact

For our client ABB Corporate Research, we created a series of alternative designs to contrast the traditional user interface and interaction design of control systems for industrial application. The Fishtank was one of the incarnations of this series. It is an interactive design exploration in the area of industrial control systems.

Conventional industrial control systems, such as ABB’s system 800xA, present the user with a panel view where machines, faceplates, sensor data, labels, etc. are organized and visualized side by side in a two-dimensional space. This design idea echoes the way in which control panels have always been designed; evolving from a non-digital era when each button, lever, label, and output device was physical and thus needed physical real-estate and a fixed location on the panel. Over the years, “the panel” as a way of framing and thinking about control room systems has formed a very strong conceptual idea for control room systems.

This is true to this day, when—at least in theory—a digitalized, computer-based control system could have any kind of user interface. Obviously, the 2D panel has not stayed on because it is a bad idea—on the contrary, there are many benefits to separating different things in two dimensions and giving them a fixed physical location in space. 

However, in this project, we wanted to explore the design space of "the possible" in this area by creating a series of radically different designs. The purpose was not necessarily that the results would aim to replace the traditional control room panel, but rather that they in different ways could come to complement, be different from, and to some extent challenge the panel as a design idea.

A typical problem in modern control rooms is the ever-expanding number of sensors that call for the operators’ attention. Relying on the quasi-physical panel as a design idea, it means that a 2D view of a factory keeps getting larger and larger. To deal with this, you either add more screens to the control room or you let the operators only see a small part of the entire factory on their personal screens.

As an alternative to this, we asked: would it be possible to design an interface in which the panel for the entire factory could fit on only one screen? 

The result from this experiment is the Fishtank prototype. It is an example of what we call an “agitational artifact”, i.e. an interactive artifact ideated, designed, and prototyped to be used using real data in real time—but where the main purpose of the artifact is to allow people to be exposed to a hands-on alternative to what they are used to; something with enough of a critical edge to shake them up a little bit, to make them think.

The Fishtank presents the user with a three-dimensional space. In this 3D space, the entire factory resides in the form of all its faceplates. A faceplate can for instance be a representation of a water tank in the form of the name and ID of the tank and its corresponding sensor data, such as water level, temperature of the water, etc.

The three dimensions in the Fishtank, i.e. X, Y, and Z space, are conceptual dimensions that can be controlled by the user. Hence, the user can decide what each of the three dimensions should represent.

For instance, the Y dimension can be made to represent the number of alarms a particular faceplate has; the X axis can be made to represent time since the last alarm; and the Z axis how far from the ideal or threshold each faceplate’s main value is.

But these conceptual dimensions can be changed easily and in real time to allow the user to interact with and play around with the factory to just monitor or to make certain parts stand out.

Unlike a traditional 2D design, the Fishtank uses movement, interaction, and conceptual dimensions—not fixed location in physical space—as the main sense making vehicle for the user. As such, it is radically different from the way in which control room software such as ABB’s 800xA has evolved and provides the user with a very different, engaging, and fun user experience. 

While an interactive artifact should be experienced hands-on, the video below gives you an idea what using this system is like.