What even IS this? Why tech companies are still failing us

Why do we know so little about the social implications of technology? It plays a starring role in everyday life, as essential as food, shelter, and clothing. A huge share (70%) of Americans use social media, and even 65% of senior citizens use Facebook  – that’s more than the number of people who eat family dinner at home, attend church, or have a pet.

Yet, we know so little about technology’s impact on everyday life. We are only just now recognizing problems like coordinated disinformation, breaches of personal data, and algorithmic discrimination.  Clearly, technology companies are falling short on understanding the social implications of their tools before and after they build them. But why? Why are tech companies failing us?

Woman in kitchen.
Woman in kitchen. Source: Art Institute of Chicago

Sadly, it is all too predictable that technologists underestimate, misjudge, or otherwise underappreciate how humans will interact with their technology. This is for one simple reason: engineering, as a discipline, does not bother to ask:  “What is this?”

Engineers are not scientists, much less social scientists. They typically have no knowledge of basic human behavior such as loss aversion or impression management, even though these are the building blocks of social interaction – and entry-level knowledge for social scientists.

Engineers could ask, “What is this?” but instead choose to ask: “Does this work?”

“Does this work?” underpins research within tech companies. Once upon a time, tech companies hired engineers they called research scientists and stuck them in labs to tinker endlessly with pieces of hardware and scraps of computer code. Even today, there are over 7800 job postings for “research scientist” on LinkedIn, which are typically engineers or computer scientists. A posting for an Uber research scientist intern is instructive. In addition to having a Master’s degree in a “technical field,” the intern is also encouraged to engage in “risk taking” and to “turn the dreams of science fiction into reality.”  Another job posting for a research scientist at Facebook asks for skills in the scientific method, but then specifically narrows that down to “evaluate performance and de-bug.” In other words: Does this work? Notably not mentioned: the ability to develop basic knowledge.

Academics would see much of this activity as more akin to prototyping than to scientific inquiry. Indeed, these engineers produced many technology prototypes, but not much in the way of generally applicable knowledge, or what the rest of us might call “science.” In other words, they never seem to stop and ask, “What is this?”

Painting by Salvador Dali
Inventions of The Monsters

Today, tech companies need to ask things like “What is a digital public sphere?” and “What is the nature of privacy?” and “What is artificial intelligence versus human intelligence?” Tech companies need typologies of human-computer interactions, motivations, fears, and human foibles. They need to create a system of knowledge around key questions of technology like artificial intelligence and social media.

Some argue that technology development doesn’t have time for “understanding,” that asking “What is this” takes too long and is too expensive. But this is a false economy. Philosopher Martha Nussbaum tells us plainly that we need that understanding, not for understanding’s sake but because it guides our planning:

“Understanding is always practical, since without it action is bound to be unfocused and ad hoc.” — Martha Nussbaum

In other words, if you don’t know “What is this” you’re probably going to build the wrong thing.

We can see this pattern of building the wrong thing in technology, over and over again. The term “user friendlywas invented way back in 1972. Curiously, “user hostile” wasn’t invented until 1996, just before Microsoft’s infamous Clippy appeared in 1997. Clippy’s abrupt entre onto the desktops of the world indicated that technology “researchers” had no idea what they had made. Word famously exploded from what appeared to be a digital typewriter, to a swollen behemoth that did everything from create a newsletter to automate mailing labels. Pick a lane, people. Clippy was there to tell users how to make Microsoft Word work, but no one bothered to find out much less explain what Microsoft Word actually was.  Word is still so swollen that a new user today can credibly ask “What even IS this?”

Clippy the paperclip
Source: NYMag.com

Flash forward to today, and the so-called “lean startup” approach to building technology is really just a faster, even more facile way to ask “Does this work.” In reality, tech companies still don’t know, “What is this?” even after they’ve built a working prototype.

In my former role as a hiring manager at a major tech company, it took an average of 100 days to hire just one ethnographer and more often than not, the job remained open much longer than that. These are the very people who can tell us, “What is this?” The demand for these social scientists only grows. Yet, the tech industry as a whole has not yet figured out they need to ask “What is this?” before they build something.

Were tech companies to ask, “what is this,” they would learn the basic properties of their tools, their coherence, intelligibility, performance, and affordances. Instead, they are fully occupied with “does this work,” and create horrific blights on our collective consciousness like Tay, the racist AI Bot on the relatively innocuous end of the scale, and Compass, the racist parole algorithm at the full-on evil end of the scale.

Technologists do not know what they do not know. Ethnographers hope for the day when they can just ask “What is this” without worrying about whether it works, because it doesn’t even exist yet. But tech development continues apace.

It’s time for ethnographers to stop this sad venture, and instead insist on asking: What IS this? Before another Tay, before another Compass. Technologists too must take responsibility because if we don’t, the 21st century will become even more technocentric, and even less intelligible. Let’s find out what’s going on before we build anything else.

The future of work: A designer’s reading list

Recently, I was asked to put together a list of reading for design students. These students are about to design something for emerging productivity needs. I looked through my academic database, I realized that much of what I’ve read is probably not appropriate for designers.

I mean, show me a designer who wants to read Marx, Weber, and Durkheim, and I’ll show you a budding sociologist!

So I changed tack. Below is the rough, popular reading list I came up with. These articles are no substitute for a robust reading of the research on work and technology, but it does reflect some of the major debates in the field.

It also is ripe for design solutions. There is tension. There are problems to solve. What problems would you solve, if you were designing for the future of work?

Understanding work in the 20th century: from hierarchy to what?

Stewart, Matthew. 2006. “The Management Myth: Most of management theory is inane, writes our correspondent, the founder of a consulting firm. If you want to success in business, don’t get an MBA. Study philosophy instead.” The Atlantic. http://www.theatlantic.com/magazine/archive/2006/06/the-management-myth/304883/

Work and technology: overworked and underproductive

Bogost, Ian. 2013. “Hyperemployment, or the exhausting work of the technology user.” The Atlantic. http://www.theatlantic.com/technology/archive/2013/11/hyperemployment-or-the-exhausting-work-of-the-technology-user/281149/

Madden, Mary, and Sydney Jones. 2008. “Networked Workers”. Pew Internet and American Life: Washington DC. http://www.pewinternet.org/2008/09/24/networked-workers/

Work and Home: blurring the boundaries

Meece, Mickey. “Who’s the boss? You or your gadget?” The New York Times. http://www.nytimes.com/2011/02/06/business/06limits.html?pagewanted=all

Duxbury, Linda, and Rob Smart. 2011. “The ‘Myth of Separate Worlds’: An Exploration of How Mobile Technology Has Redefined Work-Life Balance.” In Creating Balance? International Perspectives on the Work-Life Integration of Professionals, edited by Stephan Kaiser, Max Josef Ringlstetter, Doris Ruth Eikhof, and Miguel Pina e Cunha, 269–284. Heidelberg: Springer-Verlag.

The Aging Workforce: time bomb or blessing?

Thompson, Derek. 2012. “Gray Nation: The very real economic dangers of an aging America.” The Atlantic. http://www.theatlantic.com/business/archive/2012/03/gray-nation-the-very-real-economic-dangers-of-an-aging-america/254937/

The End of The Job: what happens when no one has a job?

Wishnia, Steve. 2012. “Temp Worker Nation: If you do get hired, it might not be for long.” Alternet. http://www.alternet.org/labor/temp-worker-nation-if-you-do-get-hired-it-might-not-be-long

The Economist. 2014. “The Onrushing Wave.” http://www.economist.com/news/briefing/21594264-previous-technological-innovation-has-always-delivered-more-long-run-employment-not-less

MBO Partners. 2011. “The State of Independence in America”. Herndon, Virginia. http://www.mbopartners.com/state-of-independence/independent-workforce-index.html

Electronic medical records and interation design

A recent study on electronic medical records (EMRs) found that they may not fulfill the promise of lowered health-care costs. This  study, and the reaction to it, illustrates much of what is wrong with technology studies, and the unintended social effects of technology itself.

Many technology studies have false ideas of how web and interaction designers actually work. We collectively tend to think of technology as a “fix” that “automagically” eliminates  “waste,” even if this is not the intent of the designers themselves (which it frequently isn’t).  But as this study points out, there are far more subtle and nuanced issues relating to technology. Specifically, technology makes it easier to do some things. Is it any surprise we end up expecting more things to be done?

Let me illustrate with EMRs.

Image licensed under Creative Commons to MC4 Army on Flickr

Researchers from Harvard Medical School found that the use of electronic medical records (EMRs) is actually correlated with a higher number of diagnostic tests, such as MRIs, which in turn implies higher — not lower — health-care costs.

The authors suggest:

These findings raise the possibility that, as currently implemented, electronic access does not decrease test ordering in the office setting and may even increase it, possibly because of system features that are enticements to ordering.

This study was a quantitative analysis of medical records so did not offer any insight into why there are more tests ordered with EMRs. The authors can only speculate that the easy availability of imaging results translates into more tests being ordered. The “enticements” to order more tests could be built into the EMR systems themselves.

Anyone who has worked in interaction design will tell you that “enticement” is precisely the kind of emotion they want their users to feel. Take, for example, Stephen Anderson’s research on on “emotional design.” Anderson argues that web and application design should be “seductive” to really be successful. Trevor van Gorp also argues that designers should be aspiring to connect “affectively” with their users, and to tap into deeply held emotional experiences.

Both Anderson and van Gorp have written and spoken extensively to the user experience designer. Their ideas are au courant in the web and application design community. It is likely that some members of that community have read van Gorp’s book or use Anderson’s psychology inspired “mental note” cards in their design practice. It is just as likely that some of these people have designed the very EMR systems that strive to, surprise surprise,  “entice” physicians to order and view diagnostic tests.

Physicians are responding to a design philosophy, which is to extract from users a deep engagement. “Good” interaction design is usable, but also engaging. Instead of boring users, contemporary web and application designers are “seducing” them. Indeed, good interaction design, according to industry leader the Nielsen Norman Group, includes the principle of “explorable interfaces.” How is it any surprise at all that physicians are “exploring the interface” by ordering more tests? Good systems are designed to entice them to do exactly that.

In their story covering the Harvard study’s findings, the New York Times reports that other researchers disagree with the conclusions. The Times reports:

Dr. David J. Brailer, who was the national coordinator for health information technology in the administration of George W. Bush, said he was unconvinced by the study’s conclusions because they were based on a correlation in the data and were not the result of a controlled test.

Dr. Brailer doubts the conclusions because he does not understand how design is currently practiced, nor does he have direct input into the design principles of EMRs. If EMRs are being designed according to current ideas, they are designed not to save money, as Dr. Brailer hopes, but to entice users to explore and be engaged. Dr. Brailer clings to his scientific method here, and rightly points out that correlation does not equal causation. However, the Harvard researchers are more in tune with current design practices.

I don’t mean to knock interaction designers. Heck, some of my best friends are interaction designers! No, really. What I’m saying here is that designers design to principles. A laudable principle is to “seduce” or emotionally affect the user. This principle creates great systems. But it results in more use of systems, not less. It should come as no surprise that imaging tests represent a “seduction” for physicians, who, like all scientists, are voracious consumers of “more data.”

This case study reminds me of how we so frequently miss the mark in understanding technology. We assume it will be “efficient,” without asking how it might actually work. Worse, we routinely ignore the normative shifts that come along with cheaper and easier labour. Take, for example, house-keeping technology. We believed that the vacuum cleaner, the washing machine and the dishwasher would lead to more leisure time. What it actually lead to was higher standards of cleanliness.

Will EMRs lead to “higher standards” of imaging desire among physicians? Perhaps they already have.