What we know about gender and performance reviews

Update December 23, 2017: New findings from an AI-based experiment on gender and performance reviews has been added, and a new evidence-based improvement strategy called the “small wins” model.

The evidence is mounting that our existing feedback systems have inherent biases that penalize women. I have a personal commitment to help improve this, and in this post, I make specific recommendations on how to do so. But first, what do we know about gender and performance reviews?

  • Women receive more criticisms of their personalities in performance reviews: A linguist did study on performance reviews. Men and women provided positive performance reviews. She found that in 83 performance reviews, men received personality feedback in only 2 cases. In 94 critical reviews, women received personality criticisms 71 times. [1] Words like “abrasive” and “strident” were present in these performance reviews.
  • Women receive less helpful feedback than men. In a study of 200 performance reviews in a tech company, researchers found that women were more likely to receive vague praise than were men (57% and 43%, respectively), which included unhelpful comments like “You had a great year.” Men were more likely to receive developmental feedback, and linked specifically to business outcomes. When women did receive developmental feedback, it tended to relate to their personalities rather than to their performance [2]
  • Women are less likely to be rewarded for good ideas. Men who provide revenue-generating ideas are given higher performance ratings. Women who provide revenue-generating ideas see no improvement in their ratings. [3]
  • Men reward other men more highly than women for achieving the same goals: 70% of men rate men more highly for achieving the same goals as women, while an algorithm rate men and women equally (as did other women).
  • The “glass ceiling” is the result of many tiny obstacles. Researchers at Harvard Business School found that there is no specific point where women face a “ceiling,” but many small instances of discrimination lead to their careers stalling over time. [4] This may be why after only 2 years with a company, women’s aspirations for career advancement fall an astounding 60%, while men’s aspirations fall negligibly.
  • Ostensibly “meritocratic” reward systems favor men over women, and whites over minorities. Researchers experimentally tested whether managers would reward people differently when explicitly creating a system of “merit.” Over 3 experiments and 445 participants, they found men were rewarded with more money than women in this supposed “meritocratic” system. They also found that ethnic minorities and non-American born people were given lower raises, even when using the same evaluative criteria. [5] In another experiment, researchers switched male and female professors of an online course in the middle of the term. Students consistently rated the male professor higher, even though they actually had a female professor without knowing it. [6]
  • Women are penalized for asking for raises: Researchers found that people judge women more harshly when they ask for a raise; women don’t ask for raises because they realistically assess the social cost of asking. [7]
  • Men are rated more highly for helping colleagues, and women are rated more negatively for not helping. In a series of experiments, researchers asked participants to rate the performance of men and women who either agreed to stay late to help colleagues, or refused to stay late and help. Men who offered to stay were rated 14% more positively (women’s rating remained the same). Women who refused to stay were rated 12% more negatively (men were not rated more negatively). [8]
  • Women pay a penalty for motherhood, while men reap a bonus for fatherhood: Researchers have found women who become mothers pay a minimum penalty of 4% decline in income. [9] The penalty is larger for more educated women. [5] By contrast, fathers reap a bonus [11]. This is not due to a lack of commitment by women, by biased perceptions of their commitment. [10]  In other words, the cost of being a parent depends on your gender, not your performance.
  • Men are penalized when they ask for family accommodation. In a study of a management consultancy, one researcher found that men who ask for flexibility to care for their families are punished in performance reviews. [12] Men who did not openly ask but made private, covert arrangements got better performance reviews.
  • Keeping track improves fairness. Just keeping track of how people get rewarded, broken down by race and gender, was enough to reduce inequality over 5 years in a single company. So know your data! How are people doing relative to each other? [5]
  • Just pointing out bias actually increases its incidence! Other research has found that pointing out bias actually increases its negative consequences. [13] This effect disappeared when researchers noted that discrimination is not desirable.
  • Training managers on potential work/life conflicts decreases employee stress. Researchers trained a single company’s managers on how to deal with work/life conflict. They found reduced employee stress, and no increase in employee hours. [14]

 

Opportunities for Improvement

  1. Examine the words you use in your performance feedback. Is it related to personality or performance? Are the words very gendered, such as “bossy”? Consider what words you might have used to describe the opposite gender.
  2. Is your feedback helpful and specific? Did you provide vague praise like “Great job this year” or did you say, “Your work on the launch plan led to greater sales”? Make sure you link the feedback to specific business goals.
  3. Did you reward fairly? Is there a systematic difference between the genders? Could this be unconscious bias?
  4. Is this a “tiny obstacle”? How many barriers has your direct report experienced in her career? Is this performance review a chance to create a “tiny ladder” through the glass ceiling?
  5. Do you expect women to be more altruistic? Reward men and women equally for the same behaviors. Consider if you expect women to be more giving of their time than men.
  6. Do you expect men to be less family-oriented? Recognize that men have families too. Are you penalizing a man for being a caring father?
  7. Is your direct report asking for something…and are you evaluating that ask fairly? Consider what kinds of requests your direct report has made. Are you judging those requests fairly, or are you penalizing the person for speaking up?
  8. Unconscious bias is wrong. Make sure you point out that bias is wrong, not just that it exists. Norms are powerful, especially for senior leaders.
  9. Keeping track improves fairness. Just keeping track of how people get rewarded, broken down by race and gender, was enough to reduce inequality over 5 years in a single company. So know your data! How are people doing relative to each other? [5]
  10. Pursue “small wins“: researchers at Stanford found that they could improve outcomes if they worked directly with managers on reducing bias. Introducing a new scorecard reduced personality-based feedback to zero.  [15]

 

 

 

References

[1]       K. Snyder, “The Abrasiveness Trap: High Achieving Men and Women Are Described Differently in Reviews,” Fortune, New York, Aug-2014.

[2]       S. Correll and C. Simard, “Vague Feedback Is Holding Women Back,” Harvard Business Review, no. April, 2016.

[3]       A. Grant, “Rocking the Boat but Keeping It Steady: The Role of Emotion Regulation in Employee Voice,” Academy of Management Journal, vol. 56, no. 6, 2013.

[4]       A. Eagly and L. Carli, “Women and the labyrinth of leadership,” Harvard Business Review, no. September, pp. 62–71, 2007.

[5]       E. J. Castilla and S. Benard, “The Paradox of Meritocracy in Organizations,” Administrative Science Quarterly, vol. 55, no. 4, pp. 543–576, Dec. 2010.

[6]       L. MacNell, A. Driscoll, and A. N. Hunt, “What’s in a Name: Exposing Gender Bias in Student Ratings of Teaching,” Innovative Higher Education, vol. 40, no. 4, pp. 291–303, 2015.

[7]       H. R. Bowles, L. Babcock, and L. Lai, “Social incentives for gender differences in the propensity to initiate negotiations: Sometimes it does hurt to ask,” Organizational Behavior and Human Decision Processes, vol. 103, no. 1, pp. 84–103, May 2007.

[8]       M. Heilman and J. Chen, “Same Behavior, Different Consequences: Reactions to Men’s and Women’s Altruistic Citizenship Behavior,” Journal of Applied Psychology, vol. 90, no. 3, pp. 431–441, 2005.

[9]       T. Street, A. Arbor, and P. O. Box, “Has the Price of Motherhood Declined Over Time ? A Cross-Cohort Comparison of the Motherhood Wage Penalty,” Journal of Marriage and Family, vol. 65, no. August, pp. 597–607, 2003.

[10]     J. a. Kmec, “Are motherhood penalties and fatherhood bonuses warranted? Comparing pro-work behaviors and conditions of mothers, fathers, and non-parents,” Social Science Research, vol. 40, no. 2, pp. 444–459, Mar. 2011.

[11]     G. Hundley, “Male/Female Earnings Differences in Self-Employment: The Effects of Marriage, Children, and The Household Division of Labor,” Labor Relations Review, pp. 95–114, 2000.

[12]     E. Reid, “Embracing, Passing, Revealing, and the Ideal Worker Image: How People Navigate Expected and Experienced Professional Identities,” Organization Science, vol. 0, no. 0, p. null.

[13]     M. Duguid and M. Thomas-Hunt, “Condoning Stereotyping?: How Awareness of Stereotyping Prevalence Impacts Expression of Stereotypes,” Journal of Applied Psychology, no. October, 2014.

[14]     E. L. Kelly, P. Moen, J. M. Oakes, W. Fan, C. Okechukwu, K. D. Davis, L. B. Hammer, E. E. Kossek, R. B. King, G. C. Hanson, F. Mierzwa, and L. M. Casper, “Changing Work and Work-Family Conflict: Evidence from the Work, Family, and Health Network,” American Sociological Review, vol. 79, no. 3, pp. 485–516, May 2014.

[15]   Correll, S. J. (2017). SWS 2016 Feminist Lecture: Reducing Gender Biases In Modern Workplaces: A Small Wins Approach to Organizational Change. Gender & Society, 31(6), 725–750. https://doi.org/10.1177/0891243217738518

 

 

 

 

Mobile productivity: it ain’t about doing more

The primary unmet need for mobile productivity is managing the torrential onslaught of constant communication. Apps and tools that aim to help users “do more” are likely to be self-defeating. On the contrary, we need tools to help us do less.

In my last post, I pointed out that the real problem in productivity technology today is that users need ways to seamlessly share information across their cross-discipline teams.I noted that our collaboration tools ironically create more work. Likewise, many mobile productivity tools actually amplify this problem, by reaching users with the most useless notifications from the most tangential acquaintances, at any time of the day. We need to get smarter about what we deliver to mobile users, by properly managing push notifications, intelligently reading user priorities, and helping mobile workers stay focused on what’s important.

First a little context.

Smartphones are ubiquitous and deeply disruptive

Smartphones are now the majority of cell phones in the developed world. 58% of Americans , 55% of Canadians, and over 50% of people in Norway, Sweden, Denmark, the UK, and the Netherlands have smartphones. At first glance, we might be tempted to see this shift as just another type of phone. This shift from feature phones to smartphones represents a qualitatively different business landscape and a different set of behaviors.

The transition from cellphones to smartphones is not trivial; today’s smartphone has the same computing power as a laptop manufactured as recently as 2006.[*]  Smartphones are little computers, while feature phones are simply communication devices. This has clear business implications — as Mary Meeker’s famous operating systems chart shows us.

Mary-Meeker-slide-006

Smartphone Behavior Change

Smartphone growth also has deep implications for everyday behavior. We have rapidly become a society in which the majority of people have tiny computers with them at all times. The majority of people are now constantly receiving email, social media notifications, in addition to phone calls and texts. This means the average smartphone user is now reachable not just to his intimate friends and family, but to even the most casual acquaintance. With feature phones, a typical user could expect to be reachable by her partner, and potentially her boss, or her babysitter. Now she is reachable by an old work colleague, a high school friend, or even someone she has never met but who shares her interest in golf. Having dinner, driving home, or working out at the gym were once private affairs. They are now all susceptible to interruptions.

To see how far our communication practices have changed, consider the eeriness of BlackBerry messages emerging, just as the Twin Towers fell. The New York Times interviewed corporate lawyer Lynn Federman, as she recalled sending frantic messages to her husband as she escaped from the World Trade Center:

“I had my cellphone in one hand, and it was useless, and my BlackBerry in the other, and it was my lifeline that day,” Ms. Federman recalled.

At the time, only about 1 million BlackBerrys were in use, worldwide.

Imagine if the same event were to happen today. Millions of tweets would emerge within moments. By way of comparison, the 2014 World Cup final game alone generated 280 million Facebook interactions, and 618,725 tweet PER MINUTE during the game. Clearly mobile technology has already arrived in the workplace — what is this shift doing to productivity?

Granted, smartphone users can turn off notifications, but we have good evidence to suggest they don’t. 4 out of 5 smartphone users check their phones within 15 minutes of waking. The average person checks their smartphone 150 times a day. Researchers have found all that checking is usually related to “information rewards.” 

The transformation at work

Many of these people use their smartphones for work, regardless of where they are. Technology research company IDC estimates that 900 million workers, 35% of the global workforce, is a “mobile worker,” meaning that they use mobile technologies such as laptops, tablets, or smartphones, for work purposes at least occasionally. In the U.S., at least 72% of workers are mobile.  An estimated 174 million people use their smartphones for work purposes. 43% of executives report that they allow employees to work anywhere, on any device they choose and 44% are actively investing in mobile collaborative tools for their employees.

These new streams of information are shifting existing productivity practices. Consider the changes to email alone, which is now 40 years old. In 1997, a prominent scientist told famed “flow” researcher Mihaly Csikszentmihalyi that email was getting in the way of her productivity:

“On bad days, I have seventeen or twenty-four email messages.”

See how much has changed, looking at email alone. Users are clearly overwhelmed:

  • Morgan Stanley’s average employee receives 625 emails a week. Intel employees spend 20 hours per week just managing email [1]
  • Part of the problem with email is “waiting to hear back” [2]
  • Email doesn’t help people organize across multiple social “streams” [3]
  • Email has increased the size of the network of people that can communicate with a user, but to a point where ordinary users have a hard time keeping up [4]
  • Email may be “addicting” [5]
  • Email usage dropped by 5% from 69% of all users to 64% from 2007 to 2011 [6]

Clearly, productivity is changing, but we have very little insight into how and in what ways.

What is mobile productivity?

Mobile technology makes workers available, wherever and whenever. Researchers have found consistently that mobile technology makes people more available to workplace demands. Research on managers found that they are available to work demands on average 72 hours per week.   My own research has shown repeatedly that when workplaces have no policies around expected availability, “always available” becomes the norm. I found that among design workers, 44% reported being available to work demands, during the night while they slept!

Rethinking mobile productivity needs

So is “being available” really the most unmet need for mobile workers? Clearly, mobile productivity today means being able to manage the constant torrent of workplace, personal, and news information. Mixing all these streams together onto a single device makes it difficult for users to discern the importance of any one news item. It also trains workers to expect a constant flow of information, instead of taking regular breaks from the news vortex and actually spending time thinking.

At least one new app has found this need and is trying to solve for it. Appfluence attempts to help users separate the “important” from the merely urgent by keeping users focused on their self-defined priorities. Critically, Appfluence isn’t just an app, but integrates into the desktop and mobile spaces equally.

We need other tools that synthesize, minimize, and simplify our working lives. We need tools to help us adroitly opt out of availability demands. We need tools to surface only the  most significant, and to delete the useless. In short, mobile productivity is not about doing more, but consistently doing less.

[*] The BlackBerry Bold 9900, released in 2011, has a 1.2 gHz processor, which would have been the processor speed of the Dell Latitude D420, which was released in 2006. The iPhone 4S has an estimated speed of 800 mHz. Granted, processor speed is not the only measure of computing power. In particular, smartphones are hampered by a lack of reliable network access or slow network speeds. However, the BlackBerry Bold’s processor, given good network access and battery life, can perform as quickly as the Dell Latitude D420 on mundane tasks, such as checking one’s email – a central function we examine in this paper.

 

[1]       L. Conrow, “Developing a Taxonomy for Office Email : A Case Study,” Rochester Institute of Technology, 2010.

[2]       M. Dredze, J. Blitzer, and F. Pereira, “Reply Expectation Prediction for Email Management,” in 2nd Conference on Email and Anti-Spam, 2005, pp. 2–3.

[3]       F. K. Ozenc and S. D. Farnham, “Life ‘ Modes ’ in Social Media,” in CHI 2011, 2011, pp. 561–570.

[4]       M. Madden and S. Jones, “Networked Workers,” vol. 2008, no. 24 September. Pew Internet Project, Washington, DC, 2008.

[5]       O. Turel and A. Serenko, “Is mobile email addiction overlooked?,” Commun. ACM, vol. 53, no. 5, pp. 41–43, 2010.

[6]       Comscore Inc., “Emal Usage,” New York, N.Y., 2011.

Empathy in Action: the idea of Accompaniment

We talk a lot about “empathy” in the design world. But we don’t have a great deal of clarity about what empathy actually is, and what it costs us as both designers of products, or as human beings.

What is empathy? Nursing theorist Theresa Wiseman argues that empathy involves the following:

  1. To be able to see the world as others see it
  2. To be non-judgmental about what you see
  3. To understand another’s feelings
  4. To communicate your understanding of those feelings to others

This notion of empathy goes well beyond what most designers can legitimately claim to do, even with the best of their intentions. Empathy requires us to be alongside someone for the long term.

WP_20130611_022

When we talk about the failure to budget the time or money for user research, what we’re really talking about is the failure to prioritize empathy. We don’t need a large budget to see how others view the world. We don’t need a lot of time to be non-judgmental about that. We don’t need time or a separate budget to understand someone else’s feelings, or even to communicate those feelings to others.

But what we do need is the moral conviction that those things matter. It’s hard to consider that when we do not leave our Ivory Towers, or our industrial chic design studios. We need to be out in the world, alongside the people who use the products we design. How can we know how others see the world? How can we  understand those feelings, in genuine and open ways? We must accompany our customers through their journey. We must be with them.

Accompaniment

Dr. Paul Farmer is the founder of Partners in Health, the Haitian based NGO that has tried, for decades, to bring some measure of dignity to the lives of the Haitian people. Farmer describes his approach to helping the people as “accompaniment,” or the act of being there, along with them.

For Farmer, accompaniment is:

to go somewhere with him or her, to break bread together, to be present on a journey with a beginning and an end.

Farmer argues that to accompany someone is to be there

Accompaniment means committing to helping people with AIDS for the entirety of their lives, or to see someone with terminal cancer through to their death. It means:

Accompaniment is an elastic term, but not too elastic. It is not the same as a paid consultancy or a one-off project to help certain institutions or certain individuals for a little while.

You can see how hard it is for designers, particularly those who work in agencies for clients, to accompany their end customers throughout the design process. For this reason, I argue that user research is more akin to a lifelong mission than it is an entry in a project plan.

User Researchers as Accompagnateurs

I came across this notion of accompaniment in a recent book on social movements, written by a lifelone labor lawyer who has recently become an advocate for prisoners in Supermax prisons. He writes that he is still a lawyer. He doesn’t forget his expertise or leave it aside when he’s accompanying workers or prisoners. What he does do, however, is leaves aside his ego, and his desire to flex his expertise.

This is a lesson for designers, first, in that a designer who accompanies would never add aesthetic flourish just for the sake of it. No, she may hold aesthetic appeal as a priority personally, but would be willing to leave it aside if it were no benefit to her user.

Likewise, many researchers spend a great deal of time adding analytic flourishes to their research, but this pleases no one but themselves. It does nothing for the users, but may (in all honesty) simply make the researcher more analytically grand than her colleagues.

This is precisely what academic researchers are guilty of. This is why the Ivory Tower of academe has failed ordinary people. The human drama of any organization is about status attainment and status maintenance, and the university is no exception to this. But even in the private sector, user researchers are more akin to inspectors than accompagnateurs.

Anyone having seen the results of an overly “scientific” usability test will see this in action. Researchers are not immune to vanity, and they may use statistics or meaningless notions like “time on task” to demonstrate their rigor – without ever once accompanying the user on his journey.

Accompaniment and ethnography

This brings me to ethnography. Ethnographic research holds so much more potential for accompaniment than other forms of research because of its essential nature. Ethnography is, at its heart, about losing one’s own viewpoint and embracing the participant’s. It is about representing the journey that others take, not your own. It is about continually interrograting one’s own position in the world, to understand others.

If you sense a purpose higher than merely “building good technology,” you’re right. My mission as a researcher is to understand the experiences of everyday people, and to communicate those experiences to my engineering and design colleagues. The sociologist C. Wright Mills once argued that there is no such thing as “value-free” sociology, and I would extend that to say there is no such thing as “value-free” design.

All of us who fail to do research because there is “no budget,” or “no time” are incrementally eroding the idea that design is for others, not for us.

The future of work: A designer’s reading list

Recently, I was asked to put together a list of reading for design students. These students are about to design something for emerging productivity needs. I looked through my academic database, I realized that much of what I’ve read is probably not appropriate for designers.

I mean, show me a designer who wants to read Marx, Weber, and Durkheim, and I’ll show you a budding sociologist!

So I changed tack. Below is the rough, popular reading list I came up with. These articles are no substitute for a robust reading of the research on work and technology, but it does reflect some of the major debates in the field.

It also is ripe for design solutions. There is tension. There are problems to solve. What problems would you solve, if you were designing for the future of work?

Understanding work in the 20th century: from hierarchy to what?

Stewart, Matthew. 2006. “The Management Myth: Most of management theory is inane, writes our correspondent, the founder of a consulting firm. If you want to success in business, don’t get an MBA. Study philosophy instead.” The Atlantic. http://www.theatlantic.com/magazine/archive/2006/06/the-management-myth/304883/

Work and technology: overworked and underproductive

Bogost, Ian. 2013. “Hyperemployment, or the exhausting work of the technology user.” The Atlantic. http://www.theatlantic.com/technology/archive/2013/11/hyperemployment-or-the-exhausting-work-of-the-technology-user/281149/

Madden, Mary, and Sydney Jones. 2008. “Networked Workers”. Pew Internet and American Life: Washington DC. http://www.pewinternet.org/2008/09/24/networked-workers/

Work and Home: blurring the boundaries

Meece, Mickey. “Who’s the boss? You or your gadget?” The New York Times. http://www.nytimes.com/2011/02/06/business/06limits.html?pagewanted=all

Duxbury, Linda, and Rob Smart. 2011. “The ‘Myth of Separate Worlds’: An Exploration of How Mobile Technology Has Redefined Work-Life Balance.” In Creating Balance? International Perspectives on the Work-Life Integration of Professionals, edited by Stephan Kaiser, Max Josef Ringlstetter, Doris Ruth Eikhof, and Miguel Pina e Cunha, 269–284. Heidelberg: Springer-Verlag.

The Aging Workforce: time bomb or blessing?

Thompson, Derek. 2012. “Gray Nation: The very real economic dangers of an aging America.” The Atlantic. http://www.theatlantic.com/business/archive/2012/03/gray-nation-the-very-real-economic-dangers-of-an-aging-america/254937/

The End of The Job: what happens when no one has a job?

Wishnia, Steve. 2012. “Temp Worker Nation: If you do get hired, it might not be for long.” Alternet. http://www.alternet.org/labor/temp-worker-nation-if-you-do-get-hired-it-might-not-be-long

The Economist. 2014. “The Onrushing Wave.” http://www.economist.com/news/briefing/21594264-previous-technological-innovation-has-always-delivered-more-long-run-employment-not-less

MBO Partners. 2011. “The State of Independence in America”. Herndon, Virginia. http://www.mbopartners.com/state-of-independence/independent-workforce-index.html