Making hard choices

Over the holidays, I had a chance to reflect and consider my future. I found myself unsatisfied with my daily work. Somewhat serendipitously, I stumbled across a TED Talk from philosopher Ruth Chang. Chang helped me decide to leave Microsoft and take a job at Amazon. I start on Monday, January 25.

Dr. Chang’s research focuses on the nature of hard choices. She argues that hard choices are hard because we tend to try to resolve them by using quantitative techniques. For example, adding up all the pros and cons of each choice theoretically should indicate the superior choice. But it doesn’t, in practice, because human experience is not quantitative.

Instead, Chang tells us, we should see competing choices as “on par” or “in the same neighborhood” as each other. From there, how do you decide? Chang urges us, in all existentialist glory, to embrace a choice for what it represents. Choose a job that declares who you are, she says. Move to the country because you declare you are “for the country.” Make a decision that symbolizes the self you want to be. And embrace that hard choice!

So I have done exactly that. I am choosing to be urban (Amazon is a 40-minute walk from my home). I’m choosing to be a little bit chaotic. I’m choosing crazy unknowns (I have no idea what I’ll be working on). I’m choosing to jump with both feet.

I was not unhappy at Microsoft; I just wasn’t growing. So above all, I am declaring myself in favor of growth. I am choosing new.

DO you have a hard choice? Chang tells us to welcome it:

“Let us not resolve to work harder at being the selves we already are. Instead, let’s resolve to make ourselves into the selves we can commit to being.”

Qualitative research and innovation

Qualitative research is not generally considered “real” research, and this has terrible implications for innovation. Companies’ thirst for operational effectiveness begs for quantitative data. But quantitative data does not and cannot form strategy. Qual data are a key ingredient to strategy, or the development of new and differentiated products.

Many people are familiar with Michael Porter’s famous paper, “What is strategy?”  Porter famously argued that many companies mistake operational effectiveness for corporate strategy. Operational effectiveness, according to Porter, is about quality, productivity, and speed. It is about doing the same thing as others, but doing it better.

Strategy, by contrast, is about being different. It is about doing entirely different activities to deliver value to customers. Other authors have called this the “blue ocean” or finding a place in the market that is calm, unoccupied, and yours for the taking.  A “red ocean” is full of competitors, doing exactly the same thing as you are, and demanding ever higher performance.

porter

Framed this way, it is clear that operational effectiveness relies heavily on quantitative data. How efficient are we? How do we stack up against the competition? How good are our products? How fast do we make them?

Operational effectiveness simply begs for quantitative data, and now that we have access to petabytes of passively collected data relating to productivity, quality, and speed, it is easier than ever to be operationally effective.

Or it should be. We know that quantitative data requires a great deal of cleaning, massaging, and managing, not to mention analysis, to make it useful for operational effectiveness.

The shift to data-driven operations has demanded a great deal of companies’ attention, mostly because data collection and analysis is not as easy as most think it to be.

But let us not mistake this for strategy.

There is nothing inherent to benchmarking performance that lends itself to strategic advantage. Quantitative data does not reveal how or in what ways customers are making their own workarounds. Quantitative data shows us how many products meet a particular standard, how many products are produced or sold, or how fast a company makes them. It can tell you the average satisfaction a customer may have, but it cannot reveal any of the detail behind that satisfaction.

In their insightful Harvard Business Review article, “An Anthropologist Walks Into A Bar,” Christian Madsbjerg and Mikkel Rasmussen argue that qualitative research gives companies the ability to bridge the “complexity gap,” or what a study of 1500 CEOs revealed as their main challenge. Why do customers do what they do? You must do qualitative research to find out. And, by extension, you must do qualitative research to innovate.

mads

 

Qualitative research is explicitly about revealing detail. Qualitative research shows how people are using products, or how these products sit and gather dust in the corner of the kitchen. Qualitative research, particularly field-based research like ethnography, offers that path to delivering truly different products.

Companies that do ethnography regularly uncover entirely new or different ways to deliver value to customers. Oftentimes, this is done unsystematically. Skillful product and brand managers know that observing everyday life can reveal the how and the why of product failure.

Nike is a great example of a company that is in touch with culture. Its marketers are largely acknowledged to be among the best in the world. Its product innovation is continual, and its brand equity is unparalleled. Even their lab-based researchers like the fabulously named Gordon Valiant are active members of the running community.  His lab-based practice is complemented by regular participation in running events, where he comes into contact with other runners.

Valiant conducts in-lab studies systematically, but observes human behavior in an ad hoc way. Imagine the advantage to companies that do this research systematically. Imagine having a steady stream of insight into real people and why they do what they do. Imagine having thick description of painful workaround work, and regular replenishment of unmet customer needs.

That can only come from systematic, regular, and rigorous qualitative research.

Compare that to a company tirelessly benchmarks its quality, productivity, and speed. The company with qualitative insight into human behavior will have almost limitless potential to do things differently, to deliver new products or services, to find entirely unexamined oceans of product innovations.

Yet we spend almost nothing on qualitative research.  Esomar, the international market research association, estimates that in 2013, corporations spent $6.6B USD on qualitative research, worldwide. The vast majority of money spent on qualitative research is on focus groups, but Esomar estimates almost $1.6B is spent on the more interpretive methods of in-depth interviewing and ethnography. This amount is dwarfed by the $32.4B USD spent on quantitative market research.

It could be argued that many companies need to start with operational effectiveness. Fair enough. But no company can survive on quality, productivity, and speed alone. It is too competitive a marketplace. Qualitative research, therefore, is a cheap way to guide the company toward different product offerings, and ultimately, toward innovation.

 

Design researchers must think fast and slow

Research into brain science some surprising insights for guiding research practice. These findings suggest that the scientific method constrains our natural creativity.
Too often, researchers take their cue from the scientific method. While this method undoubtedly changed the world and our knowledge of it, it is antithetical to the creative needs of a well-rounded researcher. It is especially problematic for design research, which requires creative solutions to existing problems.

Design researchers should embrace less structure and more openness at the early stages of product design, and rigor and structure in the mature stages of product sales. As sales drop off and the product loses its natural match to the culture, design researchers should once again embrace openness in their research approaches.

research phases

Generally, we think of research as the focused, systematic collection of data, over time, in keeping with a given framework or theory. In this view, research is intended to confirm or deny given hypotheses, and incrementally improve our knowledge about a given topic.

We know from the book Thinking Fast and Slow, however, that this research approach only serves one type of thinking. Thinking Fast and Slow author Daniel Kahneman tells us that “Type 2” or “slow thinking” is a disciplined, focused, kind of thought that roughly matches the deductive reasoning of the scientific method and other traditional forms of research. It is structured and deliberate, requiring the cerebral cortex.

But Type 1 or “fast thinking” is less structured, more instinctual, and involves the more reptilian parts of the brain. At first glance, fast thinking appears to be undisciplined or even lazy – the antithesis of the scientific method. But fast thinking produces creative and intuitive leaps that are impossible with the iterative, deductive, and controlled manner of slow thinking.

Design research both thinking fast, and thinking slow. Thinking fast entails creating novel combinations, unusual interpretations, or unique syntheses. Thinking slow entails systematic evaluation and the structured contribution to a body of knowledge.

Gifted researchers engage in both thinking fast, and thinking slow. As sociologist C. Wright Mills describes, a researcher must have her “files,” which is a set of unstructured, messy, and without order:

mills

C. Wright Mills

“…You will notice that no one project ever dominates [the files], or sets the master categories in which it is arranged. In fact, the use of the file encourages expansion of the categories which you use in your thinking. And the way in which these categories change, some being dropped and others being added – is an index of your intellectual progress and breadth. Eventually, the files will come to be arranged according to several large projects, having many sub-projects that change from year to year. [1, p. 3]”

Anthropologist Bronislaw Malinowski echoes this messy disorder when he describes what will eventually become his masterwork The Argonauts of the South Pacific:

malinowskiBranislaw Malinowski

“I estimate that my future publication will be voluminous, roughly three volumes of 500 pages each at 500 words per page. It will take me about two years to get the [manuscript] ready and see it through the press. My material is now a chaotic mass of notes. To work it out and put it into the right theoretical frame is perhaps the most difficult, exacting, and important stage of research. To work it out efficiently I must give it all my time. [2, p. 582]”

Malinowski recognizes the “chaotic mass of notes” must be whipped into shape to become a manuscript, but he must first grapple with the disorder. This is precisely what psychotherapist Rollo May describes as the “creative encounter,” or the unstructured time an artist (or researcher) spends with her subject of study.

may

Rollo May

“The first thing we notice in a creative act is that it is an encounter. Artists encounter the landscape they propose to paint – they look at it, observe it from this angle and that. They are, as we say, absorbed in itOr scientists confront their experiment, the laboratory task, in a similar situation of encounter.  [3, p. 39] P. 39″

Consider also the “commonplace book,” or the kind of notebook great thinkers like John Locke and Charles Darwin used to organize their thoughts. As innovation author Stephen Johnson tells us, early modern readers did not read sequentially, but jumped around, setting the stage making creative connections.

 

johnsonSteven Johnson

“The tradition of the commonplace book contains a central tension between order and chaos, between the desire for methodical arrangement, and the desire for surprising new links of association….Each re-reading of the commonplace book becomes a new kind of revelation. [4, pp. 109–110]”

In other words, researchers who allow themselves to read out of order, or to collect without regard for structure, are able to make creative, intuitive leaps. But researchers who fail to methodically manage their knowledge fail to close the loop of production. Researchers need to think fast and to think slow. They need to think broadly and think narrowly. Type 1 and Type 2 thinking translates into 3 kinds of research: exploratory (thinking fast), evaluative (thinking fast and thinking slow), and experimental (thinking slow).

Frequently, social scientists in particular focus on “rigor” as the solution to good research. But rigor without creativity adds little to our collective knowledge.  As Heideggerian scholar Carol Steiner argues, this “fore-structure” – or predetermined way of looking at the world – stops us from conducting innovative research and producing innovative things. Instead, innovative researchers, she found, are open to “Being,” or the ability to have experiences, people, and objects reveal themselves to them.

steinerCarol J. Steiner

“The innovators I studied seemed sometimes to be attuned to that old understanding of the relationship between Being and people…Losing faith in the scientific method has allowed them to understand themselves as other than knowledge-makers. Consequently, they often project an openness that allows them a different world to shine through for them, the public world. “[5, p. 594]

In other words, researchers in particular must struggle against the “fore-structure” or their extensive theoretical and methodological training which interferes with receptivity. As Rollo May argues, being receptive does not mean lacking in rigor.

“The receptivity of the artist [or researcher] must never be confused with passivity. Receptivity is the artist holding him or herself alive and open to hear what being may speak. Such receptivity requires a nimbleness, a fine-honed sensitivity in order to let one’s self be the vehicle of whatever vision may emerge. [3, p. 80]”

Rigor must be introduced later in the process – after the researcher becomes open to a vision, after the researcher grapples with the complexities of the data and their incongruence. Rigor often comes after a period of unconscious processing of the data.  Taking walks, playing, napping, and engaging in unstructured activity have all been shown to allow synthetic ideas to emerge.

Researchers should therefore use the scientific method with caution. Be aware of when you need rigor, and when you need creativity.

 

 

 

 

References

[1]      C. W. Mills, The Sociological Imagination. New York: Oxford University Press, 1959.

[2]      M. W. Young, Malinowksi: Odyssey of an Anthropologist. New Haven, CT: Yale University Press, 2004.

[3]      R. May, The Courage to Create. New York: WW Norton, 1994.

[4]      S. Johnson, Where Good Ideas Come From: The Natural History of Innovation. New York, NY: Riverhead Books, 2010.

[5]      C. Steiner, “Constructive Science and Technology Studies: On the Path to Being?,” Soc. Stud. Sci., vol. 29, no. 4, pp. 583–616, 1999.

 

What we know about gender and performance reviews

Update December 23, 2017: New findings from an AI-based experiment on gender and performance reviews has been added, and a new evidence-based improvement strategy called the “small wins” model.

The evidence is mounting that our existing feedback systems have inherent biases that penalize women. I have a personal commitment to help improve this, and in this post, I make specific recommendations on how to do so. But first, what do we know about gender and performance reviews?

  • Women receive more criticisms of their personalities in performance reviews: A linguist did study on performance reviews. Men and women provided positive performance reviews. She found that in 83 performance reviews, men received personality feedback in only 2 cases. In 94 critical reviews, women received personality criticisms 71 times. [1] Words like “abrasive” and “strident” were present in these performance reviews.
  • Women receive less helpful feedback than men. In a study of 200 performance reviews in a tech company, researchers found that women were more likely to receive vague praise than were men (57% and 43%, respectively), which included unhelpful comments like “You had a great year.” Men were more likely to receive developmental feedback, and linked specifically to business outcomes. When women did receive developmental feedback, it tended to relate to their personalities rather than to their performance [2]
  • Women are less likely to be rewarded for good ideas. Men who provide revenue-generating ideas are given higher performance ratings. Women who provide revenue-generating ideas see no improvement in their ratings. [3]
  • Men reward other men more highly than women for achieving the same goals: 70% of men rate men more highly for achieving the same goals as women, while an algorithm rate men and women equally (as did other women).
  • The “glass ceiling” is the result of many tiny obstacles. Researchers at Harvard Business School found that there is no specific point where women face a “ceiling,” but many small instances of discrimination lead to their careers stalling over time. [4] This may be why after only 2 years with a company, women’s aspirations for career advancement fall an astounding 60%, while men’s aspirations fall negligibly.
  • Ostensibly “meritocratic” reward systems favor men over women, and whites over minorities. Researchers experimentally tested whether managers would reward people differently when explicitly creating a system of “merit.” Over 3 experiments and 445 participants, they found men were rewarded with more money than women in this supposed “meritocratic” system. They also found that ethnic minorities and non-American born people were given lower raises, even when using the same evaluative criteria. [5] In another experiment, researchers switched male and female professors of an online course in the middle of the term. Students consistently rated the male professor higher, even though they actually had a female professor without knowing it. [6]
  • Women are penalized for asking for raises: Researchers found that people judge women more harshly when they ask for a raise; women don’t ask for raises because they realistically assess the social cost of asking. [7]
  • Men are rated more highly for helping colleagues, and women are rated more negatively for not helping. In a series of experiments, researchers asked participants to rate the performance of men and women who either agreed to stay late to help colleagues, or refused to stay late and help. Men who offered to stay were rated 14% more positively (women’s rating remained the same). Women who refused to stay were rated 12% more negatively (men were not rated more negatively). [8]
  • Women pay a penalty for motherhood, while men reap a bonus for fatherhood: Researchers have found women who become mothers pay a minimum penalty of 4% decline in income. [9] The penalty is larger for more educated women. [5] By contrast, fathers reap a bonus [11]. This is not due to a lack of commitment by women, by biased perceptions of their commitment. [10]  In other words, the cost of being a parent depends on your gender, not your performance.
  • Men are penalized when they ask for family accommodation. In a study of a management consultancy, one researcher found that men who ask for flexibility to care for their families are punished in performance reviews. [12] Men who did not openly ask but made private, covert arrangements got better performance reviews.
  • Keeping track improves fairness. Just keeping track of how people get rewarded, broken down by race and gender, was enough to reduce inequality over 5 years in a single company. So know your data! How are people doing relative to each other? [5]
  • Just pointing out bias actually increases its incidence! Other research has found that pointing out bias actually increases its negative consequences. [13] This effect disappeared when researchers noted that discrimination is not desirable.
  • Training managers on potential work/life conflicts decreases employee stress. Researchers trained a single company’s managers on how to deal with work/life conflict. They found reduced employee stress, and no increase in employee hours. [14]

 

Opportunities for Improvement

  1. Examine the words you use in your performance feedback. Is it related to personality or performance? Are the words very gendered, such as “bossy”? Consider what words you might have used to describe the opposite gender.
  2. Is your feedback helpful and specific? Did you provide vague praise like “Great job this year” or did you say, “Your work on the launch plan led to greater sales”? Make sure you link the feedback to specific business goals.
  3. Did you reward fairly? Is there a systematic difference between the genders? Could this be unconscious bias?
  4. Is this a “tiny obstacle”? How many barriers has your direct report experienced in her career? Is this performance review a chance to create a “tiny ladder” through the glass ceiling?
  5. Do you expect women to be more altruistic? Reward men and women equally for the same behaviors. Consider if you expect women to be more giving of their time than men.
  6. Do you expect men to be less family-oriented? Recognize that men have families too. Are you penalizing a man for being a caring father?
  7. Is your direct report asking for something…and are you evaluating that ask fairly? Consider what kinds of requests your direct report has made. Are you judging those requests fairly, or are you penalizing the person for speaking up?
  8. Unconscious bias is wrong. Make sure you point out that bias is wrong, not just that it exists. Norms are powerful, especially for senior leaders.
  9. Keeping track improves fairness. Just keeping track of how people get rewarded, broken down by race and gender, was enough to reduce inequality over 5 years in a single company. So know your data! How are people doing relative to each other? [5]
  10. Pursue “small wins“: researchers at Stanford found that they could improve outcomes if they worked directly with managers on reducing bias. Introducing a new scorecard reduced personality-based feedback to zero.  [15]

 

 

 

References

[1]       K. Snyder, “The Abrasiveness Trap: High Achieving Men and Women Are Described Differently in Reviews,” Fortune, New York, Aug-2014.

[2]       S. Correll and C. Simard, “Vague Feedback Is Holding Women Back,” Harvard Business Review, no. April, 2016.

[3]       A. Grant, “Rocking the Boat but Keeping It Steady: The Role of Emotion Regulation in Employee Voice,” Academy of Management Journal, vol. 56, no. 6, 2013.

[4]       A. Eagly and L. Carli, “Women and the labyrinth of leadership,” Harvard Business Review, no. September, pp. 62–71, 2007.

[5]       E. J. Castilla and S. Benard, “The Paradox of Meritocracy in Organizations,” Administrative Science Quarterly, vol. 55, no. 4, pp. 543–576, Dec. 2010.

[6]       L. MacNell, A. Driscoll, and A. N. Hunt, “What’s in a Name: Exposing Gender Bias in Student Ratings of Teaching,” Innovative Higher Education, vol. 40, no. 4, pp. 291–303, 2015.

[7]       H. R. Bowles, L. Babcock, and L. Lai, “Social incentives for gender differences in the propensity to initiate negotiations: Sometimes it does hurt to ask,” Organizational Behavior and Human Decision Processes, vol. 103, no. 1, pp. 84–103, May 2007.

[8]       M. Heilman and J. Chen, “Same Behavior, Different Consequences: Reactions to Men’s and Women’s Altruistic Citizenship Behavior,” Journal of Applied Psychology, vol. 90, no. 3, pp. 431–441, 2005.

[9]       T. Street, A. Arbor, and P. O. Box, “Has the Price of Motherhood Declined Over Time ? A Cross-Cohort Comparison of the Motherhood Wage Penalty,” Journal of Marriage and Family, vol. 65, no. August, pp. 597–607, 2003.

[10]     J. a. Kmec, “Are motherhood penalties and fatherhood bonuses warranted? Comparing pro-work behaviors and conditions of mothers, fathers, and non-parents,” Social Science Research, vol. 40, no. 2, pp. 444–459, Mar. 2011.

[11]     G. Hundley, “Male/Female Earnings Differences in Self-Employment: The Effects of Marriage, Children, and The Household Division of Labor,” Labor Relations Review, pp. 95–114, 2000.

[12]     E. Reid, “Embracing, Passing, Revealing, and the Ideal Worker Image: How People Navigate Expected and Experienced Professional Identities,” Organization Science, vol. 0, no. 0, p. null.

[13]     M. Duguid and M. Thomas-Hunt, “Condoning Stereotyping?: How Awareness of Stereotyping Prevalence Impacts Expression of Stereotypes,” Journal of Applied Psychology, no. October, 2014.

[14]     E. L. Kelly, P. Moen, J. M. Oakes, W. Fan, C. Okechukwu, K. D. Davis, L. B. Hammer, E. E. Kossek, R. B. King, G. C. Hanson, F. Mierzwa, and L. M. Casper, “Changing Work and Work-Family Conflict: Evidence from the Work, Family, and Health Network,” American Sociological Review, vol. 79, no. 3, pp. 485–516, May 2014.

[15]   Correll, S. J. (2017). SWS 2016 Feminist Lecture: Reducing Gender Biases In Modern Workplaces: A Small Wins Approach to Organizational Change. Gender & Society, 31(6), 725–750. https://doi.org/10.1177/0891243217738518