Research, Design & Invention

This is an old site...please go to the new one

Please go to the new website at  www.DonRickertDesign.com.

Pick up books after 11:45 am on Monday April 2nd

I don't know what I was thinking. I have to be at Tech for an ID Faculty meeting at noon. It does not make sense for me to come to Tech early in the morning, go back to Decatur and then back to Tech again. Hope that waiting an extra hour or so does not cause undue inconvenience.

Don

Photos from Ethnography in the Fuzzy Front End of Product Development Presentation

Attached are photos of Don at the AMA's Ethnography in the Fuzzy Front-End of Product Development presentation.  Download of the presentation and links to other resources can be found in many of the entries below.

    

Acrobat Version of Ethnography in the Fuzzy Front End of Product Development

A number of attendees at the recent American Marketing Association presentation by Jay Heisler (UPS) and Don Rickert (Wiederholt & Rickert Partners) and colleagues have requested an easily printable version of the presentation. Here it is (a zipped Acrobat file):

Download Ethnography in the Fuzzy Front-end of New Product Development print version 5-22-06 (zipped)

Downloads from Ethnography in the Fuzzy Front End of Product Development

For those who attended Jay Heisler's (of UPS) and Don Rickert's (of Wiederholt & Rickert Partners, LLC) presentation on Ethnography in the Fuzzy Front-End of Product Development, thank you--you were a wonderful audience. We have broken the PowerPoint into four parts for download, as the presentation is quite large.

If you have any questions or problems with downloading, please do not hesitate to call Don Rickert at 404-713-3750 or email him at drickert@warp3insights.com.

• Download Main Presentation (PowerPoint)
• Download Examples of Traditional Techniques images (linked PowerPoint)
• Download Observations in a Mega-Store images (linked PowerPoint)

• Download What Does Ethnography Look Like? images (linked PowerPoint)

Ethnography in the ‘Fuzzy Front End’ of New Product Development

The event is sponsored by the Marketing Research SIG of the American Marketing Association, Atlanta Chapter: May 17 @ noon Maggiano's Buckhead (See www.AMA-Atlanta.com for details)

Don Rickert of WARP3 Insights and Jay Heisler of UPS will explain the emerging role of Marketing & Consumer Research with an emphasis on the new role of Ethnography in New Product Development.

This presentation is a unique opportunity to hear perspectives on the role of Ethnography in New Product Development from both the ‘supplier-side’ and ‘client side’ Marketing Research perspectives.

New Product Development (NPD) is about developing useful products that consumers want and that can be manufactured and sold at a price that makes a fair profit. Credible contemporary notions of the NPD stress the importance of forming, early on: 1) deep understanding of the people who will use products, 2) how they will use them and 3) the context in which usage occurs. This is the true “Voice of the Customer”.

The existing tools of Marketing Research, such as focus groups, surveys and concept/usability testing, when used in isolation, don’t always help us understand consumers, their needs and desires. We are faced with such issues as:

1. the limitations of recall
2. the fragile ‘belief-attitude-intention-behavior’ chain
3. people’s difficulty in articulating their needs and desires
4. External Validity (does what happens in a research setting apply to the real world?)

Failure to overcome these issues diminishes Marketing Research’s contribution to New Product Development. This comes at a time when Market Research is being called upon to take on an expanded leadership role in NPD. The first order of business is to improve the usefulness of our existing methods. One way that this is going to happen is greater emphasis on Observational methods. Observational methods go by many names, including ‘New Product Ethnography’, whose signature attributes include:

1. observation of people doing whatever it is they do on their own turf
2. qualitative in approach
3. inductive in nature (does not start with hypotheses)
4. ‘Participant-Observation’ (Interaction with Research Participants)
5. Depth Interviewing (Long-Open-ended Interviews)

Unlike traditional Anthropology, New Product Ethnography emphasizes actionable insights into consumer behavior and lifestyle activities, consumer preferences for product features, form factors, materials, color, patterns of use and purchase.

As it turns out, New Product Ethnography also helps us to improve the strength of our ‘legacy’ methods. For example, the actionable insights from field observation help us to construct the hypotheses and identify the right questions to ask on surveys, as well as whom to survey. It helps us to choose the most fruitful issues for focus groups (and who to include as well). It also helps us to select the most appropriate participants and the tasks they should perform in concept tests and usability tests.

Why Surveys and Usability Testing are Often a Waste of Time and Money

A recent rant in Interactions Magazine (January - February 2006) asks the question: are you doing 'User Research' or 'Loser Research'?  The author is not referring to researching people who are 'losers' but rather researchers who are not doing good work-- 'Loser Researchers'. That article inspired our post below.

Attempting to develop any kind of product without a clear notion of who the users are and what they might do with the product is a very bad idea. Furthermore, even starting other types of consumer research, such as surveys and usability testing before the product opportunity is fully understood, or before personas (i.e. users) and scenarios (i.e. the things they do) are defined is ineffective. Lacking a clear notion of who the users are and what they do, how could one even know what questions to ask or what to test in the usability lab? In science jargon, asking irrelevant or just plain wrong research questions is called 'Type III Error'.

The common practice of conducting usability testing prior to rigorous Discovery Research, such as Field Observation and Depth Interviews, is a principal reason for so much usability testing being a waste of time and money, and this hurts the credibility of Usability Engineering as well as its related professions.

By analogy, conducting a usability test before scenarios are developed is like booking a recording studio before any of the songs are written--very expensive and very little valuable output.

Definition of Ethnography

The term ‘Ethnography' or its many synonyms often comes up in the context of New Product Development. For example, Cagan and Vogel state that “the most powerful area of new product research is an emerging field of new product ethnography.” (Cagan and Vogel, 2003). Ethnography also plays an important role in our model; therefore, it will be useful to describe the method and its central role.

What is Ethnography?

The term ‘Ethnography' seems to have achieved the dubious status of buzz-word. There are many definitions, depending on who is doing the defining. A good definition of traditional ethnography is—

…the art and science of of describing a group or culture. It is a form of cultural anthropology using fieldwork to observe the group and derive patterns of behavior, belief, and activity (Cagan and Vogel, 2002, p. 183).

No matter what definition you get, there are several signature attributes—

• Observation of people doing whatever it is they do on their own turf —Going out into the “real world” and observing is critical.
• Qualitative in approach —For instance, statistical analyses are not normally performed on the discoveries of studies deemed to be ‘ethnographic.'
• Inductive in nature —does not start with hypotheses
• Interaction with ‘Research Subjects' is encouraged —Many, especially those with legitimate claim to the title of ‘Professional Ethnographer', stress that ‘Participant-Observation' is a necessary attribute of ‘true' ethnography.
• Depth Interviewing (Long-Open-ended Interviews) is often used
• Hypothesis generation and prediction —There is not universal agreement about whether hypothesis generation, based on observations, should be part of ethnography or whether it should be purely descriptive.

The Problem with Ethical Relativism

Note: This relates to the post entitled The Ethics Problem in Technology Companies and What Can Be Done About It

Ethical Relativism or Moral Relativism is a popular philosophy, which asserts that a universal moral or ethical code is not possible, because different cultures and individuals within cultures have their own beliefs about what is right and wrong. Widespread belief in Moral Relativism and the related philosophies of Cultural Relativism and Subjectivism have given rise to what Tavani (2004) calls “discussion stoppers” on ethical issues in business. These “discussion stoppers” are—

• People disagree about morality, so how can we reach agreement on moral issues?
• Who am I/who are we to judge others and to impose my/our values on them?
• Isn't morality simply a private matter?
• Isn't morality simply a matter that different cultures and groups should determine for themselves?

In addition to militating against ethical discourse, Relativism provides the philosophical basis for doing what one must do to “win” and rationalizing one’s choices (Maxwell, 2004). This is important to those of us who design, build and market products to consumers. Are our products good for people, or just a means to an end--profitability and happy shareholders? Any of us who possess and ounce of introspection know that this is not just some academic exercise, but rather an issue we deal with every day.

Do you think that right and wrong are completely up to the individual? Read Maxwell's and Tavani's articles before making up your mind.

WARP3 Works with All Types of Product Development

The list companies with which we have worked is quite diverse with respect to type of organization as well as products produced. Unlike many firms that tout marketing/consumer research, design or usability services, we do not limit ourselves to websites and online applications.

The key criterion is: Do people use it?

Types of Organizations   

• Broadcast media
• Technology
• Telecommunications
• Communications
• Publishing
• Energy
• Research & Development
• Interactive Television
• Consumer Electronics
• Cable TV
• CRM (Customer Relationship Management)
• Insurance
• Medical
• Higher Education
• Financial Services
• Professional Services

Types of Products

• Bank automation
• Online Rate Quote for auto insurance
• Teller systems
• Self-service point-of-sale systems
• ATMs
• Medical informatics
• Wireless devices and other consumer products
• Handheld remote controls
• Interactive television user experience
• Online insurance
• Consumer banking, and other forms of e-commerce such as online retail
• Ambient Devices
• Musical instrument design (e.g. electric violins, baritone violins and carbon fiber accoustic instruments)
• Rowing innovations

WARP3 Insights Now a VAR for Morae

Nothing is more powerful than seeing real users interact with your Web site, software or e-Commerce application. It is the only way to truly understand user needs and the "why" behind their actions.

By making usability testing accessible, Morae gives you the power to understand where problems exist for your users. With this information, you can make key design changes that will:again.

• Improve the navigation of your online store and increase conversion rates.
• Make your software easier to use and boost user satisfaction.
• Make information easier to find on your site so users will return again and
• Increase the productivity of employees using your intranet.

We will have details in the next few days about WARP3 Insights' services related to Morae, which we use exclusively for usability testing in the real world.

What is Design Research?

Design Research has been used to designate a number of activities, including the study of the designers and the design process itself. We use the phrase 'design research' in a more practical way.

When we say 'design research' we mean the application of behavioral and social science research approaches to helping designers to design the best products possible. We use the best of Marketing/Consumer Research, Human Factors, Ethnography and Ergonomics methods to uncover insights needed to develop great products that people value enough to buy at a price that makes a fair profit.

Rickert and Whiddon Ethics and Technology Paper

Don Rickert recently presented his and Alycen's paper, The Ethics Problem in Technology Companies and What Can Be Done About It, at the 8th Annual Ethics & Technology Conference, 
Saint Louis University, June 24 & 25, 2005

Downloads:

• High level Summary PowerPoint
• MS Word Version (the actual paper)

Don Rickert Addresses Computer Science Advisory Board at Faulkner University

Don is a member of the Computer Science Department Advisory Board at Faulkner University in Montgomer, Alabama. He is one of several advocates for the new track in Informatics (e.g. human factors, usability, information architecture, etc.). He was the scheduled speaker for the most recent board meeting on September 16, 2005.

The title of Don's presentation was End-User Research in the Informatics Process. It included a demonstration of Morae (by Techsmith, the SnagIt people), a very cool new system for capturing usability testing and other consumer research video and audio data in the field.

Download Faulkner_Rickert.ppt

Download Faulkner_Detailed_Background_Rickert.ppt

The Missing Link: Observation of Real People Doing Real Things

Do you ask yourself any of these questions?

What are the real problems that people want to solve with products, built environments or services like yours?

How do people live with your current offerings? Your competitors'?

What do consumers think that your products should look, feel, and even sound like?

What attributes delight customers, consumers and/or users?

What are the hidden requirements? (the “must-haves” that nobody thinks to tell you about until after you get it wrong!)

What are the "enragers" (the things that make people hate you)?

How do your marketing messages really play with potential customers? Does anybody care?

If, even after surveying your customers, conducting focus groups, or perhaps some usability testing, you still lay awake at night wondering what the answers are, you are not alone. The big problem with conventional marketing research is that people’s beliefs, attitudes, intensions and customer experience as measured by questionnaires, focus groups and interviews are notoriously unsuccessful at predicting actual adoption or actual satisfaction.

The Importance of Convergence of Method in Consumer Research

What makes Consumer Research valuable to companies is discovering what really motivates people to buy their products. No matter what else our jobs might involve, the fundamental job of all Consumer Researchers is:

Figuring out who the target consumers are

Uncovering their unmet needs and desires (what is appealing?)

Understanding what motivates people to value products

Translating discoveries (all of the above) into the development of new product ideas

Using the insights gained to support intelligent business decisions

Risks to Gaining Useful Insights from People

There are a number of risks to gaining useful insights from doing research involving people. Among these risks are:

• Asking irrelevant questions
• Not asking the right questions
• Asking questions in such a way as to create attitudes and opinions where none previously existed
• Assuming that what happens in a formal research setting applies in the “real world”
• Confusing what people are willing to say in a group interview with what they really think
• Observing people doing things that they would not really do
• NOT observing them doing the things that they really would do
• Talking to, surveying or observing the wrong people
• Confounding people’s remembered or imagined beliefs, attitudes, intentions or experiences with actual beliefs, attitudes, intentions and experiences.

In his aptly-named article in Marketing Research ("Consumer Research in the Land of Oz”, Marketing Research, Vol. 15, No. 1, Spring, 10-15) John Liefeld tackles several of these risks, particularly the last one listed above. He points out that...

Marketing Research or Human Factors?

Several of us are consumer researchers who work primarily in product development. A constant source of debate (even with ourselves) is the matter of whether consumer-oriented research in a product development setting is marketing research at all. I am not sure that I have a definitive answer; however, I will continue to be active in the American Marketing Association (AMA), particularly the Atlanta AMA Marketing Research SIG and Atlanta Marketing Research Executives Roundtable, the Interactive Marketing Research Organization (IMRO) as well as SIG-CHI (the Association for Computing Machinery Special Interest Group on Computer-Human Interaction) and Human Factors & Ergonomics Society.

While some would call me a Human Factors Engineer or "HCI type", I will also answer to "Marketing Researcher." Indeed, there is a fuzzy line between Marketing Researchers  and Usability/Human Factors/HCI people.

Here are my thoughts...

Thoughts on Design Patterns

For the past several years, design patterns have made their way into the design of products, particularly interactive systems.

Patterns are formal descriptions of best practices within a given design domain. They capture common solutions to design tensions (usually called "forces" in the pattern literature). The idea of patterns actually started in the field of architecture (as in physical buildings). The first book (in the 1970s) on patterns and pattern languages (collections of related patterns) was Christopher Alexander's A Pattern Language. The “Alexandrian” format for documenting patterns is still very popular today. Patterns were introduced to the field of commercial software engineering in the mid-1990s by the publication of Design Patterns by Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides.

Method Overview: Usability Testing

Key Points

• What people do in a controlled setting

• When you are asking the questions

• Relatively unobstrusive observation of semi-realistic behaviors

Method Overview: Ethnography and Related Field Methods

Key Points

• Happens on people’s own turf
• Observation of what they really do
• Often supplemented by field interviews and quantitative
• Expensive!
• Data analysis is time-consuming.
• Can have high external validity (generalizable conclusions)

Interactive Marketing Research Organization (IMRO)

On a periodic basis, we will post short descriptions and links to organizations and journals that we think readers of our weblog should know about. This is the first such posting.

The Interactive Marketing Association, or IMRO, is a:

A worldwide association of researchers dedicated to providing an open forum for the discussion of best practices and ethical approaches to research being conducted via the Internet.

IMRO has recently launched the Journal of Online Research at www.ijor.org. The IMRO website describes JOR as--

... an online source of "research on research" and targets individuals who are active in the use of the Internet and Computer-assisted methods for conducting marketing research. It is specifically designed to showcase fast breaking study results with a focus on "best practices" information. The JOR encourages IMRO members to submit articles and papers for publication. Visit the JOR site for details.

IMRO is to be praised for its Code of Ethics. Anyone doing research involving online surveys should study it carefully!

Recently, IMRO has become a division of the Marketing Research Association (MRA). This seems to be a good thing.

Usability Reductionism (draft in progress)

Usability is a key attribute of many, it not most, of the things that people use in everyday life—from toothpaste tubes to the cars they drive. Usability is a pivotal concern in the design of products of all kinds, despite the fact that the emphasis seems to have been on Internet applications in recent years. I think that that is where the jobs have been, even after the dot.com crash.

Why care about usability?—because usability matters to the people who buy stuff (consumers). Increasingly, consumers are putting a high value on ease of use and quality (perceived as well as real), as well as other attributes such as attractiveness, emotional appeal (even seductiveness, some would argue), efficiency and usefulness.  With the Internet in particular, many users find applications to be absolutely unusable.  The common response is to simply stop using the offending web sites, or the Internet altogether!

Method Overiew: Analysis of Artifacts

• Artifacts reviewed depend on the domain (e.g. consumer durables, online system, etc.)
• In technologhy companies, the artifacts are usually—
  • Customer Service logs
  • Complaint emails
• Review of artifacts provides good context for understanding actual observations of customer behaviors.
• Can lead to wrong conclusions if interpreted ‘stand-alone’ (not in the context of actual observation).
  • I.e. should be done in concert with ethnography or other observational research

Focus Groups, Theory or the Kid in the Garage?

Based on an article published by the author and his doctoral advisor in Communications of the ACM, Vol. 42, No. 3 March,1999, Focus Groups, Theory or the Kid in the Garage?

Kids in the Garage

Here is a notion that is not often well-recieved by those of use who consider ourselves Market or Consumer Researchers (and usability specialists, human factors engineers, product designers as well). The basic idea is this-we can talk about survey results, conjoint analyses, focus group findings, participatory design sessions, theories, methods, statistics, designs until either a) the end of time or b) the money runs out. In the meantime, some kids in a garage may invent the NEXT BIG THING right under our noses.

Method Overview: Interviews

• Can be…
  • In a facility or
  • In the field or
  • Over the telephone
  • Do not generate easy to analyze data.

Typical purpose

• You guessed it—measure beliefs, attitudes, intentions and experiences

Why researchers use them

• Probing by a good interview can draw out much more information than questionnaires
• Convenience—can even be done over the telephone or online

The risks

• Same as questionnaires and focus groups, plus
• Asking questions can create attitudes and opinions where none existed before.

The possibility

• Hearing what consumers say can add to insights gained from actual observation of behavior.