Characteristics of Research Tools

Current Usage

Article – 21th Century Technology for Usability and User Interface Design Activities.

Some survey…

Data Collection – Software designer tend to use online survey and online analytics more than product designer. But pretty consistent data collections used.

Analysis – spreadsheet most common. Very few using qualitative analysis software.

Knowledge and Satisfaction – big lack of knowledge and satisfaction in the research field. (the gap and need presents an opportunity).

The basic characteristics of effective user research tools

• Documentation
• Measurement – it can be a sub-category of documentation.
• Efficiency
• Enhancement – i.e. eye tracking.

Mora 3 is a multidimensional tool that has many if not all of the research characteristics.

Reviews of New Technologies

Livescribe Pulse

• It writes and record. also recording what’s being written. Both actions are sync up.
• Flaw: it rolls!

Many Eyes (by IBM)

• Nice visualization features both quantitative and qualitative.
• Flaw: it’s free, and it’s public. You can create it and immediately delete it.

High-speed Imagery

• Casio EXF1 Pro recommended for its versatility
• 30 images/second to capture study’s process in details (something they can’t describe well ‘casue it’s done subconsciously).
• High-speed video don’t look as good under poor lighting. Slow motion not as good unless it’s high-end video.

User Research Technology Concept

Field CREW – contextual research workstation

• wireless, multi-channel video management
• touch screen, writable interface
• Configurable hard-keys for rapid event tagging
• Speech to text transcription
• Sub-vocalization to text transcription
• Integrated storage/recharge for wireless components

Train a team of research, there’s usually a facilitator/note taker and video-grapher. It’s hard to do it all by 1 person.

The tool focus on data collecting, but subsequently it also improve the data anglicizing.

Waterfall approach

• Predictable and structured approach
• Human-centered: solutions are based on upfront research and strategy
• Team roles are clearly defined (designers design, engineers build)
• Rigid, heavy, and slow
• Documentation and specifications

“Waterfall method is best when you can’t afford to learn from your mistakes. You don’t design software for the Space Shuttle in an Agile manner.” Alon Salant, engineer

When Waterfall method works better
Type of projects:

• Creating experiential, immersive, branded experiences that are not module-driven.
• High-risk or complex projects that require breadth and deep thinking.
• Findings through design research and insight are key to project success.

Team dynamics:

• Engineering platform and team is not defined early on
• Team is remote, making collaboration and quick decision-making difficult

Client situations:

• Clients that don’t understand user’s needs, want and desires and lack domain knowledge
• Cannot be involved face-to-face, day-today or make decisions quickly
• Many stakeholders that require documentation to provide context around decision-making

Agile Manifesto, 2001

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

Individuals and interactions over process and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

Agile approach
Deliver useful, working software early and continuously through iteration.

• Trust
• Flexibility and ability to handle changing requirements
• Client-driven or engineer-driven design, designers as consultant
• Solutions largely based on intuition or hunches
• Priorities are focused on addressing business needs and value
• Lightweight
• Fast but not necessarily cheaper

Agile methods

• EX – Extreme Programming
• FDD – Feature Driven Development
• Crystal
• Scrum
• DSDM – Dynamic System Development Method
• Adaptive Software Development
• Lean Software Development
• Rational Unified Process

“All the engineers need is a picture and a conversation.” Josh Damon Williams, user experience architect

When Agile works better
Types of projects:

• Applications with an existing conceptual framework already in place; adding features
• Start-ups with a very clear vision of their product, its features and goals
• Early working prototypes are that required for funding, speed to market is critical

Team dynamics:

• Trust is paramount
• Experienced, embedded team that can collaborate and actively participate
• Designers (user experience and visual) who are comfortable with constant iteration
• Engineers who are human-centric

Client situations:

• Clients who can make quick decisions and are available every day (but can change their mind)
• Business goals and vision are clear and will not change
• Client understands users needs, subject matter, and domain knowledge

Big disconnects – Waterfall vs. Agile
Holistic vs. modular thinking
Research-based assumptions / decisions vs. hunches / intuition
Enumeration and alternatives vs. iterations
User research vs. user validation
Scheduled changes vs. continual change
Artifacts vs. working software
Quality vs. quantity

Similarities
Full team involvement and participation is important on day one
User feedback is valued at all points in the process
Collaboration and respecting individual contribution is key to good design
Prioritization methods help set expectations
Design for flexibility; expect change and evolution
Communication is critical and needs to be transparent

Lesson learned

• Redefined the definition of “designer” – include everyone in the design process, but still be clear on ownership of decisions, including clients.
• Communicate and collaborate early and often.
• Be flexible, trust your team, and don’t let your ego get the best of you.

Advices: Step out of your problem solving role

Empathy
Vicarious experiencing of the feelings, thoughts, or attitudes of another.
Feelings, thoughts, attitudes, emotions, beliefs, behaviors, reactions, philosophies, motivations

Example – Mental model courtesy of Google Analytics Team, 2007
Understand My visitors (use first-person verbs). Make sure you are in the same mental model as they (users) are, wearing their shoes.

Advices: Play Psychotherapist to explore root causes
Don’t conduct interviews, don’t write interview questions, have conversation with people. Find out what makes them tick. Ask open questions, ask them in relation to what’s going on in the conversation. Listen, and think.

The Hallway Test – what is she thinking while walking down the hallway?

Advice: Build a Model of Behaviors and Harness it for perspective
Find patterns and similarities in the transcripts. working it from bottom to top, group and label patterns into table. bubbled up from the bottom.

Use mental models to compare and find out: How well we are supporting people (not users) in real life.

Example: Engage
Challenges are some of the emotions happens in real life, don’t happen online.
Find out the most shocking when meeting for the first time, doesn’t look like what they think.
Solution: “where i’m going to be”

Strategy is now

• Is it worth building?
• Can we trump competition in a different mental space?
• What’s the value of this to the user? To us?
• How feasible/fast is it to build this? Are there dependencies?
• When should we build it?

- – -
Book: Mental Models: Aligning Design Strategy with Human Behavior
Essay: Look at it another way on a list apart

The origins of scenario planning

Developing and communicating scenario worlds

Frameworks

• Make sense of what you see
• Build mental models, maps and metaphors
• Use needs to highlight opportunities

Imperatives

• Suggest directions and principles for development
• Offer guidelines for prioritizing and making decisions

Observations

• See the world with fresh eyes
• Listen and Learn
• Find points of view

Solutions

• Make concepts tangible
• Have an impact
• Get feedback

Seven Steps to planning scenarios:

1. Focal Issue – A specific, high-level strategic challenge facing the organization.
2. Key factors and Driving Forces – Key Factors (impacting the business) and Driving Forces (going on in the macro-environment) influencing the focal issue.

Common Practice – Focus on understanding the problem at hand
Reframe – Look broadly to expand and inform our thinking

“What will our business look like in 2025? Should we make an acquisition in a new industry?”

3. Critical Uncertainties – Understanding which forces will happen, which won’t and which will be critical and less critical?

Common Practice – Uncover the most relevant trends that exists today
Reframe – Imagine where trends are going, not just where they are now

“How will Millennials change the business culture? What role will the new administration have on the economy?”

4. Scenario Logics – The underlying rationales that make up a scenario’s plot.

Common Practice – Use imperatives as a tool to guide ideation
Reframe – Use ideation as a tool to explore what’s most important

“How plausible is this scenario? Does it challenge conventional thinking? Does it help us make decisions?”

5. Scenario Experiences – Illustrate the key critical uncertainties in action, describing how they unfold over the next seven to ten years.
6. Implications – How does the focal issue look in light of the scenarios? What are the strategic implications? What should we do?

Common Practice – Visualize ideas to make them tangible and increase impact
Reframe – Develop rich experiences to give context to solutions

“Write a story. Bring each scenario to life to people”

“How the focal issue look in light of the scenarios? What are the strategic implications? What’s next?”

7. Leading Indicators – What trends to monitor along the way?

Common Practice – step through the process to reach a great solution
Reframe – The strategic conversation never ends

“What are the indicators to monitor along the way”?

Strategies to make confident, smarter decisions in times of changes

• Focus on trends that are highly unlikely, yet impactful.
• Opportunities for the future of business travel.
• Cross different driving forces to ideate potential solutions

• Facilitating multi-sensory experiences
• Monitor how the future is shaping up
• Map the trends within opportunity spaces

Seminar video available – Go to video.

Focus on Latin America multi-cultural segment

Assimilation vs. Acculturation
Assimilation – the abandonment of one’s original culture in favor of a second or host culture. In the US, some immigrants gradually lose their original culture in favor of the second, thus assimilating.

Acculturation – in contrast to assimilation, is considered to be the process of learning a second culture while preserving or retaining their original culture. In the US, some immigrants acculturate by adding a second culture, not replacing their first culture.

Define and Articulate
Hispanic Dominant – people who have not adopt to US culture
Culturally Unique – somebody doesn’t adopt to mainstream culture and not maintaining their original culture (very few of these kind, not-relevant)

Findings to save you $500,000 – Culturally Unique and US Dominant are not relevant groups

Acculturation Factors
Technology/Transportation

• Long distance rates
• Internet/chat/email
• Air transportation costs and routes

Changing U.S. Society

• Acceptance of multi-ethnicity
• Coolness
• Benefits of cultural melting pot

Age and Education

• Use of English
• Knowledge of U.S. culture
• Openness to integration

Reasons for Immigrating

• Living the American dream vs. layover
• Poverty vs. Prosperity
• Family reunification

Acculturation Summary
Hispanic Dominant
Language skills | Spanish
Media | Spanish-language TV (particularly commercials with famous Hispanics); Hispanic-oriented publications.
Marketing | Word of mouth: New arrivals in the US will ask those already established for recommendations on where to shop and what to buy.
Shopping habits

• Buy products imported from home country
• Don’t use coupons
• Tend to buy same brand as last time
• Trust brand names
• Language barriers may limit the types of stores they visit

Bicultural
Language skills | Spanish and English
Media | Consume roughly equal amounts of media in both languages, but English competency doesn’t amount to preference – Spanish-language ads may speak more to them.
Marketing | Word of mouth will also work, but will also respond to general market techniques; however, Spanish-language or Hispanic-oriented marketing may resonate better.
Shopping habits

• Heavily influenced by price and family
• Many use coupons

US Dominant
Language skills | English
Media | Similar to general US population; English-language advertising through standard channels
Marketing | Similar to general US population
Shopping habits

• Heavily influenced by price and easy preparation
• Many use coupons
• More mainstream US food purchases

To save you $750,000
1. Language at home and with friends – determine if they are hispanic or us dominant
2. Media Usage – length of time will effects
3. Country of Birth (length of time in US)
4, 5. Closeness to US culture, Closeness to Hispanic or other Original Culture – ask use scale of 1-5 how close they are

Cultural Values
Optimism is common amongst immigrants
Collectivism – commonly most important in hispanic and asian cultures

Example: Humana, the health insurance company facing challenges in the hispanic culture as people relies on spirituality, fatalism. Everything is in god’s hands.
Solution: design frameworks to educate the population, not to throw brochure blinded at them.

Example: American Airlines approaching to hispanic business travelers.
Solution: approach them from a journey, progress, sense of self sense. “I don’t know where i’ll go…i think i’m going there.”

Example: Little Debbie to latch on sweet 16 tradition (huge business in hispanic family tradition). Competitors are Bimbos, Mariaras (bread products).
Solution: Home-country tradition, family

Example: Ford. American Airlines images and case study around progress and journey. A terrible word of mouth about Ford’s poor quality products.
Challenges: Oral traditions, folklore and myths.
Solution: counter-effects with oral traditions (blog,…) Will it work? still in progress…

Trends, Innovation and Multicultural
WE KNOW: For GM “Green is the new black”
- expression of self
- badge
- sophistication

- Prius
- BP Helios HOuse
- Ecopalms

- beyond ‘trendiness”
- economic imperative?

WE’VE HEARD: Hispanic Market = Organic Growth
But, it there a strong relationship between hispanic group and environment awareness??

As the result…they are more eco-friendly (i.e. turn-off tv to preserve power)

To find out if there’s a relationship, the Answer may lie in ethnographic research…ex: health

Usage of mobile technology

Q&A
[observation] Mexicans tends to be very reserved during focus group. while say, Columbians are more vocal and willing to share their opinion.

Q: What’s nostalgic and what’s current?
A: The culture here in the little italy/little village are definitely different from back at home. It is not the best practice to observe little village hoping to be representatively of the entire country.

Design = Problem solving
Design Research = Problem seeking

Principles of Insight Translation:

• Aspirational – expresses what people wish.
• Actionable – data that affords translation. Concrete. Sensory. Direct. (this one is tough)
• Meaningful - clearly linked to the research data. ( back up with numbers)
• Inspirational – descriptive not prescriptive. (combination of different inspirations show possibilities)

Examples: What is a rugged phone?

• Stimulus planning
• Collaborative image
• Sample kit
• Colleague
• Increase meaning (connection to the date) by shortening the leap between actionable stimulus and inspirational

Big Ideas

• Provides a clear transition from customer meaning to product attributes – focusing team creativity for market success.
• Requires visual skills and concrete thinking to embody abstract ideas and principles.
• The object is not to solve problem, but to create a framework to focus creativity.
• Planning is critical to getting the right input from people to effectively translate insights for design teams.

• In the end how to present the ideas in a concrete matters.
• Question the users what’s important.
• Before use the words, find out what that means.
• Materials, forms and shapes are carefully chosen for he sample kit.
• You need to discuss the “concrete” thing first before you can run the principles of insight translation.

How to identify the word “roughed”
Find out what’s matter to users and what’s important to them based on their explanation.

The design and research boundaries are blurring
Design and research are blurring at the front end of the design process. It is a time of disruption and confusion.

Changes can be seen
The roles people play in the design processes are changing. These changes can be seen in the language we use.

The name that we used to refer of the people we are serving having changed and become more and more proactive. (customer -> consumer -> user -> adapter -> participant -> co-creator -> )

Co-creation – Expert-driven and user-centered is not going away. The new thing is not replacing the old thing. But we need to know when and which is taken place.

“Co-creation is a new and innovative marketing mantra intended to keep consumers brand loyal.” www.admark.wordpress.com

We’ll focus on the user-centered related co-creation.

A map of design practice and design research

A map of design practice and design research is evolving globally.

Design lead (top to bottom) vs. research lead approaches (bottom to top)
Expert mindset (users seen as subjects) vs. Participatory mindset (users seen as partners)

• Human factors and ergonomics
• User-centered design
• Scandinavia
• Participatory design
• Critical design (a challenge to user-centered design, design lead)
• Generative design research (also to challenge user-centered design, use design lead tool to bring users into design in the early beginning) – this is Liz’s area she is focus on

There’s no good or bad in the map. The key is to figure out where you are. Where you have to be in order to do your work better.

Transformations in design practice

Co-designing teams are expanding to include all the relevant stakeholders, particularly in the front end of the design process.
We see this particularly on design challenge of large scale and complexity.

Old design discipline (deterministic outcomes based on discipline)

• Visual communication design
• Interior space design
• Product design
• Information design
• Architecture
• Planning

New design discipline (focus on purposeful and holistic outcomes)

• Design for experiencing
• Design for emotion
• Design for interacting
• Design for sustainability
• Design for serving
• Design for transforming

Collective creativity

Design is taking on increasingly collective forms. The new domains of collective creativity will require new tools and methods for designing and researching.

Example: Generative tool is used in group level

Consultant, researchers, health-care professions are involved

Tools are not only being designed, but play-acting in actual environment where the tools will be used

Tool-kit (doll house) – collective creativity

Even userability testing is a collaborative activity

Discovery: A new view on the blur

Let’s not try to classify it as if it’s design or research, but rather call it Discovery.

There are 2 kinds of discoveries that are going on in the front end:
Exploration

• serendipitous
• opportunistic
• open-ended
• future-oriented

Research

• systematic
• rigorous
• structured
• knowledge-seeking

The new map:

• Design exploration: this is what the designers are trained for
• “Real” research
• Research using design thinking
• Experiments and interventions (rare to see)

Future challenges
In the future, designers and researchers will be the creators of scaffolds upon which everyday people express their creativity.

Currently we are on collective generativity, the future in at cultural sustainability.

Bryce Glass, Sr. Interaction Designer at Yahoo!, shared his presentation on building reputation system at the 2008 IA Summit. Highlights from the slide presentation:

Yahoo! Design Pattern Library: Reputation Patterns

Define your business goals

• Drive user engagement
• Promote a specific feature
• Acknowledge top contributors
• Increase content quality
• Promote user retention

Community Spirit
The Competitive Spectrum

• Caring (i.e. Support Groups, A Live Journal blog)
• Collaborative (i.e. Wikipeida, dating sites)
• Cordial (i.e. Yahoo! Answers, Message Boards)
• Competitive (i.e. Fantasy Sports, Casual Gaming)
• Combative (i.e. Halo on Xbox Live, Hot or Not)

People write reviews

• Self-interest
• Interest in others
• Interest in the Object being rated

Entities accrues reputation

• People (video, blog posts, bookmarks, article submissions, collections, playlists, message board post, reviews, social objects)
• Content

Takeaways

• Rate the thing, not the person
• Beware of ambiguity in semantic
• Contexts should generally be as specific as necessary, but still general enough to apply widely

Global health expert Hans Rosling’s presentation on debunking third-world myths with Gapminder. Developed by Hans himself, Gapminder shines new meaning into stats and set right myths due to preconceived ideas.

More on Hans Rosling and his contributions here.

During the keynote, Paul presented the14 distinct gamer groups. Of the 14 segments, Heavy Action, Slow Strategists, Fantasy Worlds, and Virtual Life are “Core” gamer groups, the rest are “Casual” groups. Core gamers are also considered as the target audience as the study showed there’s a overlapping 58% of gamer play both casual and core games.

Gamer profile – Nancy Drews
Older female gamers that prefer to play alone and like to think, relax and rejuvenate.

Primary Genres:
- Match 3 Puzzle Games
- Brain Teasers/Logic Games
- Mahjong
- Word games
- Hidden Object Puzzles
- Jigsaw Puzzle Games
- Trivia Games
- Puzzle Adventure
Key Findings:
- 63% female
- 59% over 35
- Highest concetration of retirees and unemployed but not looking for work
- Over-represented in Old School and Clicker segments
- Least likely to have a console of any kind, except a handheld console
- More likely to have reading books and visiting eBay

Gamer Profile – Heavy Action
The mainly man gamer that plays for conquest and adrenaline.

Primary genres
- Shooter Games
- Racing
- War Games
- Driving Mayhem Games
- Fighting Games
- Flying Simulations
- Realistic Sports Simulations
- Heavy PRG or RPG

Key Finding
- 73% male
- 45% 18-34 years old
- A surprising 56% also play in the Nancy Drews segment
- Over-represented in Fantasy Worlds and King of Kongs
- Under-represented in Old School Segment
- High concentration in military, craftsman, and students
- Under-represented in retirees and home makers

Heavy Action vs. Nancy Drew

Gamer Profile – King of Kongs
Attracts the middle age male tech-savvy Internet surfer longing for the good old days of quarter arcades.

Primary Genres
- Classic Action Arcade
- Classic Arcade inspired