Mind on Marketing

In her all-day day session at EDUI2009, “Usability Testing Without the Scary,” Dana Chisnell laid out the entire process for usability testing. Breaking the process down into small parts that are easy to understand, Dana showed us how to be comfortable with the process, rather than intimidated. Here’s what she shared about creation of the plan for usability testing.

Someone who will try the design, somewhere to test and something to study.

As Dana puts it, the essence of testing is “sit next to someone and watch them do stuff.” Pretty straightforward. You just have to know what you want them to try out, and how to watch and ask questions so you can learn what they are thinking as they work with your design and functionality.

To support great site design, each phase of the development process should be supported by input from users. It’s also essential to have multidisciplinary teams working on development (ie. marketing, IT, management, etc.).

Although it’s highly useful to look at best practices, guidelines and conventions for usability, they aren’t enough to guarantee usability and accessibility.The nuances of implementation can conflict with best practices, and best practices can even conflict, cancel each other out, or magnify certain issues.  “Informed designs come from data,” says Dana.

The goals of usability testing should be to identify problems in design that lead to

• misinformation
• incomplete transactions
• need for support from administration, management or staff

What tests and measurements to do when

–When you want to map out what the design should do, you observe and listen to users through user testing, focus groups, participatory design, surveys, heuristic evaluations and setting benchmarks.

–To figure out how the design should work, use participatory design, paper prototyping, walk-throughs, usability testing and heuristic evaluation

–To determine whether the design actually does what you want it to do, conduct usability tests, do heuristic evaluation, follow-up studies and comparison of benchmarks.

In the early part of the process, do things that help you learn, that are exploratory and formative. In the middle, do things that help you assess and summarize. And at the end, validate and verify when you’re close to launch but have enough time to incorporate changes.

The User Testing Plan

It’s important to create a test plan. It serves as your blueprint for testing, is a communication vehicle, clarifies needed resources, is a focal point for each test, and lays out milestones. It should include:

• Goals and objectives**
• Research questions**
• Participant characteristics**
• Description of method**
• List of tasks**
• Description of test environment
• Say what the moderator will do
• List of the data you’ll collect
• Description of how the results will be reported

**If your time is limited, focus on the starred tasks.

Next: Selecting participants

Tweet This Post 

In her EDUI 2009 Workshop, Usability Testing Without the Scary, Dana Chisnell discussed how to best support great site design. User testing is essential, even if you only have time to spend one hour with one person. If one person has a problem, it’s legitimate to assume that others do too, and to make design changes based on that data. It’s also important to have management buy-in to doing testing–they must be able to see the benefits of doing the research and implementing what is learned from the tests. Multi-disciplinary teams also support implementation of great design, provided they speak one anothers’ languages and understand one anothers’ skills. Finally, it’s essential to be willing to learn as you go and make changes accordingly. (Note that these criteria are completely in alignment with Jared Spool’s research showing that vision, feedback and culture are the most important elements of developing great design.)

Great designs are :

• Useful–What’s valuable to your user about having this information or this tool.
• Efficient–Users can accomplish their goal quickly (that means you have to know what your users’ goals are).
• Learnable–It should be easy for users to apply what they’ve learned about how to use other sites to using your site (your site should conform to usabilities conventions as much as possible).
• Satisfying–Your site should be one that people are happy to use, one that seems to be easy and doesn’t take too much time.
• Accessible–Implementing accessibility guidelines actually makes your site easier for everyone to use.

You should identify design problems that lead to misinformation, incomplete transactions, and/or necessitate support from administrators or staff.

What kind of assessment to use to answer fundamental site design questions

Dana segments analysis into three areas and defines certain types of assessments to use for each area.

What should the site design do? When you want to answer this question, do usability testing, conduct focus groups, have users participate in the design process, do surveys, conduct heuristic evaluations and set benchmarks.

How should the site design work? To answer this question, use participatory design, paper prototyping, walk-throughs, usability testing and heuristic evaluation.

Does the site design do what we want it to do? Answer this via usability testing, heuristic evaluation, follow-up studies and comparison to benchmarks.

Next: Dana’s pointers on putting together a test plan

Tweet This Post 

Dana Chisnell presented an all-day session at the EDUI Conference held in Charlottesville  September 21 & 22, 2009. She founded Usability Works and does usability research, user interface design, and technical communication consulting and development. She is co-author of the Handbook of Usability Testing.  The focus of the workshop was to show how eminently doable user testing can be, if you let go of the idea that you have to do it “by the book.”  Classic, by-the-book user testing can seem intimidating and scary. Instead, Dana spent the day showing how to identify and do the essentials and make it easier to test, and to do it  on an on-going basis. Her approach takes into account that there usually isn’t a lot of time to test. The following is a list of all elements “by the book,” and I’ve noted which ones Dana says are essential:

• Develop a test plan ESSENTIAL
• Choose a testing environment–Don’t test in the lab—go to where the users are
• Find and select participants–ESSENTIAL
• Prepare test materials
• Conduct the sessions–ESSENTIAL
• Debrief with participants and observers–ESSENTIAL–Debrief only with observers
• Analyze data and observations
• Create findings and recommendations

User testing works best when you’re early in the design process (that’s where testing using paper prototypes comes in very handy). What you need is someone who will try the design; somewhere to test (preferably where the user typically would use your site, such as at home, the office, or in a dorm room); something to study. Neat and simple. And you’ll learn things about your website interface that you would never have seen on your own.

For example, in conducting user tests of the University of Pennsylvania website,  Dana learned that users were looking for admissions “dates.” But they couldn’t find the information because the University used a different term: “timetable.” User testing provided the context in which they learned about this very simple labelling problem. Your site has to accommodate the way your users think and work, and you can’t be certain about that unless you actually watch them use your site. You must learn whether the user grasps the conceptual design of your site and if not, what the user’s concept is.

 Pointers for moderating user tests

There’s tremendous value in sitting next to someone and watching them do something. Dana recommends saying to the tester things like “tell me what you’re doing–think aloud… tell me what you’re thinking…” When the tester is quiet, be careful about when you ask a question–sometimes they’re quiet because they’re working on a problem, and you may not want to interrupt that process. At the end of the test, review the experience with the tester. Walk through what happened, what happened next. Ask “how did that go? What was confusing or frustrating.” Dana conducted a demonstration, working with an attendee. I found the moderator questions she asked to be very helpful:

• How would you describe the information you’re looking for?
• How far do you think you are from getting it?
• Are you warm or cool?
• You were hovering there–tell me what you were thinking.
• What question do you have about the site right now?
• What do you think the site is about?
• What one main thing should be improved?

It’s essential for the moderator to talk with the tester and create a situation that feels natural to him or her. As moderator, you should be neutral and objective, but also friendly. Remember that this is not a psychology experiment, it’s a test of user-centered design.

NEXT: What Dana says about design.

Tweet This Post 

Tricks and techniques work better for improving website usability than do structured methodologies, Jared Spool (CEO, User Interface Engineering) said at the EDUI conference last week. Spool says that there is no evidence that methodologies result in quality designs, and that design can’t be institutionalized to conform to usability templates. He equates methodology with dogma, defined as “an unquestioned faith independent of any supporting evidence, using the TSA’s airport screening procedures as a prime example of methodology’s lack of value. His research found that:

• The best teams didn’t have a methodology or dogma they followed
• The struggling teams often tried following a methodology, withou success
• The best teams all focused on increasing the techniques and tricks for each team member
• They were constantly exploring new tricks and techniques for their toolbox
• Struggling teams had limited tricks and techniques

He recommends that usability development teams focus on developing great techniques and tricks and avoid get dragged down by methodologies. Techniques, he says, are things that you can master through practice, building blocks that can be applied to any methodology.

How the best teams develop great design

Vision, feedback and culture are the most important variables in the way the best teams develop great design:

• Vision: Can everyone on the team describe the experience of using your design five years from now? Spool says that five years is an optimal time period becaue the team’s thinking is not constrained by current legacy ( ie.technology limitiations), freeing them to focus on the aspiration of the design. Everyone on the team should have the exact same vision. Focusing the vision gives the team the chance to know where they’re going. They can deal with how to get there later on in the process.
• Feedback: In the last six weeks, have you spent more than two hours watching someone use either your design or a competitor’s?Everyone should answer “yes” to this question. You don’t know what’s happening with your design until you watch people use it. Spool points out that great chefs go out into the dining room and ask people about the food.
• Culture: In the last six weeks, have you rewarded a team member for creating a major design failure? Spool says that we only learn by making mistakes. He advocates making a huge investment in order to a failure, mentioning that Scott Cook, founder of Intuit (Quicken and Quickbooks), gives a big party for someone who has “screwed up big,” that celebrates all the things they learned. Spool asserts that organizations that are risk averse “produce crap.”

Spool says that user testing can be as speedy as the “Five-Second Page Test” that he uses. Have the test participant look at a page for five seconds (not the home page, but one with a specific purpose, such as providing customer support information or making a donation). Then ask the participant to write down what they remember aboutthe page and whether they would do business with the organization. The test is particularly useful when a page is too cluttered or confusing, and identifies whether pages quickly communicate their purpose.

On the other hand, Spool says that paper prototype testing is especially useful when the design is in flux. The team can participate in the sudy at a point where they can make changes before going into technical implementation. Paper Prototyping, by Carolyn Snyder, is a recommended resource.

What a couple of Jared Spool's paper prototypes look like

Virtual seminars and many other free resources are available on Spool’s company website and on his blog, Brainsparks.

Tweet This Post 

 Twitter’s limit of 140 characters is one way that the medium shapes the message and the conversation. Wesch talked about the ways that digital media affect self-awareness and the construction of identity. Twitter’s “What are you doing?” contributes to “lifecasting” or “mindcasting.” In creating YouTube videos, we have no idea who, exactly, will be watching. We talk to the webcam. And when we look at YouTube videos, we are watching others without staring at them. Context collapses, as we saw in the “Bomb Iran” performance of John McCain, who thought his behavior was perfectly acceptable to the audience he was addressing in person, military veterans. But when the performance, captured digitally, appeared on YouTube, the context changed completely, as did the reception of his act.

I was particularly struck by Wesch’s comment that the anonymity and physical distance inherent in both traditional and digital media have enabled what he calls “hatred as public performance.” We see this on reality TV, in flame wars and vicious comments posted on YouTube, blogs and other places. It’s one outcome of the tide of narcissism that afflicts our culture, combined with a sense of hopelessness and impotence, the behavioral amplification of “whatever.” At the same time, he says that it’s important to have sites where comments can be posted anonymously, as a mechanism for great creativity and as a safeguard for free speech.

In spite of what seems to be a bleak analysis, Wesch is moved by what he sees as real and caring community on YouTube, and the emergence of more civil discourse through crowd ratings of the content of posts on Reddit (Wikipedia notes that “When there are enough votes against a given comment, it will not be displayed by default… Users who submit articles which other users like and subsequently “vote up” receive “karma” points as a reward…”). He’s amused and encouraged by the Free Hugs video that went viral on YouTube (as of this writing, the video has been viewed 50,748,157 times), and the kindness off the responses to a YouTube creator who spoke about her hopelessness.

There’s opportunity and danger in digital media, Wesch says, and we need to see and seize the opportunity or we’ll be washed up thanks to mainstream media. We need to be aware how the machine is using us, so we can guide the future.

Tweet This Post 

Professor Michael Wesch’s talk at EDUI 2009, “Mediated Culture,”  may well have been attendees’ favorite session. Looking at relationships between individuals, communities, traditional mass media and the state, Wesch showed how blogging, YouTube, Twitter and other social media provide us with a new way of relating to one another, a way that has not been available in television and print.

Wesch builds on his own anthropological research in New Guinea, explaining that the creation of a state book of law changed the way individuals related to one another. When the law book was published, communities began to gather to measure up individuals against the rules and apply punishment accordingly. The book shifted focus from individuals’ relationships to one another to individuals’ relationship to the state. He says that the media are not just tools or means of communications but that they mediate our relationships. When media change, our relationships change.

Drawing on Neil Postman’s book, Amusing Ourselves to Death, Wesch says that our culture is far more the way Aldous Huxley envisioned it, rather than George Orwell. Huxley commented that it wouldn’t be necessary to ban books, because no one would want to read them.

Television became the primary medium for  the conversations of our culture, controlled by a few people and designed for “the masses.”  The conversations are always entertaining, even the serious ones. They’re punctuated by commercials and they create a culture of irrelevance, incoherence and impotence. As Huxley said, we are amusing ourselved to death. At the same time, because so much of the media is aimed at entertaining us, we feel special, flattered, as if shows have been created just for us.

 Wesch shows a funny picture of school children demonstrating various states of boredom in the classroom and explores the origins and meanings of the word, “whatever.” He explains that reality TV has shaped the meaning of the word as, “who cares about you–I’ll do whatever I want.” In addition, kids have been told by their parents the world is their oyster. Becoming a contestant on reality TV has become the way to prove that it’s true. And, as all but the shows’ winners discover (and as they head into the world in their twenties), they see that it isn’t true. We see that discovery happening on every episode of American Idol and the other reality shows.

Moving to a discussion of digital media, Wesch comments that it is fundamentally different from anything we’ve seen before. We still use concepts from the past, such as referring to web “pages.” But Tim Berners-Lee said, long ago, that the Web is not supposed to be a glorified television channel.” And thanks to blogging, YouTube and other social media, anybody can create and publish anything for “the masses.” Evolving Web standards for web browsers have helped make that possible. It’s no longer necessary to know HTML and other aspects of digital technology to have visibility online.

(To be continued…)

Tweet This Post 

The University of Virginia hosted an outstanding usability-focused conference yesterday and today, EDUI 2009. The Virginia Foundation for the Humanities and the University of Virginia, in partnership with User Interface Engineering, produced the conference, featuring a star-studded roster of presenters (forgive the hyperbole, but this conference was an amazing value for the cost of attendance). Today I’m acting as your amateur reporter, providing a quick summary of the conference. I’ll share info from the sessions in forthcoming posts, in the interest of brevity.

I spent yesterday in an all-day session with Dana Chisnell, an independent usability consultant and user researcher who founded UsabilityWorks. She co-authored the Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests.  The session was titled Usability Testing Without the Scary. Dana reviewed the entire process of user testing, from developing a test plan to recruiting users, working with stakeholders, summarizing and prioritizing issues, to reporting on results and making recommendations. The day ended with a reception hosted by Intalgent at their new venture/venue, OpenSpace. Wonderful networking, great food and wine, and a very funny improv performance.

Today began with a presentation by Michael Wesch, who teaches Cultural Anthropology and Digital Ethnology at Kansas State University. Dubbed “the explainer” by Wired magazine, Michael Wesch is a cultural anthropologist exploring the impact of new media on society and culture. The EDUI session was titled Mediated Culture, and showed how the media are not just tools or means of communications—they mediate our relationships. When media change our relationships change.

I attended “How Your Design Feels,” presented by Dan Rubin, founder and principal of Webgraph (a design and branding studio), co-founder of Sidebar Creative (a design collective) and Sinelogic (UI and usability consulting) as well as a web standards consultant and speaker.

Jared Spool, founder of User Interface Engineering, the largest usability research organization of its kind in the world, presented the conference keynote address, Cooking Up Gourmet User Experiences on a Fast-Food Budget.

I’ll share details on these sessions starting tomorrow.

Tweet This Post 

What is the goal of your website? That’s the first question Seth Godin says you must ask yourself when you’re getting ready to overhaul your website or launch a new one. And I’ve seen first-hand how many site owners have difficulty extending the answer beyond “because we have to have one.” Yes, having a site in order to provide basic information about your business is important. But how will you gauge your site’s effectiveness? Expect your site to perform for you. Set goals. Review performance. Tweak content and navigation to better meet goals. Repeat.

“What outcomes do you expect?” asks Godin. Take your goals beyond just putting up brochure-ware. The web is interactive. Use your site to establish and nurture relationships that lead to ongoing business. How many leads do you want to generate for your business pipeline? How many do you want to convert to customers? How will you use your site to do that? If you’re driving people to your site via Twitter and Facebook, what’s waiting for them when they arrive? How well does it work?

These are tough questions for many business owners. Although they’re accustomed to looking hard at the performance of other elements of the business, the web site is often launched and forgotten. And forgotten sites get boring quickly. Not a good thing if you’re trying to engage people. Strong content that gives people the gift of deeper or broader knowledge, and/or that entertains is an important element of user engagement. Content that they can read, listen to or watch. Tipsheets, white papers, podcasts, YouTube videos all provide engagement on a variety of levels, as long as the content is relevant and well-presented. One great way to engage visitors is to put a blog on your site, using your own URL, and add to it frequently. The added bonus is that Google loves frequently-updated content, and your site will rise in their organic search results.

Tweet This Post 

I really like Hubspot’s Blog and also the many webinars they offer. They know how to market their business. Today’s post got my attention via a ReTweet from HootSuite, “How Do You Market Successfully Today? Same as Always: Creatively.” They make a point that took me awhile to learn on my own–social media, which was new to me many months ago, is just another tool for inbound marketing (what we used to call in the old days, prior to 1996 or so, direct marketing). There’s no pixie dust. Darn. I love technology and often fall in love with the next new thing, hoping for for magic.

The thing is, a lot of businesses engage in social media and other inbound marketing channels without paying enough attention to their websites–the home base for most of these activities. You drive prospects to your website, right? What shape is it in? Neglecting your website is like inviting prospects to your office. You put away files, make sure they have a place to sit, and offer a cup of coffee when they arrive. You made a great first impression. But deals get done when you show you care and good relationships are forged.

Hubspot points out that content creation is the marketing strategy. It’s what makes an impact on the first visit, and what compels people to come back for more–provided you continually refresh your website with new and relevant content. Blogs are a great way to do that and help your SEO position–I’ve learned that by experience. Google loves fresh and frequent content. As one comment in response to the blog post observes, static sites don’t cut it.

Tweet This Post 

Just came across this post from Harvard Business Publishing, thanks to my Linked-In Marketing Executives Group. Alexandra Samuel comments in a piece published July 27  that there are three great options for “robust social media presences that let you manage cost and isk by building on existing tools and established best practices.” She describes how to execute each idea, where to spend and where to save, and where to get help.

1. Set up a suggestion box and ask customers, employees and others to submit ideas for new products, services or improvements.  Then have the community rate submission to identify the top ideas. Then implement them.
2. Create an interactive widget for your supporters to put on their Facebook pages or blog. It can display your latest news, deals or contests, solicit donations, etc.
3. Offer special, limited-time deals–a product available in limited quantities, a discounted service, or donation matching. Update frequently to give your audience a reason to visit your site often.

In another essay published August 3, Samuel makes a great case for giving up trying to stay up with social technology and instead to focus on your core business challenges: Don’t Keep Up With Social Technology.

Tweet This Post