Practical Paper Prototyping

Thursday, June 18, 2009 - 22:49

Practical Paper Prototyping

These are my unedited conference notes. Soon I'll post a better article, with video from the conference.

We fail at the wrong time

Here's the problem: we fail at the wrong time.

We fail at the wrong time

invent design build test deploy fail

This is what happened to the Tacoma Narrows Bridge.

All of us fail; all of us are wrong and make mistakes. Far from being something to shy away from, we should rush to and embrace mistakes, for it is only by making mistakes that we truly learn.

Worst place to fail?

fail invent design build test deploy

You could argue that this is the worst place to fail, before we even try.

But then what is the best place to fail?

Best place to fail?

invent design fail build test deploy

Here, I think; this is the sweet spot. We have the rush of invention and the pleasure of design, but have not yet incurred the cost of construction.

If we always fail here, if we always catch our mistakes here, we will always fail as cheaply and usefully as possible.

So that's what this talk is about: how to fail in the right place, consistently.

"Like me"

Frank Herbert once said that the only useful definion of "human" is "like me." This talk is for people like me.

The first software I wrote that was used by another person was a small UI enhancement for my father, on a Tandy 1000. Like everything I've done since then, I care less about how to build the guts than about how to make them accessible.

I've been with OpenSourcery for four years, initially as Lead UI Designer and now as Director of Engineering. I'm part process geek and part union representative for our developers.

Today, my work varies. I've been on one project for over 5 years, and regularly on some new projects I spend only a handful of hours. Regardless of size, these projects have common problems.

Challenge: enough time

For me it's always hard to get what I feel is enough time for quality user-facing design.

Challenge: solving the user's real problem

It's very easy to solve the wrong problem; there are so many. If you hear the user's story only once, and filter it through your own perceptions, as we all do, and there's no telling where we can end up.

Challenge: embrace being wrong

I hate being wrong. Hate it hate it hate it. I hate failing.

But this is one type of work where we must embrace failure, for the faster and more often we fail the closer to the truth we get. Assuming we have a good process.

So how do we fail at the right time? How do we do it quickly, cheaply, and in a way guaranteed to reflect our audience's real needs?

Prototype, Paper, Practical

We create the illusion before the reality, and we test the illusion, and it's surprisingly effective.

We prototype on paper because it's very, very fast. I can sketch an interesting widget in a minute. Humans are fundamentally pattern recognition devices, and we only need enough fidelity to trigger that pattern recognition, and allow people to fool themselves.

By which I mean, you can do it yourself, practically for free, using nothing more than what you have on hand, and what we're going to discuss in the next 40 minutes.

What is Paper Prototyping?

"Paper prototyping is a variation of usability testing where representative users perform realistic tasks by interacting with a paper version of the interface that is manipulated by a person 'playing computer,' who doesn't explain how the interface is intended to work."

Carol Snyder, "Paper Prototyping"

Let's unpack this. The ideal paper prototyping session has several components.

The user

First, the user. The user you choose should bear some fair resemblance to the target demographic of your application.

The computer

The computer is a person manipulating the created prototypes as indicated by the user's interaction. This person should be as robotic as possible, not responding to anything the user says or does that isn't directed at the prototype.

The observers

The observers are often your team members, other people who are creating the software with you.

The facilitator

The facilitator's job is to keep the computer safe. Users often get frustrated interacting with a prototype, and as the facilitator you can prevent an angry user from throttling your poor computer.

I still keep a paper prototype I made 4 years ago; it's half a page, and it's pretty mangled. It looks almost as if someone grabbed it in her fist and shook it in my face and said, loudly, "I can't work like this! This will ruin my life!"

Don't underestimate how frustrated users can get. Regardless of what you say, they will feel tested.

The facilitator's job really is to focus on the user to avoid frustration. Also to provide instructions, answer questions if appropriate, and push the user a little to discover why they're having trouble.

And they will have trouble because, as we've discussed, it's our job to fail.

The video crew

Video-taping a session can be very useful. One good setup is to have two cameras, one pointed at the user's face and one at the prototypes being manipulated by the computer.

If you search YouTube, you'll find good examples of this.

That's a lot of infrastructure

I'm convinved that many people don't start paper prototyping because they don't think they can get this infrastructure together. And you know what? You don't need it.

Sure, it's useful; I don't have a bad thing to say about any of it. But you can still get excellent results without it.

Let's strip this down to its essence.

Don't be completionist

  • Paper
  • Pen
  • Scissors (optional)

There's not much you really need: paper, pen, scissors.

The essence?

  • Simple
  • Interactive
  • Fast

There's nothing here that can't be done pretty well by one person, that is, you. You can be observer, computer, video crew, and facilitator at the same time.

Not for every context

Now, you're not likely to be able to prototype an entire online banking application this way. After an hour of playing these three roles you'll be tripping over your notes, begging for a video camera.

But online banking isn't the problem we're trying to solve.

What is the problem?

To give us a little taste of how this works, and to try to establish some general axioms, we're going to walk through prototyping and testing a little application.

Now, if I'm going to spend 45 minutes up here talking about something, it had better be funny. And for my money there's nothing funnier than a duck.

A duck

And here's my second favorite duck joke:

Q: Why do ducks have little flat feet? A: To put out forest fires.

Q: And why to elephants have big flat feet? A: To put out flaming ducks.

We're going to be building an application for Duck Delivery.

Duck Delivery

We paradrop firefighting ducks anywhere in the Pacific or Mountain time zones.

But can't ducks fly?

Sales call

Sales walks up to me one day and says, "We've got a new client. It sounds crazy, but he rents out ducks and elephants, something to do with fighting forest fires. He wants to get online before the summer fire season. I told him you were our rock star designer, and that you'd come to the meeting with some good ideas."

And he'll be here in an hour

Now, let me emphasize that none of the sales guys at OpenSourcery have ever done this. Seriously, those guys are pros. I always get at least two hours.

An hour, huh? That's not much time.


And now's where we start talking about ego. What do you want here? Do you want to look like a rock star this first time, right now, in front of this client? Or do you want to solve his problem in the best way possible?

Because if you want to solve his problem you have to let go and stop being afraid of being wrong. Embrace being wrong.

Prototype #1

So we take what our sales guy told us and start sketching. Our goal here is to spend 30 minutes on something quick to show one of our colleagues. I don't know about you, but I haven't designed a shopping cart for a while, and I know that Alex next door has done a lot of ecommerce work. We don't need full prototypes for him, just the sketch of an idea.

What we're going for is a simple home page and then a shopping cart. This seems like enough to cover it: what we do, how to find out more, how to get some ducks.

  • Clearly indicate affordances

We have a mental model of how this works: user asks for ducks, we see if we have them, we take their shipping information and money, we send the order. And this seems like a simple way to do it: ducks and elephants, date range. That should be enough to check availability.

  • Leave the user space to write
  • Preserve your originals

We're going to steal Amazon's login page here.

  • Steal, steal, steal

And we assume that the user doesn't have an account already.

They tell us where they want the ducks, and how they're going to pay for them.

  • Decide your level of detail

And we give them a nice confirmation page.

Prototype #1: colleague feedback

  • Availability location

We walk next door to Alex and run him through. He finds a problem on the first page. "You're talking about two time zones, right? That's a lot of ground. What if availability depends on location?"

Oooh, that's a good point, we think.

  • Stay cheap

Prototype #1: colleague feedback

  • Change on confirm

"Huh, what if they want to change some of this information? Using the back button through multiple form submissions is a pain in the butt, and if your cart's not designed for it things might break. You should give them some way to change information on this page."

Another good idea, and one we can implement without creating a new prototype. Sweet.

Prototype #2: Walk-through with client

We make these changes, and our time's up. The client walks in, you sit down and chat for a minute, and he says, "Your sales guy told you all about our business. Let's see what you've got."

You explain what you're up to. "These are paper prototypes, quick sketches of possible functionality. I'd like us to talk through them as if they're a real application, to see what meets your needs and what does not. I'm sure they'll be wrong in a number of ways. I'll incorporate your feedback into the next round of sketches, which we'll test with some of your users."

"This home page will be controlled by a content management system; that is, you'll be able to change it as you like. For now, let's start with ordering some ducks."

And you put the paper in front of him.

Right away there's trouble.

Prototype #2: client feedback

  • Elephants are project managers

"Ah, see, we supply the elephants based on the number of ducks. One elephant to 25 ducks. The elephants are project managers; ducks just don't self-organize very well. Or self-extinguish. Can you just force this number? Make it so that we set the calculate the number of elephants based on the ducks?"

Sure, you say.

Prototype #2: client feedback

  • Use GPS

"Also, city and state won't help. These fires are out in the wilderness, mostly, and people do it all by GPS these days. Take some GPS coordinates instead."

Oh, that's great feedback.

Prototype #2: client feedback

  • No logins; real clients are state agencies

"Login? People don't login. We deal with state agencies and they just give us a purchase order number. We need a PO number and someone in the agency to authorize it; that'll do it."

"Yeah, the confirmation page looks good. Listen, when can you have this done?"

Let's reflect a minute

Prototype #3: Test with real user

We have some really good feedback from the client, and our first user tests should be much better because of it. Now we just need to get some users. This really is the most difficult part.

Iteration number three. Keep in mind that so far we've spent about two hours on this. One hour for the first sketches, a hallway usability test, and resketching. Another hour talking with the client.

It's hard to see how this could be more efficient.

  • Setup the user to succeed

Now we're in front of a real user. We give him a similiar spiel. This time we really emphasize that we're testing the prototypes, not him. Any frustration at all is our fault, not your fault. You know your business and we do not, and the fastest way we can learn what's right is to present you what's wrong and have you tell us."

  • Have user think out loud

They click "order ducks" and away we go.

Very soon we have some trouble.

Prototype #3: user feedback

  • Need more elephants

"I think that in our environment ducks are more successful when there are at most 20 ducks per elephant. I understand you have a minimum number, but can I get more elephants?"

Prototype #3: user feedback

  • Start date & end date?

"Start date? I need them now! I can't imagine a situation when I wouldn't need ducks immediately. And end date? I never have any idea how long I'll need them."

Prototype #3: user feedback

  • "Place order"

"Place order? I need to know if you have ducks and how soon I can get them before I want to order them."

Prototype #3: user feedback

  • Boss out to lunch

Prototype #3: user feedback

  • PO number?

Prototype #3: user feedback

  • Agency > me?

Prototype #3: user feedback

  • "How will I know?"

"Will you call me when you're on your way?"

  • Always review the whole deck in detail

What have we learned?

Here's the result of everything we learned.

A user can still specify the number of ducks, but now they can also add more elephants. There's a clear message explaining this.

We changed the label from "Location" to "GPS Coordinates," so it's unambiguous.

We changed the button label from "Start order" to "Check availability," and ...

...added an explicit availability screen.

We've expanded the previous "Agency and Order Information" page into two pages, and started with the user's contact information. We've added a message telling the user how important this is, and because of the number of options and the fact that there's nothing else on the screen, we're also trying to /show/ the user how important it is.

Agency information is on its own page, and we've also expanded the possible contact methods.

We've added a much better confirmation page, that explicitly lays out the next steps once the user completes the order.

Add Your Comment Here

Randall Hansen
OpenSourcery Alum