Posted by: Arild | March 24, 2010

So, I’ve done the testing…now what?

The testing is done, and you have several pages of notes from the sessions. Now it’s time to put things into bullet points. Right after the test sessions are done, you should summarize your notes while you still remember things. Too many times I have waited a day or two with writing up notes and found that I don’t remember what I meant by something I had written. If you have a recording of the sessions you may be able to go back through the sessions and figure out what you meant, but this can be both tedious and time consuming.

Write up all the issues in point form. Most likely many of your testers will have come across some of the same problems, and you can probably group some issues together to tidy things up. Now you’ve got a list that you can present to the team of developers. Just do a 10-15 minute session where you all agree on the top 3-4 issues that will give you the most gain from the least effort. If you are using an agile project methodology (such as Scrum), you simply put these issues into your backlog and treat them as any other.

However, if you use expert users that take part in many rounds of testing or in the sprint demos, I have found that it can help to show them that you are actually doing something about their feedback. One way to do this in a Scrum project is to have a sprint backlog item each sprint that allows the developers to use a few hours or function points to implement a couple of “quick win” fixes to motivate the end-user testers. This can be its own item or part of the user-testing item. In any case you should aim to fix the 3-4 main issues before the next round of testing.

If you complete the user testing a few days before the end of the sprint, you can have a version of the application or web site that includes most of the functionality in the sprint, while still allowing some time to fix a few of the most obvious usability issues before the demo.

If you as the usability resource on the team are also doing most of the interaction design (like me in my current project), it can be difficult to admit that the users didn’t find your design as intuitive as you had thought it would be. In my case I have had to swallow my pride several times, but of course you have to consider the user feedback and try to figure out another way to do it, or at least tweak it so that it will be more obvious to the users how t use it.

One thing is to take the critique yourself; another thing is to convey it to someone else. It can be very difficult to tell someone in the development team that their “ingenious” solution to something was not understood by the users. Usability testing guru Steve Krug suggests a very easy way to solve this issue: Let the development team follow the test session using screen sharing software and a set of speakers so that they can listen to the comments from the user. When the developers see how the users struggle to use what they have made, it is usually also apparent how they can fix the problem.

I would recommend reading “Rocket Surgery Made Easy” by Steve Krug if you would like to learn more about usability testing. There is an entire chapter called “make it a spectator sport” that highlights many of the benefits of letting people see how useful doing usability testing can be.

Happy testing! 😀



  1. Actually scheduling usability improvements during the normal course of development is something I would like seeing in more projects. Too often, usability improvements are left for the “final touch”-phase (which tend to not happen at all).

    • I couldn’t agree more!

      We usually try to do usability testing with a couple of users at the end of each sprint (we do 2 week sprints), or at least once a month (depending on the functionality we have implemented). In the early stages of the project I think it can be very useful to do it towards the end of each sprint.

      My best tip is to make usability testing an integrated part of each sprint (just like the planning, retrospective and so on), and book the date for the next few months as a recurring event. It is much easier to cancel one session if you don’t have any changes to the user interface.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: