Step 1: Convince People.
I have found that it can be difficult to convince folks to start user testing at big companies. Here are some tips and tricks to convincing people that you need to do user testing:
- Repeat yourself. When I first started working in a 9-5, I gave up really quickly on convincing people of things. This is because, in my youthful arrogance, I would blame other pepole for my failures in persuasion (signs of this problem saying, "Everyone is illogical"). Sometimes you just gotta say the same things in a couple different contexts in a few different ways.
- Offer user testing as a solution. What finally convinced my team to get budget for user testing was the anxiety right before launch. Everyone was saying things like: "Are we launch ready?" and "What if conversion rate goes down?" User testing was way to know with a good amount of certainty if we could move forward with launching the new site.
- Make a budget and a plan. Is this obvious? Probably. Did I do it before I said we needed to do user testing? No. That's why it is here. For the budget portion you will need to think about materials (see below) and paying users for their time. Budget will largely depend on your audience. Students might give you an hour of their time for $20, but lawyers will need a bigger incentive. If you have customers that love you, a giftcard will work.
- Research the case for user testing: have the stats memorized.
- Give a presentation. I didn't do this, but I have seen web developers convince our companies to build our new site responsively with the power of PowerPoint.
Step 2: Determine your Audience.
Once you have interest or agreement to do user testing, you will need to figure out who will be testing your site with you.
You'll need at least 5 people per each segment (Source), and if you testing on multiple devices you'll need 5 people from each segment per device.
Step 3: Recruitment.
We recruited existing customers, since most of our revenue came from returning customers. We did this by emailing users that we had in our email list (other ways to recruit users and the pros and cons of each method here).
If you work in a large organization, sometimes getting work done from different teams takes longer than what you would expect. Make sure you know how long each step in the process will take. Because getting an email out was the longest step in the process, I kicked off the email portion of the project first. You will have to look at your own company to figure out the timing of the steps. Things you will need to determine before you write the email:
- Email audience. How much of your email audience will you target? You probably want to segment your list to only people in the area of your office (flying people out for user testing is a bit extreme). Of course - this isn't a problem if you decide to do user testing remotely.
- How will you convince people to sign up? Will you pay them, or rely on their goodwill and desire to improve your company?
- How many users do you want?
- What time and date do they have to be available
- Where will you hold user testing?
- What's the tone, look, and feel of the email?
Step 3.5: Write a screener
You will need to write and build a screener survey: this should be the landing page for your email. Make sure you remember to ask folks when they are available and their preferred method of contact. Make a nice thank-you page with when they can expect to be contacted.
Step 4: Build your tasks and questions.
This should be fairly simple: identify the major tasks on your site and write them down. For example, an eCommerce site task could be something like: "Find a product you are interested in." Here's the Nielsen Norman Group's post on how to write tasks if you are unsure.
For each task make sure you have a method to evaluate the success of the task or not. This can be pretty tricky. If your office thinks in numbers and analytics, you will want to measure the outcomes of your test quantatively. Plan how you are going to do that at this step. Here's a book on the subject.
Step 3: Acquire Materials.
Purchase or make use of what is in your office. When I was doing this, I was lucky enough to find all the materials in house. I asked our video team to lend me a mini-tripod for our QA team's iPhone (which I used to film the sessions). Mini tripods are available at Best Buy (or just do a google search).
At this point you will need to think about how you will record the user reactions and what is on the screen. We set up our user testing so that users faced the interviewer and the camera. The device they were using was plugged into our projection system. The projection was behind them. That way the camera captured the screen and their reactions to the device. I don't have any advice on screen recording software - so if you do - message me and I'll update the article.
Step 4: Book Interviews.
I was lucky enough that the admin assistant for my department did this for me. Make sure you build time into your schedule to do this work (it takes awhile).
Step 5: Prep for the Testing.
Don't forget about ports, questions, release for video if you filming, run a practice session, have someone taking notes / keeping track, decide if you want to broadcast the interviews live, and don't wear a low-cut shirt because sometimes you might have to bend down in-front of the camera (true story).
Thanks for reading, and write to me if I missed anything. Back to top