Category: User Testing

During my time at the City of Durham, I participated in IdeaStarter, a program that gave employees an opportunity to propose and explore an idea. Through the process I got to explore usability testing and identify several ways to enhance the site based on user feedback.

The program, led by the Office of Performance and Innovation (OPI) encouraged City employees to bring their ideas to receive funding for prototype testing and potential implementation, and brings participants together as they build out their ideas through a hands-on human-centered curriculum.

Below is a recap of my exploration of usability testing to enhance the City’s website.

The Idea

With more than 6 million pageviews on DurhamNC.gov in FY20-21, we know a lot of people use the City’s website. However, we don’t know much about their experience finding information or completing tasks.

• How can we capture and use feedback from website users to help improve their experience?
• How might we work usability testing into our web development/content creation process?

The Process

• Interviewed peer stakeholders, including Durham County Library and State of North Carolina;
• Researched available testing tools and best practices;
• Drafted and conducted two rounds of usability testing for top five web tasks on the City’s website;
• Compiled and shared test findings to make web updates/improvements;
• Created presentation to share usability testing tips, tools, and resources with colleagues.

The Testing

Participants

• Group 1: IdeaStarter employees from various departments (5)
  • Helped with run-through testing
    
• Group 2: Durham Neighborhood College participants (7)
  • Offered $10 restaurant gift cards
  • Used Microsoft Bookings tool to easily schedule sessions

Test Scenarios

For the tests, I identified five of the most common tasks completed on the site. In order to observe the full user journey, users were not guided to start at durhamnc.gov.

Key Learnings

• Many peer organizations utilize usability testing, some have dedicated roles/resources for this purpose
• Testing can be done fairly easily with existing tools and resources
• Ideal to test during web or app ideation/creation vs. testing after the fact
• Almost any testing is better than no testing
• Usability testing should be viewed as a learning opportunity and approached with an open mind
• Ideal to include affected staff and stakeholders throughout the testing process

One of the exciting projects I had the pleasure to work on at the Durham Convention & Visitors Bureau, was the development of a new overview video for the city.

We had an existing video that served this purpose but was ready for an update. It was more than ten minutes long, included some dated footage, and utilized a testimonial format that was effective but lacked emotional appeal.

The goal was to create a 2-3 minute video that captured the personality of the city through its sights, sounds, and people. A video that left the viewer wanting to learn more and excited to check out Durham, particularly someone who might not be familiar with the city.

We were lucky to partner with Saleem Reshamwala, a local videographer with a deep understanding of the city and a highly creative vision. His ideas included working with local musicians to create a custom soundtrack, and incorporating the actual sounds of the city.

Once we were clear on the content goals and shot list, Saleem started filming and we got to see several rough cuts along the way. As the video started to take a more finished form we were excited and knew it was almost there, but had some questions:

• We knew WE enjoyed the video, but what would viewers think?
• What would someone outside of Durham think? Would they be excited by the video?
• With the fast-paced footage, would the viewer get a sense of what the city offers?
• And maybe most importantly, would it make them interested in visiting?

These were not questions we could answer without feedback, so I proposed using User Testing to get input from some of our ideal viewers. We did not have a large budget to use, but I developed a short test to get feedback from five users who met our desired profile. (See the target audience and test plan at the end of the post).

The results of the test were extremely valuable and helped internal stakeholders feel confident that we could wrap up the project with some minor changes. All five testers were excited by the energy of the video and surprised by everything that Durham has to offer, and 4/5 said it made them more interested in visiting. We also got specific feedback to act on, such as the suggestion for additional food footage.

What scene(s) from the video had the greatest impact on you (positive and/or negative)?

“Loved the guy picking the guitar. The opening photo of the city did not look very appealing. I’m sure you could come up with a better one. Show more food.”

Would you be likely to share the video with a friend?

“This one yes, cuz I have family who live in Durham, so I’d send it to them to show them how awesome it is. I’d probably send it to my husband as well, to help convince him we should go there. :)”

When the video debuted on Facebook it was a hit and received more than 350,000 views and 5,000+ shares. With the positive feedback we are confident that the video met its goals and can be used to introduce Durham to new audiences.

Below you can see the targeted audience for the test as well as the questions.