Usability Testing Script

The following is a script for doing usability testing on Kiwix - specifically browsing and searching. It is designed for use with a pre-installed copy Wikipedia 0.8 (which should already be indexed).

Introduction and setup

• Thanks for agreeing to help us test.
• Get the participant to sign the consent form.
• Explain what Kiwix is.
• We're collecting feedback on Kiwix and testing its usability. Basically I'll ask you to perform a task and then watch you use the software.
• This will take about 15 minutes.

Explanation of roles

• Your job is really easy, you just have to be yourself and act as you naturally would. As you interact with the program, please be honest with your positive and negative thoughts. There’s no right or wrong answers, and nothing you say will hurt my feelings.
• You are not being tested, the software is being tested.

Background questions

• Tell us a little about yourself, like where you're from and what kind of work you do.
• Have you ever used an offline Wikipedia reader before?

Think aloud

• Next I’d like to have you open up Kiwix to do a couple of tasks.
• There’s one thing that I would like you to do differently. Please think aloud as you use the program today. Feel free to say anything that comes to your mind - for example "That’s interesting" or "What were they thinking?" Talk out loud while you’re doing each task, so I can understand what you are thinking and doing - for example "Now I’m going to try to use the search bar".
• We're going to be focusing on using Kiwix for basic browsing and searching.
• Do you have any questions before we continue? Okay, let’s open the program.
• Launch Kiwix.

Tasks

Find an article

• Pull up the article for Berlin, Germany

Find specific information

• Find out who the current King of Thailand is.

Bookmarking

• Bookmark the article on Bhumibol Adulyadej (King of Thailand).
• Close the bookmarking interface (if you haven't already).
• Go back to the Main Page.
• Go back to the Bhumibol Adulyadej article using the bookmark you created.
• What do you think about the bookmarking interface?

In-page searching

• Within this article search for information on the "Finland Plot".
• What do you think about the in-page search interface?

Full-screen mode

• Switch the interface to full-screen mode.
• Switch the interface back to regular-screen mode.

Tabs

• Bring up three articles in separate tabs.
• What do you think of the tab interface?

Final questions

• Is there anything that you would want to see changed/done differently in Kiwix? Particularly related to the tasks you completed today? If so explain.
• Any other likes or dislikes that you’d like to mention?
• Anything else you’d like to share with me or ask me?
• Would it be okay for us to contact you if we have any follow up questions or information?

Results

Hackathon 2011

Liam

• search bar too long
• no bookmark bar which allows a simple drag&drop like in FF
• create new bookmark not so intuitive (too complicated)
• bookmarks: replace sets by folders
• bookmark sidebar croping the windows on the right (also on the left)
• search in place: position indicator in the vertical scrollbar (like in google chrome)
• fullscren button tooltip does not change at all
• On Mac this is only possible to open a new tab with CTRL+T and tabs header if visible (add context menu ofr ctrl+click)
• on Macos tabs are unfamiliar
• X button on tabbar is confusing because it does not close the tabbar, but only the current tab.
• closing tab is not buggy
• Interface does not follow mac app. standarts.

See also

• [1]
• [2]