|
Report on Usability Testing
Brown University Library Web Site
Session #1, July 2006
Overview
Methodology
Key Findings
Communication Preferences
Finding their way to the library web site
Moving through the library web site
Searching Josiah
InfoGate
General Preferences
Summary
Appendix A: Usability Testing Script
Appendix B Usability Testing Questions and Tasks
In July 2006, library staff conducted usability testing on the BUL web site, the purpose of which being to solicit feedback from users. Such testing can be an effective tool in identifying sources of user frustration and confusion. In addition, it highlights the need for new or altered content and services. Additional sessions will be held on a regular basis in the future, and the cumulative results will play a role in the iterative and redesign process for the library web presence.
top
Five library student workers (from the Bindery and Media Services ) spent 45 minutes being interviewed by Whitney Wilson, Simmons intern and ARL fellow. The interview process included an overview of the purpose of the testing and a series of questions about their experiences at Brown and personal web use preferences. Students then attempted eleven individual tasks on the library web site at the prompting of the interviewer, who encouraged the students to talk through their actions and thought processes and ask questions, in order to create a record of their experiences.
Interviews were videotaped and broadcast on a closed circuit to observers in an adjacent room. Screen actions were recorded using Camtasia software on the testing computer. Observers of the closed circuit broadcast were Sarah Bordac, Lori Jargo and for some of the sessions Jean Rainwater and Bonnie Buzzell. Each student was compensated with a $15 Starbucks gift card.
The student testers were:
- Rising 2 nd year, humanities focused
- Rising junior, PreMed-Psychology concentrator
- Rising senior, Linguistics concentrator
- Newly graduated Art History concentrator
- 2 nd Year graduate student, English
top
- 0 of 5 students knew there were staff in the library who were dedicated to a subject (subject specialist is an unfamiliar term)
- 0 of 5 students were unsure how to search for books and articles at the same time and did not notice the InfoGate search area on the homepage
- 1 of 5 students had never contacted a librarian before (graduate student)
- 5 of 5 students knew how to access ILL information and understood the service
- 2 of 5 students suggested increased visibility and functionality of a “your library” account
- 4 of 5 students said they would not call the library or a librarian – preferring email
- Preferred eResources were: JSTOR, ProjectMuse, ProQuest
top
Students were aware that they could ask for help, but two of the five specifically said they would prefer to find answers on their own. Receiving a quick response to requests was a priority for those who said they would contact a library staff member.
- Email is the preferred format to get help finding information in the library, although only one of five had used this functionality.
- Opinions of IM communication were mixed, with one student stating that it is a nice option although she wouldn't use it. Another student described librarians with an IM account as “creepy” and said she would prefer to come in to talk to someone.
- Two students did not easily find the Ask a Librarian link from the homepage.
- Students assumed the options on the Ask a Librarian page will provide them with quick responses.
- Upon seeing the librarians by subject phone list, one student said, “this document doesn't look like a list of librarians,” but quickly recognized a name and felt she had found what she was looking for.
- One student stated that he didn't know what subject specialists did and didn't know how he could be helped by librarians. He acknowledged that “they probably could be helpful but I've never made contact.”
top
Brown.edu was set as the default location for the browsers on the test computer. Students were asked to find the library homepage. The five students demonstrated four preferred ways to get to the library web site.
- From the Brown homepage used the pull down menu in the upper right
- Brown homepage A-Z list
- Typed in http://library.brown.edu
- From the Brown homepage clicks on Academics on the left and then on Libraries on the right. (this student wished the library has a user-friendly URL like UCLA's – www.library.ucla.edu)
top
While students moved through the eleven tasks presented to them, the team observed overall patterns of use.
- 3 students used FAQ link in lower left (e.g. to contact a librarian and to find out hours).
- 3 students used the pull-down menus as a last try after not finding what they were looking for elsewhere.
- 3 used the A-Z list (e.g. to find information on renewing books).
- 1 student used the “search library web” feature, which returned a lot of information without a clear relevance to the search. The student was confused by the results and returned to the homepage to look at the A-Z list.
- Some of the students said they did not use the icons in the upper right of the screen because they thought they were decorations. Another student did use the icons.
top
All students were aware that Josiah was the library catalog and knew how to do a basic author or title search. Four of the five students used the quick search box on the homepage, and one selected the http://library.brown.edu interface to do her search.
- Students use the Quick Search box on the BUL homepage. One of these changed the pull down choices from Title to Words.
- One student typed an author in the (default) Title box for the quick search and the correct information was returned.
- One student treated http://library.brown.edu as the library homepage. She says that she uses “Josiah as a starting point for all things that I do.” When she was shown the http://www.brown.edu/library/ page she didn't know how to access Josiah.
top
While on the homepage, students were asked “to search for books and articles at the same time.” None of the students saw the search box on the homepage, and none of them knew about InfoGate before the testing sessions.
- Students expressed frustration finding and using InfoGate.
- Comments include:
- Efficiency: “It takes a long time for results.”
- Application: “Why you would want to search both books and articles?”
- Usability: One student was unsure how to read the results, and assumed there would be some indicator (icon or color) to differentiate books from journals, etc.
- One user said he would most likely just use the catalog because he knows it works.
top
The usability testing provided insights into preferences for research and general web use. These responses came up in conversation during the sessions, particularly during the discussions in the first part of each session.
Sites used a lot:
- Facebook (3)
- Allrecipies.com
- Lotsofnoise.com
- Runnersworld.com
- Itunes.com
- Gawker.com
Sites/Features that annoy students:
- Google – “it doesn't give me what I want; even when I am using quotes I don't get what I want. It is too general. Scholar is better.”
- Verizon wireless site – I don't use the service anymore. I got a new cell phone.
- Don't like having to log on to get to content (i.e. New York Times)
- RISD site is annoying
top
Two of the students shared their preferences for getting started with research projects.
- One student starts her work in Google Scholar, because she finds it difficult to research titles of articles through the eresources interface. Once she gets the titles in Scholar, she clicks on the title there or goes to the eresources page and finds the journal. This student doesn't use databases and instead goes directly to the journals. She wasn't aware there were databases. (Pre-Med concentrator)
- One student begins research on a popular media source or database (e.g. Yahoo) to find topics and then go to a database such as ProQuest. He also scans the stacks near books he knows are relevant for other possible books. (Linguistics concentrator)
top
Based on direct observation of the usability testing sessions, the following findings are identified.
- All users are different. Each of our student testers had different methods for finding information on the web and the library web site.
- Students want easy access to the best tool for the job at hand. They want to know how and where to find things. One student suggested adding that content to the homepage and said “I wish someone had told me how to find information well.”
- A helpful tool would be a small guide that provides essential information on library use, services and information skills
- Students identified the core content that FYS students should have access to as: Josiah, eResources, hours/locations
We need to increase and improve awareness of our services, such as:
- Identify staff as experts. There are people who work at the library dedicated to your subject area (subject specialists).
- General information. Make the contact and hours information more prominent
- Course pages. All students liked the idea of course pages and are surprised that librarians know about their subject area.
- WebCT access. Improve the connection/transparency between library content and WebCT
top
This script covers how the usability testing process works and the role the student testers play in the process.
Hi, my name is Whitney Wilson, and I'm going to be walking you through this session.
You probably already know, but let me explain why we've asked you to come here today: we're testing the existing library web site before we begin to make large scale changes to it this fall. Your experience today will help us create a more user-friendly site.
I want to make it clear right away that we're testing the site , not you. You can't do anything wrong here. This session isn't a test of your library skills; in fact, we're happy to have users who are less familiar with the site. Either way, your participation today will be really helpful to us.
We want to hear exactly what you think, so please don't worry that you're going to hurt our feelings. In fact, I'm not even a permanent staff member at Brown, and I have had no role in the development of the existing site. I do know that the librarians really want to improve the site, so we need to know honestly what you think.
As we go along, I'm going to ask you to think out loud, to tell me what's going through your mind. This will help us. I might prompt you or ask what you're thinking from time to time.
If you have questions, just ask. I may not be able to answer them right away, since we're interested in how people do when they don't have someone sitting next to them, but I will try to answer any questions you still have when we're done.
We have a lot to do, and I'm going to try to keep us moving, but we'll try to make sure that it's fun, too.
You may have noticed the camera. With your permission, we're going to videotape the computer screen and what you have to say. The video will be used only to help us figure out how to improve the site, and it won't be seen by anyone except the people working on the project. It also helps me, because I don't have to take as many notes. There are also some people watching the video in another room.
If you would, I'm going to ask you to sign something for us. It simply says that we have your permission to tape you, but that it will only be seen by the people working on the project.
Do you have any questions before we begin?
top
The following questions and tasks were used as the structure of each of the sessions.
Before we look at the site, I'd like to ask you just a few quick questions. First, tell me about your experience at Brown. Are you a full-time student? ________
Good. Now, roughly how many hours a week would you say you spend using the Internet, including email? ________
How do you spend that time? In a typical day, for instance, tell me what you do? ________
Do you have any favorite Web sites? Can you think of a site that you find difficult to navigate, but that you use anyway because of its content? ________
OK, great. We're done with the questions, and we can start looking at things. ________
And again, as much as possible, it will help us if you can try to think out loud so we know what you're thinking about.
- You want to take a look at the Brown University library web site. Could you find it for me? Please begin by opening a browser.
- You need research help, but can't make it to the library in person. How would you contact a librarian?
- You go to the library all the time during the regular academic year, but now it's summer semester. What time does the Rockefeller close on Tuesday?
- You want to see if the library has a copy of Me Talk Pretty One Day by David Sedaris. Where would you go?
- All of our copies of Me Talk Pretty One Day are checked out. How could you get a copy delivered from another library?
- Your copy of Me Talk Pretty One Day is due tomorrow, but you need it for another week. How would you renew it?
- You need a few articles for your “Biotechnology and Medicine” paper. How would you find them?
- You're writing a political science paper. Who would you contact for research help?
- You're taking a class called “The Campus on Fire.” Your professor has told you that the library has created a page of resources that will be helpful to you during the semester. Where would you find it?
- You'd love to search for books and articles at the same time. How would you do that?
- You're writing a paper about Dante and have been told that Brown owns some rare editions of The Divine Comedy . Where would you find them?
Now I'd be happy to answer any questions you might have about the questions I asked you. If you are curious about something that seemed frustrating to you, I'll explain it. Remember, there's no grade for this experience. In fact, any difficulties you had will be addressed in the new version of the library web site.
top
Return to the production homepage
|
