Tag Archives: User Study

Checkout System: Trial Run Results/Lessons Learned

In the last post, I described both the steps taken to prepare for the trial run and the trial run itself. In this post, I would like to discuss some of the results of that run and the lessons learned. I will approach each stage separately and will leave until a later post some overall thoughts.

Stage 1: Stabilize and Maintain the Hardware

As one might recall, this stage happens before a request can even be made as the collection items have to be assessed and processed. BUT, the Checkout System returns to this stage after the items have been successfully reshelved. As such, I will revisit the results applicable to this stage later in this post. Stay tuned…..

Stage 2: Request the Hardware

The major issue with this step was getting started. The pages had trouble remembering where to begin, which resource to use. This is understandable because only one had experienced a single training session and a cursory one at that. Once the pages navigated to the UTVA Wiki page, the same problem occurred: which resource in the UTVA Informational Packet to use first?

I mentioned in an earlier post that I was concerned about UTVA documentation becoming unwieldy. Item level control and description can lead to an explosion of records. In this stage of the Checkout System, I created the opposite problem. By trying to (gently) cram all of the relevant information in one ‘convenient’ location, the Pages were overloaded with too many options. It reminds me of shopping for toothpaste…my mind shuts down whilst looking at the wall of choices. My instructions and labels were less than clear. The Pages mentioned that there were “lots of links”, a “lot of steps”, and the process was a “little confusing.”

As I described in an early post, the best part of the Wiki is its currency. As a living document, we can revisit it and make improvements. For example, we plan to create a uniform starting position by removing some options and bringing others forward.

The Pages did successfully complete the UTVA Hardware Request slip by filling in the SKU#s, names, and locations of the console and its peripherals.

Stage 3: Locate & Retrieve the Hardware

The major result – and likely the one that most surprised me – for this step was that the Pages pulled the boxes instead of individual items. This was totally unexpected for me and it should not have been. The peripherals were individually bagged and placed within their appropriate boxes. With the item level control, I assumed they would grab only the console and the peripherals and move back downstairs.

When Pages at the Briscoe service archival requests (compared to maps or photographs as examples), the entire box is retrieved and brought to the patron. Since the peripherals were in boxes, naturally they pulled the boxes. When I asked the Pages about pulling individual items instead, one responded that the Pages reshelving the next morning would have no clue where to return the items since the individual archival ID tags did not come with the item’s AR# and/or location. In an earlier post, I described how Pages can ‘deduce’ a box’s home location by its tag. In the case of individually pulled items, the reshelving Pages would have to use the original request slip or the UTVA Master Hardware Inventory List. It simply did not make sense to the Pages to leave the box behind.

The Pages located the objects and brought them downstairs.

Stage 4: Assemble & Use the Hardware

This stage went quite smoothly. The major issue was the setup location. I had originally planned to assemble the system in the Winkler Study Room instead of the Reading Room because I was imagining a scenario in which we want to minimize the potential for distraction for the other patrons. Unfortunately the assembly instructions and digital images are all on the computers in the Reading Room. So…we set up there instead. One page would read aloud the instructions while the other was looking at the digital images. I’ll put in a request to get the Pages some IPads so we are not tethered to the desktops.

The Pages assembled the system and we played some 1983 Frogger with some incredibly unresponsive controllers.

Stage 5: Reshelve the Hardware

The major issue here was the written instructions on the Wiki. The instructions were written with respect to reshelving the individual items, not the boxes. This created some confusion especially if the researchers wish to place a hold on the items and use them the following day. What happens to the system? What happens to the boxes? Do we leave the system assembled yet detached from the TV? Etc.

Since we had only an hour, we ended the scenario by imagining that the researchers are indeed finished and did not place a hold. The white UTVA Hardware Request slip proved useful because the box ID tags do not intuitively point to a home location.

The Pages successfully returned all the items and boxes.

And back to Stage 1: Stabilize & Maintain the Hardware

After wrapping up with the Pages, two suggestions were provided. If the UTVA items are to be pulled individually, the AR# and location need to be added to the items’ archival ID tags. If not, the boxes’ tags need to be amended to include location.

Second, we bandied about the idea of adding an item’s digital image to the tag as well to help with identification purposes. We take the photos of the items when processing but we don’t know how big the ID tags would need to be to accommodate photos.

This was an incredible experience for me. Watching the Pages interact with the Checkout System shined a massive spot light on pain points to which I was utterly blind. Due to creating and working with the system for months, it ‘seemed’ perfectly natural to me. Now, I can revisit the access tools and make them even more perfectly natural.

In the next post, I will discuss my overall thoughts and impressions.


Checkout System: Trial Run

In the last post, our hypothetical team of researchers finished its emulation research and the pages successfully reshelved the console and its associated peripherals.

In this post, I will discuss the trial run that we conducted at the Briscoe Center in which we actually requested the Mattel Intellivision II and watched the process throughout. At this point, I would like to say a HUGE “Thank You” to the Briscoe staff and Pages for helping me with this aspect of my project. The trial run (user study) was an incredibly valuable learning experience and I am grateful for everyone’s participation.

Preparing for the Trial Run:

Before we could test the system, I had to get the items ready and create the access tools I mentioned in earlier posts. A quick note for the reader: many (if not all) of these access tools have been discussed in our blog, especially in the last 5 posts. Please revisit those posts for actual images. 

Stage 1: Stabilize & Maintain the Hardware

I ‘bagged and tagged’ all collection materials, created the abbreviated hardware list, and took pictures of the hardware. The hardware list was available on the UTVA Wiki and all images were uploaded to the computers in the Reading Room.

Stage 2: Request the Hardware

I created (and printed out) the UTVA Hardware Request form and the UTVA ‘Informational Packet’, also on the UTVA Wiki. One might recall that the packet contains, among other things, instructions, guidelines, resources, and helpful tips.

Stage 3: Locate & Retrieve the Hardware

I took images for navigation including ones of the Reading Room, the hardware’s location in the Briscoe, and the shelving system. And again the UTVA Informational Packet.

Stage 4: Assemble & Use the Hardware

I took images of the console and its peripherals during the assembly process and created written step-by-step instructions to accompany the images….and the UTVA Informational Packet.

Stage 5: Reshelve the Hardware

I wrote instructions for the Informational Packet.

With respect to the Informational Packet, I used the Mattel Intellivision II when creating the different components. At the same time, most of the instructions and resources are system agnostic and can be applied to UTVA hardware requests in general. The gaming system worked for me leading up to the test which gave me hope it would work during the trial run.

Training the Pages

I conducted a training session with two pages the morning of November 9th. I showed the available resources to them, including the images, the UTVA cabinet, the UTVA collection in the Briscoe, and the UTVA Informational Packet on the Wiki (after getting them access). In an earlier post, I mentioned undergoing a brief training session similar to one the Pages experience when hired. Training is also revisited (when needed) anytime collections are added to the Briscoe or procedures change.

The Trial Run

The trial run took place the morning of November 12th. Due to an earlier cancellation, we had a different set of pages, one of whom had been trained while the other had not. I feel this worked in our favor as it provided us the opportunity to see how a person with virtually no prior UTVA experience would interact with the system. We had only one hour to test the system, the hour after reshelving and before the Briscoe opened.

I “requested” the Mattel Intellivision II for emulation research by filling in the top of the UTVA Hardware Request slip with the console’s name and SKU#. I handed the slip to the pages and they got to work. Please note that, by design, I stayed in the background taking notes and stepped in only when it was clear that the process needed a nudge. And since we had only one hour, I was a touch proactive in “nudging” so we could hit all of the stages. I asked the pages to narrate the process and talk openly about what they were doing and thinking during each stage.

In the next post, I will discuss the results of the trial run.