Think Aloud -- Mengdi Huang

In order to identify potential usability problems of interaction design products, think aloud helps designers to know the thoughts of users by letting them say what they are thinking when conducting a certain task by product. Since our product has no complex functionality, the task is simply successfully create a postcard. There will still be some silent moments, due to people's customs. Despite of this, designer can capture users much preciser than imagining.

Pros:
Good to have a brief tutorial

Map taking place of list is interesting

Cons:
Tutorial is a bit boring
Buttons not so obvious

A little confusing when first see the pins on map
Too many spots to choose. Annoyed to click each of them to see which is the best. And forget which one has been clicked.

Stickers are not enough.
Stickers are not adjustable in size or angle.
                 

Didn't know how to get video clip until the postcard is printed out.

Think-aloud, by Hui Pang

Think-aloud Evaluation

Think-aloud is a technique where users interact with the product in order to be observed so that the researchers can test the usability goals and user experience goals of a design.

I asked a fellow student to perform the think-aloud evaluation since she can also be viewed as an international tourist.

Prior to the evaluation, we did not specify the functionality of this app, but to see how well she can make use of the tutorials to also evaluate the placement and clarity of it.

Below are the stages of evaluation.




On the introduction page: 

 "Those portrait slides illustrate this app is for snapshots."
"There are two buttons, one for going straight ahead, the other for reading instructions."
"The text below are not showing completely." [We edit that part later]

("By clicking the 'tutorial' button, we go to the tutorial page. ")


On the tutorials page: 



“The tutorials seem clear enough.”
"Too many texts with no pictures though."
"Also it would be lovely to add a 'go straight ahead' button, so we don't have to go back to intro page."[There is a link to the map page now.]

("By clicking 'back to home' button on the upper right corner to go back to into page, and then 'go ahead' on the home page, we go to the map page. ")

On the map page: 



"The explanation seems clear. "
“There is pan function for the map so we can see all the pinpoints on the map. ”

("By clicking one of the pinpoints, we go to a scenery page. ")


On the scenery page: 



"There is a short description for Vasa Museum."
"Also multiple scenes to choose from. "

("By clicking the 'apply' button, we go to the video capturing page.")


On the video capturing page: 



“By clicking the start button, we see a demonstration of an AR video”

("When the video capturing ends, it will automatically switch to snapshot page")


On snapshot page: 




“This page shows several snapshots, and there is clear instruction for how to choose from them.”

("By clicking 'confirm' button, a green 'print' button appears on upper right corner, and we click that to go into the sticker page.")



On sticker page: 



“This page has a menu to choose stickers from, which is nice. ”
"However, one can only use one sticker for a single time."
"It would be better to add more options. "

("Once we are happy with the decorations, we click the 'print' button to go to the printing process.")

On waiting for printing page: 





Think-aloud comments

What do you like about the app? 

It is easy to use, and has interesting applications.

What do you dislike about the app? 

It would be more fun with more scenery and sticker options.

Think aloud + Extra Field Study at Slussen using modified prototype[Pavithren]

I wasn't present during this week, therefore I am not able to comment on the Think Aloud test conducted on the participant. I would just give my reflection about the think aloud technique instead.

Thinking aloud is a cheap but valuable tool in usability testing. It encourages users to speak out their actions while using a product. This allows designers to identify what kind of thoughts go through a user's mind while navigating through the application.

Personally, I was a volunteer during one of the HCI lectures where the lecturer instructed me to use a website to book a badminton court for a particular day. I had to verbalize all my actions while trying to complete this task. I was having issues finding out if my actions had been received by the system as when I changed the dates while searching for a booking slot, i didn't receive any feedback or wasn't well phrased so I was stuck in the page for quite some time. After some random clicking, I managed to select my slot.
Even though I was a evaluator for this task, I can see that asking someone to think aloud exposes the emotional responses by users. Some might even get annoyed if they are not able to achieve a simple task.
But thinking aloud technique might not always work as some might not want to say everything they are thinking of and "filter" out their responses. Also we need to make sure that we keep quiet during the entire process so as to make sure that we don't feed words into the evaluator's mouths causing a bias.

In conclusion,  think aloud is a cheap and quick way to get feedback from users but we shouldn't totally depend on the findings from this technique only.


Field Study in a natural setting(20/11/2016)

Before we presented the low fi prototype during our exercise, Agneev, Shi Ping and I went to Slussen to conduct a field study about the prototype we had created in a natural setting. We conducted a usability testing on 5 families with children to get their feedback. Overall they were quite pleased with the minimalistic interface and suggested they would pay up to 3-5 euros for a postcard. They also feedback that a timing of about 20-30 seconds of the video was suitable. A demo of the app can be seen below(if it doesn't work watch here(mute audio please:) ): https://youtu.be/oXrh59JAcCs).

Please mute audio

 Gathering results from this study, we made some changes to the prototype for the presentation during the in-class exercise as mentioned in Agneev's post

Exercise 4 Group Evaluation [Group Post]

During the evaluation exercise, Group 3 gave us many valuable feedback on how our prototype could be improved. We provided the group with rough sketches of our intended UI, in design terms referred to as Low-Fidelity prototype. Before we asked them to try it out, we explained them the context of the problem we were trying to solve. Thereafter, they were asked to perform a walk through of the application to perform a heuristic evaluation. 




Feeeeling
After performing the walkthrough, the evaluators felt that they weren't clear if our prototype was well adjusted for the target group. They highlighted potential issues such as the following
-lack of interest by kids in joystick/buttons
-not something users would want to use again(mainly for tourists) - maybe I wasn't clear when explaining about the personas

Primary and Secondary Functions
As mentioned previously, the evaluators were not clear about the functions of our prototype. Probably because we tried to integrate the trip planner feature together with the souvenir machine. We agree with their feedback as we were quite ambitious trying to solve several features at once. This made us realize that we should focus on a single problem and ace in it.

Interaction
The evaluating team highlighted that the prototype didn't have a clear objective it was trying to achieve. They suggested that instead of trying to solve many problems at once, we could focus on the main issue we are trying to solve i.e. create something memorable for tourists who visit djugarden and they can share this memory with their loved ones back home.

After looking at the feedback, our group decided that we should forego some of the features to streamline the final design and improve the usability for the tourists.

Conclusion

The evaluation stage was quite vital in the design process as it enabled us to uncover problems we wouldn't have thought of due to the designer bias.

Evaluation for Group A5: Agneev Guin

The following is an evaluation of the app prototype of group A5.

User group

The user group consists of Tourists from all backgrounds. The group focuses on everyone carrying mobile devices who are eager to learn about the nearby places of interest. 

"The feeling"

The pluralistic walkthrough of the app is quite appealing as it's user centered approach doesn't allow any unnecessary stuff, thus focusing on its usability. The augmented reality section will be quite useful while riding on the ferry, however the use of the map view is constrained while in the ferry as the first time tourists might not be able to relate to the map. The information along with the interest points might help the tourists choose their next destination point. 

Interaction and over arching structure

The one touch selection to display the details of the tourist spots seems simple and easy to operate. There are no unnecessary options to mess around, thus focusing on the easy to grasp capability. 

Primary and secondary functions

The main function, to use the Augmented Reality,  is the main focus of the design prototype. The AR is the catchy part where information for the important points is easy to capture from your current location. The secondary function, to see the map view, is more valuable to those part of the target group who are on the shore. 

Design/composition

An interesting splash screen of AR and Map is a clear segregation. The heuristic approach of having a layout to provide the user an impression of a first time user for a fresh app helps keep the design simple. The design is thus, quite appealing. People might want to explore more based on the simple nature.

Suggestions for improvement

• The tag AR or Augmented Reality may not be clear to all. The app seems quite attractive once someone is using it but it might be a bit of a confusion for the first time users. 
• The impression that its a tourist guide can be explained more.
• The settings for AR and GPS on/off might not be required. It can be like Google maps that if GPS is disabled, it will popup a message to enable.
• The pop up window should remain open even after moving away the mobile device. Else the user might have to hold it that place and direction to get all the info.
• In case more info needs to be provided like information about the place or public transport timings to reach there may be provided with easy to grasp icons. 
• Ratings may be provided based on previous tourist travelers.

Help

No further support is provided or needed for any tricky stuff.

Sketches

The sketches clearly justify their plans. The pop up information shows that they cover a part of the screen. This needs to remain even after removing the device is moved. They have even provided an idea of the settings section to describe the possibilities.

Summary

The usability of using a mobile device to explore the nearby places has been widely explored. The simplicity provides a good user experience focusing primarily on their target groups.

Prototype Design [Group Post]

Design Sketches:

For the product we design the paper prototype for the interactive operation interface.

Start Page:

The start include the Welcome message and main functions.

Main Functions:

• ACTIVITIES IN Djurgården
• FERRY TRIP

ACTIVITY:

• Photobooth
• game

Below the category of  activity, there are selfie and games function

Selfie:

It uses an height-adjustable camera to take selfies.

Users can apply filters to the photos, generating electronic postcard with the selfie just taken, with several postcard formats available

It can print the postcard immediately or send an e-postcard via email or twitter.

Our persona Mariana takes her children in a stroller, so the adjustable camera will increase the usability among users of various heights.

Our another persona are from a foreign country. We think it will improve their travel experience if they can get free souvenir as it gives them something memorable which they can share with friends when they return to their country.

Game:

FERRY TRIP

Below this category we offer information of places of interests on Djurgården.

We use an interactive map as our main page to organize information. The sketches below serveas a low-fidelity prototype.

There are several choices on the top. When clicked, the spots of that kind will be lighten up by different color or brightness.

Users can know more about a spot on the map by clicking on the particular dot.

The Conceptual design is as follows:

For the information of the Museum we provide introduction of current exhibitions illustrated with pictures, opening hours and entrance fees.

For the restaurant part, taking users’ requirements into consideration, we will mainly include the cuisine type and the opening hours. We also offer the reservation information (if the restaurant needs reservation) so that they can know whether restaurant will be avalible when they visit. Besides, we also give the information of approximate fee per person.

As you noticed, there is a pink heart at the bottom, which means you want to go there. Then the corresponding spot will be marked out in the main interactive map, making users’ next decision easier if they want to go to the spots not so far away from each other. We believe this can help users minimize memory load. Recognition rather than recall makes the product easier to use.

As for the information we provide, we use the heuristic of minimalist, tailored to be the most information users want to get, avoiding to mess up users’ mind.

We will make the height of touchscreen adjustable, so that children can choose activities together with parents. On the top of the screen there is a camera for our selfie function.

This children-friendly product has two main functions: activities and infomation.

Of course our product support Swedish and English at the same time, to match the system between real-world. We will use simple vocabulary, and put graphic icons beside texts to make children can understand as much as possible.
Moreover in case gloves do not work with touchscreens during winter, we will be adding a joystick and buttons to navigate the menu and play the games.

Reading Seminar 2 Notes: Agneev Guin

Chapter 10: Introducing Evaluation

The example on the HutchWorld Usability Study clearly states that any number of usability tests cannot give a real scenario perspective. No system is ever perfect. The integration of the HutchWorld with the real clinical environment gave some insights on the natural setting. Also, one important aspect is "if users don't like a system, it doesn't matter how successful the usability testing is: they probably won't use it."

Chapter 11: An Evaluation Framework

The evaluation needs to address all aspects and can be captured as a data in the design process. Thus the evaluations are classified as Objective and Subjective, Quantitative and Qualitative and Laboratory and field or naturalistic studies.

The DECIDE framework focuses on the goals and applies the logical and practical implementation which are to be dealt while planning an evaluation. The acronym DECIDE expands to the steps carried in the process.
1. Determine the overall goals that the evaluation addresses.
2. Explore the specific questions to be answered.
3. Choose the evaluation paradigm and techniques to answer the questions.
4. Identify the practical issues that must be addressed, such as selecting participants.
5. Decide how to deal with the ethical issues.
6. Evaluate, interpret, and present the data.

The focus of the framework on the practical issues constitutes of users, facilities, schedule, budget and expertise while the ethical issues focuses on the code of ethics, informed consent and deceptive experiment. The reliability, validity, bias and scope can only be observed using the data captured.

Question:
Is it necessary to tell the users that they are being logged? Should double blinding be a general aspect of evaluation?
What if the ethical aspects cannot be met?

Reflection for Reading Seminar 2 - Evaluation [Pavithren]

Chapter 13 introduces us to the different ways designers can evaluate their design. It also emphasizes the importance of conducting evaluation through the Why, What, Where, and When of evaluation. Evaluation allows us to collect information on how the end user will use our product. We can conduct studies in a controlled or uncontrolled environment. This could depend on whether we want to test a hypothesis and perform statistical analysis or determine how the product is going to be used.  With the emergence of big data, now we can collect massive amounts of data and perform data analytics to aid us in the design process. For example, in android smartphones or software on computers, prompts may be shown to the user asking about collecting anonymous data for improving the software. Some of the data collected could be tracking crashes, analyzing user behavior etc. One importance aspect of conducting evaluation is to respect user privacy and confidentiality. When researchers might have to obtain a CITI certification and permission from universities’ institutional review board(IRB) prior to conducting the study.

Chapter 14 delves into the mechanics of evaluation studies such as laboratory studies and field studies. For example, a researcher might want to evaluate how kids learn using a computer or a tablet. To achieve this, they might observe their touch patterns or gaze using eye tracking equipment. The data collected from these techniques could then be used to further improve the way the kids are taught using a computer/tablet. Tools such as IBM SPSS helps researchers analyze data and reach conclusions much easily. Another interesting thing highlighted in this chapter is the possibility of discovering surprising findings during a field study. During a controlled study, there may be certain restrictions on the user’s behavior or interaction with the product, but when maybe its tested in the WILD, researchers might come across interesting findings they wouldn’t have anticipated.

Chapter 15 introduces heuristic evaluation which is based on a set of guidelines/checklist created by some researchers to test UI and so on conform to well-known principles. This is a much cheaper and quicker option as compared to hiring users to testing the product. Walkthroughs are another method of conducting testing without the involvement of users. It involves going through a task in the product and identifying problematic features. Data collected from software or websites such as user behaviors and errors encountered could be useful in improving future versions of the software and could also be used for customizing the product per user’s preferences.

All in all, evaluation is an important aspect of user-centric design which helps to improve user experience.

Questions

How do I identify users who might not be acting normally during a field study after they realize that the researchers are observing them? The Dillema: If I inform the user that he is participating, he might not behave naturally. In the other hand if I record data without informing the user, its an ethical issue as I have not gotten their consent

With the increased usage of Big Data to perform analytics, will researchers be missing out as they are merely interpreting data and there might be misinterpretation of data?

Given our time constraint, can we look at previous research work done in this field and support our design ideas using them?

Solutions for the problems found in the first interview - Suping

--LED information board instead of Swedish broadcasting

When we are travelling, the most important thing is to make sure that we on really on our way. This sounds like nonsense but things may go wrong when you are not clear if there are multi-destination of the ferry you took.

                                                               Notice with only Swedish on it on the ferry

On the ferry, we first introduced a Swedish couple carrying their kids to the amusement park. The ferry was very crowded at that time. For that reason, it is also very noisy in the ferry.

For this interview, we set up some questions before. Thus it is quite a semi-structured interview so that it will be more flexible. And for the consideration of triangulation, we interviewed parents from sweden and from tourists travelling to Sweden who can not speak Swedish. As for the method for data recording, we chose the audio recording. And for the ethics problem, all the interviewees are informed that the audio recording will be published.

                                 

                                               LED information board that can be changed each time

We first ask them their purpose to take the ferry instead of bus. They think that they choose to take the ferry mainly because of the fancy view. But the space is usually quite limited on weekends. As can be heard from the audio recording, the background is quite noisy so that it is hard to hear the speaking from others and the broadcasting. We first interviewed a parent from sweden. He told us that, they should always be aware of the broadcasting, because there are several destinations for this ferry. They have already taken off at the wrong destination twice. Since there is quite noisy inside the ferry, it is not that easy to always concentrate on what is broadcasting. The next parents who also have this trouble are from Korea. They speaks little Swedish. But the broadcasting on the ferry is all in Swedish. So they feel that sometimes they will be quite confused. What is more, there is usually only one stuff on the ferry so it is hard to find someone to ask about the information.

In order to solve that problem, we suggest there should be a LED board in the ferry. Because the destination of the ferry is not always the same. So it is a good idea to have a board can not only shows a fixed information for the destination. Ofcourse, it is better to show the information not only in Swedish since Stockholm is quite an international city where a lot of international tourists are visiting.

In China, in the train or boat, we usually have a LED board to shows the information including the destination and the notice for the passengers in Chinese and English. So that even if it is very noisy or crowded, people can still know the important information. And it is easy to change the destination information or the notice for the passenger.

Reflection for the reading seminar 2---Jinchen Liu

Chapter 13:

The aim of the Evaluation is to integrate whole design procedure, it will combine the information from the users ' experiences about the related process. Evaluation also focus on the usability of the whole system, such as how easy the system is to learn and to use. Sometimes designers may invent products based on their own preference, evaluation enables to help them check whether the product is acceptable by wider user population or not.

Depend on the evaluation goals and the researcher's or evaluator's expectations,  the methods for evaluating can be changed a lot. Frequently, it involves observing participants and measuring their performance.  To sum up, different methods are usually showing the different perspectives in the range of the study.

Finally, ethical problem about how the participants are treated and theirs rights to privacy should also be taken into consideration. The more comfortable and nature we set our evaluating environment, the more accurate and reliable information we will get for developing the product.


Chapter 14:
The main aim of this Chapter is to introduce the laboratory evaluation studies and field evaluation studies, which are controlled experiments and natural settings respectively. Usability testing always takes place in the labs which enables evaluators to control the test parameters. This approach focus on testing how usable the product is. The users will be given specific requirements or tasks and the evaluators collect data from their responses to examine the product.

Before the testing begin, we need to form the design of the experiment. A concern in experiment design is to divide the different participants condition individually, which result in different experiment design. Each of them has their advantages and drawbacks. Finally, result analysis always use statistical methods such as t-tests to gain the probability of a result occurring.

Field studies is conducted to evaluate the user experiences. Although the naturing settings always tend to be messy and constantly interrupted, this follows the people real reaction for using products therefore the result should be more realistic and meaningful. But sometimes the findings are unexpected.

Chapter 15:

In this chapter,  the main content is to further introduce the evaluation methods,   these methods are based on understanding users through knowledge codified in heuristics or data collected remotely, or modes that predict user's performance. They are called inspection evaluation methods. The most different feature compares to the other methods is that they will not require users to be present during  the evaluation. The two common methods, heuristic evaluation and walkthroughs, offer evaluator a structure to guide the evaluation process.

Heuristic evaluation is a usability inspection method. It collects the feedback from the professional point of view based on the certain design principles.
The cognitive walkthrough is to make sure that how easy the task can be carried out.  This method could be very funny. For every step of the process of the evaluating procedure, the user could be stopped by testing whether the users be aware of what is going on.



Questions:
1. How do evaluators deal with ethical problem during evaluation procedure?