Evaluation from D3----2016/11/07

Group 2

User group

Is it clear to whom the service is made for?

Is functionality well adjusted to the particular user group? (language, difficulty level, interests)

• -  Tourist families on ferry from Djurgarden (?)
• -  We weren’t sure of their target group
  
  
  “The feeling"
  What does the first impression tell?
  Will the user stay? return?
  Does the design keep the users attention/interest?
  
  

• -  Kids attention won’t be captivated with joystick/buttons
• -  People can do these things on their phone
• -  Will cause a long line, people won’t like that
• -  There is no advantage to the hardware
• -  Could just make this into an app or something
• -  This isn’t something people would return to, it’s for tourists
• -  This will get people’s attention, but not keep it
  
  
  Interaction and overarching structure
  Is it a clear flow?
  Is it clear what options and possibilities the interface provides?
  
  

• -  Unclear why “Ferry Trip” leads to Tripadvisor thing
• -  Not clear what this does, user would have to play around with it
• -  What is the purpose of this? Why will people want to use it?
  
  
  Primary and secondary functions
  Is the main functions in “focus” in the design?
  Is the the secondary functions valuable or unnecessary?
  
  

• -  What are the primary and secondary functions?
• -  It seems like the Tripadvisor stuff is unnecessary. People can do this on their
  phone
• -  Maybe just focus it to a game thing for kids
  
  
  Design/composition
  Is functions and elements naturally grouped?
  Does the design have a good structure and layout? Suggestions for change? Remove objects? Simplify? Is the design appealing to the user group?
  
  

• -  Very simple design was shown, two options either “activities” or “Ferry Trip”
• -  Unclear why “Ferry trip” leads to Tripadvisor stuff
• -  Simplify it, maybe just pick one feature to focus on

Help

Is any type of support for tricky stuff provided? Is it needed?

- No, nothing is tricky

Sketches

Clear enough? Self explaining? Well thought through?

• -  The sketches were not very detailed
• -  Not sure what the UI for game or activities would look like
  
  
  Messy?
  Other important questions/problems/tips?
  
  

• -  People won’t be waiting around before the ferry with extra time for games
• -  Kids will play the game for too long, people wanting to see the activities will be annoyed
• -  There is no real purpose, the functions of the device seem kind of random

- What is the problem you’re trying to address?

• -  Where would this be? Portable?
• -  There is wifi on the ferry, or that would be a better solution
• -  How can the Tripadvisor stuff be updated? 

Summary of think aloud--- Group post

Think-aloud is a technique when users interact with the product in order to be observed so that the researchers can test the usability goals and user experience goals of a design. We set up an interview for the think aloud feedback.

We asked seven users to think aloud for us. Here are the feedbacks that are listed in each page.

Tutorial page:

  Positive feedback:

• Like the tutorial page that can guide the user well
• The steps for using are quite easy, no confusing instructions

  Criticism:

• Kind of boring
• Should be more clear about how to get the video
• The button is not that clear

Information page:

  Positive feedback:

• Like the design of each page, very clean and clears
• Appreciate the multiple choice of backgrounds
• Like the information given a basic idea of that place

  Criticism:

• Hope to have more choices of the background

Video page:

  Positive feedback:

• Really like this idea, it is quite a new one
• The guide is really clear
• It will be a fancy souvenir to be a memory of stockholm

  Criticism:

• The video backgroud is a bit dark

Sticker page:

   Positive feedback:

• Like the stikers, very interesting for children to play with

   Criticism:

• Do not like the given stickers, hope to have more choices

Post card:

   Positive feedback:

• Like the design of the post card
• Would be a quite special and fancy souvenir

  

   Criticism:

• Need a better design for the post card

Overall:

For our first version, we asked several users to try our first prototype. Our users thinks that we should have more choices for the backgroud. And stickers should be added to the printed postercard. What is more they hope the instruction will be more clear.

We made some changes due to these comments.

After the experiencing of the new version, our users gave the feedbacks as following:

“ I quite like this idea, hope I can use it soon”

“ I think kids will like it much, I will certainly try it. Also, I quite like the design, it is clean”

“The instructions are clear, I quite like this idea, it will be a great souvenir for me. But I hope the post card can be better designed ”

“There are still a lot to be improved. But I quite like this idea. And I would like more choices for both the background and stickers”

Think aloud + Extra field study with modified prototype--- Suping Shi

Think aloud is a direct observation method of user testing that involves asking users to think out loud as they are performing a task. Users are asked to say whatever they are looking at, thinking, doing, and feeling at each moment. This method is especially helpful for determining users' expectations and identifying what aspects of a system are confusing. A qualitative data gathering, the think-aloud is a technique which focuses on understanding the designs actual usability in a real context of use. It encourages users to speak out their actions while using a product. This allows designers to identify what kind of thoughts go through a user's mind while navigating through the application. This also gives observers insight into the participant's cognitive processes (rather than only their final product), to make thought processes as explicit as possible during task performance.

For our first version:

We set up an interview weeks ago, looking for people to experience our prototype. That was the first version of our prototype. Though we have not totally accomplished all the function by then, it could still shows the basic ideas to the interviewee. We interviewed 5 people. During their interview time, we let our users to speak out what they think about our prototype instead of asking them in the end. We kept ourselves quiet so that we would not influence our user's ideas causing any bias. Most of them felt that they were sometimes confused about the next action. They would feel more comfortable under our instruction. So we changed this in our new version of the prototype. We added a tutorial page in our new version.

Here are what they respond:

Positive feedback:

“I quite like this idea, I think the child gonna like it”

“Oh, I would like to spend much time on it, it is a pretty cool souvenir!”

“I really appreciate the idea of post card, I like to have myself on it”

“Woo, I like this idea, the video is quite fancy for me to be a memory of stockholm ”

Criticism:

“Oh, what is the next step, not that sure what I should do”

“Well, I think the choices of the background should be more, so that children will like it more”

“You should send the video to our email”

“You should add some kind of tutorial thing into it”

“That is fancy, but I would like something like stickers to make it more special (towards the post card)”

Adjusting in the new version and think aloud

Tutorial page

For the criticism about tutorial, we added a tutorial page into out prototype

Think-aloud:

“I quite like this tutorial page, it is really clear about the whole process”

“Oh, it is neat and clear, like it”

“May be the tutorial page can be more fancy and more structured, but I quite like it”

“The tutorial page is a bit boring”

Scenery page

And according to the quote for multi-background, we added several different backgrounds for each site.

Think-aloud:

“Yeah, I quite like this, I like the feeling of having more choices.”

“Good, I like these backgrouds, I hope there can be more.”

“I like this page, the structure is pretty good.”

Sticker page

Think-aloud:

“Still that words, I like that when I can have choices, I think more stickers will be better.”

"I think the kids will definitely like to play with it"

"Oh, adorable, I like those. But I would expect more choices"

Overall Think Aloud:

Positive:

“I would like to spend time on this machine. Of course my kids will also like it, hope to see it in Sweden soon.”

“Can not wait to try a real one, hope the progress is good, I would even want to see this in other country. I think that is pretty fancy, and I quite like the video part.”

“I have seen something similar with this, but they do not have the post card printed like you guys, really like this”

Criticism:

“ I like this but this prototype is not that completed now, hope you can make it better”

“ I think it will be better if the post card is better designed, I would also recommend more choices of stickers.”

“ I so not like the QR code for getting the video, I hope it can be sent to my email. But I really like this idea.”

Think aloud--Jinchen Liu

Think-aloud protocol is a protocol used to gather data in usability testing in product design and development, participants are asked to say whatever comes into their mind as they complete the task. This might include what they are looking at, thinking, doing, and feeling.

I interview an international student from the Asian country, Here is some questions and evaluation.
Initial page: 
Pros:   - Compact and clear 
Cons:  - Confuse about the functionality. need to click on the tutorial.

Tutorial page:
Pros: Clear sentence to conclude each steps. 
Cons: Simple, hard to pay attention. 

Map page and Selection page: Design as normal, can understand the function.

 

The video page and Sticker page 

Pros:  - sticker function is really cute

           - time count down for preparing the pose 

Cons: - not so clear for adding the stickers 

           - the background is fragmented and ugly..

Ending page: design as normal.

In conclusion:

The final design is clear and compact, have the basic functionality. But some parts still puzzle the user about how to use it; To sum up, this prototype is well designed, the user has the interest to have a try in the real equipment.

Think Aloud: First prototype: Agneev Guin

Due to the feasibility of the presentation slides, we designed the first prototype in the form of powerpoint slides demonstrating the various usability perspectives and allowing the low fidelity prototype to provide most of the functionalities.

The user page consisted of the type of the memoir to be presented which highlights the Postcard in this case.

The simple map interaction of zoom and swipe functionalities were provided and the blinking light displaying the current location.

• The placement of the booths was suggested to be placed at all the tourist spots
• Additional info was required stating what would be the next steps.

Clicking on the important places would provide the pop up displaying the details about the place along with a running video which would be eventual background. In case they wanted to select the video, they could click on the like or dislike buttons.

• The icons of thumbs up and down were unclear as they didn't signify the selection but rather if they likes the place or not. This clearly deviated from our idea.

The next step is to pose which the recording takes place.

• Some kind of prior notice needs to be indicated which warns the starting of the recording.
• The length of video highly reflects the duration of the people standing at the booth.

The pose selection page was quite straightforward and there was a provision to change the picture if required.

• An initial suggestion could be provided considering the eye blinks and the facial clarity which could save time for the user and be a nice perspective for the user.

The users can then sketch or color by selecting the options on the sides.

• Here, a wonderful suggestion was to include standardized stickers of the place which could indicate the speciality of the place.
• This could also be used as an advertising tool to promote brands for companies.

The final page reflected the QR and a 30 second duration to scan the QR code to obtain the video.

• The duration was quite short considering the fact that there was no prior information and the users might not be able to open QR reader apps in time.
• The QR may be added to the postcard itself to scan at any further stage as well.
• This could be just a thank you page and a welcome notice for all tourists like a See you again page.

This is the postcard that would be printed after the entire process. This could be replaced by other types of memoirs as well.

Evaluation for group D1 [Group-post]---2016/11/07

User group

is it clear to whom the service is made for?

Their target group is tourist.
is functionality well adjusted to the particular user group? (language, difficulty level, interests)
They have language options, their accessibility for old people need to improve. 
And as for our group, we do not have much interests for using this equipment instead of our phones. 

"the feeling"

will the user stay? return?

the same as the last questions, not so much interest because we can search all the stuff on our mobile phone. does the design keep the users “all the way”?
the alphabet search functionality does not serve enough information. The better one would be replaced by the functionality search, like what kind of museum.

Interaction and over arching structure

 it’s always clear what options and possibilities the interface provides?
there is a change screen button on the corner, it is not so clear the functionality. 

Design/composition 

 Does the design have a good structure and layout? Suggestions for change? Remove stuff?
The design has a good layout and structure, one suggestion is that, some screen is not necessary to separate, can be integrated together to make it more clear.

Primary and secondary functions 

there is no secondary function, and primary function fulfils the goal.

Conclusion

The function for map and path planning is normally design on the phone software, we suggest to add some second functions to make it more interest. On the other side, some functions need to be modified as described above, also the accessibility for disabled or old people also need to consider.

Think Aloud -- Mengdi Huang

In order to identify potential usability problems of interaction design products, think aloud helps designers to know the thoughts of users by letting them say what they are thinking when conducting a certain task by product. Since our product has no complex functionality, the task is simply successfully create a postcard. There will still be some silent moments, due to people's customs. Despite of this, designer can capture users much preciser than imagining.

Pros:
Good to have a brief tutorial

Map taking place of list is interesting

Cons:
Tutorial is a bit boring
Buttons not so obvious

A little confusing when first see the pins on map
Too many spots to choose. Annoyed to click each of them to see which is the best. And forget which one has been clicked.

Stickers are not enough.
Stickers are not adjustable in size or angle.
                 

Didn't know how to get video clip until the postcard is printed out.

Think-aloud, by Hui Pang

Think-aloud Evaluation

Think-aloud is a technique where users interact with the product in order to be observed so that the researchers can test the usability goals and user experience goals of a design.

I asked a fellow student to perform the think-aloud evaluation since she can also be viewed as an international tourist.

Prior to the evaluation, we did not specify the functionality of this app, but to see how well she can make use of the tutorials to also evaluate the placement and clarity of it.

Below are the stages of evaluation.




On the introduction page: 

 "Those portrait slides illustrate this app is for snapshots."
"There are two buttons, one for going straight ahead, the other for reading instructions."
"The text below are not showing completely." [We edit that part later]

("By clicking the 'tutorial' button, we go to the tutorial page. ")


On the tutorials page: 



“The tutorials seem clear enough.”
"Too many texts with no pictures though."
"Also it would be lovely to add a 'go straight ahead' button, so we don't have to go back to intro page."[There is a link to the map page now.]

("By clicking 'back to home' button on the upper right corner to go back to into page, and then 'go ahead' on the home page, we go to the map page. ")

On the map page: 



"The explanation seems clear. "
“There is pan function for the map so we can see all the pinpoints on the map. ”

("By clicking one of the pinpoints, we go to a scenery page. ")


On the scenery page: 



"There is a short description for Vasa Museum."
"Also multiple scenes to choose from. "

("By clicking the 'apply' button, we go to the video capturing page.")


On the video capturing page: 



“By clicking the start button, we see a demonstration of an AR video”

("When the video capturing ends, it will automatically switch to snapshot page")


On snapshot page: 




“This page shows several snapshots, and there is clear instruction for how to choose from them.”

("By clicking 'confirm' button, a green 'print' button appears on upper right corner, and we click that to go into the sticker page.")



On sticker page: 



“This page has a menu to choose stickers from, which is nice. ”
"However, one can only use one sticker for a single time."
"It would be better to add more options. "

("Once we are happy with the decorations, we click the 'print' button to go to the printing process.")

On waiting for printing page: 





Think-aloud comments

What do you like about the app? 

It is easy to use, and has interesting applications.

What do you dislike about the app? 

It would be more fun with more scenery and sticker options.

Think aloud + Extra Field Study at Slussen using modified prototype[Pavithren]

I wasn't present during this week, therefore I am not able to comment on the Think Aloud test conducted on the participant. I would just give my reflection about the think aloud technique instead.

Thinking aloud is a cheap but valuable tool in usability testing. It encourages users to speak out their actions while using a product. This allows designers to identify what kind of thoughts go through a user's mind while navigating through the application.

Personally, I was a volunteer during one of the HCI lectures where the lecturer instructed me to use a website to book a badminton court for a particular day. I had to verbalize all my actions while trying to complete this task. I was having issues finding out if my actions had been received by the system as when I changed the dates while searching for a booking slot, i didn't receive any feedback or wasn't well phrased so I was stuck in the page for quite some time. After some random clicking, I managed to select my slot.
Even though I was a evaluator for this task, I can see that asking someone to think aloud exposes the emotional responses by users. Some might even get annoyed if they are not able to achieve a simple task.
But thinking aloud technique might not always work as some might not want to say everything they are thinking of and "filter" out their responses. Also we need to make sure that we keep quiet during the entire process so as to make sure that we don't feed words into the evaluator's mouths causing a bias.

In conclusion,  think aloud is a cheap and quick way to get feedback from users but we shouldn't totally depend on the findings from this technique only.


Field Study in a natural setting(20/11/2016)

Before we presented the low fi prototype during our exercise, Agneev, Shi Ping and I went to Slussen to conduct a field study about the prototype we had created in a natural setting. We conducted a usability testing on 5 families with children to get their feedback. Overall they were quite pleased with the minimalistic interface and suggested they would pay up to 3-5 euros for a postcard. They also feedback that a timing of about 20-30 seconds of the video was suitable. A demo of the app can be seen below(if it doesn't work watch here(mute audio please:) ): https://youtu.be/oXrh59JAcCs).

Please mute audio

 Gathering results from this study, we made some changes to the prototype for the presentation during the in-class exercise as mentioned in Agneev's post

Exercise 4 Group Evaluation [Group Post]

During the evaluation exercise, Group 3 gave us many valuable feedback on how our prototype could be improved. We provided the group with rough sketches of our intended UI, in design terms referred to as Low-Fidelity prototype. Before we asked them to try it out, we explained them the context of the problem we were trying to solve. Thereafter, they were asked to perform a walk through of the application to perform a heuristic evaluation. 




Feeeeling
After performing the walkthrough, the evaluators felt that they weren't clear if our prototype was well adjusted for the target group. They highlighted potential issues such as the following
-lack of interest by kids in joystick/buttons
-not something users would want to use again(mainly for tourists) - maybe I wasn't clear when explaining about the personas

Primary and Secondary Functions
As mentioned previously, the evaluators were not clear about the functions of our prototype. Probably because we tried to integrate the trip planner feature together with the souvenir machine. We agree with their feedback as we were quite ambitious trying to solve several features at once. This made us realize that we should focus on a single problem and ace in it.

Interaction
The evaluating team highlighted that the prototype didn't have a clear objective it was trying to achieve. They suggested that instead of trying to solve many problems at once, we could focus on the main issue we are trying to solve i.e. create something memorable for tourists who visit djugarden and they can share this memory with their loved ones back home.

After looking at the feedback, our group decided that we should forego some of the features to streamline the final design and improve the usability for the tourists.

Conclusion

The evaluation stage was quite vital in the design process as it enabled us to uncover problems we wouldn't have thought of due to the designer bias.