Category Archives: RAA

Research Article Analysis

An Interesting Read: ID Construction on Facebook

I am starting exploring a very possible topic of my thesis: building an online identity management tool. Though my focus will be put more on the design and development of the tool, it is essential to understand theories and current practice of online identity management.

Last weekend, I came across a very interesting and informative paper researching online identity construction on Facebook: Zhao, S., Grasmuck, S., & Martin, J. (2008). Identity construction on Facebook: Digital empowerment in anchored relationships. Computers in Human Behavior, 24(5), 1816–1836. doi:10.1016/j.chb.2008.02.012. As published in 2008, this paper is an early effort to research on online identity management in the context of nonymous Social Networking Sites (SNS).

Important things to notice before reading: the Facebook in 2012 is quite different from the version back in 2008. The most important difference is that now Facebook is not entirely nonymous: you can create fake account and you don’t have to be a college students. However, my interests will still be on those who use SNS to extend offline life and communication, and thus will need so called “identity management”; but not those who creates forged identities that have nothing to do with their offline presentation.

Back to this paper. The methods they used are content analysis of Facebook accounts and follow-up structured interviews. The literature review part of this paper gives very comprehensive review of theories of identity construction. I will list some important ones here as an index that can help to go back to the paper:

  • “Identity is an important part of the self-concept. … and identity is that part of the self ‘by which we are known to others’ (Altheide, 2000, p.2)”
  • Construction of identity = identity announcement & identity placement. Identity announcement is made by the individual to claim who she is while identity placement is made by others to endorse the claim. When there is intersection between identity announcement and identity placement, this intersection will be the constructed identity.
  • Difference between identity construction through localized interactions and online interactions: here the authors discussed the famous work from Goffman (The Presentation of Self in Everyday Life). Basically the localized (offline) interactions place many constrains to hinder individuals from displaying inconsistent selves while anonymous online environment detaches the embodiment of an individual and frees one to present a totally new self. This new mode of presenting a “hidden self” online shows the ability of Internet to empower identity construction.
  • A nonymous online environment is between those two extremes discussed above, since it is built upon offline relationships. The authors coined the term “anchored relationships” to describe this kind of offline-based online relationships. In this anchored online relationships, people are nonymous online and can be traced to their offline identities. Unlike anonymous online society, this nonymous online environment also places constraints on identity claims.
  • The authors then introduced the concepts of “now selves” and “possible selves”. The previous one is who you are in others’ eyes right now and the latter refers to who you want to be in the possible future, which is an image unknown to others at this stage (Markus and Nurius, 1986). They then argued that the nonymous online environment differs from localized interactions and anonymous online environment, in that it empowers a new self-presentation as “hoped-for possible selves“.
  • The concept of this “hoped-for possible selves” is so important that I list it alone here and quoted the definition given in this paper: “Hoped-for possible selves are socially desirable identities an individual would like to establish and believes that they can be established given the right conditions” (p.1819).
  • As the authors review the dating-site (as a form of nonymous sites) research, they found that these nonymous dating sites provide opportunities to users to make public “identity statements“, which can be implicit and explicit. This is an important path for people to construct the “hoped-for possible” selves that are not known offline.

The results of the paper are discussed based upon these theoretical framework covered in the literature review. The major findings are two fold. First, the identity statements form a continuum of implicit to explicit claims. At one end, people adopt “showing without telling” strategy, using visual presentations such as photos to present themselves. This is an implicit way, which is the most common among participants. At the other end of explicit expression, people use narrative format to tell and label themselves. This is the least popular among participants. In between, there is an enumerative way to show the “cultural self” through listing one’s tastes on movies, songs, and hobbies etc. Second, the authors examined the types of self claims. Most participants chose to project themselves as “socially desirable” and hid pessimistic personas and academic identities. Though most results fit the hypothesis of showing the “hoped-for possible” selves that are positive, they did find some presented some types of “hidden selves”, through publishing “superficial or hedonistic images”, “less socially sanctioned” quotes, and “sexually provocative statements”.

Some reflections: As mentioned in the discussion part, living in this nonymous online environments, online and offline world is not separated anymore. The negative “hidden selves” being shown online is exactly the reason why we want to build this online identity management tool. People, especially students need to learn how to coordinate their identity claims in online and offline worlds, since they are highly connected. The “hoped-for possible selves” could be the ideal image we would like people to cast online. In my previous study of identity construction on another popular SNS Twitter, I also found the similar strategy continuum of implicit claims to explicit claims, which is very exciting. After reading is paper, I have several research ideas: 1. As suggested in this paper, we should compare the “objective” coding of online content with the “subjective” self-reported online identity construction methods. Basically compare “what people thought they did” with “what they actually did”.  2. Continue my previous study of identity construction on Twitter and compare it with the case on Facebook. To better compare, it will be better if the participants group are comparable. Understanding how people are presenting themselves online helps to discover areas that are commonly failing and yelling “SOS”.

RAA: A Practical Open Card Sorting Study

RAA stands for: Research Article Analysis

Paper discussed:

Lewis, K. M., & Hepburn, P. (2010). Open card sorting and factor analysis: a usability case study. Electronic Library, The, 28(3), 401-416. doi:10.1108/02640471011051981

1. Purpose of the research:

This paper describes the process, analysis, results, and implications of a card sorting usability study conducted for redesigning the library Website of University of Illinois at Chicago.

2. Methods:

Totally 18 participants were recruited and 15 were completed the task at the end. Among these 15 participants, 7 are undergraduate students, 7 are graduate students, and 1 is faculty member.

The open card sorting was done individually with each participant.

  • The researchers created 93 numbered index cards. Label on each card was one of the existing or potential content from the library website.
  • The participants were allowed to create anything missing or duplicate cards where they felt the cards belonged to multiple categories, using blank index cards.
  • The participants were also allowed to discard cards that they felt redundant or named a category with “other”  or “general” for cards seems necessary but not fit into any other categories.
  • Speak aloud protocol was carried out to gain the participants’ rational for sorting decisions.
  • The researchers didn’t answer any question to define the label terminology or indicate which category should a card go.

Post-analysis using “factor analysis” was done to indicate the association of a card with a category.
3. Main Findings:

Using the methods above, the researchers got the final result of sorting these 93 cards into 11 categories, with 27 cards not sorted into any categories. Besides the final sorting result, the researchers also concluded some qualitative findings as follows:

  • Participants tend to group together cards that have the same words on them.
  • Participants were sorting not only by format but by processes or tasks.
  • The 27 cards that were not associated to any categories were due to different reasons: should be piled to themselves; redundancy; vague meaning; meaninglessness.

4. Analysis:

I found this paper particularly interesting because the up coming card sorting project. This 2010 paper not only described the project process and findings, but also did a good job reviewing previous card sorting study done by several university libraries. The qualitative findings of this paper reminds me of some essentials of designing card sorting experiment, mentioned by Gergle & Wood (2002), such as “listen to other comments about the content” and “include a brief description on each card”. Overall, it is a well-written paper with considerable details and discussions, which could be used as a good reference to our project. What I am going to dig deeper is their statistical method to combine the results. This seems to make much more sense than just eyeballing the results. I will write about different methods of combining card sorting results in a future post.

 

References:

Gergle & Wood (2002), Usability for the Web: Designing Websites that Work. Morgan Kaufman.

RAA: It’s hard to tell a good story — exploring persona-scenarios

RAA stands for: Research Article Analysis

Paper discussed:

Madsen, S., & Nielsen, L. (2010). Exploring Persona-Scenarios – Using Storytelling to Create Design Ideas. In D. Katre, R. Orngreen, P. Yammiyavar, & T. Clemmensen (Eds.), Human Work Interaction Design: Usability in Social, Cultural and Organizational Contexts (Vol. 316, pp. 57-66). Berlin, Heidelberg: Springer Berlin Heidelberg. Retrieved from http://www.springerlink.com/content/6nh13h2684v154lm/

1. Purpose of the research:

Explore the persona-scenario method and propose guidelines about how to construct persona-scenarios as good and coherent stories for IT system design.

2. Methods:

Review of persona, scenarios, and narrative theory was introduced and followed by a case study. Review part is heavy in this paper because of its theoretical characteristic. Case study was presented as a verification as well as supplement for their conclusion.

The case study was reported as group tasks from a persona-scenarios workshop. The task was to build persona-scenarios to facilitate the redesign of Virk.dk, a portal that enables companies to submit forms digitally to government. Sixteen participants were chosen, as key stakeholders in development process, to attend the workshop. They were than divided into 4 groups and worked on building persona-scenarios. Each group was given a short text with a start situation for their persona, and asked to write and present their scenarios based on the situation given. The authors also provided a brief analysis of the case.

3. Main Findings:

The authors brought up their “theory” of building persona-scenarios in narrative theory perspective, mainly based on their review of different methods and theories. Since the nature of scenarios is story-telling, they believed that follow and modify the narrative theory could help to build well-constructed persona-scenarios. They then gave a mapping between narrative elements (characters, time, problem, setting, opening episode, episodes, resolution, plot, overall story, and narrator’s perspective) in the story form and narrative elements in a persona-scenario. In this way, they tried to define the scope and structure of a persona-scenario.

In the next stage, through analyzing the case study, they found out that “for scenario writers, once the story is started, it develops in its own course”, which means they might have different focus and extra preference developing the story. This will inevitably harm the credibility of the scenarios. Based on the case study, they further provided the following instructions as supplements to their previous mapping:

  • In the scenarios, while the persona is the protagonist, the future IT system has to play a prominent role as well — it could be part of the events — rather than a character or tool-like object.
  • In design scenarios, the problem should always be solved and goal should always be reached. Because “the more they address obstacles and design-oriented ways of overcoming the obstacles, the more concrete the future IT system and design ideas for the future IT system stand out within the story and get validated from the persona’s point of view”.

At the end, they gave a comprehensive table of components in a design-oriented persona-scenarios, developed from the original mapping table and the case study results.
4. Analysis:

This paper is well executed in terms of providing very good points for constructing persona-scenarios. The correlation of persona-scenarios and narrative theory was wisely found and presented to construct the structure of persona-scenarios. Analysis of case study was brilliant too, with good points found to further modify the theoretical structure. I enjoyed the analyzing process that they finally come up with the opinion that persona-scenarios should all have a happy-ending in order to provide more insight for design.

The only thing weak in this paper is the way they presented their case study. It is unclear in several critical points, for example, in what degree did these participants already knew about persona and persona-scenarios; at which stage of the workshop did they started to carry out group tasks; how much user research results were given in that “short text”? Without knowing these information, we could not fully trust on the authors conclusion about persona-scenarios couldn’t be well written without their guidelines. In another way, though good points were presented, the credibility of the paper is lowered due to lacking of these information.

RAA: Using Design Patterns in Heuristic Evaluation

RAA stands for: Research Article Analysis

Paper discussed:

Botella, F., Gallud, J. A., & Tesoreiro, R. (2011). Using Interaction Patterns in Heuristic Evaluation. In A. Marcus (Ed.), Design, User Experience, and Usability. Theory, Methods, Tools and Practice (Vol. 6769, pp. 23-32). Berlin, Heidelberg: Springer Berlin Heidelberg. Retrieved from http://www.springerlink.com.login.ezproxy.lib.purdue.edu/content/t346743643602746/

1. Purpose of the research:

Proposes a method to use interaction patterns in heuristic evaluation, in order to facilitate the process of heuristic evaluation as well as improve the output of heuristic evaluation in terms of providing redesign advice.

2. Methods:

The authors overviewed the use of heuristic evaluation as usability inspection methods and the prevalence of using design patterns in interface design respectively, and came up with the idea of mapping Nielsen’s heuristics with subsets of design patterns from Welie’s library. This approach seeks to find the correspondence between each heuristic and one or more design patterns, as showed below. The paper also claimed after several evaluation cycles, a refined correlation will be gained.

Mapping of heuristics and design patterns

Mapping of heuristics and design patterns

3. Main Findings:

A case study of heuristic evaluation of a university website using the proposed method shows it is easier to find direct solutions for usability problems found by evaluators.

4. Analysis:

After I re-read Nielsen’s procedure of conducting heuristic evaluation, I figured out that this paper is trying to fill the gap Nielsen mentioned as “Heuristic evaluation does not provide a systematic way to generate fixes to the usability problems or a way to assess the probable quality of any redesigns”. I appreciate the authors effort in mapping Nielsen’s heuristics with design patterns, trying to offer a systematic solution to the problem, but I think it a little bit forced to combine them in this way.

First of all, I think seeking redesign solutions through the successful examples is a common sense, and the use of pattern library as a reference could be a good idea (which might have already been widely used). But I am not sure how practical and efficient to use the mapping methods proposed in this paper as an evaluation step. After all, Nielsen’s heuristics is subjective, Welie’s patterns categories are subjective, and this mapping between them is subjective, it might be hard and unnatural to force people understanding and accepting the mapping. At least, as I looked at the mapping figure, I could not tell directly and clearly the relationship between the left column and right column. With a specific usability problem found, I would say it is more straight forward to find corresponding examples directly in the pattern library.

Anyway, I will try to test this idea when I do my heuristic evaluation, with questions like this: Is the mapping a redundant step? Will it be easier to locate solutions directly in the pattern library without mapping to their categories first?

RAA: Customize your heuristics?

Paper discussed:

Kientz, J.A., Choe, E.K., Birch, B., Maharaj, R., Fonville, A., Glasson, C. & Mundt, J. (2010) Heuristic evaluation of persuasive health technologies. Proceeding of the 1st ACM International Health Informatics Symposium (IHI ’10), 555-564. doi: 10.1145/1882992.1883084.

1. Purpose of the research:

Develop a set of 10 heuristics intended to find problems in persuasive technologies, and compare with Nielsen’s heuristics to see if specific designed heuristics could be more helpful for persuasive technologies.

2. Methods:

2.1 How to Define Heuristics

The research group firstly reviewed related literature and compile a master list of all usability guidelines and heuristics for persuasive technologies. Then they narrow down the list by combining the similar guidelines, prioritizing them, and discussing them using a process similar to affinity diagramming and ultimately came up with 10 heuristics. The list of 10 heuristics enable evaluators to focus on the most important aspects and also allow researchers do a comparison to Nielsen’s 10 heuristics.

The list of 10 heuristics are as follows (you could find explanation for each one in the paper):

Appropriate Functionality; Not Irritating or Embarrassing; Protect Users’ Privacy; Use of Positive Motivation Strategies; Usable and Aesthetically Appealing Design; Accuracy of Information; Appropriate Time and Place; Visibility of User’s Status; Customizability; and Educate Users.

As you might see these heuristics had some overlap with Nielsen’s. This was intentional and necessary because Nielsen’s list reflects the fundamental usability principles.

2.2 How to Conduct Evaluation

The researchers chose two web-based applications to evaluate: Mindbloom and MyPyramid BlastOff. The former is a website designed to track progress of users’ life goals, including health goal; while the latter is a online game aiming to educate children about healthy food choices. These two examples of persuasive technologies were chose because they could be accessed easily by any evaluators at any places with internet connection.

The researchers also recruited 10 evaluators, among who there were graduate students in HCI-related program and one game designer and one web coordinator. They were then randomly assigned to 2 groups: experimental and control group, corresponding to evaluate applications using new heuristics and Nielsen’s heuristics, respectively. The evaluation process was basically identical to Nielsen’s instruction, however, the researchers define the severity rating afterwards, instead of evaluators.

3. Main Findings:

There were several interesting findings.

The researchers claimed that the designed heuristics could discover more sever issues, more severe issues more frequently, and more issues that are useful in improving persuasive aspects of the interface evaluated.

What’s more interesting to me is that they found out the first two heuristics in the list had the highest number of issues correlated. This phenomena was consistent in both experimental and control groups. This finding suggested the order of heuristics we gave to evaluators might influence their findings. Thus we could intentionally randomize the order or place the heuristics in order of importance to get better data.

4. Analysis:

I will give 3 out of 5 points to this paper. Though the findings are kind of interesting, and valuable for evaluation of persuasive technologies, I doubt its scientific significance for the following reasons:

First, as the authors stated at the beginning of the paper, design specialized heuristics was already a trend in usability evaluation area. What this paper did was duplicate the research procedure of similar papers and applied it to evaluate persuasive technologies.

What’s more, I saw too much manipulations from researchers in the experiment. For example, since the evaluators recruited were not expert-level (with average self-rate experience = 2.3, 1=no experience, 4=very experienced), the researchers had to re-group the issues they found during evaluation and gave them severity rating. Without any verbalized comments from observer (see also Nielsen’s instruction), it was hard to tell how much subject opinions had been added during this process, which might skew the final conclusions.

Generally, I don’t doubt the overall conclusion of this paper, and agree that specialized heuristics could facilitate the evaluation for specific applications than Nielsen’s heuristics. However, as a research paper analyzed with statistics, I would like to see more rigid control of the experiment.

RAA: Care more about intended users rather than general public

RAA stands for: Research Article Analysis

Paper discussed:

Das, A., Faxvaag, A., & Svanas, D. (2011). Interaction design for cancer patients: do we need to take into account the effects of illness and medication? Proceedings of the 2011 annual Conference on Human Factors in Computing Systems, 21-24. doi: 10.1145/1978942.1978946

1. Purpose of the research:

Examine cancer patients’ ability to use a patient-centered information system and their need for the system, in order to improve the usability of the next version of the system.

2. Methods:

One way to accomplish the research goal is to establish if there are significant differences in task performance between particular patient groups and average computer users. Authors proposed a hypothesis that the group of cancer patients would have significantly more difficulties using a web-based healthcare system compared to a control group of healthy individuals.

The study was set up as an observational case-control study with an experiment where cancer patients and healthy controls were observed while they conducted tasks by using a web-based healthcare system. Semi-structured interviews and questionnaires were used afterwards to collect additional information. The whole precess was captured on video and analyzed to evaluate the usability based on the definition of usability by ISO (effectiveness, efficiency, and satisfaction).

3. Main Findings:

Effectiveness: the cancer patients experienced more difficulties compared to the healthy controls for the entire task and for all its five subtasks (lower completion rate). Efficiency: measured “time on task” was not quite useful because cancer patients quickly gave up and was given assistance while healthy controls quickly finished the tasks, which led to similar time on task. Satisfaction: SUS score might be affected by cancer patients’ motivation to use the system.

In conclusion, authors claimed effectiveness is the main issue faced by cancer patients due to their impaired physical and cognitive ability. A patient-centered systems should be designed with the intended users in mind, rather than average, general public.

4. Analysis:

I will give 4 out of 5 points to this paper. Four points for its novel and sympathetic consideration towards the design of patient information system, as well as the clear-conveyed experiments. One point is lost for the relatively small size of sample and lack of in-depth suggestions about specific improvements should be made. For future study, research could be carried out to examine the exact problems that cancer patients face. Possible approaches could be eye-tracking experiments and think-aloud method, in the way that both objective and subjective descriptions could be gained.