Paper Reading #27- Sensing cognitive multitasking for a brain-based adaptive user interface

Title: Sensing cognitive multitasking for a brain-based adaptive user interface
Reference Information:
Erin Solovey, Francine Lalooses, Krysta Chauncey, Douglas Weaver, Margarita Parasi, Matthias Scheutz, Angelo Sassaroli, Sergio Fantini, Paul Schermerhorn, Audrey Girouard, and Robert Jacob, " Sensing cognitive multitasking for a brain-based adaptive user interface". CHI '11 Proceedings of the 2011 annual conference on Human factors in computing systems. ACM New York, NY, USA. ©2011. ISBN: 978-1-4503-0228-9.
Author Bios:
Erin, Francine, Krysta, Douglas, Margarita, Matthias, Angelo, Sergio, and Robert are all associated with Tufts University in Massachusetts, USA.
Paul is associated with Indiana University.
Audrey is associated with Queen's University in Ontario, Canada.
Summary:

• Hypothesis: If the authors can create a system to detect a user's "to-do list" and allow them to multitask on different things at once, then those tasks will be completed faster, the user will become "understood" by the system, and a new kind of technology will be effectively used.
• Methods: While constructing their functional near-infrared spectroscopy (fNIRS), the authors took into account the three multitasking scenarios of the brain: branching, dual task, and delay. Branching is when you hold in mind goals while exploring and processing secondary goals. For example, you are working on your homework when a friend e-mails you and you begin to read the e-mail (while still remembering to return to your homework after you finish). Dual Task is when you have two tasks that require additional resources to complete. For example, a network technicial is fixing issues with his company's network while responding to important e-mails (possibly answering questions about the network). Finally, Delay is when a primary task is being worked on and a secondary task gets ignored. For example, you are watching a movie on your laptop when you see a notice come up that you have an e-mail. You ignore this notice. This is called delay because the secondary task gets delayed by the primary task. The authors were curious to see if they could tell the difference, cognitively, between the three kinds of multitasking. So they strapped some volunteers with gear and tested to see if they could while users performed given tasks (obviously multitasking was involved). After the preliminary study, the authors conducted a second study with the same methodologies in different spaces more relevant to HCI (for example, users were required to sort rocks by their type from Mars while keeping track of the position of a robot). The authors also launched a third study involving random vs predictive branching (the robot would move randomly in terms of the number of rock types were displayed vs the robot would move after every three presentations of rock types).
• Results: The preliminary study returned a recognition accuracy of 68%. To the authors, this was promising. In the second study with the robot and the rocks, any result where the participant achieved less than a score of 70% were discarded because it was seen as the task being done incorrectly. In the last study, there was no significant statistical difference found between the random and predictive branching. The authors were able to construct a proof-of-concept model because they were able to differentiate between the three types of tasking and incorporate machine learning into it.
• Content: The authors wanted to be able to create a system to measure and handle multitasking mechanisms. They were able to differentiate between three types of multitasking, create studies to measure efficiency for each kind of multitasking technique, test their system, and prove that it works as well.

Discussion:
I think these guys are geniuses. The technology is super advanced and the methodologies were complex when dealing with the proof-of-concept system. I don't know how much effect this will have in the HCI field (at least that I can see) because it didn't seem to me from reading this that there was much of a "new invention" or "new technology" here. They were able to recognize and "quantify" multitasking, but aside from that, I'm not sure how this could be applied. Maybe I missed it. I think the authors definitely achieved their goals, though. They said that all they wanted to do was be able to handle cognitive multitasking, which they were able to do. I'm indifferent about this article, honestly.

Paper Reading #26- Embodiment in brain-computer interaction

Title: Embodiment in brain-computer interaction
Reference Information:
Kenton O'Hara, Abigail Sellen, and Richard Harper, "Embodiment in brain-computer interaction". CHI '11 Proceedings of the 2011 annual conference on Human factors in computing systems. ACM New York, NY, USA. ©2011. ISBN: 978-1-4503-0228-9.
Author Bio:
Kenton O'Hara- Microsoft Researcher in Cambridge in the Department of Socio Digital Systems.
Abigail Sellen- Abigail Sellen is a Principal Researcher at Microsoft Research Cambridge in the UK and co-manager of Socio-Digital Systems, an interdisciplinary group with a focus on the human perspective in computing.
Richard Harper- Richard Harper is Principal Researcher at Microsoft Research in Cambridge and co-manages the Socio-Digital Systems group.
Summary:

• Hypothesis: If BCI technology can be effectively used and harnassed, then it can be used outside of the realm of just video gaming and possibly with the combination of other technologies.
• Methods: The authors decided to test the effectiveness of BCI with a video game called MindFlex. In the game, you keep a ball floating by concentrating (not just on the ball..on anything really. Higher concentration sends a higher "fan level" to the game. The higher the fan level, the higher the ball was kept up). The authors conducted a user study on participants (of at least two in a normal social setting) playing the game in a trial run to measure social relationships, interactions with the game, and coordination of the gameplay.
• Results: The results of the video stream and data from gameplay indicate strategies were taken by participants to keep the ball up (they would hold certain poses, do certain things, etc...because they thought it would help them). Other things such as gaze, intent, vision, imagination, proximity to the ball, and gestures all played roles in gameplay. Spectators also played roles in how the players played the game. Sometimes they would joke with the player or sometimes they would help them play.
• Content: The authors wanted to create an environment where the limits and applications of a BCI game were recordable. To do this, they took a game called MindFlex and gave it to participants to play for a week to later study how the game was played in terms of group size, strategy, focus, gestures, time spent playing, etc. They found that each user would approach the game uniquely and had a different style of play. Even the spectators of the game had a role to play in the gameplay of the player.

Discussions:
I thought this was kind of cool. I think I would have fun if I were able to play a game like this. I wonder if it would be difficult or not. I noticed that some players had some difficulty in controlling the ball sometimes. I wonder what kind of gestures / movements / strategies I would take to do well in the game. I don't think spectators would have an effect on me at all. I think that the authors definitely showed the the power of BCI technology is tangible and applicable to (at least) video gaming. Like the authors said, there are many other avenues of application for technologies like BCI. I think the authors achieved their goals for sure.

Paper Reading #25- Twitinfo: aggregating and visualizing microblogs for event exploration

Title: Twitinfo: aggregating and visualizing microblogs for event exploration
Reference Information:
Adam Marcus, Michael Bernstein, Osama Badar, David Karger, Samuel Madden, and Robert Miller, "Twitinfo: aggregating and visualizing microblogs for event exploration". CHI '11 Proceedings of the 2011 annual conference on Human factors in computing systems. ACM New York, NY, USA. ©2011. ISBN: 978-1-4503-0228-9.
Author Bio:
Adam Marcus- I am a graduate student at the MIT Computer Science and Artificial Intelligence Lab (CSAIL), where I am part of the Database group and the Haystack group. My advisors are Sam Madden and David Karger.
Michael Bernstein- I am a final-year graduate student focusing on human-computer interaction at MIT in the Computer Science and Artificial Intelligence Lab. I work with Professors David Karger and Rob Miller.
Osama Badar- Graduate student at MIT in the CSAIL.
David Karger- I am a member of the Computer Science and Artificial Intelligence Laboratory in the EECS. department at MIT.
Samuel Madden- Sam is an Associate Professor in the EECS department at MIT. He is also a part of the CSAIL group.
Robert Miller- I'm an associate professor in the EECS department at MIT, and leader of the User Interface Design Group in the Computer Science and Artificial Intelligence Lab.
Summary:

• Hypothesis: If the authors can organize and effectively communicate the timeline-based display information of TwitInfo, then Twitter's existing implementation of information will be able to be more effectively manipulated and enhanced.
• Methods: The authors create an algorithm to automatically "browse an event" and label the created event (which was done using a keyword) if a certain event is tweeted about over a certain amount of times per time unit. This will allow the activity on a timeline to "peak" to show that users are tweeting about an event. The authors were also able to implement a "sentiment analysis" about certain events (i.e. is this event "positive" or "negative" judging by user's comments and feedback about the event). TwitInfo also allows the creation and browsing of subevents. TwitInfo was evaluated against three soccer matches as well as a month-long collection of raw data. The authors then recruited 12 people to evaluate the UI of TwitInfo.
• Results: TwitInfo is biased by the Twitter users' interests. It was effective in measuring events for the soccer matches as well as the earthquakes. Occassionally, TwitInfo returned false positives such as anyone tweeting about a soccer term that wasn't necessarily about the soccer match being observed. In the study of understanding the UI, the results concluded that TwitInfo is an effective source for news without even any prior knowledge of events. Users were able to find events on the timeline and read summaries of sub-events. Users didn't necessarily agree with the sentiment analysis, however.
• Content: The authors wanted a better way to browse Twitter events and effectively get caught up on events and relevant news for a keyword event. The created a UI which incorporated a timeline-based display where users could browse events with "high peaks" of interest and get caught up on them. The authors also created an engine to determine whether an event was a positive thing or a negative thing for each part of the world.

Discussions:
I think this is very interesting, but as an avid anti-Twitter-er, I won't be using this technology. I have nothing against Twitter, I just got a Facebook first. The way the authors were able to take all of the information from Twitter and personal tweets and organize it all into meaningful, timelined events that users could browse is a very good idea though. I believe the authors weren't compltely happy with their results as far as the user evaluation and journalist evaluation of TwitInfo went, but I'd say that they achieved their goals. The technology that they created is definitely better than what was already in-place for Twitter, anyway.

Paper Reading #24- Gesture avatar: a technique for operating mobile user interfaces using gestures

Title: Gesture avatar: a technique for operating mobile user interfaces using gestures
Reference Information:
Hao Lu and Yang Li, "Gesture avatar: a technique for operating mobile user interfaces using gestures". CHI '11 Proceedings of the 2011 annual conference on Human factors in computing systems. ACM New York, NY, USA. ©2011. ISBN: 978-1-4503-0228-9.
Author Bios:
Hao Lu- I am a graduate student at University of Washington Computer Science & Engineering and DUB Group. I work on technologies that change and improve the way how people interact with computers.
Yang Li- Yang is a Senior Research Scientist at Google. Before joining Google's research team, Yang was a Research Associate in Computer Science & Engineering at the University of Washington and helped found the DUB (Design:Use:Build), a cross-campus HCI community. He earned a Ph.D. degree in Computer Science from the Chinese Academy of Sciences, and then did a postdoctoral research in EECS at the University of California at Berkeley.
Summary:

• Hypothesis: If using gesture methods to operate mobile interfaces performs optimally over finger-based touch input, then in a dynamic, mobile environment, it will prevail.
• Methods: To correctly and accurately model a user's intent for interaction on a page, the shape of the gesture and the distance from the gesture to the objects aer taken into consideration. After the user draws a gesture, an "avatar" bounding box appears translucently behind the gesture. The user can then interact with that avatar the same way as they would their desired object. Only this avatar is larger than the object on the page so it will be easier to interact with. If the object that gets highlighted is not the desired object, the user may disgard that by using several techniques. The authors conducted a study among 20 participants involving half of them learning gesture avatar and then shift, and vice versa. They were asked to complete tasks using each software. For some tasks, users were sitting down, and for others, they were on a treadmill (to simulate using the device while walking).
• Results: From the study, the authors found that Gesture Avatar was slower than Shift for 20px targets, but faster for 10px targets. Users using Shift while sitting were much faster than while they were walking on the treadmill, but for Gesture Avatar there was no significant difference in time reported. Gesture Avatar's error rates were lower than Shift's for all target sizes. 10 out of the 12 participants preferred using Gesture Avatar over Shift. The authors found that increasing the size of unique characters that would be drawn had no affect on Gesture Avatar's performance. Gesture Avatar supports one-shot interaction and acquiring moving targets also. There have been some integration issues with using Gesture Avatar in existing systems, however.
• Content: The authors wanted to create a way to effectively use your mobile device accurately while in a mobile environment. Touch-activated UIs can be highly inaccurate because of "fat-finger" and "occulation" problems. To correct for these, the authors created a technique to allow the user to "call" a desired object to interact with by "describing it" with a gesture in order for Gesture Avatar to enlarge it so it can be mroe accurately interacted with (and to ensure the correct object is being interacted with). After studies, the authors showed that Gesture Avatar is superior to other similar existing systems.

Discussion:
I thought this was kind of a neat idea. At first, I thought "Oh man, Yang Li is at it with another dumb gesture recognition idea again..." but I was not entirely correct. This time, I believe the proposed system can be extremely useful- especially in a mobile environment. I also liked how the different ways to ensure the correct object is selected is implemented as well. Gesture Avatar allows for disregarding incorrect objects, finding the next best match, interacting more accurately with desired objects, and allowing existing features of a UI to continue to be used if the user didn't want to use Gesture Avatar techniques on a particular page. I might consider using something like this if I did have a smartphone / touch-screen phone. I think Yang and Hao achieved their goals. It always feels good to know something you created out-performs something in existance already.

Paper Reading #23- User-defined motion gestures for mobile interaction

Title: User-defined motion gestures for mobile interaction
Reference Information:
Jamie Ruiz, Yang Li, and Edward Lank. "User-defined motion gestures for mobile interaction". CHI '11 Proceedings of the 2011 annual conference on Human factors in computing systems. ACM New York, NY, USA. ©2011. ISBN: 978-1-4503-0228-9.
Author Bios:
Jamie Ruiz- I'm a fifth-year doctoral student in the Human Computer Interaction Lab in the Cheriton School of Computer Science at the University of Waterloo. My advisor is Dr. Edward Lank.
Yang Li- Yang is a Senior Research Scientist at Google. Before joining Google's research team, Yang was a Research Associate in Computer Science & Engineering at the University of Washington and helped found the DUB (Design:Use:Build), a cross-campus HCI community.
Edward Lank- I am an Assistant Professor in the David R. Cheriton School of Computer Science at the University of Waterloo.
Summary:

• Hypothesis: If smartphones contain devices that allow for 3D tracking and sensing of the phone, then a set of optimal gestures to invoke commands with very natural mapping exists.
• Methods: The authors conducted a "guessability study" on participants by having them perform tasks and asking them what gesture / motion allows for the optimal mapping. For the experiment, users were told to treat the smartphone as a "magic black brick" because the authors removed all recognition technology from it so the users wouldn't possibly be influenced by anything of that nature. They were told to create gestures for performing tasks from scratch. The participants were recorded via audio and video. Data was also collected from a software on the phone for a "what was the user trying to do?" perspective. The study was conducted over all users who have previous and relevant smartphone experience.
• Results: Users most commonly used gestures not related to smartphone gestures. For example, viewing the homescreen for the smartphone had a very popular gesture as a result involving shaking the phone (as with an etch-a-sketch). Generally, the results gathered from the authors had a general agreement among gestures as well as the reasoning for the gestures. The authors therefore were allowed the luxury from the video "out-loud" process data to understand the user's thought process when creating the gesture. Tasks that were considered to be "opposite" of each other had similar gesture motions but were performed in "the opposite direction" for example. For example, zooming in and out from a map would involve moving the phone closer to you or farther away, respectively. The authors then went into a detailed study about what kind of gesture was created for performing tasks and what was involved / how the phone was treated for the gesture. "Agreement scores" were then calculated for each gesture to quantitatively find how "good" each individual gesture users made were.
• Content: The authors wanted to be able to create more natural, easy-to-use gesture sets for interactions with motion for smartphones versus plain gesture interactions. To create such a set, the authors studied volunteers who use smartphones which created their own gestures for performing tasks on smartphones. These gestures were than individually taken and measured against every other participants to see how "good" each gesture created was. From here, the specific kind of interaction and what all was involved/manipulated was taken into account also to see what general kind of pattern was recognizable.

Discussion:
I love studies like this where there is no field-specific jargon, no technical processes, no high-level fancy vernacular to have to learn, and no difficult end-goal in mind- this was very straightforward. The authors wanted to create a way to have simple and optimal motion gestures to interact with a smartphone. What better way to do it than to let a sample set of smartphone users "create" such a set? I definitely believe the authors achieved their goals and I would definitely consider using a smartphone with motion-based capabilities of the motions were based from studies like these. I enjoyed reading this paper.

Paper Reading #22- Mid-air pan-and-zoom on wall-sized displays

Title: Mid-air pan-and-zoom on wall-sized displays
Reference Information:
Mathieu Nancel, Julie Wagner, Emmanuel Pietriga, Olivier Chapuis, and Wendy Mackay. "Mid-air pan-and-zoom on wall-sized displays". CHI '11 Proceedings of the 2011 annual conference on Human factors in computing systems. ACM New York, NY, USA. ©2011. ISBN: 978-1-4503-0228-9.
Author Bios:
Mathieu Nancel- I am a Ph.D. student in Human-Computer Interactions in the in | situ | team (INRIA/LRI) since september 2008. I work on distal interaction techniques for visualization platforms: precise pointing, navigation in very large datasets, menu techniques, etc.
My Ph.D. supervisors are Michel Beaudouin-Lafon and Emmanuel Pietriga.
Julie Wagner- She is a Ph.D Student at INRIA. Her supervisor is Wendy Mackay.
Emmanuel Pietriga- Chargé de Recherche - CR1, interim leader of INRIA team In Situ.
Full-time research scientist working for INRIA Saclay - Île-de-France.

Olivier Chapuis- Chargé de recherche (Research Scientist) CNRS at LRI (CNRS & Univ. Paris-Sud).
Member and team co-head (by interim) of the InSitu research team (LRI & INRIA Saclay Ile-de-France).
Wendy Mackay- She is currently a Research Director with INRIA Saclay in France, currently on sabbatical at Stanford University. She runs a research group called in|situ|. Their focus is on the design of innovative interactive systems that truly meet the needs of their users.

Summary:

• Hypothesis: If the authors can study and trial different methods for large-scale wall-display-sized navigation, then they will create an optimal gesture set suitable for real and complex applications dealing with such a problem space. The authors had 7 separate hypotheses about each particular area of their design, but I combined this to make one general hypothesis.
• Methods: The authors conducted a series of pilot tests, pursued research, and performed empirical studies to narrow down all possible gesture and input methods for this system down to 12. They took into account performance (cost and accuracy), fatigue over periods of use of the input, ease of use, and natural mapping. The authors conducted an experiment to evaluate each of the 12 factors they discussed using in their system to see which were optimal. The authors had their ideas about which of the 12 were optimal to begin with, but they tested their ideas to see if this was the case.
• Results: After the experiment, the authors found that the two-handed gesture tasks were performed faster than the one-handed gesture tasks, involving smaller muscle groups for input interactions improves performance (providing higher guidance further improves it), and linearly-performed tasks were generally performed faster than circular ones. Circular gestures were slower because it was more often that participants overshot their target with circular gestures than with linear ones. After receiving feedback from the participants, they found that what the users were saying were (in general) in agreeance with their results from the data gathered.
• Content: The authors wanted to study a field that has historically received little attention. They wanted to figure out a way to effectively explore a wall-sized interaction surface with respect to performance, ease, and minimal fatigue. The authors eventually found (via studies and pilot testing) an optimal set of interactions and gestures suitable for such an interaction space.

Discussion:
I thought this paper was pretty interesting. This field of study has a lot of real-world applications such as (as mentioned in the paper) crisis management/response, large-scale construction, and security management areas. I thought their studies were accurate, appropriate, and extensive enough to gather relevant and meaningful data for designing a suitable system for the display. They definitely achieved their goals, in my opinion. I would have loved to have been a participant in this particular experiment; I think it would have been fun. I would consider using a technology like this if I was a billionaire and wanted to play a video game on my wall where I was a controller or something.

Paper Reading #21- Human model evaluation in interactive supervised learning

Title: Human model evaluation in interactive supervised learning
Reference Information:
Rebecca Fiebrink, Perry Cook, and Dan Trueman. "Human model evaluation in interactive supervised learning". CHI '11: Proceedings of the 2011 annual conference on Human factors in computing systems ACM. New York, NY, USA. ©2011. ISBN: 978-1-4503-0228-9.
Author Bios:
Rebecca Fiebrink- I will be joining Princeton as an assistant professor in Computer Science and affiliated faculty in Music in September, 2011. I have recently completed my PhD in Computer Science at Princeton, and I will be spending January through August 2011 as a postdoc at the University of Washington. I work at the intersection of human-computer interaction, applied machine learning, and music composition and performance.
Perry Cook- Professor Emeritus* (still researches but no longer teaches or accepts new graduate students) at Princeton University in the department of Computer Science and Dept. of Music.
Dan Trueman- Professor in the department of Music at Princeton University.
Summary:

• Hypothesis: If the user can be allowed to iteratively update the current state of a working machine learning model, then the results and actions taken from that model will be improved (in terms of quality).
• Methods: The authors conducted three studies of people applying supervised learning to their work in computer music. Study "A" was a user-centered design process, study "B" was an observatory study in which students were using the Wekinator in an assignment focused on supervised learning, and study "C" was a case study with a professional composer to build a gesture-recognition system.
• Results: From the studies, the authors gathered results and analyzed all of the results. From study "A", the authors saw that participants iteratively re-trained the models (by editing the training dataset), for study "B", the students re-trained the algorithm an average of 4.1 times per task, and the professional from study "C" re-trained it an average of 3.7 times per task. Cross-validation wasn't used in study "A", but for studies "B" and "C", cross-validation was used an average of 1 and 1.8 times per task, respectively. Direct evaluation was also present in the evaluation metric of the system. This was used more frequently than cross-validation. Participants in "A" strictly used this measure, while the students and the professional in studies "B" and "C" used direct evaluation on an average of 4.8 and 5.4 times per task, respectively. Using cross-validation and direct evaluation, users were able to receieve feedback on how their actions affected the outcomes. The overall results were that users were able to fully understand and use the system effectively. The wekinator allowed users to create more expressive/intelligent/quality models than with other methods/techniques.
• Conent: The authors wanted to create some method to allow users to provide feedback to the system iteratively while it is being used in order to hopefully create some iterative machine learning mechanism. The authors conducted user studies to test their hypothesis and collected results that suggested that the methods provided by the authors to perform tasks were superior to other techniques.

Discussion:
I thought this article was sort of interesting. It is a really good idea to have a system where you can tell the machine what is "good" or "bad" before it even spits out the final result. Being able to re-train your algorithm (mid-computation) is really beneficial to have. The cost-benefit for it seems reasonable, so I could see this idea becoming more widespread before long. The authors, in my opinion, definitely achieved their goals and proved their hypothesis to be true. I didn't understand cross-evaluation or direct evaluation in terms of the actual methods too much, but I know those factors were taken into consideration when collecting data for "satisfaction" of the system.