Paper Reading #27: Sensing cognitive multitasking for a brain-based adaptive user interface

• Title: Sensing cognitive multitasking for a brain-based adaptive user interface
• Reference Information:
• Erin Treacy Solovey, Francine Lalooses, Krysta Chauncey, Douglas Weaver, Margarita Parasi, Matthias Scheutz, Angelo Sassaroli, Sergio Fantini, Paul Schermerhorn, Audrey Girouard, and Robert J.K. Jacob. 2011. Sensing cognitive multitasking for a brain-based adaptive user interface.  In <em>Proceedings of the 2011 annual conference on Human factors in computing systems</em> (CHI '11). ACM, New York, NY, USA,  383-392. DOI=10.1145/1978942.1978997 http://doi.acm.org/10.1145/1978942.1978997
• UIST 2010 New York, New York.
• Author Bios:
• Erin Treacy Solovey with Tufts University.  Fifteen publications.
• Francine Lalooses with Tufts University.  Two publications.
• Krysta Chauncey with Tufts University.  Six publications.
• Douglas Weaber with Tufts University.  Two publications.
• Margarita Parasi with Tufts University.  First publication.
• Matthias Scheutz with Tufts University, Indiana University and the University of Notre Dame.  Thirty seven publications.
• Angelo Sassaroli with Tufts University.  Six publications.
• Sergio Fantini with Tufts University.  Seven publications.
• Paul Schermerhorn with the University of Notre Dame and Indiana University.  Twenty four publications.
• Audrey Girourd with Tufts University and Queen's University.  Nineteen publications.
• Robert J.K. Jacob with Tufts University and MIT.  Two publications.Seventy five publications, and 1,004 citations.
• Summary
• Hypothesis:
• An fNIRS tool will be able to capture the tasking state of a human mind as effectively as required by an HCI application standing, around the same level as an fMRI machine.  Researchers also hypothesized that a system can be designed to capture this tasking state and facilitate it by aiding the user in their tasks.
• Methods
• The first hypothesis was tested by seeing how accurately an fNIRS machine could classify a user when they were in various tasking states (branching, dual or delay).  They simply asked users to perform a variety of tasks and analyzed how frequently their system was correctly classifying the current state.  The second test involved participants instructed to perform various activities with a robot, and they analyzed how their system facilitated this interaction.  
• Results
• The fNIRS machine showed some recognition between various states, above 50%, but was not extremely accurate.  The researchers noted that this was a small group of testers and that other various improvements could be made to improve this accuracy.  Building off this the researchers built a tool that attempted to facilitate these changing states.
• Contents
• The researchers noticed that the fNIRS machine is not as effective at collecting data as the fMIR machine is, but it is much more practical in a real world environment.  Researchers built a proof of concept system that showed promise.  
• Discussion
• These researchers proposed an interesting system and were effective at providing proof for the second hypothesis (the first one still needs to be analyzed some more).  I like this idea because I am constantly attempting to multitask and the little tasks that can easily be automated are the ones that take up the most time (switching contexts etc.).  If this is effectively implemented, then the work place should increase in productivity by a significant amount.



Picture Source: "Sensing cognitive multitasking for a brain-based adaptive user interface"

Paper Reading #26: Embodiment in Brain-Computer Interaction

• Title: Embodiment in Brain-Computer Interaction
• Reference Information:
• Kenton O'Hara, Abigail Sellen, and Richard Harper. 2011. Embodiment in brain-computer interaction.  In <em>Proceedings of the 2011 annual conference on Human factors in computing systems</em> (CHI '11). ACM, New York, NY, USA,  353-362. DOI=10.1145/1978942.1978994 http://doi.acm.org/10.1145/1978942.1978994
• UIST 2010 New York, New York.
• Author Bios:
• Kenton O'Hara has been cited in nearly 500 articles published through the ACM in the last 18 years.  He is affiliated with Hewlett-Packard as well as Microsoft Research.
• Abigail Sellen is a Principal Researcher at Microsoft Reserach Cambridge.  She joined Microsoft Research after working for Hewlett Packard Labs.
• Richard Harper is a Principal Researcher at Microsoft Research in Cambridge.  
• Summary
• Hypothesis:
• Researchers hypothesize that Brain-Computer Interaction can have a social impact when used in different environments.  In particular, when a BCI is used in a gaming environment, the interactions between the people involved change fundamentally.  The researchers hope to examine exactly how this interaction changes.
• Methods
• The researchers sent the MindFlex game home with three different groups of people and asked them to record their gaming experience.  Each of the groups was supposed to choose when, where and with whom to actually play the game.  This created a very realistic, and fluid environment in which people freely came and went.  
• Results
•  The analysis showed a few novel differences, such as the unnecessary mental imagery created in an attempt to properly control the game.  Many users would frequently think 'up, up, up' to raise the ball when in fact all they had to do was concentrate a little more.
• Contents
• The paper shows results that can be used to extend the use of BCI into other environments. These environments will not be typical of previous social interaction space since there are new problems such as users not being able to acknowledge feedback from other people around them.
• Discussion
• The researchers hypothesis was very open-ended, simply that BCI interaction is something that needs to be studied in order to be expanded.  The researchers were able to effectively study these interactions, and presented several clear findings.  I had never thought about the fact that simply moving your hand or responding to a question could have such a profound effect on concentration.  I hope that this research is continued so that more fluid invisible computing can be accomplished in the future.



Picture Source: "Embodiment in Brain-Computer Interaction"

Paper Reading #25: TwitInfo: Aggregating and Visualizing Microblogs For Event Exploration

• Title: TwitInfo: Aggregating and Visualizing Microblogs For Event Exploration
• Reference Information:
• Adam Marcus, Michael S. Bernstein, Osama Badar, David R. Karger, Samuel Madden, and Robert C. Miller. 2011. Twitinfo: aggregating and visualizing microblogs for event exploration.  In <em>Proceedings of the 2011 annual conference on Human factors in computing systems</em> (CHI '11). ACM, New York, NY, USA,  227-236. DOI=10.1145/1978942.1978975 http://doi.acm.org/10.1145/1978942.1978975
• UIST 2010 New York, New York.
• Author Bios:
• Adam Marcus is a graduate student at MIT.  He received his undergraduate degree from Rensselaer Plytechnic Institute.
• Michael S. Bernstein researches crowdsourcing and social computing.  He is in his final year at MIT.
• Osama Badar is a graduate student at MIT.
• David R. Karger is a member of the AI laboratory at MIT.  He has spent time working for Google.
• Samuel Madden is an associate professor at MIT.  Has developed systems for interacting with mTurk.
• Robert C. Miller is affiliated with Carnegie Mellon University.  He has 71 publications in the ACM over the last 15 years.
• Summary
• Hypothesis:
• The researchers hypothesize that information aggregated from microblog sources, Twitter in particular, can be used to study events.  This should be accomplished in real time and produced by a system that makes data visualization and exploration simple and intuitive.
• Methods
• Researchers developed a tool called 'TwitInfo' to implement the goals set forth in their hypothesis.  They then evaluated the effectiveness of their system by letting average users of twitter and an award winning journalist test it.
• Results
•  The evaluation showed that TwitInfo effectively analyzed events based on spikes in tweets and allowed users to easily gain a shallow understanding of a chain of events.  The journalist emphasized that this knowledge was only shallow, but that the tool still allows people to gather an understanding of events from the first person point of view as they unfold.
• Contents
• The paper presents a tool that is able to analyze twitter information that is not domain specific in real time which has not been effectively accomplished before.  The major limitations of this system are that not all interesting events are flagged when analyzing peaks in the number of tweets (such as a yellow card in a soccer game) and that the information available is generally more shallow than what a standard news report would generate.  
• Discussion
• The researchers were certainly able to create a tool that performs the intended functionality.  It is interesting that this article was mentioned since Dr. Caverlee just recently gave a speech to UPE about a project involving Twitter.  Datamining this expansive source of information can certainly produce some interesting results if it can be done correctly.  I'm actually a little surprised that Twitter itself doesn't do more to support such functionality.



Picture Source: "TwitInfo: Aggregating and Visualizing Microblogs For Event Exploration"

Paper Reading #24: Gesture Avatar: A Technique for Operating Mobile User Interfaces Using Gestures

• Title: Gesture Avatar: A Technique for Operating Mobile User Interfaces Using Gestures
• Reference Information:
• Hao L&#252; and Yang Li. 2011. Gesture avatar: a technique for operating mobile user interfaces using gestures.  In <em>Proceedings of the 2011 annual conference on Human factors in computing systems</em> (CHI '11). ACM, New York, NY, USA,  207-216. DOI=10.1145/1978942.1978972 http://doi.acm.org/10.1145/1978942.1978972
• UIST 2010 New York, New York.
• Author Bios:
• Yang Li received his Ph.D. from the Chinese Academy of Sciences which he followed up with postdoctoral research at the University of California at Berkeley. Li helped found the Design Use Build community while a professor at the University of Washington before becoming a Senior Research Scientist at Google.
• Hao Lu is a graduate student at the University of Washington.  His research interests include improving interactions between humans and computers.
• Summary
• Hypothesis:
•  The researchers had three hypotheses.  Gesture Avatar would be slower than Shift on large targets but faster on the smaller targets.  Gesture Avatar will have a lower error rate than Shift.  Finally, the error rate for Gesture Avatar will not be affected as much by walking as Shift's will be.
• Methods
• The researchers designed an experiment which required users to select targets using both methods (Shift and Gesture Avatar). Half of the participants learned Shift first while the other half learned Gesture Avatar first.  The variables were the two different techniques, the state of the user (sitting versus walking), the size of the targets being selected and the number of repeated letters in the selection group.
• Results
•   The results show the following facts.  Shift was significantly faster for larger targets, but significantly slower for smaller targets.  The error rate for Shift increased as the target size decreased, while the error rate for Gesture Avatar remained nearly constant.  Only one user in the study preferred Shift over Gesture Avatar.  Finally, surprising to the researchers was that the number of repeated letters had almost no effect on the accuracy of Gesture Avatar.
• Contents
• This paper presented one implementation of Avatar Gesture.  Minor modifications can be made, such as displaying a magnified version of the selected target as opposed to the gesture created.  This system works the best when the maximum amount of information is available about the underlying UI.  Essentially, it has been programmed into an API that provides a set of wrapper functions to embed the functionality to transform the user experience.
• Discussion
• I want to begin the discussion by thanking Yang Li.  Every single one of the researcher papers that he has authored have been presented in an extremely clear and efficient manner.  This makes reading the papers and drawing conclusions exceedingly easy.  The researchers were certainly able to provide support for all three of their (very clearly stated) hypotheses.  This is also one of the few papers that solves a current problem that I have personally experienced.  Many of the papers focus on solutions to problems in the future or for a select group of people (eg. how to control a wall-sized display).  This problem is widely experienced, with approximately 50% of Americans owning a smartphone.  The proposed technique seems very intuitive (especially the re-selection swiping) and it would be great to test this idea out in a real world environment.



Picture Source: "Gesture Avatar: A Technique for Operating Mobile User Interfaces Using Gestures"

Paper Reading #23: User-Defined Motion Gestures for Mobile Interaction

• Title: User-Defined Motion Gestures for Mobile Interaction
• Reference Information:
• Jaime Ruiz, Yang Li, and Edward Lank. 2011. User-defined motion gestures for mobile interaction.  In <em>Proceedings of the 2011 annual conference on Human factors in computing systems</em> (CHI '11). ACM, New York, NY, USA,  197-206. DOI=10.1145/1978942.1978971 http://doi.acm.org/10.1145/1978942.1978971
  
• UIST 2010 New York, New York.
• Author Bios:
• Jaime Ruiz is a fifth-year doctoral student at the University of Waterloo.  Ruiz plans to graduate in December 2011.
• Yang Li received his Ph.D. from the Chinese Academy of Sciences which he followed up with postdoctoral research at the University of California at Berkeley. Li helped found the Design Use Build community while a professor at the University of Washington before becoming a Senior Research Scientist at Google.
• Edward Lank is an Assistant Professor at the University of Waterloo.  Lank received his Ph.D. in 2001 from Queen's University. 
• Summary
• Hypothesis:
• Researchers hypothesized that actions can be performed efficiently on a mobile device by utilizing 3D gestures recognized by sensors located on the device such as an accelerometer. 
• Methods
• The researchers designed an experiment to allow users to freely create their own gestures.  The screen on the phone was locked so that it wouldn't display any feedback to the users.  The participants were presented with sets of tasks and were asked to design an easy to use and remember gesture for each of them, and were not required to commit until all of them had been designed.
• Results
• The data collected was then analyzed which resulted in some classifications.  When mapping the gestures they were classified into four dimensions of the nature of the action: metaphor, physical, symbolic or abstract.  Other classification descriptions were developed which resulted in a gesture taxonomy.  
• Contents
•  Researchers hope that this taxonomy will aid in the creation of gesture interactions for phones in the future.  The researchers are unclear whether these gestures will be used in a generic fashion, with multiple applications supporting similar motions, or whether developers will use these to create their own arbitrary gestures for different applications. The hope is that representative motions will be utilized for similar functionality.
• Discussion
• The researchers seem to have presented a proposal for new navigational techniques which may be used in future generations of mobile devices.  The researchers proposed further research to investigate gesture delimiting techniques, so that fluid interactions can be achieved when performing tasks.  I believe this paper was also accepted to the same conference and would be an interesting read to determine the feasibility of this idea.



Picture Source: "User-Defined Motion Gestures for Mobile Interaction"

Paper Reading #22: Mid-air Pan-and-Zoom on Wall-sized Displays

• Title: Mid-air Pan-and-Zoom on Wall-sized Displays
• Reference Information:
• Mathieu Nancel, Julie Wagner, Emmanuel Pietriga, Olivier Chapuis, and Wendy Mackay. 2011. Mid-air pan-and-zoom on wall-sized displays.  In <em>Proceedings of the 2011 annual conference on Human factors in computing systems</em> (CHI '11). ACM, New York, NY, USA,  177-186. DOI=10.1145/1978942.1978969 http://doi.acm.org/10.1145/1978942.1978969
• UIST 2010 New York, New York.
• Author Bios:
•  Mathieu Nancel is a Ph.D. student in HCI.  Focuses on distal interaction techniques.
• Julie Wagner is a postgraduate research assistant.  Wagner currently works with Wendy Mackay on new tangible interfaces.
• Emmanuel Pietriga is the interim leader of INRIA team in Situ where he is a full-time reasearch scientist.  Works on interaction techniques for wall-sized displays.
• Oliver Chapuis is a research scientiest at LRI.  Received his Ph.D. in Mathematics in 1994.
• Wendy Mackay is a research directory with INRIA Saclay in France.  Focuses on the design of interactive systems.
• Summary
• Hypothesis:
• Researchers hypothesized that they could improve interaction with wall-sized displays by studying the effectiveness of several factors as gesture interactions.  These factors included the number of hands, the motion of the gesture and the degrees of freedom for the gesture.   
• Methods
• The researchers designed an experiment in which all patterns of interactions were exhausted.  The participants completed this test in several sessions, with a few guidelines set to minimize fatigue and memory loss.  
• Results
• The researchers took the data collected and analyzed it using several statistical analysis techniques.  The conclusions of their study cannot prove or disprove the effectiveness of any of the techniques, but they do suggest some would be more natural and useful than others.
• Contents
•  Researchers determined that participants preferred gestures utilizing both hands as opposed to single handed gestures.  Similarly, linear motions were preferred (as well as more accurate) than circular ones.  Researchers suggested that 3D free motions as well as one handed circular motions on a 2D surface should be rejected and not used in the future.
• Discussion
• The researchers had a very interesting problem to tackle, but I am undecided as to how effectively they were in proving or disproving their hypothesis.  Regardless, the work done here is exciting because of the possibilities that it implies for the future.  As mentioned in the paper, movies already visualize humans interacting with very large displays using fluid motions as opposed to tools.  While humans have never had to do this in the past, that is not an indication that it cannot be both smooth and natural.



Picture Source: "Mid-air Pan-and-Zoom on Wall-sized Displays"

Paper Reading #21: Human model evaluation in interactive supervised learning

• Title: Human model evaluation in interactive supervised learning
• Reference Information:
• Rebecca Fiebrink, Perry R. Cook, and Dan Trueman. 2011. Human model evaluation in interactive supervised learning.  In <em>Proceedings of the 2011 annual conference on Human factors in computing systems</em> (CHI '11). ACM, New York, NY, USA,  147-156. DOI=10.1145/1978942.1978965 http://doi.acm.org/10.1145/1978942.1978965
• UIST 2010 New York, New York.
• Author Bios:
• Rebecca Fiebrink has just completed her PhD dissertation.  In September of this year she joined Princeton University as an assistant professor in Computer Science and affiliated faculty in Music.  She spent January through August of this year as a postdoc at the University of Washington.
• Perry Cook earned his PhD from Stanford University in 1991.  His research interests include Physics-based sound synthesis models.
• Dan Trueman a professor who has taught at both Columbia University as well as Princeton University.  In the last 12 years he has published 6 papers through the ACM.
• Summary
• Hypothesis:
• Researchers hypothesized that Interactive Machine Learning (IML) would be a useful tool that could improve the generic supervised machine learning methods currently in practice.   
• Methods
• The researchers developed a system to facilitate IML.  This system was then used in three seperate studies (A, B, and C).  The results of these tests were then analyzed throughout the paper.  The first test was composed of six PhD students who used the system (and its subsequent updates) for ten weeks.  The second study was composed of 21 undergraduate students using the system (Wekinator) in an assignment focused on supervised learning in interactive music performance systems.  Finally, the third study was with a professional cellist to build a gesture recognition system for a sensor-equipped cello bow.
• Results
• The results from this study show various expected and unexpected results.  One thing the system showed researchers was that it encouraged users to provide better data.  Some users 'overcompensated' to be sure that the system understood what they were attempting to do.  Additionally, the system surprised users occasionally which encouraged them to expand their attempted efforts.  Sometimes the system performed better than their initial goals which encouraged them to redefine their ultimate destination idea.
• Contents
• The researchers determined that any supervised learning models should have their model quality examined.  This is because cross-validation may not be enough to validate model quality.  Additionally, Interactive Machine Learning was determined to be useful because of its ability to continuously improve the usefulness of a trained model. 
• Discussion
• The researchers did an excellent job proving their hypothesis.  Utilizing three seperate studies, that were formatted in different ways allowed them to collect a wide range of useful data.  The real time feedback and interaction of this system is what makes it particularly appealing to me.  Since the users are allowed to see the effectiveness of the training models their providing as they provide them, rapid marked improvements can be made to the system.  This facilitates efficient development of a final system, as opposed to a slow and articulated struggle to reach an intermediate goal.



Picture Source: "Human model evaluation in interactive supervised learning"