Paper Reading #2: Hands-On Math: A page-based multi-touch and pen desktop for technical work and problem solving

• Title:
•  Hands-On Math: A page-based multi-touch and pen desktop for technical work and problem solving
• Reference Information:
• Robert Zeleznik, Andrew Bragdon, Ferdi Adeputra, and Hsu-Sheng Ko. 2010. Hands-on math: a page-based multi-touch and pen desktop for technical work and problem solving.  In <em>Proceedings of the 23nd annual ACM symposium on User interface software and technology</em> (UIST '10). ACM, New York, NY, USA,  17-26. DOI=10.1145/1866029.1866035 http://doi.acm.org/10.1145/1866029.1866035
• UIST 2010 New York, New York.
• Author Bios:
• Robert Zeleznik has been the the Director of Research for the Computer Graphics Group at Brown University since 1996.  He received his Master's Degree in Computer Science from Brown University in 1989.  
• Andrew Bragdon is a second year Ph.D. student at Brown University.  He received both his Bachelor's and Master's degrees from their in 2007 and 2008 respectively.  He has previously worked at Microsoft working on the User Experience for Office.
• Ferdi Adeputra has helped author four papers published by the ACM.  The other three are all based upon a research project called code bubbles, which is a new way to develop integrated development environment.
• Hsu Sheng Sheng Ko is another affiliate of Brown University.  His second article for the ACM was just presented at CHI '11 that also dealt with gesture techniques.
• Summary
• Hypothesis:
• Integrating traditional input of pencil and paper with the power of a Computer Algebra System would allow for free-form transformations that gain the power of computational based systems while losing the error-prone process of manual manipulation.
• Methods
• The team created a system called Hands-On Math which utilizes a Microsoft Surface as well as a light pen to allow users to perform mathematical work on a computer system using, what was designed to feel like, a more natural input system.  The system made frequent use of bi-manual input, which used both hands simultaneously to perform operations.  Often, the non-dominant hand would sit on the surface with each finger in control of a menu option while the dominant hand held and manipulated the drawing pen.  The system made heavy use of pages which helped define a clear and logical work area.  A continuous virtual environment allowed the users to place their pages however they preferred which could be brought back later for further work.  The researchers chose a small group of volunteers to get initial feedback to their fully functional system.
• Results
• Each of the participants averaged about 45 minutes testing the system and, generally, the response was positive.  The light pen and text recognition software performed well when converting the written input into mathematical expressions.  Participants had the gestures explained to them since the system did not have a built-in gesture discovery, and they usually picked them up quickly especially after a demonstration.  Until the 'proper' techniques had been seen, participants would frequently make awkward movements to perform a gesture.  Researches noted what they describe as the sandwich problem, users would prefer if their non-dominant hand were not heavily relied upon in case they wanted to use it for an unrelated task.  Also noted was the fact that the likelihood of a misinterpreted gesture decreased dramatically when the input was bi-manual.
• Contents
• These researchers are simultaneously tackling two problems at once: namely, making the computational power of an electronic system more readily available to students and mathematicians as well as developing new input methods for touch-based input in general.  Of their tested gestures, most of them were well received by the participants in the study.  Bi-manual input, however, seemed to cause the users more trouble.  The system's recognition of these bi-manual inputs was much more accurate as compared to traditional uni-manual input.  The prototype only included basic algebraic manipulation techniques and more powerful techniques were requested, particularly the full power of Mathematica.    
• Discussion
• This research is both interesting and significant for me because I have always struggled with the barrier between the freedom of drawing and erasing on physical paper and the powerful tools offered by computer programs.  Frequently, particularly after my entry into college, I would have sheets lying all over my desk while Matlab was open and awaiting a single line to execute.  As the participants noted, this research certainly has practical applications, as I could have saved both a significant amount of frustration and a great number of trees.  Furthermore, my mother is a 7th grade math teacher who attempts to introduce as much technology into the classroom as possible.  Her biggest struggle is that the software she runs, while great at manipulating expressions pre-loaded into it, lacks the ability to easily accept input from a middle school student greatly limiting the amount of examples she can present. The results of this study show that there are still effective input methods that are still waiting to be discovered that will further ease the use of technology into daily life.
  
  

Pictures Source: "Hands-On Math: A page-based multi-touch and pen desktop for technical work and problem solving"