Paper Reading #4: Gestalt: Integrated Support for Implementation and Analysis in Machine Learning

• Title:
• Gestalt: Integrated Support for Implementation and Analysis in Machine Learning
• Reference Information:
• Kayur Patel, Naomi Bancroft, Steven M. Drucker, James Fogarty, Andrew J. Ko, and James Landay. 2010. Gestalt: integrated support for implementation and analysis in machine learning.  In <em>Proceedings of the 23nd annual ACM symposium on User interface software and technology</em> (UIST '10). ACM, New York, NY, USA,  37-46. DOI=10.1145/1866029.1866038 http://doi.acm.org/10.1145/1866029.1866038
• UIST 2010 New York, New York.
• Author Bios:
• Kayer Patel is a Ph.D. student at the University of Washington.  He was previously funded by a National Defense Science and Engineering Graduate Fellowship and is currently funded through a Microsoft Research Fellowship.
• Naomi Bancroft was, at the time of publication of the paper, a senior undergraduate student at the University of Washington.  Her plans were to work for Google following graduation.
• Steven Drucker received both his Master's degree and Ph.D. from MIT.  He is an affiliate professor at the University of Washington while concurrently a Principle Researcher and manager at Microsoft Research. He researches human computer interaction dealing with large amounts of data.
• James Fogarty is an Assistant Professor at the University of Washington teaching a senior level introduction class to HCI.  He received his Ph.D. from Carnegie Mellon University.  His vast research is supported by the National Science Foundation, FXPAL, Google, Intel and Microsoft.
• Andrew Ko is an assistant professor at the University of Washington.  Ko is a member of the dub group which performs human computer interaction research and education. Ko is interested in developing software which will ease the software development process.
• James Landay is a professor at the University of Washington.  Landay received his Ph.D. from Carnegie Mellon University in 1996.  Landay is a founding member of the DUB center and has previously worked as the Director of Intel Labs Seattle.
• Summary
• Hypothesis:
• A general purpose Machine Learning tool that allows developers to analyze the information pipeline will aid in the development process by increasing efficiency and awareness.
• General purpose tools can solve many of the same problems that domain specific tools can solve.
• Methods
• After developing the Gestalt framework, the researchers added functionality to MATLAB to minimize the differences between the two programs, leaving key functionality out.  MATLAB was chosen as the comparison program because it is currently the most widely used machine learning aid.  Eight graduate students, all of whom had completed at least one machine learning class, were selected to test the two programs.  Researchers created two problems, one pertaining to movie reviews and another pertaining to gesture recognition.  Each of these had 5 bugs injected into them.  The participant's job was to find and fix these bugs.  Each participant solved each problem with both tools.  
• Results
• The study showed that participant's were able to find a significantly greater number of bugs when utilizing the Gestalt tool.  Three of the eight users took the time to create similar functionality in MATLAB that is found in Gestalt, two of which hadn't used Gestalt yet.  All eight users were unanimous in stating their preference for Gestalt.  During questioning, five users explicitly stated that they found the visual and interacting elements of the pipeline a useful feature that they would want in tools they used.  Additionally, participants were able to create their own views through Gestalt support in the limited amount of time they were using the system encouraging the idea that general purpose tools are powerful.   
• Contents
• Machine learning, which analyzes data to describe appropriate behavior, is a powerful tool for developers.  As opposed to fighting with describing appropriate behavior syntactically  developers can show desired behavior using data.  General purpose tools for this process have not been widely tested or developed.  Gestalt provides this general purpose tool that can be used for a range of different problems.  It remains to be seen as to whether general purpose tools can solve any problem that domain specific tools can. Gestalt's key strength is the its ability to aid developers in analyzing the data pipeline through visualization and interaction.
• Discussion
• Tools, such as Gestalt, are long overdue in the field of machine learning.  Demonstrated in other fields, general purpose tools are usually not quite as powerful as domain specific tools.  However, many times they are flexible enough to still be a powerful tool for a wide range of problems.  The key advancement introduced by this program is NOT its generalization of the machine learning problem.  Rather, it is their influential work in recognizing that the visualization of the data pipeline is key for efficient understanding of the problem.  Interacting with this visualized data pipeline furthers this advantage.
  
  
  

  
  

Picture Source: "Gestalt: Integrated Support for Implementation and Analysis in Machine Learning"