HCI Research Papers: Paper Reading #8: Gesture Search: A Tool for Fast Mobile Data Access

Yang Li. 2010. Gesture search: a tool for fast mobile data access. In <em>Proceedings of the 23nd annual ACM symposium on User interface software and technology</em> (UIST '10). ACM, New York, NY, USA, 87-96. DOI=10.1145/1866029.1866044 http://doi.acm.org/10.1145/1866029.1866044
UIST 2010 New York, New York.

Yang Li is the sole author of this paper. Li received his Ph.D. from the Chinese Academy of Sciences which he followed up with postdoctoral research at the University of California at Berkeley. Li helped found the Design Use Build community while a professor at the University of Washington before becoming a Senior Research Scientist at Google.

The researcher hypothesized that traditional methods used to enter queries for searching phones may be inappropriate in some situations. Voice-based queries, for example, are inappropriate in quiet or exceptionally loud locations. Keyboard based entries are difficult to use, especially with the small real estate of mobile devices. Gesture searching, on the other hand, should be natural and easy to use in any situation. ("We hypothesized that Gesture Search provides a quick and less stressful way for users to access mobile data.")
GUI-oriented touch input should have less variation in its trajectory than gestures.

The researcher developed an Android application that he made available for download within the company. The application logged user data which was then analyzed at the end of a trial period. Users were not obligated to use the application and they could quit using it at anytime. When choosing the data to analyze the researcher required that the user had used the application at least once a week for a month.
To collect data pertaining to the difference between a GUI-based interaction and a gesture, the researcher set up a study which asked participants to perform various tasks on a mobile device. The interactions were recorded without the users knowing what the study was about, this was to create natural interaction.

The data collected supported the hypothesis that Gesture-based queries were applicable for a mobile device in various situations. Users typically used the application to find contacts. Less than a third of the time was the search used to find applications with a minimal amount of queries used for web pages or bookmarks. The majority of searches were completed in under 3 seconds with 2 or fewer characters entered. The average rating was 4/5 stars. A majority of users commented that they liked the application because it alleviated the user of having to navigate through menus to find information.
The hypothesis that there was less variation in a GUI gesture was also supported in the study's results. While the application is unsure about a potential gesture, it is shown in a faded yellow line and the ambiguity is usually resolved in less than a quarter second.

This paper presented an application which combined Gestures recognition with search technology. There are several key features which feed the success of this application. The ambiguity of the gesture, such as not being able to properly identify an "A" versus an "H" is taken into account when acquiring the results, which in this case would include both "A" and "H" words. Additionally, the system learns from previous results, with selected results having a higher likelihood of being presented at the top of the results. The longer a user goes without selecting an item the more that weight fades out.

This paper combines powerful technology with a fantastic idea. Personally, navigating through a menu in some situations can be frustrating, such as when on the bus. I downloaded Gesture Search and gave it a shot in that exact situation and, true to the researcher's findings, it worked well in that environment. In fact, I did find it easier to use than launching the Contacts app to find friends.
In that particular situation, however, I was just as comfortable using the traditional text-based queries. The only reasoning for this is that the query can be entered faster, not having the down time of recognizing each letter as it is entered (and not having to spend as much time entering each letter). The point of this paper was not advocating this as a faster alternative to keyboard entered queries; rather, it allows for easier entry of queries. This goal was accomplished, as I frequently used it when walking around campus and at other times when I didn't want to pay as much attention to my phone as I did to the environment around me. The learning feature of the search as well as the ambiguity resolution make it a viable application.

Picture Sources: "Gesture Search: A Tool for Fast Mobile Data Access"

HCI Research Papers