Gestures disected: The value of posture, build, and dynamics
Team: J.Wu, P. Ishwar, J. Konrad
Funding: National Science Foundation (CISE-SATC)
Status: Ongoing (2011-…)
Summary: This project focuses on discovering how much the performance of gesture-based authentication can be attributed to pure biometric information, that a user has no control over, such as individual limb lengths, and gesture dynamics, that a user can fully control. A related question is: How easy is it to copy these dynamics? In this project, we propose a framework to decompose a gesture into three components: initial posture, limb proportions, and gesture dynamics. We then study the impact of each component and various component combinations on the performance of gesture-based user authentication using a dataset of 36 users performing 3 gestures of varying complexity. Details of this dataset can be found here.
Gesture Decomposition: We posit that a gesture contains three types of user information: initial posture, limb proportions (build), and dynamics. The first two types of information have no user intent. A posture is related to user habits, whereas limb proportions (body build) are inherent characteristics that cannot be easily changed. In contrast to this, dynamics have an intent — depending on the gesture, the dynamics change. In spherical coordinates, these 3 values can be represented as the following: length of the limbs, initial angles between limbs at each joint, and changes in these angles over time. We can suppress the effects of each component by “standardizing” the corresponding component. Limb proportions are suppressed by replacing the proportions with a standard (averaged across users) proportion. Initial posture is suppressed by replacing the initial posture with a standard (averaged across users) posture. The dynamics component is suppressed by only considering the first frame.
The following two figures show skeletal pose estimates with various components suppressed.
In the next table, we show the performance of gesture-based authentication when spoof attacks are considered. This is compared to the case when there are no spoof attacks (zero-effort attacks by the same to-be attackers). We consider two information suppression scenarios. The first is when nothing is suppressed, and the second is when we suppress posture and build — which leaves behind only dynamic information.
The authentication power of a gestures does not lie solely in posture, body build, or dynamics. Each gesture component plays a non-trivial role. For zero-effort attacks with a single component suppressed, the suppression of dynamics causes the most significant degradation followed by posture and then build. In the context of matched spoofing, although attackers may be able to improve their ability to replicate dynamics through training, posture and limb proportions serve to make the gesture more secure. This effectively comes full circle — gestures do indeed serve as a renewable biometric. Should a breach occur and dynamics get compromised, the biometric can be partially changed by performing a different dynamic.
For a more in-depth explanation of our methodology and the aforementioned results please refer to our paper below.
- J. Wu, P. Ishwar, and J. Konrad, “The value of posture, build and dynamics in gesture based user authentication,” inInt. Joint Conf. on Biometrics (IJCB), pp. 1-8, Oct. 2014.