Gestures disected: The value of posture, build, and dynamics

Team: J.Wu, P. Ishwar, J. Konrad
Funding: National Science Foundation (CISE-SATC)
Status: Completed (2011-2017)

Summary: This project focuses on discovering how much the performance of gesture-based authentication can be attributed to pure biometric information, that a user has no control over, such as individual limb lengths, and gesture dynamics, that a user can fully control. A related question is: How easy is it to copy these dynamics? In this project, we propose a framework to decompose a gesture into three components: initial posture, limb proportions, and gesture dynamics. We then study the impact of each component and various component combinations on the performance of gesture-based user authentication using a dataset of 36 users performing 3 gestures of varying complexity. Details of this dataset can be found here.

We also study spoof attacks using the same dataset and show, somewhat surprisingly, that amateurs are unable to copy gestures with sufficient accuracy so as to significantly degrade the overall authentication performance even when they are trained on users that they are closest to.

Gesture Decomposition: We posit that a gesture contains three types of user information: initial posture, limb proportions (build), and dynamics. The first two types of information have no user intent. A posture is related to user habits, whereas limb proportions (body build) are inherent characteristics that cannot be easily changed. In contrast to this, dynamics have an intent — depending on the gesture, the dynamics change. In spherical coordinates, these 3 values can be represented as the following: length of the limbs, initial angles between limbs at each joint, and changes in these angles over time. We can suppress the effects of each component by “standardizing” the corresponding component. Limb proportions are suppressed by replacing the proportions with a standard (averaged across users) proportion. Initial posture is suppressed by replacing the initial posture with a standard (averaged across users) posture. The dynamics component is suppressed by only considering the first frame.

The following two figures show skeletal pose estimates with various components suppressed.

ijcb_move — Intermediate frame of a skeletal sequence with suppressions of [from left to right]: Nothing, Posture, Build, Posture + Build.

Results: The aforementioned suppressions were evaluated on our acquired dataset. Various combinations of suppressions were tested. We show below results for the 3 considered gestures (left-right, double-handed arch, and balancing).

ijcb_zero_new — User authentication EER (average of user-specific EERs) with zero effort attacks when various components are suppressed. The best performing EER for each gesture is bolded in green.

Unsurprisingly, suppressing no information will lead to the best-performing EER (smaller is better). In terms of quantitative performance, we find from the above table that dynamics are more valuable than posture and build in gesture-based authentication.

In the next table, we show the performance of gesture-based authentication when spoof attacks are considered. This is compared to the case when there are no spoof attacks (zero-effort attacks by the same to-be attackers). We consider two information suppression scenarios. The first is when nothing is suppressed, and the second is when we suppress posture and build — which leaves behind only dynamic information.

ijcb_spoof_new — EERs for matched zero-effort attacks, and matched spoofing attacks. We show results for when no information is suppressed (Nothing), and when user-unique initial posture and build information are removed.

While training certainly improves an attacker’s ability to copy gesture dynamics, it seems that the unique limb proportions (which cannot be altered) and the initial posture (which amateurs attackers fail to pay attention to), more than make up for the loss due to compromised dynamics (which can always be renewed). This can be seen when there is a notable increase in EER from zero-effort to spoof attack only when posture and build are suppressed.

The authentication power of a gestures does not lie solely in posture, body build, or dynamics. Each gesture component plays a non-trivial role. For zero-effort attacks with a single component suppressed, the suppression of dynamics causes the most significant degradation followed by posture and then build. In the context of matched spoofing, although attackers may be able to improve their ability to replicate dynamics through training, posture and limb proportions serve to make the gesture more secure. This effectively comes full circle — gestures do indeed serve as a renewable biometric. Should a breach occur and dynamics get compromised, the biometric can be partially changed by performing a different dynamic.

For a more in-depth explanation of our methodology and the aforementioned results please refer to our paper below.

Publications:

J. Wu, P. Ishwar, and J. Konrad, “The value of posture, build and dynamics in gesture based user authentication,” inInt. Joint Conf. on Biometrics (IJCB), pp. 1-8, Oct. 2014.