BodyLogin Dataset: Multiview

Description

The BodyLogin Dataset: Multiview (BLD-M) contains 2 gesture types performed by 40 different college-affiliated users (27 men and 13 women of ages that are primarily in the range 18-33 years). Each subject was asked to perform 2 unique short gestures, each approximately 3 seconds long, each with 20 samples. Both gestures involve motion in the upper and lower body.

• S gesture: drawing an “S” shape with both hands. Although simplistic, this gesture is shared by all the users (harder for authentication).

• User-defined gesture: each user chooses his/her own gesture with no instruction. Although potentially complex, this gesture is unique for most users (easier for authentication).

Data Acquisition

Each user gesture was captured using four Kinect cameras (version 1) for Windows. Three Kinects were placed facing the user and one Kinect was directly behind as shown in the figure to the right. Of the forward-facing Kinects, one was placed directly in the front (center) and two were offset by about 35 degrees to each side of the center. All devices were set up approximately 2 meters away from the user. Users were primarily facing the center camera for the duration of the performed gesture. All the Kinects were connected to a single PC to assure time synchronization. Captured frames were synced to the frame-rate of the center Kinect. Each Kinect camera captured a 640×480 depth image and skeletal joint coordinates at 30 fps. All data was captured using the official Microsoft Kinect SDK.

The simultaneous use of multiple Kinect cameras introduces structured light interference, which reduces the quality of the estimated depth maps. In order to reduce this interference, we applied an approach similar to the one developed by Butler et al. [link]; we attached Amico DC motors with a different number of revolutions per minute (RPM) to each camera as shown in the image to the left.

Four different types of gesture acquisition scenarios were enacted to capture different types of real-world degradations: no degradations, personal effects, user memory, and gesture reproducibility.

• Personal effects: In the case of personal effects, users either wore or carried something when performing a gesture. Half of the users were told to wear heavier clothing, and the other half were told to carry some type of a bag.

• User memory and gesture reproducibility: The impact of user memory was tested by collecting samples a week after a user first performed a gesture. Users were first asked to perform the gesture without any video or text prompt. After a few samples were recorded, the user was shown a prompt and asked to perform the gesture again. These last samples measure reproducibility.

Of the 20 samples recorded for each gesture, each of the described scenarios has 5 samples recorded. The following table summarizes the degradation scenarios that were used for each gesture.

Data Definitions

Data is provided in the form of unprocessed skeletal joint coordinates. BLD-M provides 3 files (2 MATLAB .mat files and a .zip). Gesture sequences (unprocessed skeletal joint coordinates) are stored in the .mat files and can be accessed using specific indexing of the cell data structure. Each gesture has it’s own .mat file. We have also provided a .zip file with helper functions that visualize the skeletal sequences, as well as a sample script that shows how to extract and display a gesture. Index definitions used in the sample script can be found later in this section. The download form can be accessed directly here: Download Form.

If used correctly, the function sample.m provided will extract two gesture samples from the data-set and display them. The following videos illustrate the output that it should yield.

For the cell data structure data, we can access a specific gesture as follows:

data{subject_id,session_id,sample_id,view_id}

ID definitions are provided in the following tables.

User-defined Gesture Descriptions

Gesture Type

Personal Effects Listing

For samples involving personal effects, users were assigned to either hold an object or wear clothing. The assignments are shown as follows.

View ID

Missing Data

The following data is unavailable and will appear as an empty matrix in the cell array.

Off-labeled Data

The following data is available for use in the data-set, but conflicts with prior labels.

Download Form

You may use this dataset (BLD-M) for non-commercial purposes. If you publish any work reporting results using this dataset, please cite the following paper:

• J. Wu, J. Konrad, and P. Ishwar, “The Value of Multiple Viewpoints in Gesture-Based User Authentication,” in the Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Workshop on Biometrics, Jun. 2014.

To access the download page, please complete the following form. When ALL the fields have been filled, a submit button will appear which will redirect you to the download page.

Contact

Please contact [jonwu] at [bu] dot [edu] if you have any questions.

Acknowledgements

This dataset was acquired within a project supported by the National Science Foundation under CNS-1228869 grant.

We would like to thank the students of Boston University for their participation in the recording of our dataset. We would also like to thank Luke Sorenson and Lucas Liang for their significant contribution to the data gathering and tabulation processes.