Articulated Hand Posture Recognition System

Abstract :
In this project, I studied the application of inner distance shape context descriptor(IDSC) in recognizing articulated objects particularly the hand articulation. The IDSC is used as a hand shape descriptor to capture the hand state(open/closed) and to recognize between them. Further, the technique is also used to recognize sign languages. My analysis is based on the SVM classifier and contour hand images obtained from the Image and Video computing group of Boston University

Approach :

• Segment the palm region using skin color based segmentation. The set of segmented images is divided into two sets: Training set and Test set.
• Preprocess to group the training and test images into different bins on the basis of their primary orientation. This is estimated by computing the scatter direction of the images through Principle Component Analysis(PCA).
• Calculate the IDSC signnatues of the segmented images. This returns a histogram description of each point along the objects' contour to describe other points in the contour wrt to distance and angle.
• These signatures are used as feature representation for input to address the hand state classification problem.

Two Classification Problems :

• Two state classification: Hand is open/closed. For this, a trained SVM on 50 samples of each state will be used.
• Sign language pattern matching: This uses solving N class pattern matching problem. Therefore, N SVMs were used in one-against-all configurations.

Graphical Representation of points on the hand boundary:
The graph is used to find the all pair shortest distance between points.

Histogram Visualization :
The log polar histogram representation of various points on the hand contour.

Presentation : Download

Report : Download

code : Download

Dataset : Download

BibTex : bibs

References :

• Raghuraman Gopalan and Behzad Dariush [2009], "Toward a vision based hand gesture interface for robotic grasping"
  IEEE/RSJ International Conference on Intelligent Robots and Systems , 2009


• S. Belongie, J. Malik and J. Puzicha. "Shape Matching and Object Recognition Using Shape Context," IEEE Trans. Pattern Anal. Mach. Intell., 24(24):509-522, 2002.


• H. Ling and D.W. Jacobs. "Shape Classification Using the Inner-Distance". IEEE Trans. Pat- tern Analysis and Machine Intelligence (PAMI), pages 286–299, 2007.