ROI-Extraction-From-Hand-Image

Submitted by: Shubham Santosh Upadhyay

Guided by: Asst. Prof. Rahul Chatterjee (UW Madison)

---

Required Libraries:

1. OpenCV
2. Numpy
3. MediaPipe

---

Abstract:

As we all know biometric authentication plays an important role in ensuring security and extraction of biometric features is always a challenge. We have separate datasets available for separate biometrics but an attempt is being made to extract all three biometrics i.e. fingertip, Palmprint, and Hand-geometry from a single hand image. We have tried different methods and finally found the most apt solution. A significant amount of time is spent on surveying different research papers on "Authentication with the help of different biometrics" and collecting all possible datasets.

---

Datasets that can be used for further research works:

Palmprint Dataset-Hindawi

GPDS Dataset

11k Hands

IITD Dataset

Boston University Hand Login Dataset

Fingerprint and Iris Databases

University of Notre Dame Iris Dataset

CASIA

XM2VTSDB multi-modal face database

**Some sites give you datasets directly, on others you have to request.

---

Methods that didn't work:

1. Skin Segmentation->Contour detection->Finding min area contour to find fingerprint.
2. Skin Segmentation->Finding Convex hull and Contour intersection point and finally locating ROI
3. Using deep learning (limited computational power)
4. Extracting ROIs using Haar Cascade.

Codes can be found in the Methods_That_Didnt_Work folder.

---

Method That worked:

1. Using Mediapipe Hands opensource framework.

---

Methodology:

1. MediaPipe Hands is a high-fidelity hand and finger tracking solution. It employs machine learning (ML) to infer 21 3D landmarks of a hand from just a single frame.

2. We used media pipe as it solves the problem of contour detection, skin segmentation, background removal, and dynamics.

3. The hand landmarks are found and then normalized to get the coordinates of 21 points on the hand.

   

4. These coordinates are used to draw ROIs.For example- points 8,12,16 and 20 represent the index, middle, ring, and pinky finger respectively. The thumb is not considered as it is tilted during photo capture.

5. Palmprint is captured by storing and finding the centroid of all the palm boundary points (0,1,5,9,13,17).

6. Hand geometry includes Length of all fingers, palm width, and distance between knuckles.

---

Input:

Input image

Output:

Hand Landmarks:


Finger Tips:

Palmprint:

Hand Geometry Values:

---

Future Work:

Furthermore, this work can be used to extract features and use them for authentication.

For fingerprint feature extraction from a low-resolution image (The image obtained from the dataset was not meant for fingerprints and hence it is a big problem to extract fingerprint features):

1. For preprocessing, as the pic has almost no fingerprint it needed to be preprocessed a lot.
   https://answers.opencv.org/question/6364/fingerprint-matching-in-mobile-devices-android-platform/
   
2. For fingerprint Matching.
   https://github.com/Utkarsh-Deshmukh/Fingerprint-Feature-Extraction
   

For palmprint feature extraction.
https://github.com/AdrianUng/palmprint-feature-extraction-techniques

---

References:

 [1]. <https://docs.google.com/spreadsheets/d/1WyfpljI49AqvCO_lV3-XOfHVg4Ilks_yJGWhZXCLU_M/edit?usp=sharing>