Measurement of Skin Deformation Using Fingerprint Feature Tracking
By Vincent Levesque and Vincent Hayward,
 

Introduction

This project proposes a skin strain measurement technique that relies on the tracking of anatomical landmarks of a fingertip sliding over a transparent surface which can be flat or have simple geometrical features. The immediate motivation is the need to generate driving signals for the STReSS, a tactile display that relies on distributed lateral skin strain patterns to cause tactile sensations. The aim is to create ‘tactile movies’, i.e. driving signals, from observation of the fingerpad deformations during tactile exploration. This measurement technique may also help improve our understanding of the mechanical behavior of the fingerpad and of the relation between mechanical signals and tactile perception.

This web page provides a summary of the technique as well as a some interesting results. Readers are referred to [1] and [2] for more details.

Skin Strain Measurement Technique

Techniques adapted from the field of online fingerprinting are used to acquire high-contrast fingerprint images and extract salient features resulting from anatomical landmarks as they contact a surface (pores, valley endings, and valley bifurcations). Further processing involving the computation of a triangulation of these features is then used to evaluate skin strain variations over time. Please click on the following links to obtain more information about each aspect of the skin strain measurement technique.

Step 1: Fingerprint Image Acquisition

The experimental platform exploits the principle of frustrated total internal reflection (FTIR) to obtain high-contrast fingerprint images. This technique is often used in biometric fingerprint sensors. A typical FTIR fingerprint sensor uses a light source, a prism and a camera as shown in Figure 1. In the absence of contact, light is reflected at the hypotenuse face of the prism towards the camera. The contact between fingerprint ridges and the surface causes light to be scattered. This results in a high-contrast pattern of black ridges over a white background.

Figure 1. Typical prism-based fingerprint sensor.

The experimental platform is illustrated in Figure 2. Unlike conventional fingerprint sensors, this apparatus includes a removable contact surface that may have simple geometrical features such as bumps and holes. A diffuser is also required at the entry face of the prism to insure that reflections are maintained along a wide range of non-flat surfaces.

Figure 2. Experimental platform.

Geometric distortions are corrected by imaging a precise calibration grid printed on a sheet of transparency film. The pattern resulting from the application of the grid on the contact surface is analyzed to correct the perspective projection, unroll the contact surface, and measure pixel size. The intensity of fingerprint valleys is also normalized to compensate for illumination non-uniformity. Figure 3 shows a typical fingerprint image acquired with this platform.

Figure 3: Typical fingerprint image.

Step 2: Feature Extraction

Online fingerprint recognition generally relies on two types of salient features of the fingerprint called minutiae: ridge endings and ridge bifurcations. Pores, which appear as small openings on the surface of the fingerprint ridges, are also used occasionally. Here we are interested in extracting three types of features: valley endings, valley bifurcations, and pores.

The feature extraction process uses a number of image processing operations to simplify, enhance and analyze the fingerprint images. The full process is illustrated in Figure 1.

Figure 1. Feature extraction block diagram.

Figure 2 illustrates some of the operations performed on a small segment of a fingerprint. The original fingerprint consists of a grayscale image with white pores and valleys. This image is binarized to indicate which pixels belong to valleys and pores. A connected-component analysis then extracts many pores in the image. The binary fingerprint, minus pores, is then thinned to simplify processing. The resulting skeleton is analyzed to locate valley features. Syntactic rules are finally applied to correct some common artifacts. Notice that valley features have an associated direction which can be used for matching.

Figure 2. Feature extraction in fingerprint segment.

Step 3: Skin Strain Measurement

Once the features have been extracted from each image in a sequence of images, an attempt is made to match each feature to a feature from the next frame. A simple matching algorithm is applied to match as many features as possible. Figure 1 illustrates some matching successes and failures from one image to the next. This process, although imperfect, results in a large number of features being tracked for varying periods of time.

Figure 1. Selected matched (full lines) and unmatched features (dashed lines).

Changes in local skin strain are estimated by observing changes in a triangulation of tracked features. The subset of features of a frame that are tracked in the subsequent frame is used to construct a Delaunay triangulation such as shown in Figure 2.

Figure 2. Typical triangulation of tracked features.

The change in local skin strain is evaluated by measuring variations in the triangulation. A first method, illustrated in Figure 3, consists of measuring the relative increase or decrease in triangulation edge lengths. A second method, illustrated in Figure 4, relies on the relative increase or decrease in triangle areas. Each pair of successive images is analyzed, yielding a map of relative changes in skin strain over time. Skin strain measurements are illustrated with color coded triangulations. In both cases, the range of relative variations is given by a bar at the bottom of the image. A maximal increase in edge length or area will be shown in red while a maximal decrease will be shown in blue. Measurements can also be made over a span of more than one frame.

Experimental Results

Experiments were conducted with three types of surfaces: a surface with a bump, a surface with a hole, and a flat surface. All experiments were performed with the same fingertip. Each recording consists of 180 frames at 60 frames/second. Results are shown in the form of mpeg movies at a slowed-down rate of 15 frames/second.

Measurements obtained from movement over flat surfaces are generally difficult to interpret. The following two experiments provide interesting measurements obtained from image sequences for which a meaningful interpretation could be found. In the first example, a fingertip is pressed against the surface and rotated. In the second example, a fingertip is moved back and forth horizontally. Please click on the following links to see the results.

Rotation

In this example a fingertip is pressed firmly against a flat surface and rotated. At frame 70, the fingertip begins a counter-clockwise rotation. Most of the fingertip is sticking to the glass. The top part of the finger is moving up, stretching the intermediate zone between the moving and non-moving segments. The right-hand part is moving toward the upper-left corner, resulting in compression at the junction of the moving and non-moving parts. Changes in triangulation edge lengths and area from frame 70 to 71 (Figure 2(a) and 3(a)), and from frame 70 to 76 (Figure 2(b) and 3(b)) agree with these observations. Notice that the fingerprint seems to be expanding vertically but compressing horizontally.

Lateral Movement

In this example a finger pressed against a flat surface moves back and forth horizontally. A patch of skin is sticking to the glass while the surrounding skin moves with the finger. Notice the patterns of compression and expansion around the stationary patch of skin.

A second set of experiments was conducted with three contact surfaces: a flat surface, a surface with a bump, and a surface with a hole. The fingertip was moved from left to right at an average speed varying from 2.0 to 2.8 mm/s. While no pattern emerges from the flat surface, a tendency of compression can be observed on the left and a tendency of expansion on the right of the bump. The reverse can be observed in the case of a hole. Measuring the changes in edge length or area over a span of 10 frames provides cleaner results. Please click on the following links to see the results.

Flat Surface

In this experiment, a fingertip is moving from left to right over a flat surface. No coherent pattern of deformations is discernible.