2.1.

The Simplified Articulated Body Model

The selection of the human model has to be done according to the kind of information that is to be extracted from the image data, i.e., human body models used for synthesis applications may need to be more accurate and realistic than human body models used in motion capture applications. Our model is used for motion capture purposes,⁹ specifically for 3D interaction in virtual worlds. Therefore, it has to be accurate enough for representing motions by means of body postures but simple enough to make the problem easily solvable and obtain real-time feedback from the application.

As a result, the body model used here is an articulated model with nine joints (Fig. 2), where every joint has three degrees of freedom and the links between the segments are rigid. By using this model, our objective is to estimate the anthropometric measurements of the body while the user is maintaining a predefined posture at the initialization stage. The anthropometric measurements that we wish to estimate are the limbs of the model in Fig. 2 forearms (distance from wrist to elbow), arms (distance from shoulder to elbow), trunk (distance from back to neck), upper back (distance from back to shoulders), and the virtual leg (distance from virtual foot to back).

Fig. 2

Articulated body model.

2.2.

Human Body Shape Decomposition

Prior to locating the body parts it is necessary to separate the human body shape from the background scene. Two methods are used to carry out this task: a chroma-key approach¹² and background subtraction.¹³ It is true that the chroma-key approach¹² ensures real-time processing, but it is also true¹³ that it allows for background subtraction in real time as long as the environment is a controlled one. It is necessary to emphasize that the automatic initialization system will work provided that the background can be deleted, irrespective of the algorithm used.

Once the human shape has been obtained, the silhouette has to be cut into different parts so as to find the joints’ positions. According to human intuition about parts, segmentation occurs at negative minimum curvature (NMC) of the silhouette, leading to Hoffman’s and Richards’s minima rule: “For any silhouette, all negative minima curvature of its bounding curve are boundaries between parts.”¹⁴ Therefore, it is necessary to find the NMC points of the shape. It is possible to compute the shape curvature directly by means of finite differences in the discrete shape. However, this local computation of curvature is not robust on images. Hence, the human body shape is parameterized using a B-spline interpolation.

A $p$-th degree B-spline is a parametric curve, where each component is a linear combination of $p$-th degree basis curves. A set of contour points $\{\mathrm{Qk}\}$, $k=0,\dots n$ is obtained from the image and they are interpolated with a cubic B-spline.¹⁵ Figure 3 shows how the B-spline interpolation smoothes the human silhouette.

Fig. 3

B-spline interpolation, initial posture and obtained cuts.

Using the B-spline shape parameterization, it is possible to analytically compute the partial derivatives up to order 2 to obtain the curvature values along the shape.

However, the minima rule only constrains cuts to pass through the NMC points it does not guide the selection of cuts themselves. On one hand, Singh et al. noted that when the boundary points can be joined in more than one way to decompose a silhouette, human vision prefers the partitioning scheme that uses the shortest cuts.¹⁶ This leads to the short-cut rule, which requires a cut:

On the other hand, if the user’s posture is known, it is possible to predict where the cuts are. In order to easily obtain the main cuts, the user must stand in a predefined posture adequate for finding all the joints of the body model (see Fig. 3). If the user assumes the same posture shown in Fig. 3, facing the stereoscopic system, the system will be able to perform the initialization correctly since it will be able to analyze all the user’s joints adequately.

2.3.

Initialization of the Body Model

Studying the initial posture in Fig. 3, it can be found that negative and positive minimum curvature points lay near the joints that we aim to find. Therefore, according to the short-cut rule and taking into account the user’s initial posture, we propose the next rules to decompose the human shape:

where $x$ and $y$ refer to the horizontal and vertical image coordinates respectively. Figure 3 also shows the human body model that results from the obtained cuts.

Subsequently, the positions of the shoulders and elbows joints are estimated as the cut middle point. Once the joints’ 2D positions in each image of the stereo pair are established, their 3D positions can be estimated using the midpoint triangulation method. With this method, the 3D position is computed projecting each joint’s 2D position onto each image to infinity and computing the 3D coordinates as the nearest point to these two lines.¹⁷

Finally, wrist joints have to be estimated to complete the body model. However, they cannot be detected using the proposed shape analysis method. Due to this, the color cue is used to find the hands on the images.¹⁸ To locate wrists, the hands are approximated by means of ellipses. Using the previously located 3D positions of the elbows, the intersection between a 2D line defined by the elbow and center of the hand positions and the 2D ellipse hand approximation is located in the image.