JPH0620055A - Method and device for picture signal processing - Google Patents

Abstract

PURPOSE:To correctly and quickly detect a human finger used as a direct input interface device by retrieving a deep local minimum corresponding to the front end of the finger in its direction. CONSTITUTION:A video camera 2, an edge detecting unit 4, a Hough transformation calculating unit 6, a possibility retrieval unit 8, and a clustering unit 10 are provided. The human finger is read and is outputted as picture data in a form of two-dimensional pixel data, and the edge of the hand is detected based on each pixel data, The image line of the detected edge is converted into parameters in a parameter space and is suitably subjected to Hough transformation, and edge pixels are integrated in the parameter space to determine the direction of the finger. A deep local minimum corresponding to the front end of the finger is retrieved in this determined direction, and end points prescribing the front end of the finger are clustered to determine the front end of the human finger.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image signal processing method and an image signal processing apparatus, and more particularly to a three-dimensional shape restoring method and apparatus for restoring a three-dimensional shape from two-dimensional image data in real time. The present invention relates to a method for extracting a feature of an object and its apparatus, and a method for detecting a finger of a human hand used as a man-machine interface and its apparatus.

It is desired to image a target object with an image pickup means such as a CCD video camera and restore the three-dimensional shape of the target object from the picked-up image. Such a three-dimensional shape restoration method is considered to be applied to, for example, a visual recognition means for recognizing the movement of a robot hand in an industrial robot and its gripping state. As another application example of the three-dimensional shape restoration method, when a part of a human body as a man-machine interface means in a computer system, for example, an arm, a hand, a palm, or a finger is used, one of those bodies is used. For example, it may be applied to a recognition unit when the three-dimensional shape of a part is restored.

The most common human input interface to a computer system is some type of transducer or actuator operated by the user, such as a CRT keyboard, joystick, trackball or mouse.
Human arm, palm, and finger movements as target objects are man-machine interface devices, such as a mouse, in computer systems currently in use.
Direct and effective man-machine interface rather than indirect man-machine interface device such as Joystick
It is known to function as an interface device, and there is a demand for a real-time practical three-dimensional shape restoration method of a moving target object such as a part of the human body, that is, a deformable target object. Has been done.

Conventionally, in order to restore the three-dimensional shape of the target object from the two-dimensional image information of the target object having a three-dimensional shape, the vertex and edge components of the target object are detected, and based on these information. A method of reconstructing the shape of the target object so that there is no contradiction in a three-dimensional space is used.

[0005]

However, all prior art methods in which these interface devices are used require that the interface device be in physical contact with the user by the user, and such input There is a problem that it usually takes a long training period to become familiar with the operation of the device by the user. The conventional method also has a problem that a range in which information is transmitted is narrow. For example,
The mouse simply sends the position and movement vector to the central processing unit of the computer system.

In the conventional three-dimensional shape restoration method, when the number of feature information such as edges is small, the feature information cannot be clearly obtained, or when erroneous information is mixed in the feature information, the three-dimensional shape is restored. I am running into the problem of becoming difficult. It is necessary to consider all possible combinations of shapes when trying to deal with error information of feature information, and to search and check whether each input information is consistent with each other. However, the calculation time is very long, and it is required, for example, when it is applied to real-time processing for industrial robots or when a part of the human body is used as a man-machine interface means. It is not possible to satisfy the requirement for real-time realization of a three-dimensional shape. In particular, the two-dimensional image data obtained by capturing an image of a target object having a three-dimensional shape with an imaging device such as a video camera does not have depth information, and thus encounters difficulty in restoring the three-dimensional shape.

A first object of the present invention is to provide a method and apparatus for correctly and rapidly detecting a finger of a human hand used as a direct input interface device. A second object of the present invention is to provide a three-dimensional shape restoration method and apparatus capable of picking up a moving target object as two-dimensional image data and restoring the three-dimensional shape of the target object in real time. To aim. A third object of the present invention is to extend the method for detecting a finger of a human hand to a general target object and apply it to the feature point extraction in the above-mentioned three-dimensional shape restoration method to suitably restore the three-dimensional shape. A method and apparatus therefor. The fourth object of the present invention is to
When the target object is a human hand, the image signal processing method is applied to the above three-dimensional shape restoration method so that the human hand can be applied as an effective man-machine interface means.

[0008]

In order to solve the above problems, according to a first aspect of the present invention, there is provided a method and apparatus for detecting a finger of a human hand. A method of detecting a finger of a human hand includes a step of reading a human hand and outputting as image data in a two-dimensional pixel data form, a step of detecting an edge of the hand based on the pixel data,
The step of parameter-converting the image line of the detected edge (the line of the image) into a parameter space and integrating the edge pixels in the parameter space to determine the direction of the finger, corresponding to the tip of the finger along the direction. There is a step of searching a deep local minimum and a step of clustering the end points that define the tip of the finger.

Preferably, the step of detecting the edge is
Calculating a two-dimensional slope of the pixel data, each of which indicates a gray level, comparing the absolute value of the calculated slope with a threshold level, and calculating the absolute value with the threshold value. If it is larger than the level, it has a step of determining the edge. Also preferably, the parameter conversion and edge pixel integration step is a step of converting a linear coordinate system showing the detected edge, for example, a three-dimensional orthogonal coordinate system to a polar coordinate system, and integrating edge points in the parameter space. And the step of determining the orientation of the finger of the hand by obtaining the largest integrated value for each of the image lines. Preferably,
The searching step includes the step of determining the deep local minimum by a random direction determination mechanism and avoiding the shallow local minimum. Preferably, the clustering step comprises selecting an arbitrary position within the cluster, allocating the end point pixel of each search to the cluster position based on at least the Euclidean distance, and
There is the step of determining a new cluster position by calculating the centers of all the pixels assigned to the cluster.

An apparatus for detecting a finger of a human hand is configured to implement the method for detecting a finger of the human hand.

Further, according to a second aspect of the present invention, there is provided a method and apparatus for extracting a feature of a target object. The method of extracting the feature of the target object is to image the target object,
Outputting as image data in the form of three-dimensional pixel data, detecting edges of the target object based on the pixel data, converting the image lines of the detected edges into a parameter space, and converting the edge pixels into the parameter space. To determine the orientation of the target object, search for a deep local minimum corresponding to the tip of the target object along the orientation, and cluster the endpoints that define the tip of the target object. Have. The method of extracting the characteristics of the target object is a generalized version of the above-described method of detecting the finger of the human hand. The apparatus for extracting the feature of the target object is configured to execute the method for extracting the feature of the target object.

According to a third aspect of the present invention, a three-dimensional shape restoration method and apparatus are provided. A three-dimensional shape restoration method includes a step of inputting two-dimensional image data obtained by imaging a movement of a target object represented by a mechanical equation having a spring connected via a node and representing a mechanical constraint by a spring model, and
Feature extraction step of extracting features from three-dimensional image data to generate feature extraction data, and applying force from the two-dimensional coordinates of the feature extraction points to the dynamic model of the target object to deform the target object model and depth information. And restoring the three-dimensional shape of the target object. Preferably, in the step of restoring the three-dimensional shape, the target object is divided into partial regions with less deformation, and the three-dimensional shape restoration calculation is performed for each mode for each partial region.

As the feature extraction, the method of extracting the feature of the target object described above can be applied. The three-dimensional shape restoration device is configured to carry out the above three-dimensional shape restoration method. The preferred target object is the human hand.

[0014]

According to the method of detecting the finger of the human hand, the human hand is read and output as image data in the form of two-dimensional pixel data, and the edge of the hand is detected based on these pixel data. The image line of the detected edge is parameter-transformed into a parameter space, preferably Hough-transformed, and the edge pixels are integrated in the parameter space to determine the direction of the finger. A deep local minimum corresponding to the tip of the finger is searched for along the determined orientation, and the end points that define the tip of the finger are clustered. The tip of the human hand is determined based on this clustering result.

The method for detecting the finger of the human hand can be extended to the feature extraction of the target object by using the feature of the target object instead of the tip of the finger.

In the three-dimensional shape restoring method, the mechanical constraint of the target object is represented by a spring model, and the three-dimensional shape of the target object is solved by solving the dynamic equation of the target object from the feature points of the two-dimensional image of the target object. To restore. In particular, the depth information can be supplemented by applying a force from the two-dimensional coordinates of the feature extraction points to the model of the target object to deform the target object model, and the three-dimensional shape of the target object can be accurately calculated in a short time. Can be restored to. By dividing the target object into sub-regions with less deformation and performing the above-mentioned three-dimensional shape restoration calculation for each mode for each partial region, the three-dimensional shape restoration processing can be performed at a higher speed.

[0017]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A first embodiment of an image signal processing method and an image signal processing apparatus according to the present invention will be described with reference to the accompanying drawings, a method and apparatus for detecting a finger of a human hand. FIG. 1 is a block diagram of an apparatus for detecting a finger of a human hand according to the present invention. Figure 2 is to determine the orientation of the human hand,
FIG. 7 is an enlarged view of a hand for explaining the relationship between the direction of the finger when searching the tip of the finger (search) and the search start line (start line). FIG. 3 is a diagram showing a direction defining mechanism for discovering a deep local minimum. Figure 4
FIG. 3 is a characteristic diagram showing a relationship between a uniform probability distribution function PDF and a transfer function.

The tip of the finger directly designates and positions an item displayed on the display unit,
Alternatively, it can be used to perform any one operation.
The apparatus for detecting a finger of a human hand shown in FIG. 1 includes a video camera 2 as a standard image pickup means that is normally used, an edge detection unit 4, and a Hough transform ratio.
n) Computation unit 6, probabilistic search unit 8 and clustering unit 10. The video camera 2 images the human hand 12 and outputs two-dimensional image data, and the clustering unit 10 outputs the position of the tip of the finger. Edge detection unit 4, Hough transform calculation unit 6, possibility search unit 8, and
The clustering unit 10 is realized by a digital computer system.

The processing contents of each unit of the apparatus for detecting a finger of a human hand realized by a digital computer system are shown in the flow charts of FIGS. Figure 5
3 is a flowchart showing image input processing by the video camera 2 and edge detection processing by the edge detection unit 4. 6 and 7 are flowcharts showing the Hough transform process in the Hough transform calculation unit 6. 8 and 9 are flowcharts showing the possibility search processing in the possibility search unit 8. FIG. 10 is a flowchart showing the clustering process in the clustering unit 10. Tables 1 to 4 show a part of program codes of the processing in the digital computer system. Although these program codes are processed according to a series of flows, they are divided and shown for the sake of illustration.

[Table 1]

[Table 2]

[Table 3]

[Table 4] Table 1 is a program code showing the initialization process of the accumulator used in the Hough conversion process. Table 2 is a program code showing a process of searching an image for an edge. Table 3 is a program code for searching the 10 best lines in the Hough transform process. Table 4 is a program code that outputs the positions r _m and t _{m at} which the value of the accumulator is the maximum as a possible candidate for the image line in the possibility search process. Table 1
The program code shown in Table 4 shows a part of processing by the digital computer system, and does not necessarily correspond to the processing contents of FIGS. 5 to 10 one to one.

As shown in step S01 of FIG. 5, the video camera 2 having a charge couple device (CCD) reads a hand, digitizes the image of the hand,
Two-dimensional image data of the hand, that is, edge pixel (image) data I (i, j) (where I (i, j) is the gray level intensity at the image position (i, j))
Is output to the edge detection unit 4.

As shown in steps S02 to S010 of FIG. 5, the edge detection unit 4 detects the edge of the hand by the edge detection algorithm based on the inclination method. That is, the edge detection unit 4 calculates the gradient gradI (i, j) of the image based on the following equation 1. Grad is also expressed as an inverted triangle symbol, nabla.

[Equation 1] The edge detection unit 4 determines the edge of the human hand by the following method. If the gradient of the pixel gradI (i, j)
If the absolute value of is greater than a predetermined threshold T, then that pixel (i, j) is an edge pixel. That is, the edge is defined when the differential value of the pixel exceeds the threshold value. If the edge detection unit 4 determines that the pixel is an edge pixel, the edge pixel I (i, i,
j) = 1 and otherwise I (i, j) = 0. The edge detection unit 4 performs this edge detection processing on all image coordinates (i = 1 to imax, j = 1 to jmax).
Do about.

As shown in FIGS. 6 and 7, the Hough transform calculation unit 6 initializes the accumulator (see FIG. 6).
Step S21, the program code shown in Table 1), the accumulator is integrated when the pixel is above the label (steps S22 to S32, Table 2), and the 10 best lines (best lines) are extracted from the value of this accumulator. (Step S33 of FIG. 7, program code of Table 3). As shown in FIG. 2, a human hand is composed of approximately 10 lines that are substantially parallel to each other. By sloping these lines and averaging the paths in the Y direction (step S34 in FIG. 7). , Finally, it is possible to obtain a directional line indicating the direction of the hand that runs in parallel with the finger by dividing the hand into two minutes. In the Hough transform method, all possible image lines are parametrically transformed into a parameter space, and the accumulator in the parameter space maintains the path of all edge pixels that are part of each line, The line with the larger pixels is selected as the image line. The details of the Hough transform process will be described below.

An arbitrary line in the Euclidean two-dimensional space is defined by the following equation 2.

[Equation 2] Here, X and Y are coordinates of an arbitrary point on a predetermined line in a linear coordinate system, for example, a three-dimensional orthogonal coordinate system, and θ and ρ are positions on the line closest to the origin in the polar coordinate system. Stipulate. In this implementation, all possible lines that can pass through the image space are ρ _min ˜ρ _max
And vector-quantized to values of ρ and θ in the range of θ _min to θ _max . Each edge position x, y in an image
For, all possible lines are determined by varying θ, and ρ is determined based on the resulting x, y. For each determined line, the value of the accumulator for that line is increased (steps S25-S26 in FIG. 6, program code in Table 2).
After the processing for all edge points is completed, the accumulator is searched and the 10 best lines associated with the accumulator having the largest value are selected (see FIG. 7).
Step S33, the program code of Table 3).

The following equations 3 and 4 are used to convert the form showing the line into the format of y = mx + b.

[Equation 3]

[Equation 4] Moments of inertia m and b are determined for all recommended (candidate) lines, and then the average of m and the average of b are calculated, which are the pointing lines indicating the hand orientation shown in FIG.

Flowcharts of the possibility retrieval processing performed by the possibility retrieval unit 8 are shown in FIGS. 8 and 9. FIG. 8, steps S41 to S48: Possibility search unit 8
Selects the line that is orthogonal to the pointing line and has the largest moment of inertia to determine the pointing line of the hand. FIG. 9, Steps S49 to S55: Once the hand pointing line is determined, the possibility search unit 8 performs search SEARCH a plurality of times on all pixels to find a “deep local minimum”. This deep local minimum corresponds to the tip of your finger. The start position of these searches is along a line orthogonal to the directivity line indicating the direction of the finger. The crossing (orthogonal) position of the starting line and the pointing line is determined by detecting the largest amount of position with respect to the pointing line, which corresponds to the thick part of the palm just below the thumb.

Each search shows a tendency towards the direction of the fingers of the hand. Search cannot cross edge regions. To avoid the shallow local minimum, the search is provided by a random directional decision mechanism biased towards the tip of the finger. Each search has a certain amount of program processing steps for a pixel, allowing a total of 400 steps, as shown in Tables 1-4. Due to the random nature of the routing mechanism, shallow local minimums can be easily avoided, but the tip of the finger appears to be a trap.

The directionality determining mechanism will be described with reference to FIGS. 3 and 4 in addition to FIGS. For each given program processing step, the angle α is determined for a given angle by a random number generator having a given probability distribution, and the next upper, lower, left or right pixel to be entered is determined. If the next pixel is an edge pixel, the search remains at the initial pixel location during that step.

The random number generator generates a random number having a uniform probability distribution in the range of 0 to 1 (step S4 in FIG. 8).
5). As shown in FIG. 4, in order to convert this probability distribution into a desired probability distribution function PDF, a cumulative distribution function CDF regarding the desired PDF is determined. If ux is a uniform random variable and F (x) is the CDF of the desired PDF, then dx = F ⁻¹ (ux) is the random variable for the desired PDF. dx is multiplied by 2π, a new angle is obtained based on FIG. 3, and a new pixel is calculated based on FIG. 4 (steps S46 to S48). Probability distribution function P
The DF is selected so as to emphasize the direction of the finger direction. In this case, a simple probability distribution was attempted as shown in FIG. There is a finite number of probability distributions, and the selection of that probability distribution improves or degrades the search performance.

After all the searches are completed, a clustering algorithm such as the "K-Means" algorithm is used in the clustering unit 10, and there are five clusters corresponding to the tips of the five fingers. The location of the search element is clustered based on the fact that These cluster positions determine the position of the tip of the finger. The cluster means a group of image processing data.

The implementation of the "K means" algorithm will be described with reference to the flow chart shown in FIG. Step S61: Arbitrarily select the initial five cluster positions (group) z1, z2, z3, z4, z5. Here, zi is the position of the i-th cluster. Steps S62 and S63: The pixels at the end of each search are allocated to the cluster positions as the ones at the shortest distance at the same cluster position based on at least the Euclidean distance. Steps S64-S66: Determine a new cluster position by calculating the centers of all of the pixels assigned to the cluster. Step S67: If a cluster position changes,
It returns to step S63. Step S68: A trial image of the hand is used and the algorithm allows the position of the tip of the finger to be determined.

As described above, different probability distribution function PDFs
The use of will dramatically change the performance of the search, improve (reduce) operation time, and allow appropriately sized program processing steps to implement the use of scale space techniques. Although the effect of noise on the Hough transform is relatively small, background edges should be removed by signal preprocessing. If S / N ratio is high, the probability distribution function number be searched P
The DF should be modified to allow a greater degree of backtracking. The advantage of this type of interface is that the user does not need the elaborate manipulations required to easily use the manual input device. Moreover, the amount of information transmitted by the movement of the hands is so large that this kind of interface can be used in a wide range of applications.

Since the above-described method and apparatus according to the present invention is not limited to the size and shape of the hand, anyone can easily operate the method and apparatus. As described above, according to the first embodiment of the image signal processing method of the present invention, a method for easily and correctly detecting a finger of a human hand used as a direct input interface device at high speed. And a device therefor are provided.

The method for detecting the finger of the human hand described above is
As a specific example, the finger of the hand that is the source of the target object is illustrated, but the method of detecting the finger of the human hand described above is applied to a more general target object, for example, the hand of an industrial robot. You can

As a second embodiment of the image signal processing method and its apparatus of the present invention, a three-dimensional shape restoring method and apparatus for performing three-dimensional shape restoration from two-dimensional image data will be described. FIG. 11 is a block diagram of a three-dimensional shape restoration device.
The three-dimensional shape restoration device includes a two-dimensional image data input means 10
1. It has a feature extraction means 103 and a three-dimensional shape restoration processing means 105. Two-dimensional image data input means 101
Can be realized by, for example, a CCD video camera, and the feature extraction means 103 and the three-dimensional shape restoration processing means 105 can be realized by a computer system.

FIG. 12 is a flow chart showing the basic operation of each part of the three-dimensional shape restoration device shown in FIG. Step S101: Two-dimensional image data input means 101
Is a CCD video camera that images a target object in a two-dimensional state, and the two-dimensional image data of the target object imaged by this video camera is applied to the feature extraction means 103. Step S102: The feature extraction means 103 performs feature extraction from the two-dimensional image data input from the video camera,
The two-dimensional coordinates of the feature points are output. Various conventionally known methods can be applied as the feature extraction method. As such a feature extraction method, for example, a method of detecting the coordinates of a feature point by detecting the color of a marker attached to the feature point of the target object in advance and detecting the edge of the target object There is a method of detecting the coordinates of the characteristic point by discriminating the characteristic shape, and any one of these is also used in the characteristic extracting means 103 of the present invention. Preferably, the method for extracting the features of the target object is applied to this three-dimensional shape restoration device. This will be described later.

The two-dimensional coordinates of the feature points extracted by the feature extraction means 103 are applied to the three-dimensional shape restoration processing means 105. Step S103: In the three-dimensional shape restoration processing means 105, using the two-dimensional coordinates of the feature extraction points from the feature extraction means 103, the dynamic equation (dynamic model) of the target object.
Based on, the three-dimensional shape of the target object is restored. As described above, in the image processing apparatus of the present invention, the mechanical constraint of the target object is represented by a spring model, and the dynamic equation of the target object is solved from the feature points of the two-dimensional image of the target object to obtain the three-dimensional image of the target object. Restore the shape.

FIG. 13 is a more detailed processing content of the processing operation of the above-described three-dimensional shape restoring apparatus of the present invention, in particular, an operation flowchart of the first form of the three-dimensional shape restoring processing method in the three-dimensional shape restoring processing means 105. Is. FIG. 14 is a diagram showing image data of human palms and fingers as man-machine interface means in a computer system and a feature extraction result thereof as a specific example of the target object which is the target of the three-dimensional shape restoration processing in FIG. is there. FIG.
The operation contents of steps S111 and S112 in step S101 are the same as steps S101 and S102 described with reference to FIG.
The operation is similar to. The operation shown in steps S113 to S116 shows the detailed operation in the three-dimensional shape restoration processing unit 105.

The contents of the three-dimensional shape restoration processing after the processing operation shown in step S113 shown in FIG. 13 will be described.
First, we describe the general theory of dynamical equations for a target object. Generally, the dynamic equation of the target object is expressed by the following equation 5.

[Equation 5] However, the symbol F is a force, the symbol M is a mass, the symbol C is a damping (friction) coefficient, the symbol K is stiffness (stiffness), and the symbol U is a node position.

The mark (.) Attached above the node position U indicates the first derivative, and the mark (.) Indicates the second derivative.
However, in the description of this specification, the first derivative of the node position U is U due to the limitation of the notation method in the electronic filing system.
^{(.) And} the second derivative are shown as U ^(..) . The same applies to other cases. The node position U is a vector U = [Ux, U in which X, Y, Z components in the three-dimensional coordinate system are arranged for each node.
y, Uz], and if the number of nodes is n, it has 3n components as shown in Equation 6 below.

[Equation 6] However, in the description of the present specification, the notation of the vector U is not the usual notation such as a bold line or an arrow with a head because of the limitation of the notation in the electronic filing system. The same applies to other cases.

A virtual mass M is given to the node position U, and a friction (damping) coefficient C is given to stabilize the system. For example, as illustrated in FIG. 14, a joint portion such as a palm and a finger is shown as a node position U in a three-dimensional space, and a finger bone portion is shown as a spring K. The mass M, damping coefficient C, and stiffness K are
It is represented by a 3nx3n matrix. These 3nx3
Equations 7 to 9 below show one M1 of the mass matrix M, one C1 of the damping coefficient matrix C, and one K1 of the stiffness matrix K represented by the matrix of n.

[Equation 7]

[Equation 8]

[Equation 9]

In this embodiment, the force F shown in the above equation 1 is used.
Then, the force vector F _i and the force vector F _k are introduced.
The non-linear internal force vector F _k is a quantity defined by the relative position of the node and shows a non-linear value. This non-linear internal force vector F _k expresses a habit of a non-linear movement (deformation) that is not represented only by the spring model of the target object, and can be said to be “knowledge of how to deform”. The force vector F _i is 2 extracted by the feature extraction means 103.
An external force vector F _i obtained from the dimensional feature extraction coordinates is shown.
When Equation 5 is transformed using the nonlinear internal force vector F _k and the external force vector F _i obtained from the two-dimensional feature extraction coordinates, Equation 10 is transformed.
Is obtained.

[Equation 10]

The operation contents of the three-dimensional shape restoration processing means 105 will be described with reference to FIG. Step S113: The three-dimensional shape restoration processing means 105 inputs the two-dimensional feature extraction coordinates extracted by the feature extraction means 103. Step S114: The three-dimensional shape restoration processing means 105 is in the X and Y coordinates acting between the input two-dimensional feature extraction coordinates and the corresponding point in the dynamic model of the target object, that is, the two-dimensional space in the two-dimensional space. The external force vector F _i obtained from the feature extraction coordinates is calculated and determined. The external force vector F _i obtained from the two-dimensional feature extraction coordinates means a spring-like force placed on a plane parallel to the X and Y planes, which generally has no depth component, and each feature A node corresponding to a point is expressed as a model in which a spring is placed between the nodes. Step S115: In this embodiment, the model of the target object is deformed by adding the external force vector F _i obtained from the two-dimensional feature extraction coordinates, and the dynamic equation shown in Expression 10 is solved.
Normally, when solving the dynamic equation of Expression 10, convergence calculation is performed by well-known numerical calculation using a computer. Step S116: When solving the equation 10 by the convergence calculation, the convergence calculation is aborted under an appropriate condition in consideration of the number of iterations of the calculation, the deviation of the node position U from the previous result, and the obtained node position U is output. To do. Step S117: In the case of assuming online real-time processing, for example, moving image data is input to the feature extraction unit 103 by continuous video frames from the video camera as the two-dimensional image data input unit 101, and the feature extraction is performed, three-dimensional 3 at that time in the shape restoration processing means 105
When the three-dimensional shape is restored, the above processing is repeated. As a result, the node position U is continuously output from the three-dimensional shape restoration processing means 105 in correspondence with continuous video frames.

As shown in FIG. 14, when a target object captured as a moving image such as a human palm and a finger is handled, the dynamics shown in Equation 10 as a differential equation in each video frame in the image signal processing. When solving the equation, since the node position U obtained in the previous video frame can be used as the initial value of the node position U, when the node position does not change much between the preceding and following video frames,
The convergence calculation of Expression 10 can be speeded up. Equation 10
Even if there are multiple solutions of the dynamic equation shown in, the solution that is close to the solution of the previous frame is selected, so the correct solution can be obtained. Even if some feature points are hidden and the feature extraction information in the feature extraction means 3 is missing, or erroneous information is mixed in due to noise or the like, this dynamic system has a stable energy state of the dynamic model. There is an advantage that the most plausible solution is output because it tries to reach the place.

FIG. 14 shows the result of reconstructing the three-dimensional shape by the three-dimensional shape reconstructing processing means 105 from the two-dimensional feature extraction points extracted by the feature extracting means 3 by imaging the human palm and finger with a video camera. It is a figure which shows schematically. For the feature extraction by the feature extracting means 103, for example, a red mark is attached to the bottom of the palm and the tip of the fingertip (nail).

As a first embodiment of the three-dimensional shape restoration method and its apparatus, as described above, the target object is represented by a mechanical model, and the target object is imaged by the two-dimensional image data input means 101 such as a video camera. , The two-dimensional image data is feature-extracted by the feature-extracting means 103 and the feature-extraction points are set to three
It is input to the three-dimensional shape restoration processing unit 105, and the two-dimensional feature information input is applied to the model by exerting a force on the calculation in the three-dimensional shape restoration processing unit 105. as a result,
Even if the amount of characteristic information is small, if the system of the target object is stable, its three-dimensional shape can be restored. In addition, even if the characteristic information cannot be obtained clearly or incorrect information is mixed in, the influence of incorrect information is automatically reduced by the constraint from other correct characteristic information or the dynamic model, and a consistent solution can be obtained. In principle, this is a restoration method that does not use a time-consuming search operation. As a result, it does not depend on the enormous amount of combinatorial calculation, which is a problem in the prior art. Furthermore, according to the present embodiment, it is possible to reproduce the depth information with only one video camera without using a plurality of video cameras.

A second embodiment of the three-dimensional shape restoration method and apparatus according to the present invention will be described. The second embodiment is the first described above.
This further shortens the calculation time in the embodiment. In the first embodiment, it is necessary to solve the dynamic equations (differential equations) simultaneously with three times the number of nodes used in the dynamic model in the three-dimensional shape restoration, and there is a drawback that the calculation amount is still large. As a method of solving such a problem, it is possible to apply a well-known method in which a spring matrix in a dynamic model is eigenvalue decomposed, the vibration modes are independently calculated, and a solution is obtained without solving simultaneous equations. However, in this known method, when the target object is largely deformed (moved), the spring matrix is also greatly changed, and it becomes necessary to perform eigenvalue decomposition sequentially. This calculation amount cannot be neglected, and as a result, it has been found that it is not preferable to apply it to a deformable target object that is relatively moving, such as a human palm and finger, or an industrial robot hand.

From this point of view, in the second embodiment, by dividing the three-dimensional dynamic model of the target object into sub-regions with less deformation, the calculation for each mode becomes possible and the calculation time is shortened. I am trying. Figure 1
FIG. 5 is a diagram showing that a part of the target object applied to the second example is divided (divided) into a plurality of divided areas.

FIG. 16 shows a flowchart of the operation processing of the apparatus of the second embodiment for three-dimensional shape restoration. Also in the second embodiment, the configuration of the three-dimensional shape restoration device is the same as that shown in FIG. Two-dimensional image data input means 10
1 (step S121) and the feature extraction means 103 feature extraction process (step S12).
2) is similar to the processing shown in FIG. Steps S123 to S128 in the three-dimensional shape restoration processing unit 105
The processing content indicated by is different from the first example.

The three-dimensional shape restoring operation of the second embodiment will be described below. The three-dimensional shape restoration processing means 105 basically solves the dynamic equation shown in Expression 5. F is a force vector added to the model of the target object, and is a force vector including the nonlinear internal force vector F _k and the external force vector F _i obtained from the two-dimensional feature extraction coordinates, as described above. Therefore, Equation 10 is solved also in this embodiment. However, the solution is different from that of the first embodiment, as described above. For example, as shown in FIG. 15, the target object is a leg portion whose shape does not change in the human body, hands, etc.
It can be considered that partial regions such as arms and fingers are connected by joints. When such an assumption is made, each divided area in the spring matrix K is represented by m independent block matrices K1, K2, ..., Km, and the spring coupling force corresponding to the joint connecting these block matrices is expressed. Fj
Is transferred to the force vector F on the right side of Expression 1.

The term (KU) in Equation 5 is expressed as shown in Equation 11.

[Equation 11] However, Kb in Expression 11 is represented by the block matrix in Expression 12.

[Equation 12] As a result, Expression 5 is expressed by Expression 13.

[Equation 13]

The eigenvalue decomposition is carried out with respect to the equation 13, and the mass matrix M, the damping coefficient matrix C, and the stiffness matrix Kb are respectively M by known methods.
^{If the} matrix P is obtained so as to diagonalize ^(<) , C ^(<) , and Kb ^(<) , Equation 13 can be rewritten as Equation 14. Incidentally, the notation limitations in electronic filing system is herein denotes chevron representation on the M indicating the mass matrix M which diagonalized as M in ^{(<) (<).} The same applies to the other cases.

[Equation 14] Here, M ^(<) , C ^(<) , Kb ^(<) , U ^(<) , F ^(<) and F _j ^(<) are represented by the following formulas.

[Equation 15] However, P ^T represents a transposed matrix of the matrix P.

When the target object is not deformed so much, the stiffness matrix K does not change much and can be diagonalized by the same matrix P. However, when the target object changes greatly with time, the eigenvalue decomposition is re-executed again. It is necessary to find a new matrix P. Therefore, assuming that the block matrix Ki in each divided region is constant and rotates by Ai, and the mass matrix M and the damping coefficient matrix C are constant matrices, it can be expressed by Expression 16.

[Equation 16] The stiffness matrix Kb after rotational deformation is expressed by Equation 17.

[Equation 17] The mechanical equation after the deformation can be expressed as Equation 18.

[Equation 18]

Since the rotation matrix A is a unitary matrix, equation 18 can be rewritten as equation 19.

[Formula 19] Since the mass matrix M and the damping coefficient matrix C are constant matrices, Equation 19 can be expressed as Equation 20.

[Equation 20] However, U ^{(<) '} , F ^(<) ' and F _j ^{(<) '} are expressed by the following formula 21.
It is represented by.

[Equation 21] Since the equation 20 is diagonalized, it is no longer a simultaneous equation, and the node position U for the target object with deformation can be obtained at high speed without performing convergence calculation.

The processing operation of steps S123 to S128 in FIG. 16 is the three-dimensional shape restoration processing means 105.
The above-mentioned arithmetic processing in FIG. Step S123: The three-dimensional shape restoration processing unit 105 inputs the feature extraction coordinates from the feature extraction unit 103. Step S124: The three-dimensional shape restoration processing means 105 obtains the unitary matrix A shown in Expression 16. Step S125: The three-dimensional shape restoration processing means 105 obtains a force vector F such as a force acting from the feature point and a force Fj connecting the divided areas. Step S126: The three-dimensional shape restoration processing means 105 performs the conversion processing using the transposed matrix P ^T shown in Expression 21. Step S127: The three-dimensional shape restoration processing means 105 solves the dynamic equation of Expression 20. It should be noted that, since the dynamic equation of the equation 20 is diagonalized, it is no longer a simultaneous equation and can be solved at high speed without performing convergent calculation. Step S128: The three-dimensional shape restoration processing means 105 transforms the node position U from the obtained U ^(<) and outputs the obtained node position U. Step S129: Repeat the above operation for the next frame.

As described above, in the second embodiment for three-dimensional shape restoration, the stiffness matrix Kb, which is created by dividing the target object into parts that are not so deformed, is eigenvalue decomposed in advance to obtain the eigenmatrix P. Then, F ^(<) 'and F _j ^(<)' are calculated by using the rotation matrix A in which the deviation of each divided area from the initial state is calculated, and the node position U'is obtained by solving Expression 20. ， This node position U '
And the eigen matrix P and the rotation matrix A to the node position U
Ask for. The force vectors F and Fj and the node position U are F ^(<) and F with the eigenmatrix P and the appraisal matrix A, respectively.
_Although it needs to be converted into _j ^(<) and U ^(<) , the amount of calculation is smaller than that in the method of solving the simultaneous equations represented by Expression 5 as they are, and high-speed processing is possible. That is, in the second embodiment of the three-dimensional shape restoration, 3
The dimensional model is divided into sub-regions with few deformations, and it is possible to calculate for each mode assuming that the spring matrix does not change in each divided region. As a result, the three-dimensional shape restoration calculation time is shortened.

When carrying out the three-dimensional shape restoration device of the present invention, other processing than the operation processing of the feature extraction means 103 and the three-dimensional shape restoration processing means 105 described above can be applied. For example, in the above-described embodiment, the image pickup data in the two-dimensional image data input means 101 is illustrated for the two-dimensional information. However, even when the distance image data is input, the same processing as in the above-described embodiment can be performed.

FIG. 17 shows, as a specific application example of the three-dimensional shape restoration apparatus of the present invention, a three-dimensional shape restoration assuming that a human palm and a finger as man-machine interface means are used as target objects. It is a flowchart which shows a processing operation. FIG. 18 is a diagram showing a result of previously analyzing various aspects of the palm and the finger when performing the three-dimensional shape restoration process shown in FIG. FIG. 18A shows the result of feature extraction from the imaged data when the palm and fingers are opened. The left side is a surface feature extraction diagram, and the right side is a side feature extraction diagram. Hereinafter, a front view and a side view when the little finger, the ring finger, the middle finger, and the index finger are bent in order from FIG. 18 (B) to FIG. 18 (D) are shown.

The operation processing of the three-dimensional shape restoration processing means 105 will be described with reference to the flow chart shown in FIG. Step S131: The three-dimensional shape restoration processing unit 105 inputs the current feature point from the feature extraction unit 103. Step S132: The three-dimensional shape restoration processing means 105 calculates the force vector in the X and Y planes between the image feature point and the corresponding point. Step S133: The three-dimensional shape restoration processing means 105 determines the X and Y spring forces F _i and adds them to the appropriate force matrix. Step S134: The three-dimensional shape restoration processing means 105 determines the coupling force between the segments and adds it to the appropriate force matrix. Step S135: When the model of the hand is unnatural, the “power of hand knowledge” is added to the appropriate force matrix in the three-dimensional shape restoration processing unit 105. Step S136: The three-dimensional shape restoration processing means 5 calculates the matrix P and the rotation matrix A for each segment. Step S137: The three-dimensional shape restoration processing means 105 uses the matrix P and the rotation matrix A for each segment to convert the dynamic equation into the closed-form dynamic equation shown in Equation 20. Step S138: The three-dimensional shape restoration processing unit 105 solves a simple linear difference equation by using the differential (difference) time δt for each Ui ^(< . Step S139: The three-dimensional shape restoration processing unit 105 causes the node position matrix U ^{(< )} To the node position matrix U
Is determined and each joint position of the finger is updated. Step S140: The three-dimensional shape restoration processing unit 105 repeats the above processing until the hand model and the image match.

19 and 20 are diagrams showing the results of the above-mentioned arithmetic processing. FIG. 19A shows an image pickup result by the two-dimensional image data input unit 101 in a state where a finger is slightly bent as the shape of a hand as a target object. 19B to 19D are three-dimensional shape reconstructions of the imaging result shown in FIG. 19A through the feature extraction means 103 and the three-dimensional shape reconstruction processing means 105, respectively, as viewed from the top and the front. A figure and a side view are shown. FIG. 20 (A) shows a shape of a hand as a target object, in which the middle finger is bent and the tip of the forefinger is bent.
It is an image pickup result by the two-dimensional image data input means 101 when the ring finger is slightly bent and the tip of the little finger is slightly bent. Figure 20
20B to 20D show the feature extraction means 103 and the three-dimensional shape restoration processing means 1 based on the imaging result shown in FIG.
The figure, the front view, and the side view which looked at each three-dimensional shape restored through 05 are shown. As is clear from the illustrations of FIGS. 19 and 20, first, a video camera or the like 2
It can be seen that the three-dimensional shape is restored from the two-dimensional image data captured by the three-dimensional image data input means 101. Figure 1
9 (B) to FIG. 19 (D) and FIG. 20 (B) to FIG. 20 (D)
And are obviously different as the restored data, and human hands can be used as the man-machine interface means by utilizing this difference in the movement of the hands.

The three-dimensional shape restoration device of the present invention is not limited to human hands, but can be used as a target object for three-dimensional shape restoration in recognition of, for example, the hand of an industrial robot and its gripping state.

Since the feature extracting means 3 and the three-dimensional shape restoring processing means 105 shown in FIG. 11 are functionally different from each other, they are shown separately. However, in general, the feature extracting means 103 and the three-dimensional shape restoring processing are shown. Since the means 105 is realized by using a computer, it can be integrated in the same computer. Of course, the feature extraction means 103 and the three-dimensional shape restoration processing means 105 can be distributed and processed by different microcomputers.

As described above, according to the three-dimensional shape restoration method and apparatus of the present invention, the three-dimensional shape is restored from the two-dimensional image data of the target object by applying the deforming force to the dynamic equation. You can Further, according to the three-dimensional shape restoration method and the apparatus thereof of the present invention, by dividing and processing a portion of the target object which is little or not deformed, the three-dimensional image data of the target object from the two-dimensional image data can be processed at high speed. The shape can be restored.

FIG. 21 is a block diagram of a three-dimensional shape restoration device of the present invention. This three-dimensional shape restoring device has the feature extracting means 103 incorporated into the three-dimensional shape restoring device shown in FIG. 11 as a feature point extracting means 103A as a device for detecting a finger of a human hand shown in FIG. Is. This three-dimensional shape restoration device is a device suitable for extracting feature points of a human hand using the feature extraction means 103A and restoring the three-dimensional shape of a human hand using these feature points. Since the individual constituent elements in FIG. 21 and their operations are the same as those described above, their detailed description will be omitted.

The three-dimensional shape restoration device illustrated in FIG. 21 is not limited to the three-dimensional shape restoration of a human hand
It can be applied not only to the hands of industrial robots, but also to other target objects.

FIG. 22 is a block diagram of an image signal processing apparatus for simultaneously detecting the fingers of the human hand and restoring the three-dimensional shape of the human hand. The two-dimensional image data input means 101 such as the video camera 2 images a human hand. This imaging data is used as two-dimensional pixel data as the edge detection unit 4
And applied to the feature extraction means 103, respectively, to detect the fingers of the human hand and restore the three-dimensional shape of the human hand in parallel. Feature extraction means 103A including an edge detection unit 4, a Hough transform calculation unit 6, a possibility search unit 8 and a clustering unit 10.
The feature extraction means 10 outputs the output result of
It can also be used for 3. Of course, the image signal processing device illustrated in FIG. 22 can be applied to a target object other than a human hand.

The above-mentioned method and apparatus for detecting a finger of a human hand and the three-dimensional shape restoration method and apparatus are examples.
The present invention is not limited to the above examples. That is, a wide variety of different embodiments of the present invention can be configured without departing from the technical idea of the present invention and the scope of the present invention, and the present invention is not limited to the specific embodiments described above. Easy to understand.

[0067]

As described above, according to the present invention, the finger of a human hand can be detected quickly and accurately. This method of detecting a finger of a human hand can be applied to detection of a general target object. Further, according to the present invention, a three-dimensional shape can be restored from the two-dimensional image data. In particular, the three-dimensional shape restoration of the present invention shows the speed and accuracy applicable to real-time processing.

[Brief description of drawings]

FIG. 1 is a block diagram of an image processing apparatus for detecting a finger of a human hand according to the present invention.

FIG. 2 is an enlarged view of a hand for explaining the determination of the hand orientation of the present invention and the relationship between the orientation of the finger and the search start line when searching the tip of the finger.

FIG. 3 is a diagram showing a direction detection mechanism for finding a deep local minimum of the human hand of the present invention.

FIG. 4 is a characteristic diagram showing a relationship between a probability distribution function and a transfer function of the present invention.

FIG. 5 is a flowchart showing image input processing and edge detection processing of the present invention.

FIG. 6 is a flowchart showing a part of the Hough conversion process of the present invention.

FIG. 7 is a partial flowchart showing the remaining processing of the Hough conversion processing of the present invention.

FIG. 8 is a flowchart showing a part of the possibility search processing of the present invention.

FIG. 9 is a flowchart showing the remaining processing of the possibility search processing of the present invention.

FIG. 10 is a flowchart showing a clustering process of the present invention.

FIG. 11 is a diagram showing a configuration of an embodiment of a three-dimensional shape restoration device of the present invention.

12 is a flowchart showing a basic operation process of the three-dimensional shape restoration device shown in FIG.

FIG. 13 is a flowchart showing a process of the first example of the three-dimensional shape restoration device shown in FIG.

FIG. 14 is a diagram showing movements of a human palm and finger as an example of an object to be applied to the processing in FIG.

FIG. 15 is a diagram illustrating division of a portion of a target object, which is less deformed, into partial regions in the three-dimensional shape restoration device of the invention.

16 is a flowchart showing a process of a second embodiment of the three-dimensional shape restoration device shown in FIG.

FIG. 17 is a flowchart illustrating an image processing method in the three-dimensional shape restoration device of the present invention when a target object is a human palm and a finger, as a specific example of the three-dimensional shape restoration device shown in FIG. 11. is there.

18 is a feature extraction model diagram of a human palm and finger used in the process in FIG.

FIG. 19 is a diagram showing a first result based on the example of the three-dimensional shape restoration device shown in FIG. 17.

FIG. 20 is a diagram showing a second result based on the example of the three-dimensional shape restoration device shown in FIG. 17.

FIG. 21 is a configuration diagram of another three-dimensional shape restoration device of the present invention.

FIG. 22 is a configuration diagram of a device for detecting a finger of a human hand and a three-dimensional shape restoration device of the present invention.

[Explanation of symbols]

 2 ... Video camera 4 ... Edge detection unit 6 ... Hough transform calculation unit 8 ... Possibility search unit 10 ... Clustering unit 12 ... Human hand 101 ... 2D image data input means 103 ... Feature extraction Means 105 ... Three-dimensional shape restoration processing means

[Procedure amendment]

[Submission date] May 1, 1992

[Procedure Amendment 1]

[Document name to be amended] Statement

[Correction target item name] 0021

[Correction method] Change

[Correction content]

As shown in steps S02 to S010 of FIG. 5, the edge detection unit 4 detects the edge of the hand by the edge detection algorithm based on the inclination method. That is, the edge detection unit 4 calculates the gradient gradI (i, j) of the image based on the following equation 1. Grad is also expressed as an inverted triangle symbol, nabla.

[Equation 1] The edge detection unit 4 determines the edge of the human hand by the following method. If the gradient of the pixel gradI (i, j)
If the absolute value of is greater than a predetermined threshold T, then that pixel (i, j) is an edge pixel. That is, the edge is defined when the differential value of the pixel exceeds the threshold value. If the edge detection unit 4 determines that the pixel is an edge pixel, the edge pixel I (i, i,
j) = 1 and otherwise I (i, j) = 0. The edge detection unit 4 performs this edge detection processing on all image coordinates (i = 1 to imax, j = 1 to jmax).
Do about.

Claims (22)

[Claims] 1. A step of reading a human hand and outputting it as image data in the form of two-dimensional pixel data, a step of detecting the edge of the hand based on the pixel data, and an image line of the detected edge in a parameter space. Parameter conversion to determine the direction of the finger by integrating edge pixels in the parameter space, searching a deep local minimum corresponding to the direction of the finger along the direction, and measuring the tip of the finger. A method for detecting a finger of a human hand having a step of clustering defined endpoints. 2. A step of detecting the edge, a step of calculating a two-dimensional inclination of the pixel data each showing a gray level, a step of comparing an absolute value of the calculated inclination with a threshold level, A method according to claim 1, further comprising the step of determining as an edge when the calculated absolute value is greater than the threshold level. 3. The step of converting the parameters and the step of integrating the edge pixels, converting the linear coordinate system showing the detected edge into a polar coordinate system, integrating the points of the edges in the parameter space, and each of the lines. The method for detecting a finger of a human hand according to claim 1, further comprising the step of obtaining a maximum integrated value to determine the direction of the finger of the hand. 4. The method of claim 1, wherein the searching step comprises the step of determining the deep local minimum and avoiding the shallow local minimum by a random orientation determination mechanism. 5. The step of clustering, selecting an arbitrary position within the cluster, allocating the end point pixel of each search to the cluster position based on at least the Euclidean distance, and the pixel allocated to the cluster. A method for detecting fingers of a human hand according to claim 1, comprising the step of determining a new cluster position by calculating all the centers of 6. A means for reading a human hand and outputting it as image data in the form of two-dimensional pixel data, a means for detecting the edge of the hand based on the pixel data, and an image line of the detected edge in a parameter space. Parameter conversion into a parameter space, integrating edge pixels in the parameter space to determine the orientation of the finger, means for searching a deep local minimum corresponding to the orientation of the finger along the orientation, and the tip of the finger. An apparatus for detecting fingers of a human hand having means for clustering defined endpoints. 7. A step of capturing an image of a target object and outputting it as image data in the form of a two-dimensional pixel data, a step of detecting an edge of the target object based on the pixel data,
A step of parameter-converting the image line of the detected edge into a parameter space and integrating the edge pixels in the parameter space to determine the direction of the target object; a deep local corresponding to the tip of the target object along the direction; A method of extracting a feature of a target object, which comprises a step of searching for a minimum and a step of clustering end points defining the tip of the target object. 8. A means for picking up an image of a target object and outputting it as image data in the form of two-dimensional pixel data, a means for detecting an edge of the target object based on the pixel data,
A means for parameter-converting the detected image line of the edge into a parameter space and integrating the edge pixels in the parameter space to determine the direction of the target object, a deep local corresponding to the tip of the target object along the direction. An apparatus for extracting features of a target object, which has a means for searching a minimum and a means for clustering end points defining the tip of the target object. 9. A step of inputting two-dimensional image data obtained by imaging the movement of a target object represented by a mechanical equation having a spring connected to a node and representing a mechanical constraint by a spring model, and the two-dimensional image data. A feature extraction step of extracting features from the object to generate feature extraction data, and applying force from the two-dimensional coordinates of the feature extraction points to the dynamic model of the target object to deform the target object model to supplement the depth information. And a step of restoring the three-dimensional shape of the target object. 10. The step of restoring the three-dimensional shape calculates a force acting in a two-dimensional coordinate system between each feature point and a corresponding point of the target object, and adds this force to a dynamic equation of the target object. 10. The three-dimensional shape restoring method according to claim 9, wherein the three-dimensional shape of the target object is restored by deforming the dynamic equation model of the target object to supplement the depth information. 11. The step of restoring the three-dimensional shape is performed by dividing the target object into sub-regions with less deformation and performing the three-dimensional shape restoration calculation for each mode of each sub-region.
The three-dimensional shape restoration method described. 12. The step of reconstructing the three-dimensional shape, the rotation from each initial of each divided area is obtained, each rotation matrix is calculated,
Calculate the force that works from each feature point and the force that joins each subregion,
A diagonal matrix is calculated for the force acting on each feature point and the binding force acting on each divided region, and the diagonalization for the force acting on each feature point and the binding force acting on each divided region is calculated based on these diagonal matrices. The three-dimensional shape restoring method according to claim 10, wherein the three-dimensional shape of the target object is restored by calculating the calculated dynamic equation. 13. The feature extraction step is performed by the input 2
Detecting the edge of the target object based on the three-dimensional image data, converting the image line of the detected edge into a parameter space, and integrating the edge pixels in the parameter space to determine the direction of the target object. 12. The three-dimensional shape restoration according to claim 9, further comprising: a step of searching a deep local minimum corresponding to the tip of the target object along the direction, and a step of clustering end points defining the tip of the target object. Method. 14. A step of detecting the edge, a step of calculating a two-dimensional inclination of the pixel data each showing a gray level, a step of comparing the absolute value of the calculated inclination with a threshold level, 14. The three-dimensional shape restoration method according to claim 13, further comprising the step of determining as an edge when the calculated absolute value is larger than the threshold level. 15. The parameter conversion and edge pixel integration steps include: converting from a linear coordinate system indicating the detected edge to a polar coordinate system; integrating edge points in the parameter space; and each of the lines. 15. The three-dimensional shape restoration method according to claim 14, further comprising the step of determining the orientation of the finger of the target object by obtaining the largest integrated value. 16. The method according to claim 15, wherein the searching step includes the step of determining the deep local minimum by a random direction determining mechanism and avoiding the shallow local minimum. 17. The clustering step comprises the step of selecting an arbitrary position within the cluster, the step of allocating the end point pixel of each search to the cluster position based on at least the Euclidean distance, and the pixel allocated to the cluster. 17. The method of claim 16 including the step of determining new cluster positions by calculating all centers of 18. The target object is a human hand.
The three-dimensional shape restoration method described. 19. A means for inputting two-dimensional image data obtained by imaging the movement of a target object represented by a mechanical equation having a spring connected to a node and representing a mechanical constraint by a spring model, and the two-dimensional image data. Feature extraction means for extracting features from the object to generate feature extraction data, and applying force from the two-dimensional coordinates of the feature extraction points to the dynamic model of the target object to deform the target object model to supplement the depth information. A three-dimensional shape restoration device having means for restoring the three-dimensional shape of the target object. 20. The three-dimensional shape restoration according to claim 19, wherein the three-dimensional shape restoration means divides the target object into sub-regions with less deformation and performs the three-dimensional shape restoration calculation for each mode of each sub-region. apparatus. 21. The feature extracting means is configured to input the input 2
Means for detecting an edge of the target object based on three-dimensional image data; parameter conversion of the image line of the detected edge into a parameter space; integration of edge pixels in the parameter space to determine the direction of the target object Means, means for searching a deep local minimum corresponding to the tip of the target object along the direction, and means for clustering end points defining the tip of the target object, and feature extraction points based on the clustering result. And applying a force from the two-dimensional coordinates of the feature extraction points to the dynamic model of the target object to deform the target object model to supplement depth information and restore the three-dimensional shape of the target object. 20. The three-dimensional shape restoration device according to item 20. 22. The target object is a human hand.
9. The three-dimensional shape restoration device according to item 9.