JPH09198504A - Attitude detector - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the estimated precision of attitude parameters at an attitude detector using a genetic algorithm by generating the attitude of a three-dimensional model corresponding to the generic algorithm and deforming this attitude of the three-dimensional model. SOLUTION: This device is provided with a generic silhouette algorithm execution part (S-GA) 10 for estimating the attitude of a person 1 while using the silhouette of the person and a genetic position algorithm execution part (P-GA) 12 for estimating only the attitude of body of the person 1 again based on the estimated result of the S-GA 10. Further, a box generic algorithm execution part (B-GA) 14 is provided for estimating the attitude of each body part of the person 1 again based on the estimated result of the S-GA 10. Thus, based on the estimated result of attitude of a three-dimensional object using the genetic algorithm, only the attitude of main part of the three-dimensional object is estimated again while using the genetic algorithm. Therefore, the estimation precision of attitude is improved.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a posture detecting device, and more particularly to a three-dimensional object imaged by a plurality of image pickup means provided in a predetermined geometrical positional relationship with respect to the three-dimensional object. The present invention relates to a posture detection device that detects the posture of a three-dimensional object based on a plurality of images.

[0002]

2. Description of the Related Art A person image is typical of a flexible moving object whose three-dimensional shape is largely changed by the movement of joints. Therefore, in the field of image processing and computer vision, a human figure is an important target. In recent years, the establishment of a technique for automatically detecting the movement and posture of a person by a non-contact method has become more important for realizing various image communication systems and monitoring systems. That is, the main bones forming the human body are connected by joints, and it is considered that the application is possible if the rotation angle of the joints is detected.

As a conventional method for detecting the posture of a person, there is a contact type method in which a marker or a sensor is attached to the human body, but it is suitable for real-time measurement, but the application area is limited. was there. On the other hand, as a study using image processing, there is a method of analyzing a moving image sequence of a single eye, but various models and constraint conditions are required, and it is not easy to detect a posture of an arbitrary combination of joint angles. Also, research is being made on obtaining a parameter description using low-level information such as an edge of an image. If accurate fitting is performed, posture estimation can be robust, but
If the description is not correct due to image noise or the like, there is a risk of fatal error.

As a means for solving the above problem, there is a posture detecting device for estimating a posture parameter of a person from a multi-image based on a genetic algorithm.
It is disclosed in Japanese Patent Publication No. 302341. This posture detection device is a non-contact method, does not require a constraint condition, and is stable against a noise factor in image processing (Genetic Algorithm: GA) [DEGoldberg,
“Genetic Algorithmin search, optimization, and ma
chine learning ”, Addison-Wesley, 1989.] is used. That is, posture parameters are made to correspond to individual genes in a population, and a 3D human model created in advance is moved or transformed according to the genetic information. Then, the degree of overlap (fitness) of the silhouette of the synthetic human image generated by observing this with a virtual camera and the real human image from the real camera is evaluated, and a certain generation genetic operation is performed on the population. The genetic information of the individual with the best fitness at the time of repetition is used as the posture estimation result.

[0005]

However, in the above-described conventional attitude detection device, although it is possible to bring the attitude parameter closer to the true value to some extent, there is a saturation phenomenon in which the attitude parameter does not approach any more when it reaches a certain value. Can be seen. That is, as shown in FIG. 11 of the above publication, the maximum value in FIG. 10A rises up to about 200 generations, and the fitness is almost saturated by 500 generations. Further, as shown in FIG. 12, the same FIG.
Also about the maximum value of, rise up to about 200 generations, 5
The fitness is almost saturated by the 00th generation. The fitness due to the saturation is 86% in FIG. 11, and the fitness in FIG.
Is about 84%. If the generations after the 500th generation are further piled up, the fitness may increase, but if it is inferred from the increase rate of the 200th generation to the 500th generation, it seems that the process has to be repeated to a considerably older generation. However, it is not efficient to repeat the process to a generation far ahead in order to improve the fitness.

The present invention has been made to solve the above problems, and an object of the present invention is to improve the estimation accuracy of a posture parameter in a posture detection device using a genetic algorithm.

[0007]

According to the present invention, a three-dimensional object is imaged by a plurality of imaging means, each of which is provided in a predetermined geometrical positional relationship with respect to the three-dimensional object, and the plurality of images are obtained. A posture detection device that detects a posture of a three-dimensional object based on the first and second genetic algorithm execution means. The first genetic algorithm executing means is 3
A first virtual three-dimensional model provided corresponding to a three-dimensional object, and a plurality of first virtual three-dimensional models each provided with the same geometrical positional relationship as the above-mentioned geometrical positional relationship with respect to the first virtual three-dimensional model. One virtual imaging means, a first comparison means for comparing a plurality of virtual images and a plurality of images obtained by the plurality of first virtual imaging means to obtain a first fitness, and a predetermined initial value. First gene information generating means for generating first gene information capable of specifying the posture of the first virtual three-dimensional model according to the genetic algorithm according to the first fitness based on the gene information; First according to the genetic information of
And a first deforming unit that deforms the posture of the virtual three-dimensional model. The second genetic algorithm execution means has a second virtual three-dimensional model provided corresponding to the main part of the three-dimensional object, and has the same geometrical positional relationship with respect to the second virtual three-dimensional model. The second fitness by comparing a plurality of second virtual imaging means provided respectively in a geometrical positional relationship with a plurality of virtual images and a plurality of images obtained by the plurality of second virtual imaging means A second comparison means for obtaining
Based on the first genetic information from the first genetic algorithm executing means, second genetic information capable of specifying the posture of the second virtual three-dimensional model according to the genetic algorithm according to the second fitness is obtained. It includes a second gene information generating means for generating and a second deforming means for deforming the posture of the second virtual three-dimensional model according to the second gene information.

[0008] Preferably, the posture detection device further comprises
A third genetic algorithm executing means is provided. The third genetic algorithm execution means includes a third virtual three-dimensional model provided corresponding to the details of the three-dimensional object, and the same geometrical positional relationship with the third virtual three-dimensional model. A plurality of third virtual imaging means provided respectively in a geometrical positional relationship, and a plurality of virtual images and a plurality of images obtained by the plurality of third virtual imaging means are compared to determine a third fitness. Based on the second comparing means to be obtained and the second genetic information from the second genetic algorithm executing means,
Third gene information generating means for generating third gene information capable of specifying the posture of the third virtual three-dimensional model according to the genetic algorithm according to the fitness of, and the third gene information according to the third gene information. And a third deforming means for deforming the posture of the virtual three-dimensional model.

More preferably, the third virtual three-dimensional model has the same color as the three-dimensional object, and the third comparing means further compares the colors of the plurality of virtual images with the colors of the plurality of virtual images. Then, the third fitness is obtained.

[0010]

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, an example of an embodiment of the present invention will be described in detail with reference to the drawings. (1) Overall Configuration FIG. 1 is a block diagram showing the overall configuration of a posture detection device according to an embodiment of the present invention. With reference to FIG. 1, this posture detection apparatus is a target person multi-image obtained by capturing images of person 1 by real multi-cameras R _{1 to} R _n arranged at predetermined geometrical positions with respect to person 1. The posture of the person 1 is detected based on 3. The posture detection apparatus re-estimates only the posture of the trunk of the person 1 based on the silhouette genetic algorithm execution unit (S-GA) 10 that estimates the posture using the silhouette of the person 1 and the estimation result of the S-GA 10. Position Genetic Algorithm Execution Unit (P-GA) 12
And a box genetic algorithm execution unit (B-GA) 14 for re-estimating the posture of each human body part of the person 1 based on the estimation result of the S-GA 10.

The S-GA 10 corresponds to the above-mentioned conventional attitude detecting device. The S-GA 10 gives to the P-GA 12 one of the estimation results relating to the torso of the person 1, and the upper arm,
Give to B-GA14 about arms, hands and head.
The P-GA 12 re-estimates the posture of the torso of the person 1 in more detail by using the estimation result regarding the torso given from the S-GA 10 as an initial value. The B-GA 14 re-estimates the posture of each human body part in more detail by using the estimation result regarding the two arms, the arm, the hand, and the head provided from the S-GA 10 as an initial value.

(2) Posture Parameter to be Estimated The posture of the person is generated by the rotational movement of the joints connecting the main bones of the person. A person can move in a three-dimensional space mainly by using his / her feet. In this embodiment, the upper body of a person is handled. The definition of posture parameters in the upper body is shown in FIG. In this embodiment, the wrist joint is taken into consideration, but the finger joint is not handled. As shown in FIG. 2, regarding the parameters relating to the movement of the person, 3 of the reference point C1 of the chest of the person is used.
3D coordinates (X, Y,
Z) and rotations α, β, γ about this reference coordinate axis, a total of six parameters need to be estimated. In addition, as shown in FIG. 2, there are a total of seven main joints C2 to C8 in the upper half of the person. Since each of these joints C2 to C8 has a degree of freedom of 1 or 3, rotation θ1 as shown in FIG.
There is a joint angle parameter of ~ θ17.

Regarding the head, there is rotation around a three-dimensional coordinate axis whose origin is the joint (center point) C8 of the head, and θ1 (up and down) and θ2 (swing the neck left and right) for each rotation. , Θ3 (tilting the neck). Regarding the shoulder, for example, focusing on the right arm,
Rotation around the bone connecting the shoulder and elbow θ4 and two rotations θ5, θ
There are six. Further, regarding the right arm, with respect to the elbow, a rotation θ7 corresponding to one degree of freedom of the elbow, a wrist portion rotates around a bone connecting the elbow and the wrist θ8, and two rotations θ9, θ.
There is 10. Thus, for the right arm, θ4 to θ10
There is also a parameter of
There are parameters 1 to θ17. Therefore, such 23 posture parameters X, Y, Z, α, β, γ, θ
1 to θ17 are estimated by the attitude detection device shown in FIG.

(3) S-GA As described above, this posture detection device is used for the upper half of the body of a person.
It is necessary to estimate three pose parameters. In general, if no constraint condition is introduced, the combination will explode and it will be extremely difficult to estimate the correct parameters. As a powerful method for solving such a combinatorial optimization problem, there is the above-mentioned genetic algorithm. The S-GA 10 in FIG. 1 estimates the posture parameters of the upper body of the person 1 according to such a genetic algorithm.

FIG. 3 is a block diagram showing a specific structure of the S-GA 10 in FIG. Referring to FIG. 3, this S
-GA10 is captured and human model (virtual three-dimensional model) 7S corresponding to the person 1, and the virtual multi-camera V _1S ~V _NS imaging the human model 7S, the human model 7S the virtual multi-camera V _1S ~V _NS The comparison unit 11S that compares the synthesized person multi-image 9S and the target person multi-image 3 obtained by doing the above, and the gene information generation unit that generates the chromosome 15S having gene information according to the fitness obtained by the comparison unit 11S 13S and human model 7S according to chromosome 15S
And a deforming unit 17S that deforms the posture. Here, the person is imaged by N (≧ 2) stage of the real multi-camera R ₁ to R _N are those persons having an original three-dimensional structure, its operation is also because those three-dimensional . This is also to reduce the probability of occlusion as much as possible. Further, as will be described later, one of the features of the present invention is that processing such as restoration of a three-dimensional structure using stereo matching or the like is unnecessary.

The person model 7S is prepared in advance for the person 1 whose posture parameter is to be obtained. Each joint of the human model 7S can rotate similarly to an actual human. Gene information generator 13S
Is a natural selection gene manipulation unit 19S that performs gene manipulation of natural selection according to the fitness obtained by the comparison unit 11S.
And a gene pool 21S in which genetic manipulation such as mutation is performed. Chromosomal 15S produced by this gene pool 21S has a gene X ₁ to X _n, these genes X ₁ to X _n corresponds to the pose parameters.

Next, the operation of the S-GA 10 will be described.
You. Real multi-camera R₁~ R_nIs the 3D information of person 1.
The person 1 is imaged in order to obtain it. And the goals obtained
The person multi-image 3 is given to the comparison unit 11S. on the other hand,
Since posture parameters such as joint angles are detected,
The person model 7S virtually provided for 1 is created in advance.
Has been established. Gene X on chromosome 15S ₁~ X_nIs a person
1 represents each joint angle. Initially predetermined
Posture parameter is chromosome 15S as initial gene information
Given to. The deforming portion 17S has such a chromosome 15
S gene X₁~ X_nEach joint of the human model 7S according to
Rotate to transform. And the virtual multi-turtle
La V_1S~ V_NSImage of the deformed human model 7S,
The composite person multi-image 9S of is given to the comparison unit 11S.
You. Here, the virtual multi-camera is the real multi-camera described above.
Mela R₁~ R_NSame as N cameras V_1S~ V_NSFrom
The camera parameters such as the positional relationship and angle of view of each other are
The data are matched. That is, virtual multi-camera
La V_1S~ V_NSGeometrical Arrangement Function for Human Model 7S
The person in charge is the real multi-camera R provided for the person 1.₁~
R_NIs the same as the geometrical arrangement position of. Such an arrangement
To extract the 3D structure and 3D motion of a person.
It is. Also, with this arrangement, occlusion
The probability is as low as possible.

The comparison unit 11S includes real multi-cameras R ₁ -R.
Target person multi image 3 from _N and virtual multi camera V _1S
~ The synthesized person multi-image 9S from V _NS is compared, the target person multi-image 3 is considered as the environment, and the fitness to the environment is calculated and obtained. This fitness is that the area ratio of the overlapping portion generated between the real human image and the virtual human image is evaluated. FIG. 4 is a diagram for explaining the fitness. Target person multi-image 3 and composite person multi-image 9
For each of S, the difference between the person 1 and the background image without the person model 7S that have been acquired in advance is calculated, and the area that may correspond to the person 1 and the person model 7S and the background candidate area are binary. Be converted. After that, the real multi-camera R _i and the virtual multi-camera V _iS (i =
1, ..., N), the fitness SF according to the following equation (1):
Is calculated.

SF _i = A / (A + B + C) (1) where A is the person candidate region 2 from the real multi-camera R _i
3 represents the area of a portion where the synthetic human area 25 from the virtual multi-camera V _iS overlaps, B represents the area of the portion in which only the synthetic human area 25 from the virtual multi-camera V _iS exists, and C represents the actual area. It represents the area of a portion where only the person candidate area 23 from the multi-camera R _i exists. Therefore, SF _i takes a value between 0 and 1, and becomes 1 when the real person image and the virtual person image completely overlap. The fitness SF of the entire multi-camera is obtained by the average of the fitness SF _i shown in the equation (1), that is, the following equation (2).

[0020]

[Equation 1]

The calculation of the fitness by the equations (1) and (2) is performed for all P individuals in the population. Each of the P individuals has a gene representing 23 parameters relating to the joints of the upper body of the person and the position of the person as described above. Therefore, the gene information generation unit 13S generally detects a parameter based on a genetic algorithm such that a combination explosion occurs if no constraint condition is introduced. This genetic algorithm is a powerful means for solving combinatorial optimization problems. First, the natural selection gene creation 19S determines the selection probability in a form proportional to the value of the fitness calculated by the comparison unit 11S, selects two parents for leaving offspring by random extraction, and selects natural selection. Genetic manipulation of. At this time, there is a high probability that a parent with high fitness is selected.

Then, two children are born from the parents in the gene pool 21S. When such a parent is bred, cross and mutation genetic manipulation, such as occurs with probability p _c and p _m, respectively is performed. In addition, here, each gene X
_{1 to Xn} are represented by bit strings. In this way, P individuals (chromosomes) 1 of the next generation are added to the gene pool 21S.
5 occurs. The cycle of posture transformation of the human model 7S by the transformation unit 17S, fitness calculation by the comparison unit 11S, and individual generation by the gene information generation unit 13S is repeated. Then, in the individual population obtained after passing a certain generation, the genetic information of the individual that gives the maximum fitness SF is used as the estimated value of the posture parameter of the person 1.

Thus, as the image processing, the difference between the background and the human image is obtained and binarized, and the fitness SF
Since the calculation of is also performed based on the area information, the resistance to image noise is high. (4) In order to improve the accuracy of the estimation result of the P-GA S-GA 10, it is effective to re-adjust the synthetic human multi-image to the target human multi-image for each human body part based on the estimation result. In this P-GA 12, the posture parameter for the torso, which is the largest of the human body parts, is re-estimated.

FIG. 5 is a block diagram showing a specific structure of the P-GA 12 in FIG. This P-GA12 is S
-Increase the estimation accuracy of the six posture parameters (X, Y, Z, α, β, γ) related to the movement obtained by the GA10. These parameters are for the torso of person 1, as shown in FIG. The torso is larger than other human body parts, and it is thought that overlapping silhouettes is effective. Therefore, in this P-GA12, P-
Body model 7P of the body instead of the person model 7C of GA10
Is used. Except for such a torso model 7P,
Similar to the P-GA12 is the S-GA10, the virtual multi-camera V _iP ~V _NP, a comparing unit 11P for comparing the synthesized human multi-image 9P and the target person multi-image 3, the genetic information to produce a chromosome 15P The generation unit 13P and the transformation unit 17P are included. The gene information generation unit 13P includes a natural selection gene manipulation unit 19P and a gene pool 21P.

At the end of the calculation of the final generation of S-GA10, among the P individuals, the upper J (<P) values of the fitness SF of equation (2) are extracted, and the initial value of P-GA12 is extracted. (Each P / J individual generates an individual having the same initial value). The gene of each individual in P-GA12 is
These are the six posture parameters described above. This P-GA1
The genetic algorithm in 2 is almost the same as the genetic algorithm in S-GA10, but the fitness P
The following F is used.

[0026]

[Equation 2]

Here, the definitions of A and B in the equation (3) are the same as those in the equation (1). Although C is not included in Expression (3), this is because the position of the torso in the actual human image is not specified at the stage of P-GA12 and the torso is the largest human body part. This is because when the person model 7P does not match the actual person image, it is reflected in B. As described above, in the P-GA 12, the posture parameter of the body, which is the main part of the person 1, is re-estimated in detail, so that the posture estimation accuracy of the person image 1 is improved. (5) B-GA FIG. 6 is a block diagram showing a specific configuration of the B-GA 14 in FIG. After the six posture parameters of the torso are re-estimated by the P-GA 12, the posture parameters of the human body parts are re-estimated by the B-GA 14 to improve the posture estimation accuracy. In this B-GA 14, a person model 7B in which a rectangular parallelepiped box 29 is set is used. Except for such a person model 7B, the B-GA 14 is similar to the S-GA 10 in the virtual multi-cameras V _{1B to} V _NB.
And a comparison unit 11B that compares the synthesized person multi-image 9B and the target person multi-image 3, a gene information generation unit 13B that generates the chromosome 15B, and a transformation unit 17B. Further, the gene information generation unit 13B includes a natural selection gene operation unit 19B and a gene pool 21B.

In the B-GA 14, each human body part of the human model 7B is matched with the human 1 by utilizing the local information of the human body parts in order from the one close to the torso. Here, color information is used as local information. This is because it is difficult to deal with the occlusion of human body parts in the image (for example, the arm in front of the torso) only with the overlapping information of the silhouettes used in the S-GA10 and the P-GA12. Is. In order to use the local information of human body parts, as shown in FIG.
A rectangular box 29 is set for each human body part in the human model 7B. This box 29 is observed as a polygonal window 31 in the target person multi-image 3 as shown in FIG. 7 (B). The posture parameter of each human body part is re-estimated according to a genetic algorithm for the inside of this window 31.

As described above, the value of the posture parameter estimated by the S-GA 10 is updated by the B-GA 14 from the human body part close to the torso. That is, the respective posture parameters are updated in the order of second arm → arm → hand → head. Here, for the human body parts other than the head, the left and right human body parts are estimated according to a genetic algorithm. Human body parts k in B-GA14
The processing contents of are as follows. Each individual (chromosome 15
B) has genes corresponding to 23 parameters. The six parameters for the torso were given to all individuals as obtained by P-GA12 and not processed by B-GA14,
It is a fixed value. The gene to be processed in B-GA14 is for the human body part k. As shown in FIG. 2, the head is θ1 to θ3, the two arms are θ4 to θ6, and the arms are θ.
7, the hand is θ8 to θ10 (same on the opposite side). The initial value of each individual gives an estimate for k of the top J (<P) individual human body parts of the S-GA result. If a human body part k ′ closer to the torso than the human body part k exists (for example, an arm is a second arm), the best value of the result of the B-GA 14 for the human body part k ′ is used as a fixed value.

Composite human multi-image i of human body part k (i
= 1, ..., N) for window 31
Degree bf_iThe following equation (5) is used as bf_i= Cf_i× SF_i (5) Where SF_iIs the weight of the silhouette given by equation (1)
The degree is cf _iIs the color of the composite portrait image 9B
Degree of overlap between color information and color information of target person multi-image 3
In total, it is expressed as the following formula (6). cf_i= D / (D + E) (6) where D is the area of the same color portion in the window 31.
, E represents the area of the part having a different color. Wind
C. To determine the color identity of the pixels within 31, use the following equation (7).
It depends on whether or not it is satisfied.

| Rm-Rr | + | Gm-Gr | + | Bm-Br | <Th (7) Here, R, G, and B represent the three primary colors of the pixel, and the subscript m is the composite person multi-image. , R are target person multi-images. Further, in Expression (7), Th represents a threshold value.
In Expression (5), the term SF _i is multiplied by cf _i in order to prevent the case where one human body part bites into another human body part, which is impossible in reality. The value of _i becomes small, resulting in bf
_The purpose is to reduce the value of _i .

[0032] From bf _i of the thus obtained synthetic human multi-image i, fitness BF of the body part k is obtained by the following equation (8).

[0033]

(Equation 3)

In the equation (8), the so-called strict condition of taking the minimum value of bf _i is imposed, so that the human body part k is more closely matched to the target person multi-image 3. is there. As described above, according to the embodiment of the present invention, since the P-GA 12 re-estimates the posture parameter of the torso using the torso posture parameter estimated by the S-GA 10 as an initial value, the fitness of the human model is Further improve. In addition, the posture parameters other than the torso obtained by S-GA10 are used as initial values for B-GA1.
4 re-estimates those posture parameters for each human body part, so that the fitness of the human model is further improved. Moreover, since the B-GA 14 uses color information, it is possible to detect a posture such as "arms are assembled in front of the torso".

[0035]

EXAMPLES The results of the experiments according to the above embodiment will be described below. (1) Experimental condition Here, the synthetic person multi-image generated using the person model is used as the target person multi-image. Three cameras are used and each image is projected in parallel. Each camera is installed on each coordinate axis of the three-dimensional coordinate system whose origin is the center point C1 of the chest of the person shown in FIG. Here, each coordinate axis passes through the center of each image from the front, side, and overhead of the person. The size of each image is 256 × 256 pixels, and the length of the image corresponds to 173 cm.

The three-dimensional modeling of a person is performed by using the three-dimensional shape data and color information acquired by the cyberware color 3D digitizer. This modeling is performed for each human body part. That is, a wire frame model is created by approximating with a set of triangular patches based on the measurement result of the surface shape of each human body part by the digitizer. Further, since the color texture can be acquired at the same time by using the digitizer, it is mapped to the wireframe model. The human body parts are connected to each other, and by appropriately moving the vertices of the triangular patch, the wire frame can be deformed and the joints can be rotated. The color texture information is mapped to a triangular patch at a corresponding location, and color interpolation or thinning is performed according to the deformation of the triangular patch.

As the human model used for synthesizing the target three-direction image, two types, a simple model and a precise model, are used. Since the precise model is composed of 6079 polygons, it maintains an extremely smooth shape as shown in FIG. In addition, the color texture used is that of an actual person. On the other hand, in the simple model, the number of polygons is 36 based on the shape of the precise model.
The color information is reduced to 4 and the color information of the corresponding areas in the precision model is averaged to obtain the color of each polygon (see FIG. 7).

In this experiment, a precise model and a simple model were used to obtain by observing persons having three types of postures shown in FIGS. 8A, 8B, and 8C from three directions. The multi-image obtained is used as the target image. In general,
Although the composite person multi-image has a smaller noise factor than the target person multi-image, the above-described S-GA10 and P-GA12
Image processing in the B-GA 14 is less affected by noise. Further, since the value of the posture parameter of the model can be freely given, there is an advantage that the estimation accuracy can be easily evaluated. 3 types of postures (A) shown in FIG.
Table 1 shows the parameter values (true values) of (B) and (C).
The parameter values of these three types of postures are common to the target images of the precision and simple models. The postures (A), (B), and (C) of the precision model are observed from the front in FIG.

[0039]

[Table 1]

The individuals in each GA 10, 12, 14 are
It is a bit string composed of genes of 8-bit integer type data. Crossing is performed by exchanging bit strings of two individuals.
Mutation is performed by bit inversion. Note that both the precise and simple models are used for the target image, but each GA
As the human model necessary for the processing of 10, 12, and 14,
Only the simple model is used in this experiment, as the precise model causes a significant increase in processing. (2) Study on Target Image of Simple Model First, the target image generated from the above-described simple model was examined.

The number of individuals used in each GA 10, 12, 14 is 1000. S-GA10 has 1 generation
000, crossing probability p _c = 0.1, mutation probability p _m = 1
It is ^0-4 . P-GA12 has 1000 generations and S
-The initial value is the estimated value of the top 100 individuals of GA10, and p
_c = 0.1 and p _m = 0.01 were used. Here, the mutation is generated with a higher probability than S-GA10,
This is because the S-GA 10 is considered to have already collected a considerable number of individuals in the vicinity of the true value, and thus exerts the effect of fine adjustment. The B-GA14 processed 100 generations for each of the seven human body parts. As the initial value of each human body part, the above 100 pieces as a result of S-GA10 were used. Further, similarly to the P-GA12, for giving the effect of fine-tuning, and the p _c = p _m = 0.1.

Table 2 below shows the values of the fitness values obtained by the processing in each GA 10, 12, 14.

[0043]

[Table 2]

As a result of P-GA12 for the body, PF
The value of is improved to almost 100 for all three target images.
%. The fitness used for B-GA14 is
Since it is local for each human body part, Table 2 shows
The posture finally obtained as an estimation result is evaluated by the degree of overlap SF of silhouettes. First S-GA S
The values are higher than the F value. By the way, although the B-GA14 uses color information, when the degree of overlap of silhouettes is used in the window 31 as in the case of the S-GA10 and P-GA12, the SF value is (A) 78.2%, (B) 75.7%, (C) 7
This is 4.3%, which is about 20% lower than when color information is used, indicating the effectiveness of B-GA14. Further, Table 1 shows the results of posture estimation by the B-GA 14 together with the results of the S-GA 10 for each parameter. From the table, it is clear that B-GA14 gives better results than S-GA10 for almost all parameters.

(3) Examination for Target Image of Precision Model When both the target image and the human model are simple models, the P-GA 12 and the BG in the above embodiment are used.
The effectiveness of A14 was revealed. On the other hand, we examined the case of using a simple model as a human model and adapting to a target image generated from a precise model. Since the shapes of the precision model and the simple model do not precisely match in each image, the degree of overlap SF of silhouettes is about 90% even when the true value can be accurately estimated.

Regarding the posture (B) in FIG. 8, as in the case of the target image of the simple model described above, the S-GA10
->P-GA12-> B-GA14 was processed respectively. As a result, SF = 8 in S-GA10
4.2% was SF = 80.6% in B-GA14, on the contrary, the estimation accuracy was worse than S-GA10.
As a cause of this, as described above, strictly speaking, the shapes of the simple model and the precise model do not match.
Further, regarding the color information, there is a delicate tint in the precise model, whereas in the simple model, one triangular patch has only a uniform color. As described above, it is considered that there is a case where the degree of overlap in the silhouette is insufficient due to the cause of the mismatch between the shape and the color.

Therefore, in order to further reflect the degree of silhouette overlap, the second silhouette GA (S-GA2) is performed again after S-GA10 and P-GA12. Furthermore, the B-GA 14 is also replaced with the P-GA 2 for evaluating the degree of silhouette overlap, as described below. S
In -GA2, each individual has the best estimated values of the six parameters related to the trunk obtained as a result of P-GA12 as fixed values, and the remaining 17 parameters are estimated. The initial values of the 17 parameters of each individual are the first S-GA.
The top 100 results of 10 are used.

In order to reflect the silhouette information in the local color information, the B-GA2 compares the BF (formula (8)) of the child resulting from the mating with the parent, and if the child is worse than the parent, the child's BF is calculated. BF that kills the individual and if the child is higher than the parent
If you have a value of, compare the overlapping degree SF of the silhouette of the parent and child, and only when the child is better,
Leave the child as the next generation individual. Table 3 shows the results of the above GA processing. In the target image (A), SG
P-GA2 gives worse results than A10, but other 2
Two images show improvement.

[0049]

[Table 3]

As described above, the improvement of the posture detection device for estimating the posture parameter of the person from the multi-image of the person based on the genetic algorithm has been proposed. In this posture detecting device, a genetic algorithm using local information of a person is executed based on the estimation result of the method (S-GA) using the degree of silhouette overlap in the conventional posture detecting device. First, the result of S-GA is given as the initial value of the torso posture parameter, the silhouette-based genetic algorithm (P-GA) is executed, and the six parameters of the torso are re-estimated. Next, based on the results of S-GA and P-GA, a genetic algorithm (B-GA) using color information is executed for each human body part of the upper arm, arm, hand, and head.

The effectiveness of the present invention was examined for two types of three-dimensional (upper and lower) human upper body models obtained from an actual person. As the target person multi-image, a synthetic three-direction image generated from these models was used.
Here, three types of postures were examined. A simple model was used as the three-dimensional human model in each GA process.
In the case of the target three-direction image synthesized from the simple model, S
It was confirmed that better estimation accuracy was obtained with B-GA as compared with the result of -GA. On the other hand, in the case of the target image from the precision model, the result that the P-GA was worse was obtained. The cause of this may be the inconsistency of the shape and color of the simple and precise model. In order to deal with such a mismatch factor, after performing S-GA and P-GA, silhouette-based GA was performed again on the upper body using these results (S-GA2). Furthermore, B-GA
In addition, GA (B-GA2), which evaluates not only the human body parts but also the degree of overlap of the silhouette of the entire upper body, was introduced, and some improvement from the initial S-GA was confirmed.

[0052]

As described above, according to the present invention, based on the estimation result of the posture of the three-dimensional object using the genetic algorithm, the posture of only the main part of the three-dimensional object is again used by using the genetic algorithm. Therefore, the posture estimation accuracy is improved. Further, based on the posture estimation result of the three-dimensional object using the genetic algorithm, the detail algorithm of the three-dimensional object is estimated again using the genetic algorithm, so that the posture estimation accuracy is further improved. Moreover, since the color information is taken into consideration when re-estimating the detailed posture of the three-dimensional object, the posture of the three-dimensional object can be accurately detected.

[Brief description of drawings]

FIG. 1 is a block diagram showing an overall configuration of a posture detection device according to an embodiment of the present invention.

FIG. 2 is a diagram illustrating the definition of rotation parameters of joint angles in the upper body of a human body.

FIG. 3 is a block diagram showing a specific configuration of the S-GA in FIG.

FIG. 4 is a diagram for explaining the fitness calculated by a comparison unit in FIG.

5 is a block diagram showing a specific configuration of a P-GA in FIG.

FIG. 6 is a block diagram showing a specific configuration of the B-GA in FIG.

7A is a diagram for explaining a box of a human body part set in a human model, and FIG. 7B is a diagram for explaining a window set in a human figure.

8A to 8C are diagrams showing target images from the front, which represent three types of postures.

[Explanation of symbols]

1 person 7S, 7P, 7B person model 10 silhouette genetic algorithm execution unit (SG
A) 12-position genetic algorithm execution unit (PG
A) 14 box genetic algorithm execution unit (B-GA) 11S, 11P , 11B comparator unit 13S, 13P, 13B genetic information generating unit 17S, 17P, 17B deformable portion R ₁ to R _N real multi-camera V _1S ~V _NS , V _{1P to} V _NP , V _{1B to} V _NB virtual multi-camera

Claims (3)

[Claims] 1. A three-dimensional object is imaged by a plurality of imaging means, each of which is provided in a predetermined geometrical positional relationship with respect to the three-dimensional object, and the posture of the three-dimensional object is based on the plurality of images. A first virtual three-dimensional model provided corresponding to the three-dimensional object, the same geometrical relationship as the geometrical positional relationship with respect to the first virtual three-dimensional model. A plurality of first virtual image pickup means respectively provided in a physical position relationship, a plurality of virtual images obtained by the plurality of first virtual image pickup means and the plurality of images are compared to determine a first fitness. First comparing means for obtaining, first gene information capable of specifying the posture of the first virtual three-dimensional model in accordance with a genetic algorithm according to the first fitness based on predetermined initial gene information First death to generate A first genetic algorithm executing means including child information generating means and first deforming means for deforming the posture of the first virtual three-dimensional model according to the first genetic information; A second virtual three-dimensional model provided corresponding to a main part, and a plurality of first virtual three-dimensional models each provided with the same geometrical positional relationship as the geometrical positional relationship with respect to the second virtual three-dimensional model. Two virtual image pickup means, a second comparing means for comparing the plurality of virtual images obtained by the plurality of second virtual image pickup means with the plurality of images to obtain a second fitness, the first inheritance The first from the dynamic algorithm execution means
Second gene information generating means for generating second gene information capable of specifying the posture of the second virtual three-dimensional model according to the genetic algorithm according to the second fitness based on the gene information of And a second deforming the posture of the second virtual three-dimensional model according to the second gene information.
Posture detecting apparatus, comprising: a second genetic algorithm executing means including the transforming means. 2. A third virtual three-dimensional model provided corresponding to the details of the three-dimensional object, and the same geometrical positional relationship as the geometrical positional relationship with respect to the third virtual three-dimensional model. A plurality of third virtual image pickup means provided respectively in the plurality of third virtual image pickup means, and a plurality of virtual images obtained by the plurality of third virtual image pickup means are compared with the plurality of images to obtain a third fitness degree. Comparing means, the second from the second genetic algorithm executing means
Third gene information generating means for generating third gene information capable of specifying the posture of the third virtual three-dimensional model according to the genetic algorithm according to the third fitness based on the gene information of And a third deforming the posture of the third virtual three-dimensional model according to the third gene information.
The posture detecting apparatus according to claim 1, further comprising third genetic algorithm executing means including the transforming means. 3. The third virtual three-dimensional model has the same color as the three-dimensional object, and the third comparing means further calculates the colors of the plurality of virtual images and the colors of the plurality of virtual images. The posture detecting apparatus according to claim 2, wherein the third fitness is also compared to obtain the third fitness.