KR100776880B1 - System for estimating attitude at specific part, methode for estimating attitude at specific part, and program for estimating attitude at specific part - Google Patents

Abstract

The posture is detected in a short time by small hardware which is not affected by the brightness of the input image and which has a low computing power. A matching image generation unit 3 which binarizes each pixel of the input image from which the specific portion is extracted to a setting threshold value based on an image value within a predetermined coordinate range, and specifies a component whose size is within the setting range to obtain a binary image; The pattern matching part 4 which detects a posture by contrasting the binary image obtained by the above-mentioned specification with a predetermined | prescribed template was provided.

Description

Specific partial pose estimating apparatus, specific partial pose estimating method and specific partial pose estimating program

The present invention relates to an attitude estimation apparatus capable of easily estimating a posture in a short time even by an image obtained by hardware having low recognition capability, such as a mobile phone or an electronic notebook, or by a simple image processing circuit attached to such a device.

For example, in the case of driving monitoring of a car or the like, or in authentication of entering a room, specifying a person, and the like, extraction of a human face including its posture, image processing, and use for monitoring or authentication is performed. In such monitoring or authentication, the posture detection is required first or posture detection becomes important prior to the subsequent image processing.

In this case, conventionally, there is an image processing apparatus for posture extraction, which shows its configuration in Fig. 14 shown in, for example, a patent document (Japanese Patent Laid-Open No. 7-200774). In the figure, the image processing apparatus 21 includes skin color extraction means 22 for extracting skin color from an input image, binarization means 23 for extracting an area for candidates for eyes or mouth from the result, and binarization means. Eye and mouth area detection and tracking means 24 for detecting eyes and mouth areas from the moving picture.

Next, the operation will be described.

First, in order to specify a face region in a moving image, skin color pixels are detected by skin color extraction means 22 using a three-dimensional color histogram and the like, and binarization means 23 binarizes the skin color pixels and other pixels. . Next, the eye / mouth area detection / tracking means 24 extracts the hole area in the skin color area to be a candidate area for eyes and mouth. From the extracted candidate region, the eye and mouth positions are detected based on heuristics of the position of the eye and the position of the mouth relative to the face region, for example. Posture information of the head is extracted from the last detected eye and mouth positions.

The conventional posture extracting apparatus is constructed as described above, and since the skin color is extracted, the area is detected and tracked, and the posture is finally detected, the image captured by the video camera first changes the shooting location. It is not limited to including illumination, and quality image quality is not limited, but there exists a subject that favorable detection of skin color is difficult. In addition, since there is a large amount of processing including area detection and a large circuit is required, there is a problem that processing takes time with a small piece of hardware mounted on a mobile telephone.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and poses are extracted in a short time even by a small amount of hardware having low computational power without affecting the image quality of an input image by a video camera or the like.

Disclosure of the Invention

The specific partial attitude estimation apparatus according to the present invention binarizes each pixel of the input image in which the specific portion is estimated to a set threshold value based on an image value within a predetermined coordinate range, and further specifies a component whose size is within the set range. A matching image generating unit for obtaining an image, a binary image obtained by specifying this image, and a pattern matching unit for detecting a posture by comparing a predetermined template were provided.

The matching image generating unit converts the input image into a gray scale image, and sets the luminance of the gray scale image as the threshold value as the average value or the median value of the luminance of pixels within a predetermined range with the target pixel as the coordinate center. It is characterized in that it is made to be digitized.

The pattern matching unit generates a matching image from a plurality of specific input images in advance, and uses the generated binary image as an element of a template.

The pattern matching unit is characterized in that a comparison with each element of the template is obtained by a logical product of the pixels.

The pattern matching unit obtains pixels having a specific value from the binary image obtained by the matching image generation unit, and estimates the inclination of the specific portion to be detected from the distribution state in the image of those pixels.

The specific partial attitude estimation method according to the present invention binarizes each pixel of the input image from which the specific portion is extracted to a set threshold based on an image value within a predetermined coordinate range, and out of the set range from the binarized image obtained by the binarization. And deleting the portion and labeling it as an image part within a setting range, matching the binary image formed by the group of parts with the label after the deletion, and the luminance of the image present in the predetermined template. .

In the binarization, the input image is first converted into a gray scale image, and the luminance of the converted gray scale image is one of the luminance average value of the pixel within a predetermined range with the target pixel as the coordinate center and one of the image center values. It selects as a digitization threshold and binarizes.

Further, the collation is characterized in that a collation value is obtained by the logical AND of the pixels corresponding to the coordinate values of the binarized image and the template image.

The specific partial attitude estimation program according to the present invention can be read and executed by a calculator and binarizes each pixel of the input image from which the specific portion is extracted to a set threshold value based on an image value within a predetermined coordinate range. A part outside the setting range is deleted from the obtained binarized image, and labeling is performed as an image part within the setting range, and the luminance of the binarized image made up of the group of parts labeled with the label and the image present in the predetermined template are compared. Characterized in that the program having a.

BRIEF DESCRIPTION OF THE DRAWINGS The figure which shows the structure of the specific partial attitude estimation apparatus in Example 1 of this invention.

2 is a flowchart showing the operation of the specific partial attitude estimation apparatus according to the first embodiment;

3 is a flowchart showing the binarization operation performed by the matching image generation unit in the first embodiment;

4 is a diagram illustrating a hardware internal configuration of a binary image generating unit in Example 1;

5 is a diagram for explaining a range of obtaining a set threshold value in Example 1;

6 is a diagram for explaining a method for acquiring a setting threshold value in Example 1;

FIG. 7 is a view for explaining a binarization operation performed by a matching image generating unit in Embodiment 1; FIG.

8 is a flowchart of a matching operation performed by the matching unit according to the first embodiment;

9 is a view for explaining a matching operation performed by the pattern matching unit in Example 1;

FIG. 10 is a diagram illustrating a configuration of another specific partial attitude estimation device according to the first embodiment; FIG.

FIG. 11 is a diagram for explaining a template creating operation executed by another pattern matching unit in Example 1;

12 is a diagram illustrating a configuration of a specific partial attitude estimation apparatus according to the second embodiment;

FIG. 13 is a view for explaining attitude extraction by pixel distribution executed by the pattern matching unit in Example 2; FIG.

14 is a diagram showing a configuration for extracting a posture according to a conventional image processing apparatus.

Implement the invention  Best form for

(Example 1)

BRIEF DESCRIPTION OF THE DRAWINGS It is a figure which shows the structure of the specific partial attitude estimation apparatus in this Example of this invention.

In the drawing, the specific partial pose estimating apparatus 1 performs a filtering process on the image capturing unit 2 for capturing an image signal captured by a video camera or the like, and compares the captured image with a posture pattern described later. Pattern matching unit 4 for obtaining a partial posture of a head or the like, in contrast with an image generated by matching image generation unit 3 and matching image generation unit 3 for generating an image, and a posture pattern stored in advance. It consists of. In addition, the inside includes a color space conversion unit 5 for converting the color image captured by the image capture unit 2 into a gray scale image, and a binary image generation unit for converting the converted gray scale image into a binary image ( 6) The part candidate extracting unit 7 which obtains a region by integrating adjacent pixels from the binarized image by the binary image generating unit 6, and extracts only candidate regions which can be eyes or mouths, and generating a matching image. It consists of a matching part 8 for matching with the matching image produced | generated by the part 3, and the posture pattern image memorize | stored previously, and the matching pattern DB 9 for storing the pattern used by the matching part 8.

Next, the operation of the specific partial pose estimation apparatus of the present invention will be described with reference to Figs.

2 is a flowchart for explaining the operation. 3 is a flowchart for explaining the operation of the binary image generating unit 6. 4 is a diagram showing a hardware internal configuration of the binary image generating unit 6, but the other elements are the same, and FIGS. 5 and 6 illustrate the flow of processing executed by the matching image generating unit 3. It is a figure for following. FIG. 7 is a diagram for explaining a matching process performed by the pattern matching section 4. 8 is a flowchart showing the operation of the matching process executed by the pattern matching section 4. 9 is a view for explaining the operation of the pattern matching section 4.

As shown in FIG. 4, the binary image generating unit 6 is a processor 61, a memory 62, an input / output interface 64, and a binary program 63 for executing the operations shown in FIG. It is composed. The processor 61 first reads the gray scale image obtained by gray scale the captured image obtained via the input / output interface 64 into the memory 62. Then, the read gray scale image is binarized in accordance with the luminance in S1-3 of FIG. 2 by the steps recorded in the binarization program 63 as described later.

First, the video capture unit 2 captures a video signal (step S1-1), and converts the captured color image into a gray scale image by the color space conversion unit 5 (step S1-2).

The conversion from the color image to the gray scale image is performed using, for example, the following equation (1).

(식1)

(Eq. 1)

Here, Gy (x, y) is the luminance value at the coordinate values (x, y), and R, G, B (x, y) is the pixel value of the color image at the coordinate values (x, y). You may use the value of that excepting the above as the coefficient value at the time of converting from a color image to a gray scale image.

Alternatively, the color-gray scale conversion in the color space conversion unit 5 may be converted using the above expression (1) after normalizing the RGB value using the following equation (2).

(식2)

(Eq. 2)

Next, in the binary image generating means 6, the gray scale image is binarized in accordance with the luminance (step S1-3). At this time, by performing the process shown in Fig. 3, a binary image is generated in accordance with the input image. The threshold value used as the binarization standard at this time sets the window coordinates 31 of the predetermined range shown in FIG. 5, and sets the luminance average value or the median value of 25 pixels in the telephone set within the coordinate range, and in the case of FIG. It calculates | requires and makes this value a threshold value. The processing shown in FIG. 6 is executed and the luminance of the pixel 32 as a target is compared with a threshold value.

In this way, the input image is scanned, and the image is binarized by repeatedly performing the processing of steps S2-1 to S2-8 for all the pixels.

That is, a pixel whose pixel brightness level value of the pixel to be determined from now on is lower than the preset threshold value is set to 1 (step S2-2, step S2-7). If the pixel brightness level of the target pixel is larger than the threshold value, a block having a size of KxK (K = 5 in FIG. 5 as described above) centered on the pixel of interest is set (step S2). -3). Next, the average value of the luminance in the set block is obtained (step S2-4). Then, it is determined whether or not the pixel value of the pixel of interest satisfies the following condition (Equation 3) (step S2-5).

(식3)

(Eq. 3)

Here, C is a prescribed value set in advance.

If the condition of Expression (3) is satisfied, the pixel value is set to 0 (step S2-6); otherwise, the pixel value is set to 1 (step S2-7).

As described above, since the binarization processing is performed in accordance with the state of the surrounding pixels, a predetermined coordinate range is set even for an image having low contrast due to deterioration of image quality of the video camera. For example, it becomes possible to perform a binarization process from an average value.

Although the average value was calculated | required in step S2-4, the median value at the time of rearranging the pixel value in a block, for example may be calculated | required, and the following formula (4) may be used.

(식4)

(Eq. 4)

In addition, after performing contrast enhancement processing such as a histogram averaging method, it may be binarized using a fixed threshold value.

When the color image captured by the image capturing unit 2 is converted to gray scale by the above process, it becomes (10) of FIG. 7, and when the image is binarized, it becomes as (11) of FIG.

Next, in the component candidate extracting section 7, it is determined whether the same binary pixels are connected to the binary image 11 by four or eight connected up, down, left, and right inclinations, thereby integrating adjacent and meaningful neighboring pixels. Then, the area is obtained and labeled as individual areas as shown in (11-a) and (11-b) in FIG. 7 (S1-4). Further, only the area in which the size of the circumscribed rectangle of the area is contained within the preset range is extracted (step S1-5). That is, the area 11-a in FIG. 7 is excluded as an area having a size other than the estimation target.

Here, in the case of using the TV telephone function in a mobile phone or an electronic notebook, in order to capture a face of a person at an angle of view and to take a larger image, the user has a certain degree of eye size or mouth size in advance. It is possible to predict what will happen. Therefore, the above threshold processing is valid.

Thus, the result extracted in step S1-5 becomes as (12) of FIG.

Using the matching image 13 obtained above, the pattern matching part 4 estimates a head posture (step S1-6).

The operation will be described according to the flowchart executed by the pattern matching section 4 in FIG. The hardware configuration of the pattern matching section 4 is similar to that of FIG. 4, but there is a matching program that performs the operation of FIG. 8 instead of the binarization program 63.

Here, it is assumed that the template shown in (14) of FIG. 9 is stored in the matching pattern DB9. As described above, when photographing a face of a person when using a TV phone function or the like in a mobile phone or an electronic notebook, the angle of view can be assumed in advance, so that the state of the eye and mouth area along the direction of the face can be predicted in advance. have. In the matching pattern DB 9, the binary mask image of the eye and mouth area in the assumed head posture is stored.

In S3-1, P of the matching image 13 in Fig. 9 binarized in the memory via the input / output interface is read. In addition, in S3-2, the first mask image T1 as a reference is read from the template 14 of the matching pattern DB 9. Then, in S3-3, the binarized value of P (x, y) and T1 (x, y) in the region of read y = 0 to B and x = 0 to A of the captured image P and the mask image T1. The binary value of is logically operated on each coordinate value (x, y). Then, the logical product is added from coordinates (0, 0) to (A, B). In S-4, steps S3-2 and S3-3 are repeated until no mask image has been tried in the template 14 yet. In step S3-4, when the AND addition of all the mask images is completed, the mask image showing the maximum addition value in step 3-4 in S3-5, and in this example, the image 15 is selected.

That is, the logical product of the matching image 13 and each mask image of the template is calculated, the number of pixels 1 (matched images) of the resultant image is calculated and added, and the largest number is selected, so that the matching result ( 15) can be obtained. In this way, since the detection of the pattern and matching does not have to be an analog comparison by the binarization, it can be performed very simply.

As described above, only attitude information as many as the number of patterns stored in the matching pattern DB can be estimated. However, since the processing is very simple, the processing can be performed sufficiently in real time even on hardware having low computing power.

In the structure of FIG. 1, although the pattern prescribed | regulated to the matching pattern DB was stored, you may generate a template image of a user individually using the image acquired from the video camera initially.

FIG. 10 is a configuration diagram of a specific partial pose extraction in the case of creating a matching pattern from the output of the video capture unit 2. FIG. In the figure, a matching pattern generation unit 16 for generating a template image of a posture pattern from the photographed image is provided.

Next, the operation will be described with reference to FIG. FIG. 11 is a diagram showing a result of generating a template image based on the image after binarizing an image captured by the image capturing unit 2 in a normal posture.

Here, the image initially captured by the image capturing unit 2 is viewed in a normal posture (a posture facing the camera), or a user is photographed in a normal posture to obtain an image of the normal posture. .

The image 17 obtained by binarizing the image acquired as described above by the matching image generation unit 3 is, for example, affine transformed in the matching template generation unit 16 by using an affine transformation. It produces a scorching image, an image of shaking the neck from side to side, and an image of shaking up and down.

The affine transformation can be expressed as a matrix represented by the following equation.

(식5)

(Eq. 5)

The affine transformation matrix shown in (Equation 5) is prepared corresponding to each attitude, the coordinates of which the pixel value of the binary image 17 is 1 are converted in the following (Equation 6), and the pixel values are converted to the coordinates after the conversion. By setting 1 to a pixel value of 0 other than that, an image corresponding to each posture shown in Fig. 11 (18) can be generated.

(식6)

(Eq. 6)

In Equation 6, the original coordinates are X and Y, and the coordinates after the conversion are x and y. In addition, the binary image 17 is made into a plane.

After the generation of the matching pattern 18 as described above, the posture can be estimated by executing the same process as that of FIG. 1.

Moreover, although it takes time, you may make a user take the various postures shown, for example in FIG. 11 (18), and binarize it. That way, you can create a template without affine transformations.

By constructing a short time or real time head pose estimation apparatus as described above, a template suitable for the feature of the user's face is generated, so that matching accuracy can be improved.

In addition, although the estimation of the posture of a head part or a face was demonstrated as a specific part also in the following Example, it is not limited to this as a specific part, It may be other parts, such as a hand, an arm, a foot, or a torso.

(Example 2)

In the above-described first embodiment, the head posture is estimated by matching the matching pattern with the matching image, but the case where the matching means is changed to estimate the head posture from the distribution of the pixel of pixel value 1 of the matching image will be described. .

12 is a diagram showing the configuration of a specific partial attitude estimation apparatus according to the present embodiment. In the figure, a pixel distribution measuring unit 19 for obtaining a pixel distribution of a matching image and estimating a head posture in accordance with the distribution state is provided.

Next, the operation will be described with reference to FIG. FIG. 13 illustrates a map for estimating the head pose according to the pixel distribution.

The pixel of pixel value 1 of a matching image is distributed in the area | regions, such as the map 20, or counts the number of pixels which enter each area | region, and the head posture corresponding to the area | region where the count number is many is made into an estimation result.

In this way, the use of the pixel distribution makes it possible to further simplify the processing, and therefore, there is an effect that the processing can be further shortened even by hardware having low computing power.

In the above embodiment, the specific partial attitude estimation apparatus has been described as being composed of hardware. However, as shown in FIG. 4, the program can be configured to actually prepare a program and execute the program by the processor. Or it is good also as a method comprised by the step which shows the flow of FIG. 2, FIG. 3, FIG.

As described above, according to the present invention, a matching image generation unit for binarizing an input image based on an average image within a predetermined range to specify a component, and a pattern for detecting posture by comparing the obtained binary image with a predetermined template. Since a matching part is provided, there is an effect of suppressing the scale and easily estimating the partial posture.

Claims (9)

A matching image generating unit which binarizes each pixel of the input image from which the specific portion is extracted to a setting threshold value based on an image value within a predetermined coordinate range, and specifies a part whose size is within the setting range to obtain a binary image; The pattern matching unit which detects a posture in contrast with the binary image obtained by the above specification, and a predetermined template. Partial posture estimation device characterized in that it comprises a. The method of claim 1, The matching image generating unit converts an input image into a gray scale image, and binarizes the luminance of the gray scale image by using an average value or a median value of luminance of pixels within a predetermined range with a target pixel as the coordinate center. Partial posture estimation device, characterized in that. The method of claim 1, The pattern matching unit generates a matching image from a plurality of specific input images in advance, and uses the generated matching image as an element of a template. The method of claim 1, And the pattern matching unit is configured to obtain a comparison with each element of the template by a logical product of the pixels. The method of claim 1, The pattern matching unit obtains a pixel having a specific value from the binary image obtained by the matching image generation unit, and estimates the inclination of the specific portion to be detected from the distribution state in the image of those pixels. Attitude estimation device. Binarizing each pixel of the input image from which the specific portion is extracted to a set threshold value based on an image value within a predetermined coordinate range; Removing a portion outside the setting range from the binarized image obtained by binarizing and labeling as an image part within the setting range; Contrasting the binarized image consisting of a group of parts labeled with the image after the deletion and the luminance of the image present in the predetermined template; Partial attitude estimation method comprising a. The method of claim 6, In the step of binarizing, first, the input image is converted into a gray scale image, and the luminance of the converted gray scale image is one of the luminance average value and the image center value of the pixel within a predetermined range with the target pixel as the coordinate center. Selecting and binarizing as a digitization threshold value, the specific partial attitude estimation method characterized by the above-mentioned. The method of claim 6, And said collating step obtains a collation value by the logical AND of the corresponding pixels in the coordinate values of the binarized image and the image of the template. Binarizing each pixel of the input image from which the calculator is capable of reading and extracting a specific portion to a set threshold value based on an image value within a predetermined coordinate range; Removing a portion outside the setting range from the binarized image obtained by binarizing and labeling as an image part within the setting range; Contrasting the binarized image consisting of a group of parts labeled with the image after the deletion and the luminance of the image present in the predetermined template; And a recording medium for recording a specific partial posture estimation program.

Images