JPH09171560A - Device for detecting face inclination - Google Patents

Abstract

PROBLEM TO BE SOLVED: To realize a device whereby face inclination is corrected even when eyes are closed. SOLUTION: A picture rotating means 4 rotates picture data of the face by angle which is designated by a rotation angle designating means. A rotation picture storage means 5 stores picture data of the face, which is rotated by the picture rotating means 4. A center line calculating means 6 calcualtes a center line based on face symmetry from the rotation picture which is stored in the rotation picture storage means 5. A inclination deciding means 7 monitors a similar degree of the center line which is outputted from the center line calculating means 6 as against the respective kinds of angles which are decided by the rotation angle designating means, detects angle with the angle of the highest similar degree of the center line and decides face inclination.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a face inclination detecting device for detecting a face inclination by performing face image recognition.

[0002]

2. Description of the Related Art Attempts have been made such that a face is photographed by a video camera and the like, and the photographed face image is used for personal identification, or the movement of the lips is extracted to perform machine lip reading.

For example, when personal identification is performed using a front image of a face, first, an image of the face of a person to be identified and the person's name are registered in advance in a dictionary. In the case of recognition, a face area is found from the input image to create a face image, and it is found by matching whether this face image exists in the dictionary. If it exists, the person name in the dictionary is output as a result.

However, when no condition is given to the person who is the subject when the face image is taken, the face may be tilted depending on the person. When the face is tilted in this way, identification may fail in matching processing or the like. In order to solve this problem, for example, as described in "Person Identification by Integration of Side Profile Image and Front Face Image", IEICE Technical Report, IE93-76 (1993-11), the pre-processing of the matching process is performed. , The image is rotated so that the centers of both eyes are obtained and the respective positions are horizontal.

[0005]

However, in the above-mentioned conventional technique, the center of the eyes cannot be detected when the eyes are closed due to blinking or the like, and the center of the eyes cannot be detected. In some cases, rotation processing could not be performed.

From this point of view, it is desired to realize a face inclination detecting device capable of correcting the inclination of the face even when the eyes are closed or both eyes are covered with sunglasses or the like. It was rare.

[0007]

The present invention employs the following structure to solve the above-mentioned problems. <Structure of Claim 1> An image input device for outputting image data of a face image, an image storage means for storing image data of the face output by the image input device, and an angle changing within a predetermined range. Rotation angle designating means for outputting the value, image rotation means for rotating the image data stored in the image storage means based on the angle output by the rotation angle designating means,
Rotation image storage means for storing the image output by the image rotation means, and centerline calculation means for determining the centerline of the face from the rotation image stored in the rotation image storage means and outputting the degree of centerline likelihood at that time. For each angle designated by the rotation angle designating means, the degree of centerline-likeness output by the centerline calculating means is monitored, and the inclination of the face is determined by detecting the angle having the highest degree of centerline-likeness. It is characterized in that it is provided with a tilt determining means.

<Structure of Claim 2> The centerline calculating means in the invention of Claim 1 determines the centerline based on the left-right symmetry of the face.

<Explanation of Claims 1 and 2> In the inventions of Claims 1 and 2, the input face image is rotated within a certain angle range, the center line of the face is obtained for each image, and the most center line is obtained. It is characterized in that the angle when there is a high possibility that is is determined as the inclination of the face.

In order to perform such processing, first, the image data of the face is stored in the image storage means, and this image data is rotated by the image rotation means. At this time, the rotation angle is designated by the rotation angle designating means. The rotated image rotated by the image rotating means is stored in the rotated image storage means. The centerline calculation means determines the centerline of the face from the rotation image stored in the rotation image storage means, and outputs the degree of the centerline likelihood at that time. The inclination determining means monitors the degree of centerline-likeness output by the centerline calculating means for each angle designated by the rotation angle designating means, and detects the angle having the highest degree of centerline-likeness. That is, the most probable angle of the center line is obtained. Then, the angle is determined as the inclination of the face.

As a method of calculating the center line of the face in the second aspect of the present invention, for example, X is applied to face image data.
= A line is set, the distance between pixel values of pixels that are line-symmetric with respect to this line is calculated, and the sum of the distances (dist)
Is a measure of symmetry at X = a. Then, such processing is performed in a certain area to find the minimum value distmin of dist, and X at this time is output as the center line. Also, the minimum value distmin of dist is output as the degree of centerline likelihood.

Therefore, as compared with the case where face image matching is performed to detect the face inclination, the face inclination can be detected even when the eyes are closed or sunglasses are worn. You can

[0013]

BEST MODE FOR CARRYING OUT THE INVENTION The face inclination detecting apparatus of the present invention rotates an input face image within a certain angle range, obtains the center line of the face for each image, and the face center line may be the most center line. It is characterized in that the angle in the case of high is determined as the inclination of the face.

Embodiments of the present invention will be described in detail below with reference to the drawings. FIG. 1 is a configuration diagram showing a specific example of the face inclination detection device of the present invention. As shown in the figure, the device of the specific example includes an image input device 1, an image storage unit 2, a rotation angle designation unit 3, an image rotation unit 4, a rotated image storage unit 5, a center line calculation unit 6, an inclination determination unit 7, and an output. It is composed of terminals 8.

The functions and operations of each of the above components will be described in detail below in order. In this specific example, the processing target is a color image.

The image input device 1 inputs a face image and A /
It has a function of performing D conversion and outputting digital image data, and is used for inputting a face image. This device is composed of a video camera, a VTR and the like and an A / D converter. The image data of the face may be input via a communication line or the like. The image input device 1 performs A / D conversion on an input image signal at a certain fixed cycle. Here, the width and height of the input image are W and H. The image input device 1 calculates the analog value at the (X, Y) position of the input image (however,
The upper left of the image is (0, 0) and the lower right is (W, H)) A
/ D conversion is performed and three digital values of red, green and blue are output. These output values are R (X, Y) and G (X,
Y) and B (X, Y). However, R, G, and B take a value of 0 or more and 255 or less. Then, these output values are output to the image storage means 2.

The image storage means 2 stores the red, green, and blue values output by the image input device 1, R (X, Y), G (X, Y), and B.
It has a function of storing (X, Y). The image storage means 2 is composed of a semiconductor memory and an auxiliary storage device such as a disk.

The values of R, G and B stored in the image storage means 2 can be called by designating the coordinates (X, Y). The values of R, G, B stored in the image storage means 2 are
Used by the image rotation means 4. The rotation angle designating means 3 is
When an image is input to the image storage means 2, it has a function of changing the angle within a certain range and outputting the angle to the image rotating means 4 and the inclination determining means 7 each time. Here, assuming that the width of the angle change is SH, the output angle S is changed within the following range. -SH≤S≤SH

The procedure of actual processing is shown below. However,
ST is the step size of the output angle. 1. S = -SH 2. Output S. 3. S = S + ST 4. If S> SH, the process is terminated, and if the condition is not satisfied, 2. Return to The width SH of the angle change and the step size ST are appropriately determined according to the subject, the shooting conditions, and the like.

The image rotation means 4 has a function of rotating the input image stored in the image storage means 2 in accordance with the angle S designated by the rotation angle designation means 3.

FIG. 2 is an explanatory view of the rotating operation of the image rotating means 4. In the figure, a rectangle represented by a width W and a height H is an input image, and a rectangle obtained by rotating the input image clockwise by an angle S is an image obtained by rotating the input image by the angle S. In addition, in the figure, the portion shown in black is an area that disappears from the input image when rotated.

FIG. 3 shows an operation flowchart of the image rotating means 4. First, in step 101, the center coordinates (X0, Y0) when the image is rotated are calculated. That is, X0 = (W / 2) and Y0 = (H / 2) (however, the part after the decimal point is discarded).

In the following steps 102 to 112, Y is changed from 0 to H, and X is changed from 0 to W for each Y. here,
Coordinates (X, Y) are coordinate values in the rotated image.

In step 102, the Y coordinate is initialized (Y = 0), and in step 103, the X coordinate is initialized (X = 0). In the next step 104, it is determined at which coordinate the coordinate (X, Y) was located in the image before rotation. The coordinates (DX, DY) before rotation are obtained by rotating the coordinates (X, Y) by an angle (-SH) around the coordinates (X0, Y0) as shown in the equation of step 104. You can

Next, since the coordinates (DX, DY) obtained by the equation of step 104 may be projected from the input image, it is determined in steps 105 and 106 whether or not they are projected. That is, the X coordinate is checked in step 105, and the Y coordinate is checked in step 106.

Next, when it is determined in the above steps 105 and 106 that the value is within the range, step 10
By 7, the red, green, and coordinates of the coordinates (DX, DY) in the input image,
The values of the blue component (R, G, B) are converted to the coordinates (D
X, DY) red, green, and blue component values (DR, DG, DB)
Substitute for Here, the values of the red, green, and blue components of the rotated image are DR (X, Y), DG (X, Y), DB (X,
Y) is assigned to the array. On the other hand, step 105,
If it is determined in 106 that the value is out of the range, the process proceeds to step 108, and 0 is assigned to DR, DG, and DB. That is, in this case, the region disappears due to rotation in FIG. 2, and this region is black (all three values of red, green, and blue components are 0).

In step 109, it is determined whether X has become the width W. If X is less than W, the process proceeds to step 110, 1 is added to X, and the process returns to step 104 to repeat the above process. . When X becomes W, the process is moved to the next step 111.

In step 111, it is determined whether Y has reached the height H. If Y is less than H, step 1
In step 12, Y is incremented by 1, and the process returns to step 103. Then, when Y becomes H, this processing is ended.

Thus, the red, green, and blue values of the coordinates of the image rotated by the image rotating means 4 are DR (X, Y), DG.
(X, Y) and DB (X, Y) are substituted. Then, the DR, DG, and DB output from the image rotation means 4 are output to the rotated image storage means 5.

The rotated image storage means 5 stores the red, green, and blue values output by the image rotation means 4, DR (X, Y), and DG (X,
Y) and DB (X, Y) are stored. The image rotation means 4 is composed of a semiconductor memory, an auxiliary storage device such as a disk, or the like.

These DR, DG and DB values are called by designating the coordinates (X, Y). The values of DR, DG, and DB stored in the image rotation means 4 are used in the centerline calculation means 6.

Next, the center line calculating means 6 will be described. The center line calculation means 6 has a function of calculating the center line of the face based on the image data stored in the rotated image storage means 5. In the center line calculation process, the X coordinate having the highest left-right symmetry among the image data stored in the rotated image storage means 5 is obtained. In addition, the value (dis
t) is also output as a result.

FIG. 4 shows an operation flowchart of the center line calculating means 6. First, in this processing, since an area that is not affected by the image rotation processing is used, the range of the image is determined in step 201.

FIG. 5 is an explanatory diagram of a region used by the center line calculating means 6. In step 201, the range of the image used in the centerline calculation process is determined. First, (X0,
Y0) is the center coordinate for rotating the image and is set to half of the width W and the height H, respectively. In addition, coordinates (X1, Y
1) and (X2, Y2) are the upper left coordinates and the lower right coordinates of the area used in the center line calculation process. Here, these coordinates have the following values. That is, X1 is an angle (SH) centered on the coordinate (0, 0) and the coordinate (X0, Y0).
It is the value of the x-coordinate of the coordinate obtained by rotating only. Also,
Y1 is the value of the y coordinate of the coordinate obtained by rotating the coordinate (0, 0) about the coordinate (X0, Y0) by the angle (-SH). Further, X2 is the coordinate (W, H) with the coordinate (X0,
The value of the x coordinate of the coordinate obtained by rotating the Y0) about the angle (SH), and Y2 is the coordinate (W, H) of the coordinate (X0, Y).
It is the value of the y coordinate of the coordinate obtained by rotating the image around 0) by an angle (-SH).

As described above, the range of the image used in the center line calculation processing is a rectangular area having (X1, Y1) and (X2, Y2) as diagonal lines. Then, also in the processing described below, only this rectangular area is symmetrical to the processing.

FIG. 6 shows an explanatory diagram of the center line calculation processing.
As shown by the thick line in FIG. 6, in the center line calculation processing, a straight line X = a is set for the input image, and a rectangle of width (CH × 2) is considered centering on this straight line. Within this rectangle, the distance between pixel values of pixels that are line-symmetric with respect to X = a is obtained, and the sum (dist) of the distances is used as a measure of symmetry at X = a. Such processing is performed by changing the value of X to (X1 + C
H) to (X2-CH), and the minimum value of dist (d
istmin), and let X at this time be the center line. The value of CH indicating the width of the area is set to a value smaller than (X2-X1) / 2. However, this value is appropriately determined according to the size and position of the face shown in the input image. Hereinafter, these operations will be described in more detail with reference to the flowchart of FIG.

First, in step 202, X = (X
1 + CH) is substituted (initialization of X). Moreover, infinity is substituted for the minimum value distmin of the variable dist for calculating the left-right symmetry. Next, in step 203, the y coordinate is initialized (Y = Y1). Also, dist = 0.
Further, in step 204, i is initialized by substituting 1 for i. Note that i is a value indicating the distance (x direction) from the center line. Then, in step 205, the center line (x = X)
A coordinate x is obtained which is line-symmetrical with respect to (where the distance from the center line is i). Let the obtained coordinate values be X0 and X1, respectively.

Then, in step 206, the coordinates (X0,
How similar the Y) and the coordinates (X1, Y) are to each other is calculated by distance calculation. That is, in this step, the square of the difference between the red, green, and blue component values at each coordinate is added to obtain the square root. The distance obtained here is dist
Is added to. In the next step 207, it is determined whether i is CH. If i is less than CH, 1 is added to i in step 208, and the process returns to step 205. If i is CH, the process moves to step 209.

In step 209, it is determined whether Y is Y2. If Y is less than Y2, 1 is added to Y in step 210 and the process returns to step 204. If Y is Y2, the process proceeds to step 211. In this way, the series of processing in steps 203 to 210 described above makes it possible to obtain the left-right symmetry dist when the center line is assumed to be (x = X).

Next, at step 211, it is judged whether the obtained dist is smaller than the minimum value distmin of dist up to that point. If it is smaller than the minimum value, the current X and dist are stored in step 212 (KX =
X, distmin = dist). If it is greater than or equal to the minimum value, the process proceeds to step 213.

In step 213, X is (X2-CH)
Is determined. If this condition is not satisfied, 1 is added to X in step 214, and step 203
Return to If the conditions are met, step 21
Processing is transferred to 5.

At step 215, the minimum value distmi of dist
n and the x-coordinate KX at that time are output, and the process ends. Here, x = KX is the center line calculated by the above processing, and distmin is the degree of the center line likelihood at that time.
Then, the minimum value distmin of the distance and the value (KX) of the center line obtained by this processing are output to the inclination determining means 7.

Next, the inclination determining means 7 will be described.
The processing of the inclination determining means 7 finds the minimum value in the distmin output by the centerline calculating means 6 with respect to the angle designated by the rotation angle designating means 3, and judges the angle at that time as the inclination of the face. , That angle is output.

FIG. 7 shows an example of the result output by the center line calculating means 6. However, in the rotation angle designating means 3,
This is the result when SH = 4 and ST = 1 are set. In this example, when the angle S = 2, distmin takes the minimum value 77, so S = 2 is output as a result. If necessary, the value of the center line at that time is also output.

The specific example has been described above, but the following modifications are possible. Although a color image was used as the input image, a grayscale (black and white) image is used. The processing amount can be reduced by outputting only the vicinity of the rotation angle of the immediately preceding frame by the rotation angle designating means 3. The processing amount can be reduced by searching only the vicinity of the center line of the immediately preceding frame with the center line calculation means 6. In this specific example, the image in which the inclination of the face is corrected is not output, but by outputting the angle determined by the inclination determining unit 7 to the image rotating unit 4, it is possible to obtain the image in which the inclination is corrected. .

<Effect> The inclination of the face can be calculated by the above specific example, the inclination of the face image can be corrected based on the calculation result, and the processing using the face image (personal identification, machine reading, etc.) can be performed. Performance improvement can be expected. Further, it is possible to relax the constraint on the inclination of the face when the face image is captured.

[Brief description of the drawings]

FIG. 1 is a block diagram showing the configuration of a specific example of a face inclination detection device of the present invention.

FIG. 2 is an explanatory diagram of a rotating operation of an image rotating means in the face inclination detecting device of the present invention.

FIG. 3 is an operation flowchart of the image rotation means in the face inclination detection device of the present invention.

FIG. 4 is an operation flowchart of a centerline calculation means in the face inclination detection device of the present invention.

FIG. 5 is an explanatory diagram of a region used by a centerline calculating unit in the face inclination detection device of the present invention.

FIG. 6 is an explanatory diagram of a centerline calculation unit in the face inclination detection device of the present invention.

FIG. 7 is an explanatory diagram showing an example of the output result of the centerline calculation means in the face inclination detection device of the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 image input device 2 image storage means 3 rotation angle designation means 4 image rotation means 5 rotation image storage means 6 centerline calculation means 7 tilt determination means

Claims (2)

[Claims] 1. An image input device for outputting image data of a face image; an image storage means for storing image data of the face output by the image input device; and an angle varying within a predetermined range. A rotation angle designating unit that outputs a value, an image rotation unit that rotates the image data stored in the image storage unit based on the angle output by the rotation angle designating unit, and an image output by the image rotation unit. A rotation image storage unit for storing, a center line calculation unit for determining the center line of the face from the rotation image stored in the rotation image storage unit, and outputting the degree of center line likeness at that time, and the rotation angle designating unit. For each designated angle, the degree of centerline-likeness output by the centerline calculating means is monitored, and the inclination of the face is determined by detecting the angle with the highest degree of centerline-likeness. An apparatus for detecting the inclination of a face, characterized by comprising inclination determining means. 2. The face inclination detecting apparatus according to claim 1, wherein the centerline calculating means determines the centerline based on the left-right symmetry of the face.