JP2004110543A - Face image processing unit and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a face image processing unit and program for accurately detecting the center line of the face even if light is not equally applied to the whole of the face. <P>SOLUTION: In the case of setting a window 20 at the left end of a cross edge image, the line bisecting the transverse segment of the window 20 is a straight line X=K. In the case where a horizontal edge point exists in both of coordinate positions line symmetrical with respect to the straight line, a preset point is added. The sum of points acquired by performing the operation for all of pixels in the window 20 expresses the degree of centerline-likeness of the straight line X=K. With the window 20 moved little by little rightward to the right end of the horizontal edge image, the score about the straight line determined by each window position is found. The straight line having the maximum score is selected as a centerline candidate of the face. When the score in the centerline candidate is larger than a preset threshold, the centerline candidate is determined to be the centerline. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a face image processing apparatus and a program, and more particularly to a face image processing apparatus and a program that can accurately detect a center line of a face based on a horizontal edge image.
[0002]
[Prior art]
Conventionally, there has been proposed a method of detecting a center line of a face from input grayscale face images using grayscale information. In this method, the center line is calculated based on the fact that the grayscale values of the left and right pixels are closest, that is, the symmetry is high (see Patent Document 1).
[0003]
[Patent Document 1]
JP-A-9-171560
[Problems to be solved by the invention]
However, when photographing a face image indoors, since the light can be evenly applied to the entire face, the gray value of a symmetrical point on the face becomes a close value, but when considering the vehicle interior environment, There is a case where light is not evenly applied to the entire face. That is, when the sunlight shines from the side of the face, only the portion where the sunlight shines becomes bright. Therefore, there has been a problem that the difference between the grayscale values of the symmetric coordinate points becomes large, and a situation occurs in which the evaluation of the face symmetry cannot be performed accurately.
[0005]
SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and an object of the present invention is to provide a face image processing apparatus capable of accurately detecting the center line of a face even when light is not uniformly applied to the entire face. It is to provide an apparatus and a program.
[0006]
[Means for Solving the Problems]
In order to achieve the above object, a face image processing apparatus of the present invention comprises: an image input unit for inputting a face image; a horizontal edge image generation unit for generating a horizontal edge image from the input face image; And a center line detecting means for detecting a center line of the face based on the image.
[0007]
In the face image processing device of the present invention, when a face image is input from the image input means, the horizontal edge image generation means generates a horizontal edge image from the input face image. Then, the center line detecting means detects the center line of the face based on the generated horizontal edge image. A face image often has a portion where a horizontal edge occurs, such as eyes, eyebrows, and glasses. Therefore, by utilizing the symmetry of the horizontal edge image, the center line of the face can be accurately detected even when the entire face is not uniformly irradiated with light.
[0008]
In the above-mentioned face image processing apparatus, the center line detecting means sets a plurality of center line candidates in the horizontal edge image, and votes if both the symmetrical pixels have edge points with respect to each center line candidate. It can be configured to detect a high number of center line candidates as the center line of the face.
[0009]
In addition, a plurality of center line candidates are set in the horizontal edge image of the area near the center line detected by the center line detecting means, and voting is performed if there is no edge point in both symmetrical pixels for each center line candidate. A center line re-detecting means for re-detecting a center line candidate having the highest number of votes as a face center line may be further provided.
[0010]
Further, the above-described face image processing apparatus includes a vertical edge image generating unit that generates a vertical edge image from an input face image, and both ends detecting unit that detects left and right ends of the face based on the generated vertical edge image, The image processing apparatus may further include face direction detecting means for detecting a face direction from both left and right ends of the detected face and the center line of the face.
[0011]
The above-described face image processing is performed by using a computer as a horizontal edge image generation unit that generates a horizontal edge image from an input face image, and a center line detection unit that detects a center line of a face based on the generated horizontal edge image. It can be realized by using a program for face image processing to be made to function.
[0012]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
(First Embodiment)
As shown in FIG. 1, the face image processing apparatus according to the present embodiment is connected to a camera 10 for photographing a driver's face, an illumination light source 12 for illuminating the driver's face, a computer 14 for image processing, and a computer 14. Display device 16 is provided. The display device 16 includes a CRT or the like, and displays a binarized image or the like extracted from the face image captured by the camera 10.
[0013]
An image processing computer 14 includes an A / D (analog / digital) converter for converting an image signal input from the camera 10 into a digital signal, an image memory for storing image data, in addition to the CPU, ROM, and RAM. And a control bus and a data bus connecting them. The ROM stores a program for executing a later-described center line detection processing routine.
[0014]
Next, a center line detection processing routine according to the present embodiment will be described with reference to FIG. An image signal input from the camera 10 is converted into digital image data by an A / D converter and stored in an image memory.
[0015]
In step 100, the face image stored in the image memory is fetched, and in step 102, a horizontal edge is detected from the fetched face image using a predetermined threshold. An example of detecting a horizontal edge will be described using the face image (gray image) shown in FIG. 3A as an example. First, the difference between the luminance values of the pixels adjacent in the vertical direction is compared with a preset threshold, and the output value of a pixel (horizontal edge point) whose luminance difference exceeds the threshold is set to 0, and the difference between the luminance values is set to the threshold. Labeling is performed so that the output value of a pixel that does not exceed 1 is set, and the face image is converted into a binary image with labels 0 and 1. Thereby, as shown in FIG. 3B, a black-and-white horizontal edge image in which the horizontal edge points are displayed in white is obtained.
[0016]
In step 104, a center line candidate of the face is selected using the obtained horizontal edge image. As shown in FIG. 4, it is assumed that a rectangular horizontal edge image having vertical and horizontal sizes of Y1 and X1 has windows (rectangular portions hatched) with vertical and horizontal sizes of Y1, 2K + 1, respectively. Assuming that the lower left end of the horizontal edge image is the origin (0, 0) of the XY coordinates, when the window 20 is set at the left end of the horizontal edge image, the line that bisects the horizontal line segment of the window 20 is X = K straight line.
[0017]
If there are lateral edge points at both coordinate positions symmetrical with respect to this straight line, a preset score is added. For example, in the case where each of the pixels existing at positions separated by a distance K1 to the left and right from the straight line of X = K, such as pixel a and pixel b, both become horizontal edge points, the points are added. On the other hand, if there are no horizontal edge points at both of the line-symmetric coordinate positions, that is, if there is only one of them and neither of them exists, no points are added. The sum of the scores obtained by performing this operation for all the pixels in the window 20 indicates the degree of the likelihood of the center line of the straight line of X = K. In other words, the higher the score, the higher the symmetry of the edge portion with respect to the straight line, and the higher the possibility of being the center line of the horizontal edge image.
[0018]
While moving the window 20 little by little rightward to the right end of the horizontal edge image, a score is obtained for a straight line determined by each window position. Then, the straight line having the highest score is selected as a center line candidate of the face, and the X coordinate and the score of the center line candidate are obtained.
[0019]
In step 106, it is determined whether or not the selected center line candidate is appropriate as the center line. If the score of the center line candidate is larger than a preset threshold, the center line candidate is determined to be the center line, and in step 108, the center line candidate is determined to be the center line. On the other hand, if the score is smaller than the preset threshold value, it is determined that the detection of the center line is impossible, and the process returns to step 100 to perform the detection of the center line again. After the center line is determined in step 108, the process returns to step 100 to fetch the next face image and detect the center line for the next face image.
[0020]
As described above, in the present embodiment, the center line is determined so that the symmetry of the edge portion of the horizontal edge image is the highest. Face images often have horizontal edges, such as eyes, eyebrows, and eyeglasses.Therefore, by utilizing the symmetry of the horizontal edges, even if the entire face is not evenly illuminated, the center line of the face can be detected. It can be detected accurately.
[0021]
(Second embodiment)
In the first embodiment, the center line of the face is detected using only the horizontal edge image, but in the present embodiment, the center line of the face is detected using both the horizontal edge image and the vertical edge image. The center line detection processing routine in this case will be described with reference to FIG.
[0022]
In step 200, the face image stored in the image memory is fetched, and in step 202, a horizontal edge is detected for the fetched face image in the same manner as in step 102. Thereby, for example, as shown in FIG. 6A, a black and white horizontal edge image is obtained. In step 204, a vertical edge is detected for the captured face image. As in the case of detecting the horizontal edge, the difference between the luminance values of the pixels adjacent to the face image in the horizontal direction is compared with a preset threshold value, and the output of the pixel (vertical edge point) whose luminance value difference exceeds the threshold value is output. Labeling is performed so that the output value of a pixel whose difference in luminance value does not exceed the threshold value is 0, and the face image is converted into a binary image labeled with labels 0 and 1 by a vertical edge. Is detected. Thereby, for example, as shown in FIG. 6B, a black and white vertical edge image in which the vertical edge points are displayed in white is obtained.
[0023]
In step 206, the first center line candidate (center line candidate 1) is selected using the obtained horizontal edge image in the same manner as in step 104. Next, in step 208, a second center line candidate is selected in a region near the first center line candidate using the obtained vertical edge image. As shown in FIG. 6B, a rectangular window (neighboring region) 30 smaller than the window 20 is set near the first center line candidate. If there are vertical edge points at both coordinate positions that are line-symmetric with respect to the line that bisects the horizontal line segment of the window 30, a preset score is added. This operation is performed for all the pixels in the window 30. While moving the window 30 in the horizontal direction little by little within a predetermined range near the first center line candidate, a score is obtained for a straight line determined by each window position. Then, the straight line having the highest score is selected as the second center line candidate, and the X coordinate and the score of the second center line candidate are obtained.
[0024]
In step 210, it is determined whether the selected second center line candidate is appropriate as the center line by combining the following three conditions.
(1) The score of the first center line candidate is larger than a preset first threshold value.
(2) The score of the second center line candidate is larger than a preset second threshold value.
(3) The difference between the coordinate positions of the first center line candidate and the second center line candidate is smaller than a third threshold value set in advance.
[0025]
If all of the conditions (1) to (3) are satisfied, or if the conditions (1) and (2) are satisfied, the second center line candidate is determined to be the center line, and the second center line candidate is determined as the center line in step 212. Is determined as the center line. On the other hand, if the condition of (1) or (2) is not satisfied, it is determined that the detection of the center line is impossible, and the process returns to step 200 to perform the detection of the center line again. After the center line is determined in step 212, the process returns to step 200 to capture the next face image and detect the center line for the next face image.
[0026]
As described above, the face image has many portions where horizontal edges occur, but as can be seen from FIG. 6B, the number of vertical edge points increases near the center of the face due to the nose. Therefore, after detecting the first center line candidate so that the symmetry of the edge portion of the horizontal edge image is the highest, the symmetry of the edge portion of the vertical edge image is the closest in the vicinity of the first center line candidate. By selecting the second center line candidate so as to be higher, the center line of the face can be detected more accurately.
[0027]
In the above example, after the first center line candidate is detected using the obtained horizontal edge image, the second center line candidate is selected using the vertical edge image. After detecting the first center line candidate from the symmetry of the edge portions representing the left and right ends of the face outline, the second center line candidate may be selected using the horizontal edge image. Further, in this case, the direction of the face (for example, facing diagonally forward) can be detected from the difference between the coordinate positions of the first center line candidate and the second center line candidate.
[0028]
(Third embodiment)
In the present embodiment, as shown in FIG. 8A, an example of detecting a center line of a face when a driver wears a mask and the nose is covered with the mask will be described. The center line detection processing routine in this case will be described with reference to FIG.
[0029]
In step 300, the face image stored in the image memory is fetched, and in step 302, a horizontal edge is detected for the fetched face image in the same manner as in step 102. As a result, a black-and-white horizontal edge image is obtained as shown in FIG. In step 304, as in step 104, a first center line candidate is selected by focusing on the symmetry of the edge portion of the obtained horizontal edge image.
[0030]
Next, in step 306, focusing on the symmetry of the non-edge portion (the portion where no edge point exists) of the horizontal edge image, the second edge region is positioned in the vicinity of the first center line candidate (center line candidate 1). Center line candidates are selected. As shown in FIG. 8B, a rectangular window (neighboring region) 40 smaller than the window 20 is set near the first center line candidate. If there are no horizontal edge points at both coordinate positions that are line-symmetric with respect to the line that bisects the horizontal line segment of the window 40, a preset score is added. This is different from the first to third embodiments in which points are added when edge points exist at both line-symmetric coordinate positions. In other words, the higher the score, the higher the symmetry of the non-edge portion with respect to the straight line, and the higher the possibility of being the center line of the horizontal edge image.
[0031]
This operation is performed for all the pixels in the window 40. While moving the window 40 in the horizontal direction little by little within a predetermined range near the first center line candidate, a score is obtained for a straight line determined by each window position. Then, the straight line having the highest score is selected as the second center line candidate, and the X coordinate and the score of the second center line candidate are obtained.
[0032]
In step 308, the following three conditions are combined to determine whether the selected second center line candidate is appropriate as a center line.
(1) The score of the first center line candidate is larger than a preset first threshold value.
(2) The score of the second center line candidate is larger than a preset second threshold value.
(3) The difference between the coordinate positions of the first center line candidate and the second center line candidate is smaller than a third threshold value set in advance.
[0033]
If all the conditions (1) to (3) are satisfied, or if the conditions (1) and (2) are satisfied, the second center line candidate is determined to be a center line, and the second center line candidate is determined in step 310. Is determined as the center line. On the other hand, if the condition (1) or (2) is not satisfied, it is determined that the detection of the center line is impossible, and the process returns to step 300 to perform the detection of the center line again. After the center line is determined in step 310, the process returns to step 300 to fetch the next face image and detect the center line for the next face image.
[0034]
As described above, the face image has many portions where horizontal edges occur, but as can be seen from FIG. 8B, the number of horizontal edge points decreases near the center of the face. Further, when the driver wears the mask, it is not possible to extract a vertical edge near the center of the face. Therefore, after detecting the first center line candidate so that the symmetry of the edge portion of the horizontal edge image is the highest, the non-edge portion of the horizontal edge image has the highest symmetry in the vicinity of the first center line candidate. By selecting the second center line candidate so as to be higher, the center line of the face can be detected more accurately.
[0035]
(Modification)
In the first to third embodiments, the calculation is performed for all the line-symmetric coordinate positions in the set window. However, the range in which the line-symmetric coordinate position is searched may be limited. For example, as shown in FIG. 9, by limiting the search area 50 to an area where face parts such as eyes and nose are present, the features of the face parts can be reliably captured, and the detection accuracy of the center line candidate is reduced. improves.
[0036]
In the first to third embodiments, the points are added based on whether or not the edge points exist at both of the line-symmetric coordinate positions. However, the edge points are added to the coordinate positions adjacent to one of the coordinate positions. Even when points exist, points may be added. For example, as shown in FIG. 10, when the horizontal bisector of the window 20 is a straight line of X = K, when each of the pixels a and b on the same Y coordinate is a horizontal edge point, the score is In addition to the above, the points are also added when the pixel a becomes the horizontal edge point and the pixel c or the pixel d adjacent in the vertical direction to the pixel b becomes the horizontal edge point. As described above, by searching for the corresponding pixel in the vertical direction within the predetermined range, it is possible to cope with the inclination of the face to some extent, and the detection accuracy of the center line candidate is improved.
[0037]
Further, in the first to third embodiments, an example in which a vertical edge or a horizontal edge is detected has been described. However, an edge is detected using a Sobel filter that detects the edge by dividing the edge into a vertical component and a horizontal component. By monitoring the angle, edges in various directions can be detected. For example, as shown in FIG. 11, instead of using a vertical edge image and a horizontal edge image, a left 45 ° edge image and a right 45 ° edge image can be used. When an edge image other than the vertical edge image and the horizontal edge image is used, the points are added based on whether or not edge points in the same angle direction exist at both line-symmetric coordinate positions.
[0038]
【The invention's effect】
As described above, according to the present invention, there is an effect that the center line of the face can be accurately detected even when the entire face is not uniformly irradiated with light.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of a face image processing device according to the present embodiment.
FIG. 2 is a flowchart illustrating a center line detection processing routine according to the first embodiment.
3A is a diagram showing an original face image, and FIG. 3B is a horizontal edge image obtained by binarizing (A).
FIG. 4 is a diagram in which a rectangular window is set for a horizontal edge image.
FIG. 5 is a flowchart illustrating a center line detection processing routine according to a second embodiment.
6A is a horizontal edge image, and FIG. 6B is a vertical edge image.
FIG. 7 is a flowchart illustrating a center line detection processing routine according to a third embodiment.
8A is a diagram showing an original image of a face image of a driver wearing a mask, and FIG. 8B is a horizontal edge image obtained by binarizing (A).
FIG. 9 is a diagram that specifies a range for searching for a line-symmetric coordinate position in a set window.
FIG. 10 is a diagram illustrating an example in which a pixel corresponding to one coordinate position is searched for in a vertical direction within a predetermined range.
FIG. 11 is a diagram illustrating an example of replacement of an edge direction.
[Explanation of symbols]
Reference Signs List 10 Camera 12 Illumination light source 14 Computer 16 for image processing Display device 20, 30, 40 Window 50 Search area

Claims (5)

Image input means for inputting a face image;
A horizontal edge image generating means for generating a horizontal edge image from the input face image,
Center line detecting means for detecting a center line of the face based on the generated horizontal edge image;
A face image processing device comprising: The center line detection means sets a plurality of center line candidates in the horizontal edge image, and votes if both the symmetric pixels have an edge point with respect to each center line candidate. The face image processing apparatus according to claim 1, wherein the face image is detected as a center line of the face. A plurality of center line candidates are set in the horizontal edge image of the area near the center line detected by the center line detecting means, and voting is performed if there is no edge point in both left-right symmetric pixels for each center line candidate. 3. The face image processing apparatus according to claim 1, further comprising a center line re-detecting unit that re-detects a center line candidate having the highest number of votes as a center line of the face. Vertical edge image generating means for generating a vertical edge image from the input face image,
Both ends detecting means for detecting both left and right ends of the face based on the generated vertical edge image,
Face direction detection means for detecting the direction of the face from both the left and right ends of the detected face and the center line of the face,
The face image processing apparatus according to any one of claims 1 to 3, further comprising: A face image processing unit for causing a computer to function as a side edge image generation unit that generates a side edge image from an input face image and a center line detection unit that detects a center line of a face based on the generated side edge image program.

Images