JP2009086705A - Method for detecting eye position - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To automatically detect eye positions using a simple technique from an original image that includes a plurality of face images within one image. <P>SOLUTION: A method for detecting eye positions from the original image comprises: a process A including (A1) creating a blurred image obtained by blurring of the original image in two intersecting directions, and determining the position of an intersection of edges in the blurring directions, (A2) creating an image for face area detection where the gradation of the original image is broken, and (A3) in the image for face area detection, detecting face area candidates from a rectangular area one of the apexes of which is the intersection of the edges in the blurring directions; and a process B including (B1) creating a plurality of images for eye position detection with varying brightness for each of the face area candidates, (B2) as the images for eye position detection fade in from a faded-out condition, detecting the block areas of pixels that appear gradually in the images, (B3) selecting those of the detected block areas of pixels that have appeared in pairs as eye position candidates, and (B4) specifying the eye position candidates as eye positions based on their frequencies of appearance. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a method and a detection system for automatically detecting an eye position from an image including a face image.

  The position of the eye in the face image can be obtained by overlaying a hairstyle model image on the subject's face image to form a hairstyle simulation image, or by overlaying a makeup image such as lips or eyebrows on the subject's face image. In this case, the image is detected for adjusting the size of the image or aligning the image. The detection of the position of the eyes and the detection of the position of the face area are also performed in face image blindfolding processing, face image frame processing, creation of an image for personal authentication, and the like.

  Conventionally, as a method for detecting an eye position from a face image, a method of detecting a face by extracting a skin color region and further detecting an eye by pattern matching or the like has been used (Non-Patent Document 1, Patent Document 1). ).

2005 5th LSI IP Design Award Winning Paper “Face candidate point detection method that enables fast and reliable face detection” (Nikkei Business Publications) JP 2004-94917 A

  However, the extraction of the skin color area is affected by the lighting environment. Therefore, reliability becomes a problem when an unspecified number of face images taken in various lighting environments are targeted. Further, the pattern matching method has a problem that the amount of calculation becomes enormous.

  In contrast, an object of the present invention is to enable automatic and highly reliable detection of the position of an eye or the position of a face region from an image including a face image.

  The inventors of the present invention (1) create a blurred image in two directions such as horizontal and vertical intersecting from the original image, calculate the edge of the blurred image in the blur direction, and further determine the intersection between them, 2) By crushing the gradation of the original image including the face image, it is preferable to obtain an image that can discriminate between the hair area and the face area but not the face structure, and (3) Considering the intersection of the edges in the blur direction, the face image often exists in a rectangular area having this intersection as a vertex and having a higher brightness than the areas adjacent to the left and right and above, and therefore This rectangular area can be used as a face area candidate. (4) For each face area candidate, when multiple images with varying brightness are created, fade-out images on the high brightness side are sequentially faded to images on the low brightness side. First, (5) In this case, the pixels in the pupil area appear in pairs, and thus the eye position candidates can be determined, and (6) appear in pairs. It has been found that eye position candidates can be identified from the accumulation results of appearance frequencies over all eye position detection images of eye position candidates.

That is, a method for detecting the position of an eye from an original image including a face image,
A. (A1) Create a blurred image blurred in two directions intersecting the original image, obtain the position of the intersection of the edges in the blurred direction in the blurred image,
(A2) Create a face area detection image with the gradation of the original image collapsed,
(A3) In a face area detection image, a rectangular area having an intersection of edges in the blur direction as one vertex and having a higher brightness than the left and right and upper adjacent areas is detected as a face area candidate. Process,
B. (B1) For each face area candidate, create a plurality of eye position detection images with varying brightness,
(B2) As the eye position detection image fades in from a state of fading out at a high brightness to a low brightness, a cluster area of pixels that gradually appears in the eye position detection image is detected,
(B3) Select the clustered areas of the detected pixels that appeared in pairs as eye position candidates,
(B4) specifying the eye position candidate as the eye position based on the appearance frequency over all eye position detection images of each eye position candidate;
A method for detecting the position of an eye having

  Further, the present invention provides a face area position detection method for specifying a face area based on a face area candidate corresponding to the eye position after the eye position is specified by the above-described eye position detection method.

Furthermore, the present invention is an eye position detection system including an acquisition unit for an original image including a face image and a calculation unit, and the calculation unit includes:
A function that creates a blurred image that is blurred in two directions intersecting with the original image, a function that detects edges in the blur direction of the blurred image and obtains the position of the intersection, and a face area detection in which the gradation of the original image is crushed Image creation function, a face area detection image is a rectangular area whose intersection is the intersection of edges in the blur direction and has a higher brightness than the adjacent areas on the left and right and above As a function to create a plurality of images whose brightness has changed as eye position detection images for each face area candidate, as the image fades out from a state of fading out with high brightness to low brightness, A function for detecting a cluster area of pixels that gradually appear in the eye position detection image, a function for selecting a pair of detected pixel cluster areas that appear as a pair, and each eye Providing a detection system of the position of the eye has a function of specifying the position of the eye based on the frequency of occurrence over all eye position detecting image position candidate.

  Further, the present invention is a face area position detection system including an original image acquisition unit and a calculation unit including a face image, and the calculation unit functions as a calculation device of the above-described eye position detection system. In addition, the present invention provides a face area position detection system having a function of specifying, as a face area, a face area candidate whose face position is specified.

  According to the method or system of the present invention, a face area candidate that is likely to contain a face image is automatically detected in an original image including a face image, and a plurality of images with varying brightness are formed for each face area candidate. The eye position in the original image, regardless of the lighting environment, subject skin color, subject pupil color, face orientation in the face image, etc. The face area to be automatically detected can be detected automatically in a short time with high reliability.

  Furthermore, according to the eye position detection method or system of the method or system of the present invention, the formation of a hairstyle simulation image for fitting various hairstyle images to an arbitrary face image based on the eye position, Simulation image by fitting multiple pieces of image information based on eye position, such as forming a makeup simulation image that fits a partial facial image after makeup to an arbitrary face image based on eye position When forming the image, it is possible to form a natural simulation image in a short time.

  The present invention also applies to image processing such as painting the eye and its neighboring area, applying a mosaic, and applying a mosaic to the entire face area, such as a blindfold process for protecting personal information. The method or system can be suitably used.

  Hereinafter, the present invention will be described in detail with reference to the drawings.

  FIG. 1 shows an eye position detection method according to the present invention for detecting the eye position of each face image in the original image including the face image, and the face area position is detected after the eye position detection. FIG. 2 is a flowchart showing an embodiment of the method for detecting the position of the face area according to the present invention, and FIG. 2 is a block diagram of the detection system 10 for detecting the position of the eye and the position of the face area for implementing these methods.

  The system 10 includes an imaging apparatus 1 and a personal computer main body 2, and a display 3, an image scanner 4, a printer 5, and the like are connected to the personal computer main body 2.

  The imaging device 1 is provided as a means for acquiring an original image including a face image, and a commercially available digital camera can be used.

  The personal computer main body 2 is provided as a calculation means, and roughly, gray scale of a color image, gradation change, brightness correction, image size change, overall blur, blur in a predetermined direction such as horizontal and vertical, edge Image processing functions such as extraction, image superposition, figure enlargement / reduction functions, etc., and a function to create a fade-in image with a predetermined fade-in ratio by combining these functions as described later. It has a function of changing (for example, a function of decreasing the contrast as the brightness is increased and increasing the contrast as the brightness is decreased). These individual image processing functions can be obtained by, for example, installing commercially available image processing software such as Photoshop manufactured by Adobe System Co. in the personal computer main body 2.

  The personal computer body 2 has a function of managing a plurality of images with different brightness by assigning a list number in order of increasing or decreasing brightness. The brightness of a predetermined area on the image is compared with the brightness of other areas to perform predetermined extraction. More specifically, a horizontal / vertical blurred image is created as a blurred image that is blurred in two intersecting directions from the original image, and the horizontal / vertical blurred image In the function of detecting the horizontal edge and the vertical edge and obtaining the position of the intersection, the function of creating the eye position detection target area setting image in which the gradation of the original image is crushed, the eye position detection target area setting image, A function for setting a rectangular area having an intersection of the horizontal edge and the vertical edge as one vertex; a function for detecting an area having a higher brightness as compared with the adjacent area on the left and right and above the rectangular area as a face area candidate; Area A function to create multiple images with varying brightness as eye position detection images for each eye, as the image fades in from a state that fades out at high brightness to a low brightness, the eye position detection image gradually A function for detecting a cluster area of appearing pixels, a function for selecting a pair of detected pixel cluster areas appearing as eye position candidates, and an image for detecting the position of all eyes of each eye position candidate The function of specifying the position of the eyes based on the appearance frequency over the entire range.

As shown in FIG. 1, the eye position detection method using the system 10 is roughly
A. Detecting a face area candidate from an original image including a face image;
B. It consists of two steps of detecting the eye position candidate for each face area candidate and specifying it.

  In the process A, first, an original image having a face image as a detection target of the eye position is acquired in the personal computer main body 2. For example, the original image may be obtained by taking a photograph including a face image with the imaging device 1 and loading the photograph into the personal computer main body 2, or reading a photograph including the face image using the image scanner 4, such as the Internet. Such an image may be acquired using the communication line.

  As the original image, the number of face images included in one original image may be singular or plural, but the present invention is particularly suitable when there are a plurality of face images and when the sizes of the plurality of face images are different. Since the effect is exhibited, a plurality of directions are preferable.

  Next, pre-processing such as gray scaling, size adjustment, blurring, and brightness correction is performed on the original image as necessary.

  Of these, gray scale conversion is performed in order to reduce the amount of subsequent processing when the acquired original image is a color image because the eye position detection method of the present invention does not require color information.

As a gray scale method, for each pixel, (1) a method that uses the average of the maximum and minimum values of R, G, and B, and (2) the average of each value of R, G, and B A method to be used, (3) a method of multiplying each value of R, G, and B by a predetermined weighting coefficient (NTSC, etc.) and then averaging them can be used. For example, in the method (3), the following equation Y (output luminance) = 0.298912 × R + 0.568811 × G + 0.114478 × B
Can be used.

  The size adjustment is to adjust the original image to an image size suitable for the eye position detection process. Specifically, the long side is preferably 320 pixels or more from the viewpoint of detection accuracy. In view of speed, it is preferable to change the screen size so that the long side is about 480 pixels or less.

  If the face image in the original image is sharp, a lot of fine edges appear in the subsequent image processing, and blurring processing is performed as necessary because it becomes noise for detecting eye position candidates. In this case, it is preferable that the degree of blurring is weak to medium. For example, for each pixel, the weighting blur filter of 1 to 5 is used for the pixel of interest and the surrounding 25 pixels. Performs blurring to output the average value. This blurring process may be performed a plurality of times as necessary.

  The brightness correction is preferably performed when the face image detection image is created when the original image is extremely bright or dark. As a lightness correction method, for example, a lightness histogram of the original image 20 is created for the original image 20 that is too bright as shown in FIG. 3A (a) or the original image 20 that is too dark as shown in FIG. 3B (a). After the count value of the pixel value which is 1.5% to 15% of the maximum count number, preferably 3% to 8% or less is set to zero, the largest pixel value among the pixel values whose count number is not zero A value Xmax and a minimum pixel value Xmin are obtained. Next, as shown in FIGS. 3A (b) and 3B (b), the original image brightness correction is performed by substituting the pixel value Xi of the original image into the following equation to obtain the pixel value after the brightness correction.

However, if Xi <Xmin, Xi = Xmin,
If Xi> Xmax, Xi = Xmax.

  The brightness correction method is not limited to the above-described method, and various methods can be applied. For example, an arbitrary image is set as a reference image in advance, the average brightness is measured, and an appropriate brightness change amount for creating a horizontal / vertical blurred image and a face area detection image is determined. The ratio of the average brightness of the original image to the average brightness is calculated, and the brightness of the original image is corrected by changing the brightness of the original image by an amount obtained by multiplying the appropriate brightness change amount in the reference image by the average brightness ratio.

  Here, the appropriate brightness change amount in the reference image is determined as follows. First, an image in which the average brightness of the reference image is changed by one gradation in both the light and dark directions is created, a detection target area of an eye position described later is detected for each image, and the detection result is actually a face. It is judged visually whether it matches with the area where the image exists, and the success rate of the detection is calculated. Then, the average brightness of the image with the highest success rate is obtained, and the difference between this and the average brightness of the reference image is set as the appropriate brightness change amount.

  As the reference image, face images of 5 to 20 persons are shown, and the ratio of the maximum size and the minimum size of each face image is preferably within 1 to 3 times the face length. The eye position detection target area may be set by the following method.

  FIG. 4 shows the original image 20 before the preprocessing and the blurred image 21 after the preprocessing when the gray scale, size adjustment, blurring, and brightness correction are performed on the original image 20 as preprocessing.

  After pre-processing, in the step (A1) of FIG. 1, a horizontal / vertical blurred image 24 as shown in FIG. 5C is created as a blurred image blurred in two intersecting directions. The horizontal / vertical blurred image 24 is a combination of the horizontal blurred image 22 created as shown in FIG. 10A and the vertical blurred image 23 created as shown in FIG. In the present invention, the two blur directions in the blurred image are not limited to the horizontal direction and the vertical direction, and may be two oblique directions as shown in FIG. 7, for example. In this case, it is preferable that the angle θ on the acute angle side of the two directions is 60 ° or more. However, in order to simplify the subsequent calculation, it is preferable that the intersecting two directions are the horizontal direction and the vertical direction, and the horizontal / vertical blurred image 24 as shown in FIG. 5C is created.

  For example, in the case of the horizontal blurred image 22, an average pixel value of 80 to 120 pixels in the horizontal direction centered on the pixel is obtained for each pixel in the horizontal blurred image 22, and the pixel value is calculated. It can be obtained by performing the operation of setting the pixel value of the pixel for all the pixels. Similarly, in the vertical blurred image 23, the vertical direction centering on the pixel for each pixel with respect to the original image 20 It can be obtained by calculating an average pixel value of 80 to 120 pixels and setting the pixel value as the pixel value of the pixel for all pixels.

  In addition, as a method of creating the horizontal blurred image 22 and the vertical blurred image 23, first, a horizontal blurred image and a vertical blurred image are generated, and then, without combining the two steps of combining them, for each pixel, The average pixel value of the horizontal direction 80 to 120 pixels and the vertical direction 80 to 120 pixels centered on the pixel may be calculated simultaneously.

  Next, as in the horizontal / vertical edge image 25 shown in FIG. 6, the horizontal edge and the vertical edge in the horizontal / vertical blurred image 24 are obtained, and the position of the intersection of these edges is obtained. For detection of a horizontal edge or a vertical edge, it is preferable to use an edge filter having a blurring function. For example, a horizontal edge filter or a vertical edge filter of the following formula can be used.

  On the other hand, in step (A2) of FIG. 1, a face area detection image in which the gradation of the original image is crushed is created.

  As the face area detection image, as shown in FIG. 8A, by changing the brightness of the original image 20, the hair area and the face area can be discriminated, but the face structure cannot be discriminated, that is, the hair area and the face area. However, it is preferable to create an image in which the high gradation side is crushed so that the inside of the face does not appear at all, or the face, eyes, nose, and mouth can be recognized but it is difficult to determine who the face is.

  Although the method for creating the face area detection image 26 depends on the shooting conditions of the original image 20, it is basically preferable to create an image with a fade-in ratio of 25 to 35% from the original image 20.

  Here, for example, when the original image has 256 gradations, an image having a predetermined fade-in ratio can be created as follows.

  First, in an image with 0% fade-in, the pixel values of all the pixels are uniformly set to 255.

  Further, in an image with fade-in x%, pixels having a pixel value of less than 2.55x in the original image have the pixel value of the original image, and pixels having a pixel value of 2.55x or more (provided that x> 0) are uniform. Suppose that it has a pixel value of 255.

  For example, in a 10% fade-in image, pixels with a pixel value of less than 25.5 (= 255 × 0.1) remain the same in the original image, and pixels with a pixel value of 25.5 or more have a uniform pixel value. 255.

  An image with 20% fade-in has a pixel value of less than 51.0 (= 255 × 0.2) in the original image, and a pixel value of 51.0 or higher in the original image remains the same. Set to 255.

  In an image with 100% fade-in, the pixel values of all the pixels remain the pixel values of the original image.

  As shown in FIG. 8B, the face area detection image 26 may be formed as a negative image with the gradation reversed, and the subsequent image processing may be performed with the gradation reversed.

  Further, the method of creating the face area detection image 26 is not limited to a method of creating an image having a predetermined fade-in ratio, and various binarization processing methods can be applied. For example, various binarization processes (generally referred to as variable threshold processes) for determining a threshold value based on the characteristics of the original image, such as a binarization process using the central brightness value of the brightness histogram of the original image 20 as a threshold value. Thus, the face area detection image 26 can be created.

  Next, as shown in step (A3) in FIG. 1, a face area candidate expected to be a face area is detected in the face area detection image 26. The face area candidate is a face area detection image 26 and a rectangular area having one vertex at the intersection of the horizontal edge and the vertical edge of the horizontal / vertical blurred image, compared to the left and right and upper adjacent areas. To detect areas with high brightness. This is because, in a grayscale image, the face area is compared to the periphery of the head, hat, clothing, and other face areas, regardless of race or hair color, and with or without hair. This is based on the inventor's knowledge that it is bright.

  As a specific method of detecting a face area candidate, for example, as shown in FIG. 9, first, a horizontal / vertical blurred image 25 of a horizontal / vertical blurred image is superimposed on a face area detection image 26, and the horizontal edge and the vertical edge are vertically aligned. At each intersection point P of the edge, a small square 30 having the intersection point P as one vertex is assumed, and the square is defined as the following (a) or the following point with the intersection point P as a base point (that is, an expansion center when expanding to a similar shape). The image is expanded in the lower right direction, the lower left direction, the upper right direction, and the upper left direction until the condition (b) is satisfied.

(a) When the ratio of pixels having a pixel value of 255 in the square becomes 60% or more and then becomes less than 60%
(b) When the ratio of the pixels having a pixel value of 254 or less becomes 1/3 or less after the ratio of the pixel value of 254 or less in the pixels forming the vertical side or the lower side of the line width of 1 pixel in the square expansion direction

  Next, as shown in FIG. 10, in a region adjacent to the left and right and above the square 31 at the time when the expansion is stopped, each side of the square is a long side, and the length is 1/3 to 1/2 of the side of the square. Assuming rectangles 32a, 32b, and 32c having short sides as short sides, if the following condition (p) or (q) is satisfied for these rectangles 32a, 32b, and 32c, the square 31 is selected as a face area candidate. And Even when all the above three rectangles cannot be drawn because the square 31 is positioned at the corner of the image area detection image 26, the rectangle is drawn within the drawable range, and the following conditions (p) or (q) are satisfied. to decide.

(p) 10% or more, preferably 20% or more of all the pixels of the rectangles 32a, 32b and 32c have a pixel value of 254 or less.
(q) Among the long sides of the rectangles 32a, 32b, and 32c, a pixel having a pixel value of 254 or less is 1/3 or more, preferably 2/5 or more of the length of the long side on the square 31 side.

  In this face area candidate detection method, the intersection point P is an intersection point of a horizontal edge generated at the boundary between the forehead and the hair and a vertical edge generated at the boundary between the face and the right and left hair areas. Since it is often a point that falls within the outer upper hair region or a point around the bottom of the earlobe, the small square 30 from the intersection point P is moved to the lower right direction, the lower left direction, the upper right direction, or the upper left direction. When the enlarged square 31 is a face area, the square 31 becomes brighter than the area adjacent to it and satisfies the above-described condition.

  The figure to be enlarged with the intersection point P as the base point is not limited to a square, but may be a rectangle or the like. The figure when the enlargement is stopped and the neighboring area for comparing the brightness are also the three rectangles as described above. Not limited to 32a, 32b, and 32c, for example, a U-shaped region surrounding the square 31 from the left and right sides and from above may be assumed. As long as the face area candidate search process can be completely automated by the calculation means, it can be set as appropriate.

  Thus, as shown in FIG. 11, a plurality of face area candidates 33 can be detected in the face area detection image 26. As shown in the figure, a plurality of face area candidates 33 may overlap in one area.

  After the face area candidate 33 is detected, a process B for detecting the position of the eyes for each individual face area candidate 33 is performed.

  In step B, first, in step (B1), in order to detect eye position candidates, a plurality of eye position detection images with varying brightness are formed as eye position detection images for each face region candidate. . More specifically, for example, as shown in FIG. 12, a fade-in image in which the brightness is gradually decreased from 0% fade-in according to the above-described image forming method with a predetermined fade-in ratio is reduced from 10% to 80% by 1%. Create about 70 levels in increments.

  In creating the fade-in image hierarchy, in order to increase the processing speed while maintaining the detection accuracy, a face area candidate 33 having a side larger than 80-120 pixels in advance is It is preferable to adjust the image size of each face region candidate 33 so that the size is 80 to 120 pixels, and it is preferable to perform blurring processing as necessary to reduce noise.

  In forming a fade-in image, it is preferable to lower the contrast as the lightness is increased and raise the contrast as the lightness is lowered because the eye part appears more clearly in the fade-in image.

  In addition, as the eye position detection image, for example, by gradually changing the γ value, a plurality of images whose brightness is changed as shown in FIG. 12 may be formed. It is possible to use.

  As can be seen from FIG. 12, when the brightness of a detection target region is increased and the brightness is reduced from an image that is completely faded out to form a fade-in image hierarchy, normally, first, a cluster of pixels is first formed in the pupil. Then, a block of pixels is detected in the nose and mouth. The cluster of pixels in the pupil region appears as a pair of left and right.

  Therefore, in the step (B2), as the eye position detection image formed from the face area candidate 33 fades in from the state of fading out with high lightness to the low lightness, the pixels that gradually appear in the eye position detection image are displayed. Detect mass area. In step (B3), a cluster of a pair of left and right pixels is selected as an eye position candidate. Depending on the hierarchy, pixel cluster areas appear in pairs in areas such as the mouth, eyebrows, and forehead in addition to the pupils. At this stage, these pixel cluster areas are also selected as eye position candidates.

In this case, it is preferable that the following (1) and (2) are set in advance as a condition for selecting a candidate eye position among the clustered regions of the paired pixels.
(1) The lateral distance of the cluster area of a pair of left and right pixels is within a certain range.
(2) The vertical distance between the left and right pixel clusters is within a certain range.

  FIG. 13 shows a block area of pixels appearing in a face area in a fade-in image in a hierarchy with a face area candidate 33 as a rectangle, and a pair of these areas connected by a straight line. .

  Next, in step (B4), eye position candidates are specified as eye positions based on the appearance frequencies of all eye position candidates over all eye position detection images. More specifically, as shown in FIG. 14, a pair of eye position candidates appearing for each fade-in image is accumulated over all layers of the eye position detection image, and the appearance frequency is counted. Create a ranking list in descending order of count. In this count, the eye position candidate that appears first when the brightness is gradually reduced from the faded-out face image and continues to appear until the final stage is normally the maximum count (ranking first). Therefore, those with a count value of 2 or less are excluded from eye position candidates, and in principle, the position of the eye position candidate with the maximum count is narrowed down as an eye position candidate.

However, if the following conditions (a) to (c) are all satisfied, or if the condition (d) is satisfied, the position of the eye position candidate with the second largest count is set as the eye position. As specified.
(a) When the second position candidate for the ranking is higher than the first position candidate for the ranking
(b) When the distance between the center of the eye position candidate of the 2nd ranked eye is longer than the distance between the centers of the position candidate of the 1st ranked eye
(c) When the left and right pupil regions of the second ranking eye position candidate are both outside the positions corresponding to the left and right pupils of the first ranking eye candidate
(d) The vertical distance between the first position candidate of the ranking and the second position candidate of the ranking is about the distance between the eyes and the eyebrows, and the first position candidate of the ranking is ranked first. When the position is above the second position candidate

  Of these, (a) to (c) are intended to prevent erroneous determination of the position of the mouth as the position of the eye, while the first ranking in the ranking may rarely be the mouth area. d) is for preventing erroneous determination that the eyebrows around the eyes are the positions of the eyes.

As described above, as shown in FIG. 15, it is possible to accurately detect eye position candidates (more precisely, the position of the pupil) in the detection target area of each eye position.

  The eye position candidate specifying method described above is performed for all face area candidates set in step A. Further, when the position of the eye cannot be specified in any face area candidate, it is handled that there is no eye in that area.

  In this way, the eye positions specified in the detection target areas of the individual eye positions are finally superimposed on the corresponding positions of the original image as shown in FIG. 16, and set as the eye positions of the individual face images in the original image. It becomes possible.

  In addition, after specifying the position of the eye in step (B4), the position of the face area in the original image is determined by specifying the face area based on the face area candidate corresponding to the position of the eye in step (B5). Can be detected. In this case, the rectangular area of the face area candidate corresponding to the specified eye position does not need to be used as the face area as it is, but a polygon area including the detected eye position according to the purpose of specifying the face area in the original image. Or a circular region or a flesh-colored region. Here, the size of the face region can be appropriately set according to the detected distance between both eyes, the size of the face region candidate, and the like. In addition to detecting the position of eyes in each face region candidate, the position of the mouth and nose is also detected using the fade-in method (Image Recognition and Understanding Symposium (MIRU2006), Proceedings CD-ROM, IS1-18 “ Face parts (hair, eyebrows, eyes, nose, mouth) are highly accurate and put into practical use ", published 2006.7.19), and at least an area that includes the nose or mouth in addition to the eyes may be used as the face area.

  In addition, when a plurality of face areas corresponding to the position of the eye and having different sizes or positions are overlapped, the face area may be determined based on these sum areas. For example, when the purpose is to perform mosaic processing of the face area in the original image, it is preferable to use the sum area in order to reliably perform mosaic processing on the original face area.

  The eye position detection method and face region position detection method according to the present invention are based on the skin color, pupil color, face orientation, and illumination environment of each subject to be detected as the eye position in a plurality of face images. Regardless of the change in position, the eye position can be detected at a high processing speed with a simple and accurate calculation method. Therefore, it is possible to detect the position of the eyes from a plurality of face images regardless of whites or blacks.

  The positions of the eyes detected by the present invention are the following: when forming a hairstyle simulation image, when forming a makeup simulation image, when performing face image blindfold processing, when performing face image frame processing, It can be used in various scenes where the detection of the eye position is required, such as when creating.

  In addition, the position of the face area detected by the present invention is also applied to the case where the face image is blindfolded, the face color correction at the time of digital camera shooting or the face color correction of the shot image, the face color correction at the time of printer printing, etc. Can be used.

3 is a flowchart of an eye position detection method and a face region position detection method according to the present invention. It is a block diagram of the detection system of the position of an eye position and a face area. It is explanatory drawing of brightness correction. It is explanatory drawing of brightness correction. An original image before preprocessing and a blurred image after preprocessing. They are a horizontal blur image, a vertical blur image, and a horizontal / vertical blur image. It is an image of a horizontal edge and a vertical edge detected from a horizontal / vertical blurred image. It is a blurred image in two diagonal directions. It is a face area detection image (positive image). It is a face area detection image (negative image). It is explanatory drawing of the square which makes the base point the intersection of a horizontal edge and a vertical edge for the detection of a face area candidate. It is explanatory drawing of the detection conditions of a face area | region candidate. It is the figure which showed the face area candidate in the image for face area detection. It is an eye position detection image in which the brightness of a face area candidate is changed. It is the figure which showed the position candidate of the eye to the fade-in image of a certain hierarchy. It is the figure which accumulated and displayed the eye position candidate over all the hierarchies in the eye position detection image of a certain hierarchy. It is the figure which displayed the position of the specified eye on the face area candidate. It is the figure which superimposed the position of the detected eye and the position of the face area on the original image.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Imaging device 2 Personal computer body 3 Display 4 Image scanner 5 Printer 10 Eye and face area position detection system 20 Original image 21 Pre-processed blurred image 22 Horizontal blurred image 23 Vertical blurred image 24 Horizontal / vertical blurred image 25 Vertical edge image 26 Face area detection image 30 Small square 31 Squares 32a, 32b, and 32c that have stopped expanding Rectangle 33 Face area candidate P Intersection of horizontal edge and vertical edge in horizontal / vertical blurred image

Claims (9)

A method for detecting an eye position from an original image including a face image,
A. (A1) Create a blurred image blurred in two directions intersecting the original image, obtain the position of the intersection of the edges in the blurred direction in the blurred image,
(A2) Create a face area detection image with the gradation of the original image collapsed,
(A3) In the face area detection image, a rectangular area having an intersection of edges in the blur direction as one vertex and having a higher brightness than the left and right and upper adjacent areas is detected as a face area candidate. The process of
B. (B1) For each face area candidate, create a plurality of eye position detection images with varying brightness,
(B2) As the eye position detection image fades in from a state of fading out at a high brightness to a low brightness, a cluster area of pixels that gradually appears in the eye position detection image is detected,
(B3) Select the clustered areas of the detected pixels that appeared in pairs as eye position candidates,
(B4) specifying the eye position candidate as the eye position based on the appearance frequency over all eye position detection images of each eye position candidate;
A method for detecting the position of an eye.   The method of detecting an eye position according to claim 1, wherein in step A (A2), an image for face area detection is created from the original image whose brightness has been corrected.   In step A (A2), when the original image has 256 gradations, an image with 0% fade-in has all pixels having a pixel value of 255, and an image with fade-in x% (where x> 0) A pixel having a pixel value less than 2.55x in the original image has a pixel value of the original image, and a pixel having a pixel value of 2.55x or more has a uniform pixel value 255. The eye position detection method according to claim 1 or 2, wherein an image having a fade-in of 25 to 35% is formed with respect to the original image.   The method for detecting an eye position according to claim 1, wherein two intersecting directions are a horizontal direction and a vertical direction. In step A (A3), assume a small square with the intersection of the horizontal edge and the vertical edge as one vertex, and use the intersection as a base point until the next condition (a) or (b) is satisfied. Expand downward, lower left, upper right and upper left,
(a) When the proportion of pixels having a pixel value of 255 in the square becomes 60% or more and then becomes less than 60%
(b) When the ratio of pixels with a pixel value of 254 or less becomes 1/3 or less after the ratio of the pixel value of 254 or less in the vertical or lower side of the line width of 1 pixel in the square expansion direction

In a region adjacent to the left and right and above the square at the time when the expansion is stopped, a rectangle with one side of the square as a long side and a length of 1/3 to 1/2 of one side of the square as a short side is assumed. The eye position detection method according to claim 4, wherein those rectangles satisfying the following condition (p) or (q) are set as face area candidates.
(p) 10% or more of all pixels have a pixel value of 254 or less
(q) Pixels having a length equal to or more than 1/3 of the long side of the rectangular square have a pixel value of 254 or less   A method for detecting the position of a face area, which specifies a face area based on a face area candidate corresponding to the eye position after the eye position is specified by the method according to claim 1. An eye position detection system including an original image acquisition unit including a face image and a calculation unit, wherein the calculation unit includes:
A function that creates a blurred image that is blurred in two directions intersecting with the original image, a function that detects edges in the blur direction of the blurred image and obtains the position of the intersection, and a face area detection in which the gradation of the original image is crushed Image creation function, a face area detection image is a rectangular area whose intersection is the intersection of edges in the blur direction and has a higher brightness than the adjacent areas on the left and right and above As a function to create a plurality of images whose brightness has changed as eye position detection images for each face area candidate, as the image fades out from a state of fading out with high brightness to low brightness, A function for detecting a cluster area of pixels that gradually appear in the eye position detection image, a function for selecting a pair of detected pixel cluster areas that appear as a pair, and each eye The position detection system of the position of the eye has a function of specifying the position of the eye based on the occurrence frequency over position detecting images of all eye candidates.   The eye position detection system according to claim 7, wherein two intersecting directions are a horizontal direction and a vertical direction.   9. A system for detecting a position of a face area including an original image acquisition unit including a face image and a calculation unit, wherein the calculation unit is a function of the calculation unit according to claim 7 or 8, and an eye among face area candidates. A face area position detection system having a function of specifying a face area based on the position of the face.