JP4510562B2 - Circle center position detection method, apparatus, and program - Google Patents

Description

  The present invention relates to a method and apparatus for detecting a center position of a circle, for example, a center position of a circle such as a pupil from a digital photographic image, and a program therefor.

In various fields, such as the determination of the center position of a circular part in a machine and the determination of the position of a pupil in a face photographic image, it is necessary to detect the center position of a circle from a digital photographic image. Conventionally, various methods such as template matching and Hough transform are used to detect a circle from a digital photographic image by photographing a subject. Among these methods, Hough transform is used as a method for detecting an incomplete circle, for example, a circle damaged by noise or the like, or a pupil that appears only partially due to blinking (see Patent Document 1). Specifically, first, a binarization process is performed on a digital photographic image to obtain a binary image, an edge is detected from the binary image, and a ring is formed using pixel data of the detected edge. Voting is performed on the Hough space corresponding to the three parameters for detection (the X coordinate of the center, the Y coordinate of the center, and the radius r). As a result of voting, obtain the coordinates with the highest number of votes, determine the values of the three parameters of the ring based on the coordinates, and determine the center coordinates (X, Y) of the determined circle Thus, the center position of the circle is detected.
JP 2003-70742 A

  However, the method of detecting the center position of the circle by performing the Hough transform as described above has a great effect on detecting the circle having the defect and the center position thereof, but only the pixel data of the edge in the digital photographic image. Therefore, there is a drawback that it is easily affected by noise.

  SUMMARY OF THE INVENTION In view of the above circumstances, an object of the present invention is to provide a circle center position detection method and apparatus that can detect the center position of a circle having a defect and is resistant to the influence of noise, and a program therefor. is there.

A first circle center position detection method of the present invention is a circle center position detection method for detecting a center position of a circle included in a photographic image from a photographic image.
The coordinates of each pixel of the photographic image are voted on an annular Hough space (X coordinate of the circle center point, Y coordinate of the circle center point, radial coordinate r), and each vote on the voted Hough space. The voting value at the voting position obtained by determining the number of times the position (X, Y, r) has been voted is defined as the first integrated voting value,
The position indicated by the (X, Y) coordinate value of the voting position having the largest first integrated voting value is the center of the circle.

  That is, in the first circle center position detection method of the present invention, the center position of the circle is determined by voting to the Hough space of the ring using only the edge pixels and finding the vote position having the largest vote value as the ring. Unlike the detection method, the circle is regarded as a large number of concentric rings, and the voting value (first value of each voting position) is voted on the Hough space of the ring using the data of all pixels of the grayscale photographic image to be processed. Based on the integrated vote value), the position indicated by the (X, Y) coordinate value of the vote position having the largest vote value is detected as the center position of the circle. In this method, since many concentric rings have different radii, but the positions of the circle centers are the same, the voting positions having the (X, Y) coordinate values corresponding to the circle positions should be voted the most. It is based on something.

A second circle center position detection method of the present invention is a circle center position detection method for detecting a center position of a circle included in a photographic image from a photographic image.
The coordinates of each pixel of the photographic image are voted on an annular Hough space (X coordinate of the circle center point, Y coordinate of the circle center point, radial coordinate r), and each vote on the voted Hough space. Obtain the vote value of the voting position by determining the number of times the position (X, Y, r) has been voted,
Of each of the voting positions, the voting value of the voting position having the same (X, Y) coordinate value is added to obtain a second integrated voting value,
The position indicated by the (X, Y) coordinate value corresponding to the largest second integrated vote value is the center of the circle.

  That is, according to the second circle center position detection method of the present invention, the voting position having the largest vote value is found as a ring by voting on the Hough space of the ring using only the pixel data of the edge. Unlike the method of detecting the center position, the circle is regarded as a large number of concentric rings, and the data of all pixels of the grayscale photographic image to be processed is used to vote for the Hough space of the ring, and the (X, Y) coordinate value Based on the second integrated vote value obtained by adding the vote values of the same vote position (they have the same (X, Y) coordinate values and therefore have different concentric radii). Thus, the position indicated by the (X, Y) coordinate value corresponding to the largest second integrated vote value is detected as the center position of the circle.

  In the second circle center position detection method of the present invention, only the voting values of the voting positions within the radius range of the circle among the voting positions having the same (X, Y) coordinate values. Is preferably added to obtain the second integrated vote value.

  In the photographic image, there may be other circles in addition to the target circle. Therefore, when the radius range of the target circle is known, the vote value is added within the radius range. Thus, it is desirable to obtain the second integrated vote value. In this way, there is a high probability that the center position of a circle detected from a photographic image having a plurality of circles will be the center position of the target circle.

  Here, the radius range of the circle is not limited to a range defined as upper and lower limits such as a predetermined size or more and a predetermined size or less, but a predetermined size or less, a predetermined size or more, It may be a range defined by only the upper limit or only the lower limit.

The circle center position detection method of the present invention can be applied to the detection of the center position of a pupil from a photographic image including the pupil. 2
When the center position of the pupil is detected by the circle center position detection method of the present invention, the number of votes of each pixel is obtained by weighted addition so that the weight becomes smaller as the chromaticity of the voted pixel increases. The value is preferably the vote value.

  Here, the “chromaticity” of a pixel may use any parameter as long as the color of the pixel can indicate the distance away from the gray axis. For example, the following formula (1) May be used to determine the chromaticity of the pixel.

U = (R−G) ² + (GB) ² + (B−R) ² (1)
Where U: Pixel color
R, G, B: R value, G value, B value of the pixel

That is, according to the circle center position detection method of the present invention, if there is a vote from one pixel for one voting position (one pixel votes only once for one voting position), the vote value It is possible to obtain a vote value at each voting position by accumulating 1 as 1 (in this case, the voting value at the voting position corresponds to the number of pixels that have voted at this voting position), but including the pupil In the case of a photographic image, that is, a facial photographic image, the chromaticity of the pupil part is lower than the other parts, so the number of votes for each pixel is weighted and added so that the weight becomes smaller as the chromaticity increases. By using the voting value at the voting position, the effects of noise such as the color frame of the eyeglasses included in the face photo image, the dyed hair, and the skin can be reduced for more accurate detection. The result can be obtained.

  The circle center position detection method of the present invention can deal with not only grayscale photographic images but also color photographic images. Note that the color photographic image may be subjected to gray conversion to obtain a grayscale photographic image, and the circle center position may be detected using the grayscale photographic image.

  Further, when detecting the center position of a circle using a grayscale photographic image or a grayscale photographic image obtained by converting a color image into gray, the grayscale photographic image is binarized with a predetermined threshold value to obtain a binary value. It is preferable to obtain an image and detect the center position of the circle using the binary image. By doing so, the processing can be speeded up, and the memory required for calculation for detection can be saved. it can.

  The predetermined threshold for binarization used for binarization is preferably obtained based on the number of pixels of the grayscale photographic image and the luminance of each pixel.

  When detecting the center position of the pupil by the circle center position detection method of the present invention, a value obtained by weighted addition of the number of votes of each pixel so that the weight increases as the luminance of the voted pixel decreases. Is preferably set as the vote value.

  That is, according to the circle center position detection method of the present invention, if there is a vote from one pixel for one voting position (one pixel votes only once for one voting position), the vote value It is possible to obtain a vote value at each voting position by accumulating 1 as 1 (in this case, the voting value at the voting position corresponds to the number of pixels that have voted at this voting position). When determining the voting value at each voting position for a circle whose luminance is smaller (ie, darker) as it is closer to the center, the number of votes of each pixel is such that the lower the luminance of the pixel that has voted at the voting position, the greater the weight It is preferable to weight and add (although the vote count of one pixel is 1) to obtain a vote value at the vote position.

  Further, the photographic image including the pupil can be a trimmed image obtained by trimming a face photographic image with the vicinity of the eyes as an outer frame.

A first circle center position detection device of the present invention is a circle center position detection device that detects a center position of a circle included in a photographic image from a photographic image,
The coordinates of each pixel of the photographic image are voted on an annular Hough space (X coordinate of the circle center point, Y coordinate of the circle center point, radial coordinate r), and each vote on the voted Hough space. A voting value acquisition means having a voting value at the voting position obtained by calculating the number of times that the position (X, Y, r) has been voted as a first integrated voting value;
Center position determining means for determining the position indicated by the (X, Y) coordinate value of the voting position having the largest first integrated voting value as the center of the circle. Is.

The second circle center position detection device of the present invention is a circle center position detection device for detecting the center position of a circle included in the photographic image from a photographic image,
The coordinates of each pixel of the photographic image are voted on an annular Hough space (X coordinate of the circle center point, Y coordinate of the circle center point, radial coordinate r), and each vote on the voted Hough space. Voting value acquisition means for obtaining the voting value of the voting position by determining the number of times the position (X, Y, r) has been voted;
Among the voting positions, voting value integrating means for adding the voting values at the voting positions having the same (X, Y) coordinate value to obtain a second integrated voting value;
Center position determining means having a position indicated by the (X, Y) coordinate value corresponding to the largest second integrated vote value as the center of the circle is provided.

  The voting value integration means adds only the voting values of the voting positions within the radius range of the circle among the voting positions having the same (X, Y) coordinate value. It is preferable to obtain an integrated vote value of 2.

  The circle center position detection apparatus of the present invention can be applied to detecting the center position of a pupil from a photographic image including the pupil.

  When detecting the center position of the pupil, the circle center position detection unit according to the present invention weights the number of votes of each pixel so that the vote value acquisition unit decreases the weight as the chromaticity of the pixel increases. It is preferable that a value obtained by addition is used as the vote value.

  The circle center position detection apparatus of the present invention can be applied not only to a gray scale photographic image but also to a color photographic image. The circle center position detection device of the present invention includes image conversion means for obtaining a grayscale photographic image by converting a color photographic image into gray, and the voting value acquisition means performs the voting using the grayscale photographic image. You may do it.

  The circle center position detection apparatus of the present invention includes binarization means, and by the binarization means, a grayscale photograph image or a grayscale photograph obtained by performing gray conversion on a color image by the image conversion means It is preferable that a binarized image is obtained by binarization with a threshold value, and the vote value acquisition means performs the vote using the binarized image.

  When detecting the center position of the pupil, the circle center position detection means of the present invention is configured so that the vote value acquisition means counts the number of votes of each pixel so that the weight increases as the brightness of the voted pixel decreases. It is preferable that a value obtained by weighted addition is used as the vote value.

  The center position detection device of the present invention for detecting the center position of the pupil further comprises trimming means for using the trimmed image obtained by trimming the face photo image with the vicinity of the eyes as the outer frame as the photo image. Is preferred.

  Further, in the circle center position detection device of the present invention, the “near eye” may be manually designated by an operator, but the position of the eye is detected from the face photograph image. Further, eye position detecting means for providing information indicating the eye position to the trimming means may be further provided, and the “near eye” may be defined based on the information indicating the eye position.

  Further, the circle center position detection apparatus of the present invention for detecting the center position of the pupil is the center position of the left pupil and the center of the right pupil determined using the trimmed image of the left eye and the trimmed image of the right eye, respectively. Whether or not the center position of the pupil is correct is determined based on whether the difference from the position is within a predetermined range according to the distance between the left eye and the right eye, and the center position is correct Until the next largest integrated vote value corresponding to the previously determined central position is obtained, and the (X, Y) coordinate value corresponding to the next largest integrated vote value is obtained. It is preferable to further include collation means for re-determining the indicated position as the center of the pupil.

  Here, the “integrated vote value” means any one of the first integrated vote value (that is, the vote value) or the second integrated vote value.

  The left eye and the right eye on the same face usually have a slight difference in the position of the eyes in the vertical direction. Since the pupil is located in the eye, the vertical difference between the center position of the left pupil and the center position of the right pupil is also in a slightly small range (for example, the range D is the distance D between the left eye and the right eye). / 50) should be within. Further, since the difference between the center position of the left pupil and the center position of the right pupil, that is, the distance between the center positions of both pupils is substantially the same as the distance between the left eye and the right eye (number of D pixels), the center of the left pupil The difference between the position and the center position of the right pupil can be set within a range of 0.8 × D to 1.2 × D, for example. The collation means in the circle center position detection apparatus of the present invention pays attention to this point, and calculates the center position of the left pupil and the center position of the right pupil (the vertical position and / or the horizontal position) in the same face image. Whether or not the center position of the pupil is positive is determined based on the fact that the difference should be within a predetermined range corresponding to the position of both eyes.

  Further, the collating means, when obtaining the integrated vote value next to the integrated vote value corresponding to the center position obtained last time, the center obtained last time, not from all the integrated vote values. It is preferable that the next largest integrated vote value is obtained from each of the integrated vote values corresponding to the (X, Y) coordinate values indicating a position separated from the position by a predetermined distance or more.

  That is, when it is determined that the obtained center position is the center position of another circle such as a eyelid near the eyes and not the center position of the pupil (the center position of the pupil is not correct), Since it is unlikely that there is a center position of the pupil near the position, when determining the next largest integrated vote value of the integrated vote value corresponding to this center position, this center position is not from all the integrated vote values. If the next largest integrated vote value is obtained from each of the integrated vote values corresponding to the (X, Y) coordinate values indicating a position more than a predetermined distance from the center position of the true pupil more quickly Can be detected.

  The “predetermined distance” is preferably a value obtained based on the distance between the eyes, and for example, D / 30 (D is the number of pixels indicating the distance between the eyes) can be used.

  A circle center position detection apparatus of the present invention for detecting the center position of a pupil performs pre-filling processing on the photographic image, and uses the photographic image subjected to the hole filling processing for the vote value acquisition means. Is preferably further provided.

  In the case of the circle center position detection device of the present invention having a trimming means for trimming a face photo image and an image conversion means for performing gray conversion, trimming may be performed after gray conversion, but the amount of calculation is reduced. However, for fast processing, it is desirable to perform gray conversion by the image conversion means after the trimming by the trimming means.

  The circle center position detection method of the present invention may be provided as a program to be executed by a computer.

  According to the circle center position detection method, apparatus, and program of the present invention, when detecting the center position of a circle included in a photographic image from a photographic image, the circle is regarded as a set of many concentric rings, Vote the coordinates of all pixels in the Hough space of the ring, and the voting value of each voted position itself or the same among the voted voting positions (the X coordinate value of the circle center, the Y coordinate value of the circle center) ) Are added to the voting values at the voting positions having the integrated voting values (first integrated voting value and second integrated voting value, respectively) and correspond to the largest integrated voting value (X, Y). Since the position indicated by the coordinate value is detected as the position of the center of the circle, the conventional technique using only the edge pixel while taking advantage of the Hough transform that can also detect the center position of a circle with a defect Less susceptible to noise It is possible to detect the center position of more accurately circular.

  Further, when obtaining the second integrated vote value, if only the vote values at the voting position within the radius range of the target circle are added, in a grayscale photographic image in which a plurality of circles having different radii exist, The probability of detecting the center position of the target circle is increased.

  When applying the circle center position detection method and apparatus of the present invention and the program therefor to the detection of the center position of the pupil from the face photograph image, each feature is utilized by utilizing the feature that the brightness is smaller as it is closer to the center of the pupil. Since the value obtained by weighted addition of the number of times of voting of each pixel is used as the voting value of the voting position so that the weight increases as the luminance of the voting pixel decreases, the voting position of the voting position becomes more accurate. The center position can be detected.

  Further, when applying the center position detection device of the present invention to the detection of the center position of the pupil, a collating means is provided, and the difference between the center positions of the eyes of both eyes on the same face is within a predetermined range. The center position of other circles such as eyelids in the vicinity of the eyes is determined as the center of the pupil by determining whether the center position of the detected pupil is correct. Since it can prevent detecting as a position, the center position of a pupil can be detected more correctly.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing an embodiment of a pupil center position detection apparatus as a first embodiment of a circle center position detection method and apparatus and a program therefor according to the present invention. As shown in the figure, the pupil center position detection apparatus of the present embodiment identifies whether or not a face part is included in the input photographic image S0, and if the face part is not included, the photographic image S0 is identified. Is output to the output unit 50 to be described later, and when the face portion is included, the left eye and the right eye are further detected, and information Q including the position of both eyes and the distance D between the eyes is trimmed by the trimming unit 10 and Based on the detection unit 1 output to the collation unit 40 and the information Q from the detection unit 1, trimming the photographic image S0 and trimming images S1a and S1b including the left eye and the right eye respectively (hereinafter, need to be described separately) If not, the trimming unit 10 that obtains both of them is referred to as S1, and the trimmed image S1 is subjected to gray conversion, and the grayscale image S2 (S2a, 2b), the pre-processing unit 14 that pre-processes the grayscale image S2 to obtain the pre-processed image S3 (S3a, S3b), and binarizes the pre-processed image S3. A binarized threshold value calculating unit 18 for calculating a threshold value T for the binarized image, and binarizing the preprocessed image S3 using the threshold value T obtained by the binarized threshold value calculating unit 18 The binarization unit 20 that obtains S4 (S4a, S4b) and the coordinates of each pixel of the binary image S4 are voted on the Hough space of the ring to obtain the vote value of each voted position and the same circle center A voting unit 35 that calculates an integrated voting value W (Wa, Wb) of a voting position having coordinates, and the center integrated coordinate corresponding to the largest integrated voting value among the integrated voting values obtained by the voting unit 35 The position candidate G (Ga, Gb) and a collation unit to be described later When instructed to search for the next center position candidate from 0, the center position candidate acquisition unit 35 for obtaining the next center position candidate, and whether the center position candidate acquired by the center position candidate acquisition unit 35 satisfies the collation criteria If the collation criterion is satisfied, the center position candidate is output to the fine adjustment unit 45 described later as the center position of the pupil. If the collation criterion is not satisfied, the center position candidate acquisition unit 35 receives the center position. Collation that causes a candidate to be reacquired and causes the center position candidate acquisition unit 35 to repeat reacquisition of the center position candidate until the center position candidate acquired by the center position candidate acquisition unit 35 satisfies the verification criteria Fine adjustment is performed on the center position G (Ga, Gb) of the pupil output from the unit 40 and the collation unit 40, and the final center position G ′ (G′a, G′b) is output to the output unit 50. A fine adjustment unit 45 to be applied, and a photographic image S0 (a photographic image that does not include a face part) output from the detection unit 1 and an output unit 50 that outputs the center position G from the fine adjustment unit 45. It will be.

  Details of each component of the pupil center position detection device shown in FIG. 1 will be described below.

  First, details of the detection unit 1 will be described.

  FIG. 2 is a block diagram illustrating a detailed configuration of the detection unit 1. As shown in the figure, the detection unit 1 includes a feature amount calculation unit 2 that calculates a feature amount C0 from a photographic image S0, a storage unit 4 that stores first and second reference data E1 and E2 described later, A first identification unit for identifying whether or not a photographic image S0 includes a human face based on the feature amount C0 calculated by the feature amount calculation unit 2 and the first reference data E1 in the storage unit 4. 5 and when the first identification unit 5 identifies that the face is included in the photographic image S0, the feature amount C0 in the face image calculated by the feature amount calculation unit 2 and the first in the storage unit 4 On the basis of the second reference data E2, a second identification unit 6 for identifying the position of an eye included in the face and a first output unit 7 are provided.

  The eye position identified by the detection unit 1 is the center position (indicated by x in FIG. 3) between the corners of the eyes and the eyes on the face, and the eyes facing directly in front as shown in FIG. In this case, the position is the same as the center position of the pupil. However, as shown in FIG. 3B, in the case of the eyes facing right, not the center position of the pupil but the position deviated from the center of the pupil or the white eye portion. To do.

  The feature amount calculation unit 2 calculates a feature amount C0 used for face identification from the photograph image S0. When it is identified that the photograph image S0 includes a face, a similar feature amount C0 is calculated from the extracted face image as described later. Specifically, the gradient vector (that is, the direction in which the density of each pixel on the photographic image S0 and the face image changes and the magnitude of the change) is calculated as the feature amount C0. Hereinafter, calculation of the gradient vector will be described. First, the feature amount calculation unit 2 performs a filtering process on the photographic image S0 using a horizontal edge detection filter shown in FIG. 4A to detect a horizontal edge in the photographic image S0. Further, the feature amount calculation unit 2 performs a filtering process on the photographic image S0 by a vertical edge detection filter shown in FIG. 4B to detect a vertical edge in the photographic image S0. Then, a gradient vector K at each pixel is calculated from the horizontal edge size H and vertical edge size V at each pixel on the photographic image S0, as shown in FIG. Similarly, the gradient vector K is calculated for the face image. Note that the feature amount calculation unit 2 calculates a feature amount C0 at each stage of deformation of the photographic image S0 and the face image, as will be described later.

  Note that the gradient vector K calculated in this way is an eye in a dark portion such as the eyes and mouth as shown in FIG. 6B in the case of a human face as shown in FIG. It faces the center of the mouth and faces outward from the position of the nose in a bright part like the nose. Further, since the change in density is larger in the eyes than in the mouth, the gradient vector K is larger in the eyes than in the mouth.

  The direction and magnitude of the gradient vector K are defined as a feature amount C0. The direction of the gradient vector K is a value from 0 to 359 degrees with reference to a predetermined direction of the gradient vector K (for example, the x direction in FIG. 5).

  Here, the magnitude of the gradient vector K is normalized. This normalization obtains a histogram of the magnitude of the gradient vector K in all the pixels of the photographic image S0, and the distribution of the magnitudes is a value that each pixel of the photographic image S0 can take (0 to 255 if 8 bits). The histogram is smoothed so as to be uniformly distributed, and the magnitude of the gradient vector K is corrected. For example, when the gradient vector K is small and the histogram is distributed with the gradient vector K biased toward the small side as shown in FIG. The magnitude of the gradient vector K is normalized so that it extends over the region so that the histogram is distributed as shown in FIG. In order to reduce the amount of computation, as shown in FIG. 7C, the distribution range in the histogram of the gradient vector K is divided into, for example, five, and the frequency distribution divided into five is shown in FIG. 7D. It is preferable to normalize so that the value of 0 to 255 is in a range divided into five.

  The first and second reference data E1 and E2 stored in the storage unit 4 constitute each pixel group for each of a plurality of types of pixel groups composed of a combination of a plurality of pixels selected from a sample image to be described later. The identification condition for the combination of the feature amount C0 in each pixel is defined.

  In the first and second reference data E1 and E2, the combination and identification condition of the feature amount C0 in each pixel constituting each pixel group may not be a plurality of sample images and faces that are known to be faces. This is determined in advance by learning a sample image group including a plurality of known sample images.

  In the present embodiment, when the first reference data E1 is generated, the sample image known to be a face has a 30 × 30 pixel size, and as shown in FIG. The distance between the centers of both eyes in the image of one face is 10 pixels, 9 pixels, and 11 pixels, and a face standing vertically at the distance between the centers of both eyes is stepped in units of 3 degrees within a range of ± 15 degrees on the plane. Sample images that have been rotated (that is, the rotation angles are −15 degrees, −12 degrees, −9 degrees, −6 degrees, −3 degrees, 0 degrees, 3 degrees, 6 degrees, 9 degrees, 12 degrees, and 15 degrees). Shall be used. Therefore, 3 × 11 = 33 sample images are prepared for one face image. In FIG. 8, only sample images rotated at −15 degrees, 0 degrees, and +15 degrees are shown. The center of rotation is the intersection of the diagonal lines of the sample image. Here, if the distance between the centers of both eyes is a 10-pixel sample image, the center positions of the eyes are all the same. The center position of this eye is set to (x1, y1) and (x2, y2) on the coordinates with the upper left corner of the sample image as the origin. In addition, the eye positions in the vertical direction in the drawing (ie, y1, y2) are the same in all sample images.

  Further, when the second reference data E2 is generated, the sample image known to be a face has a 30 × 30 pixel size, and as shown in FIG. The distance between the centers is 10 pixels, 9.7 pixels, and 10.3 pixels, and the face standing vertically at the distance between the centers of each eye is rotated step by step in a range of ± 3 degrees on the plane. It is assumed that the sample image (that is, the rotation angle is −3 degrees, −2 degrees, −1 degrees, 0 degrees, 1 degree, 2 degrees, 3 degrees) is used. Therefore, 3 × 7 = 21 sample images are prepared for one face image. Note that FIG. 9 shows only sample images rotated to −3 degrees, 0 degrees, and +3 degrees. The center of rotation is the intersection of the diagonal lines of the sample image. Here, the positions of the eyes in the vertical direction in the drawing are the same in all the sample images. In order to set the distance between the centers of both eyes to 9.7 pixels and 10.3 pixels, the sample image whose distance between the centers of both eyes is 10 pixels is enlarged or reduced to 9.7 times or 10.3 times. Thus, the size of the sample image after enlargement / reduction may be set to 30 × 30 pixels.

  Then, the center position of the eye in the sample image used for learning the second reference data E2 is set as the eye position to be identified in the present embodiment.

  As a sample image that is known not to be a face, an arbitrary image having a 30 × 30 pixel size is used.

  Here, as a sample image that is known to be a face, learning is performed using only a center image whose distance between the centers of both eyes is 10 pixels and the rotation angle on the plane is 0 degree (that is, the face is vertical). When performed, the face or eye position is identified with reference to the first and second reference data E1 and E2 only for a face that is not rotated at all with a center-to-center distance of both eyes of 10 pixels. is there. Since the size of the face that may be included in the photographic image S0 is not constant, when identifying whether or not a face is included or the position of the eyes, the photographic image S0 is enlarged or reduced as described later. The position of the face and eyes that match the size of the sample image can be identified. However, in order to accurately set the distance between the centers of both eyes to 10 pixels, the size of the photographic image S0 needs to be identified while being enlarged or reduced in steps of, for example, 1.1 units as an enlargement ratio. Will be enormous.

  Further, the face that may be included in the photographic image S0 is not only rotated at 0 degree on the plane as shown in FIG. 10A, but also rotated as shown in FIGS. 10B and 10C. Sometimes it is. However, when learning is performed using only a sample image in which the distance between the centers of both eyes is 10 pixels and the rotation angle of the face is 0 degrees, FIGS. As shown in (), the rotated face cannot be identified.

  Therefore, in this embodiment, as a sample image known to be a face, the distance between the centers of both eyes is 9, 10, 11 pixels as shown in FIG. 8, and ± 15 degrees on the plane at each distance. In this range, a sample image obtained by rotating the face step by step in increments of 3 degrees is used to allow the learning of the first reference data E1. As a result, when the identification is performed in the first identification unit 5 described later, the size of the photographic image S0 can be set to, for example, the enlargement ratio because the photographic image S0 can be enlarged or reduced in steps of 11/9. For example, the calculation time can be reduced as compared with a case where the enlargement / reduction is performed in units of 1.1. Further, as shown in FIGS. 10B and 10C, a rotating face can also be identified.

  On the other hand, in learning of the second reference data E2, as shown in FIG. 9, the distance between the centers of both eyes is 9.7, 10, 10.3 pixels, and each distance is within a range of ± 3 degrees on the plane. Since the sample image obtained by rotating the face step by step in units of 1 degree is used, the learning tolerance is smaller than that of the first reference data E1. Further, when the identification is performed by the second identification unit 6 to be described later, the photographic image S0 needs to be enlarged / reduced in units of 10.3 / 9.7 as an enlargement ratio. It takes a longer time to calculate than the identification. However, since only the image in the face identified by the first identification unit 5 is identified by the second identification unit 6, the eye position is identified as compared with the case where the entire photographic image S0 is used. The amount of calculation for performing can be reduced.

  Hereinafter, an example of a learning method for the sample image group will be described with reference to the flowchart of FIG. Here, learning of the first reference data E1 will be described.

  The group of sample images to be learned includes a plurality of sample images that are known to be faces and a plurality of sample images that are known not to be faces. As described above, the sample image that is known to be a face has 9, 10, 11 pixels in the center position of both eyes for one sample image, and is 3 in a range of ± 15 degrees on the plane at each distance. Use a face rotated stepwise in degrees. Each sample image is assigned a weight or importance. First, the initial value of the weight of all sample images is set equal to 1 (S1).

  Next, a discriminator is created for each of a plurality of types of pixel groups in the sample image (S2). Here, each discriminator provides a reference for discriminating between a face image and a non-face image by using a combination of feature amounts C0 in each pixel constituting one pixel group. In the present embodiment, a histogram for a combination of feature amounts C0 in each pixel constituting one pixel group is used as a discriminator.

The creation of a classifier will be described with reference to FIG. As shown in the sample image on the left side of FIG. 12, each pixel constituting the pixel group for creating the discriminator is a pixel at the center of the right eye on a plurality of sample images that are known to be faces. P1, a pixel P2 on the right cheek, a pixel P3 on the forehead, and a pixel P4 on the left cheek. Then, combinations of feature amounts C0 in all the pixels P1 to P4 are obtained for all sample images that are known to be faces, and a histogram thereof is created. Here, the feature amount C0 represents the direction and magnitude of the gradient vector K. Since the gradient vector K has 360 directions from 0 to 359 and the gradient vector K has 256 sizes from 0 to 255, If used as they are, the number of combinations is 360 × 256 four pixels per pixel, that is, (360 × 256) ^four , and the number of samples, time and memory for learning and detection are large. Will be required. For this reason, in this embodiment, the gradient vector directions are 0 to 359, 0 to 44, 315 to 359 (right direction, value: 0), 45 to 134 (upward value: 1), and 135 to 224 (left). Direction, value: 2), 225-314 (downward, value 3), and quaternarization, and the gradient vector magnitude is ternarized (value: 0-2). And the value of a combination is computed using the following formula | equation.

Combination value = 0 (when gradient vector size = 0)
Combination value = ((gradient vector direction + 1) × gradient vector magnitude (gradient vector magnitude> 0)
Thus, since the number of combinations is nine patterns ^4, it can reduce the number of data of the characteristic amounts C0.

  Similarly, histograms are created for a plurality of sample images that are known not to be faces. For the sample image that is known not to be a face, pixels corresponding to the positions of the pixels P1 to P4 on the sample image that is known to be a face are used. The histogram used as the discriminator shown on the right side of FIG. 12 is a histogram obtained by taking logarithmic values of the ratios of the frequency values indicated by these two histograms. The value of each vertical axis indicated by the histogram of the discriminator is hereinafter referred to as an identification point. According to this classifier, an image showing the distribution of the feature quantity C0 corresponding to the positive identification point is highly likely to be a face, and it can be said that the possibility increases as the absolute value of the identification point increases. Conversely, an image showing the distribution of the feature quantity C0 corresponding to the negative identification point is highly likely not to be a face, and the possibility increases as the absolute value of the identification point increases. In step S <b> 2, a plurality of classifiers in the above-described histogram format are created for combinations of feature amounts C <b> 0 in the respective pixels constituting a plurality of types of pixel groups that can be used for identification.

  Subsequently, the most effective classifier for identifying whether or not the image is a face is selected from the plurality of classifiers created in step S2. The most effective classifier is selected in consideration of the weight of each sample image. In this example, the weighted correct answer rate of each classifier is compared, and the classifier showing the highest weighted correct answer rate is selected (S3). That is, in the first step S3, since the weight of each sample image is equal to 1, the number of sample images in which the image is correctly identified by the classifier is simply the largest. Selected as a valid discriminator. On the other hand, in the second step S3 after the weight of each sample image is updated in step S5, which will be described later, a sample image with a weight of 1, a sample image with a weight greater than 1, and a sample image with a weight less than 1 The sample images having a weight greater than 1 are counted more in the evaluation of the correct answer rate because the weight is larger than the sample images having a weight of 1. Thereby, in step S3 after the second time, more emphasis is placed on correctly identifying a sample image having a large weight than a sample image having a small weight.

  Next, the correct answer rate of the classifiers selected so far, that is, the result of identifying whether each sample image is a face image using a combination of the classifiers selected so far, is actually It is ascertained whether or not the rate that matches the answer indicating whether the image is a face image exceeds a predetermined threshold (S4). Here, the sample image group to which the current weight is applied or the sample image group to which the weight is equal may be used for evaluating the correct answer rate of the combination. When the predetermined threshold value is exceeded, learning can be completed because it is possible to identify whether the image is a face with a sufficiently high probability by using the classifier selected so far. If it is less than or equal to the predetermined threshold, the process advances to step S6 to select an additional classifier to be used in combination with the classifier selected so far.

  In step S6, the discriminator selected in the most recent step S3 is excluded so as not to be selected again.

  Next, the weight of the sample image that could not be correctly identified as a face by the classifier selected in the most recent step S3 is increased, and the sample image that can be correctly identified as whether or not the image is a face is increased. The weight is reduced (S5). The reason for increasing or decreasing the weight in this way is that in selecting the next discriminator, an image that cannot be discriminated correctly by the already selected discriminator is regarded as important, and whether or not those images are faces can be discriminated correctly. This is to increase the effect of the combination of the discriminators by selecting the discriminators.

  Subsequently, the process returns to step S3, and the next valid classifier is selected based on the weighted correct answer rate as described above.

  By repeating the above steps S3 to S6, the classifier corresponding to the combination of the feature amount C0 in each pixel constituting the specific pixel group is selected as a classifier suitable for identifying whether or not a face is included. If the correct answer rate confirmed in step S4 exceeds the threshold value, the type of the discriminator used for discriminating whether or not a face is included and the discriminating condition are determined (S7). The learning of the reference data E1 is finished.

  Then, the second reference data E2 is learned by obtaining the classifier type and identification conditions in the same manner as described above.

  In the case of adopting the above learning method, the discriminator provides a reference for discriminating between a face image and a non-face image using a combination of feature amounts C0 in each pixel constituting a specific pixel group. As long as it is not limited to the above histogram format, it may be anything, for example, binary data, a threshold value, a function, or the like. Further, even in the same histogram format, a histogram or the like indicating the distribution of difference values between the two histograms shown in the center of FIG. 12 may be used.

  Further, the learning method is not limited to the above method, and other machine learning methods such as a neural network can be used.

  The first identification unit 5 configures each pixel group with reference to the identification conditions learned by the first reference data E1 for all combinations of the feature amounts C0 in the respective pixels constituting the plurality of types of pixel groups. An identification point for the combination of the feature amount C0 in each pixel is obtained, and all the identification points are combined to identify whether or not a face is included in the photographic image S0. At this time, the direction of the gradient vector K, which is the feature amount C0, is quaternized and the magnitude is quinary. In the present embodiment, all the identification points are added, and identification is performed based on the positive / negative of the added value. For example, when the sum of the identification points is a positive value, it is determined that the photograph image S0 includes a face, and when the sum is negative, it is determined that no face is included. The identification performed by the first identification unit 5 as to whether or not a face is included in the photographic image S0 is referred to as a first identification.

  Here, the size of the photographic image S0 is different from the sample image of 30 × 30 pixels and has various sizes. When a face is included, the rotation angle of the face on the plane is not always 0 degrees. For this reason, as shown in FIG. 13, the first identification unit 5 scales the photographic image S0 stepwise until the vertical or horizontal size becomes 30 pixels and rotates it 360 degrees stepwise on the plane. However, a mask M having a size of 30 × 30 pixels is set on the photographic image S0 enlarged and reduced at each stage, and the mask M is set to 1 on the enlarged photographic image S0. While moving pixel by pixel, it is identified whether the image in the mask is a face image, thereby identifying whether the face is included in the photographic image S0.

  Since the sample image learned at the time of generating the first reference data E1 has 9, 10, and 11 pixels at the center position of both eyes, the enlargement ratio at the time of enlargement / reduction of the photographic image S0 is 11 / 9. In addition, since the sample image learned at the time of generating the first and second reference data E1 and E2 is a sample image whose face is rotated in a range of ± 15 degrees on a plane, the photographic image S0 is 30 degrees. What is necessary is just to rotate 360 degree | times in a unit.

  Note that the feature amount calculation unit 2 calculates a feature amount C0 at each stage of deformation of enlargement / reduction and rotation of the photographic image S0.

  Then, whether or not a face is included in the photographic image S0 is identified for the photographic image S0 at all stages of enlargement / reduction and rotation. If it is identified that the face is included even once, the photographic image S0 includes a face. And a 30 × 30 pixel region corresponding to the position of the identified mask M is extracted as a face image from the photographic image S0 of the size and rotation angle at the stage where the face is identified. .

  The second identification unit 6 learns the second reference data E2 for all the combinations of the feature amounts C0 in the respective pixels constituting the plurality of types of pixel groups on the face image extracted by the first identification unit 5. With reference to the identification conditions, the identification points for the combination of the feature amounts C0 in the respective pixels constituting each pixel group are obtained, and the positions of the eyes included in the face are identified by combining all the identification points. At this time, the direction of the gradient vector K, which is the feature amount C0, is quaternized and the magnitude is quinary.

  Here, the second discriminating unit 6 enlarges / reduces the size of the face image extracted by the first discriminating unit 5 stepwise and is enlarged / reduced at each step while rotating stepwise 360 degrees on the plane. A mask M having a 30 × 30 pixel size is set on the face image, and the eye position in the image in the mask is identified while moving the mask M pixel by pixel on the enlarged / reduced face.

  Since the sample image learned at the time of generating the second reference data E2 has the number of pixels at the center position of both eyes of 9.07, 10, and 10.3 pixels, enlargement when the face image is enlarged or reduced The rate may be 10.3 / 9.7. Further, as the sample image learned at the time of generating the second reference data E2, a face image rotated in a range of ± 3 degrees on the plane is used, so the face image is rotated 360 degrees in units of 6 degrees. Just do it.

  Note that the feature amount calculation unit 2 calculates the feature amount C0 at each stage of deformation such as enlargement / reduction and rotation of the face image.

  In this embodiment, all the identification points are added at all stages of deformation of the extracted face image, and the upper left corner of the face image in the mask M of 30 × 30 pixels at the stage of deformation having the largest added value is obtained. The coordinates corresponding to the coordinates (x1, y1) and (x2, y2) of the eye position in the sample image are obtained, and the position corresponding to this position in the photographic image S0 before deformation is set as the coordinate. Identify the location.

  When the first identification unit 5 identifies that the photographic image S0 does not include a face, the first output unit 7 outputs the photographic image S0 as it is to the output unit 50, while the first identification unit 5 Recognizes that a face is included in the photographic image S0, the distance D between both eyes is obtained from the position of both eyes identified by the second identification unit 6, and the position D of both eyes and the distance D between both eyes are used as information Q. The data is output to the trimming unit 10 and the collation unit 40.

  FIG. 14 is a flowchart showing the operation of the detection unit 1 in this embodiment. For the photographic image S0, first, the feature amount calculation unit 2 calculates the direction and size of the gradient vector K of the photographic image S0 as the feature amount C0 at each stage of enlargement / reduction and rotation of the photographic image S0 (S12). . Then, the first identification unit 5 reads the first reference data E1 from the storage unit 4 (S13), and performs first identification as to whether or not a face is included in the photographic image S0 (S14).

  When determining that the face is included in the photographic image S0 (S14: Yes), the first identification unit 5 extracts the face from the photographic image S0 (S15). Here, not only one face but a plurality of faces may be extracted. Next, the feature amount calculation unit 2 calculates the direction and size of the gradient vector K of the face image as the feature amount C0 at each stage of enlargement / reduction and rotation of the face image (S16). Then, the second identification unit 6 reads the second reference data E2 from the storage unit 4 (S17), and performs second identification for identifying the position of the eyes included in the face (S18).

  Subsequently, the first output unit 7 outputs the eye position identified from the photographic image S0 and the distance D between both eyes obtained based on the eye position as information Q to the trimming unit 10 and the collation unit 40. (S19).

On the other hand, if it is determined in step S14 that no face is included in the photographic image S0 (S14: No), the first output unit 7 outputs the photographic image S0 as it is to the output unit 50 (S19).

  The trimming unit 10 cuts out a predetermined range including only the left eye and only the right eye based on the information Q output from the detection unit 1 to obtain trimmed images S1a and S1b. Here, the predetermined range at the time of trimming is a range in which the vicinity of each eye is an outer frame. For example, the eye position (eye) identified by the detection unit 1 as shown by the hatched range in FIG. Centered around the center point), the lengths in the X direction and the Y direction in the figure are D and 0.5D, respectively. The hatched area shown in the figure is the trimming range of the left eye in the figure, but the same applies to the right eye.

  The gray conversion unit 12 performs a gray conversion process on the trimmed image S1 obtained by the trimming unit 10 according to the following equation (2) to obtain a grayscale image S2.

Y = 0.299 × R + 0.587 × G + 0.114 × B (2)
Y: Luminance value
R, G, B: R, G, B values

The preprocessing unit 14 performs preprocessing on the grayscale image S2, and here, smoothing processing and hole filling processing are performed as preprocessing. The smoothing process is performed by applying, for example, a Kaussian filter, and the hole filling process can be an interpolation process.

  As shown in FIG. 3, in the pupil portion of the photographic image, there is a tendency that the portion above the center is partially brightened. Therefore, the detection accuracy of the center position of the pupil is obtained by performing hole filling processing and interpolating the data of this portion. Can be improved.

  The binarization unit 20 includes a binarization threshold value calculation unit 18, and uses the threshold value T calculated by the binarization threshold value calculation unit 18 to store the preprocessed image S <b> 3 obtained by the preprocessing unit 14. The binary image S4 is obtained by digitization. Specifically, the binarization threshold value calculation unit 18 creates a luminance histogram shown in FIG. 16 for the preprocessed image S3, and is a fraction of the total number of pixels in the preprocessed image S3 (in the drawing, The luminance value corresponding to the appearance frequency corresponding to 1/5 (20%) is obtained as the threshold T for binarization. The binarization unit 20 binarizes the preprocessed image S3 using this threshold T to obtain a binary image S4.

  The voting unit 30 first votes the coordinates of each pixel (pixel having a pixel value of 1) in the binarized image S4 to the annular Hough space (circle center point X coordinate, circle center point Y coordinate, radius r). Then, the voting value at each voting position is calculated. Normally, when one vote position is voted by a pixel, the vote value at each vote position is obtained by adding 1 to the vote value as if it was voted once. When one vote is voted for a certain pixel, instead of adding 1 to the vote value, the brightness value of the voted pixel is referred to, and the smaller the brightness value, the higher the weight is added. The voting value at each voting position is obtained. FIG. 17 shows a table of weighting coefficients used in the voting unit 30 in the pupil center position detection apparatus of the embodiment shown in FIG. Note that T in the figure is a binarization threshold T calculated by the binarization threshold calculation unit 18.

  After the voting unit 30 obtains the voting value of each voting position in this way, among these voting positions, the coordinate value of the center point of the ring, that is, (X, Y, r) in the ring Hough space (X, Y, r). Y) The vote values of the vote positions having the same coordinate value are added to obtain an integrated vote value W corresponding to each (X, Y) coordinate value, and corresponding to the corresponding (X, Y) coordinate value Then, the data is output to the center position candidate acquisition unit 35.

  The center position candidate acquisition unit 35 first acquires (X, Y) coordinate values corresponding to the largest integrated vote value as the center position candidate G of the pupil from each integrated vote value from the voting unit 30. Output to the verification unit 40. Here, the center position candidate G acquired by the center position candidate acquisition unit 35 is two, that is, the center position Ga of the left pupil and the center position Gb of the right pupil, and the collation unit 40 is output by the detection unit 1. Based on the distance D between the eyes, the two center positions Ga and Gb are collated.

  Specifically, the collation unit 40 performs collation based on the following two collation criteria.

1. The difference in Y coordinate value between the center position of the left pupil and the center position of the right pupil is (D / 50) or less.

2. The X coordinate value difference between the center position of the left pupil and the center position of the right pupil is within the range of (0.8 × D to 1.2 × D).

The collation unit 40 determines whether or not the two pupil center position candidates Ga and Gb from the center position candidate acquisition unit 35 satisfy the above two collation criteria. The pupil center position candidates Ga and Gb are output to the fine adjustment unit 45 as the pupil center position. On the other hand, if one of the two criteria or one of the two criteria is not satisfied (hereinafter referred to as not satisfying the collation criteria), the center position candidate acquisition unit 35 is instructed to acquire the next center position candidate. , For the next center position candidate acquired by the center position candidate acquisition unit 35, the above-described collation, the center position output when the collation criteria are satisfied, and the center position candidate when the collation criteria are not met are reacquired. Processing such as instructions is repeated until the verification criteria are satisfied.

  When the acquisition of the next center position candidate is instructed from the collation unit 40, one of the center position candidate acquisition units 35 first fixes the center position of one side (here, the left pupil) and the other side (here, the left center position). The (X, Y) coordinate value of the voting position satisfying the following three conditions is acquired as the next center position candidate from each integrated voting value Wb of (right pupil).

1. Finally, it is separated from the position indicated by the (X, Y) coordinate value of the center position candidate output to the collating unit 40 by D / 30 or more (D: distance between both eyes).

2. The corresponding integrated voting value corresponds to the (X, Y) coordinate value of the center position candidate that is finally output to the collation unit 40 among the integrated voting values corresponding to the (X, Y) coordinate value satisfying the condition 1. Next to the integrated vote value.

3. The corresponding integrated voting value is 10% or more of the integrated voting value (the largest integrated voting value) corresponding to the (X, Y) coordinate value of the center position candidate output to the collation unit 40 for the first time.

The center position candidate acquisition unit 35 first fixes the center position of the left pupil and searches for a center position candidate of the right pupil that satisfies the above three conditions based on the integrated vote value Wb obtained for the right pupil. If no candidate satisfying the above three conditions is found, the center position of the right pupil is fixed, and the left pupil satisfying the above three conditions is determined based on the integrated vote value Wa obtained for the left pupil. Find the center position.

  The fine adjustment unit 45 performs fine adjustment on the pupil center position G output from the collation unit 40 (center position candidate satisfying the collation criteria). First, fine adjustment of the center position of the left pupil will be described. The fine adjustment unit 45 repeats the mask calculation three times using a mask of size 9 × 9 and all 1 for the binary image S4a of the left-eye trimmed image S1a obtained by the binarization unit 20, Based on the position (Gm) of the pixel having the maximum result value obtained as a result of this mask calculation, fine adjustment is performed on the center position Ga of the left pupil output from the matching unit 40. Specifically, for example, the average position obtained by taking the average of the position Gm and the center position Ga may be set as the final center position G′a of the pupil, or the center position Ga is weighted. The average position obtained by the average calculation may be used as the final center position G′a of the pupil. Here, an average calculation is performed with weights applied to the center position Ga.

  Further, the fine adjustment of the center position of the right pupil is performed in the same manner as described above using the binary image S4b of the trimmed image S1b of the right eye.

  In this way, the fine adjustment unit 45 provides the output unit 50 with final center positions G′a and G′b obtained by performing fine adjustment on the pupil center positions Ga and Gb output from the collation unit 40. Output.

  The output unit 50 attaches identification information “no face included” to the photographic image S0 (photo image that does not include a face) from the detection unit 1 and outputs and records it on a recording medium. The center position G ′ from the adjustment unit 45 and the photographic image S0 corresponding to the center position G ′ are output in association with the recording medium and recorded.

  FIG. 18 is a flowchart showing processing of the pupil center position detection apparatus of the embodiment shown in FIG. As shown in the figure, the photographic image S0 is first discriminated whether or not a face is included in the detection unit 1 (S110). As a result of the determination, if the photographic image S0 does not include a face (S115: No), the photographic image S0 is output from the detection unit 1 to the output unit 50, while if the photographic image S0 includes a face ( Further, the position of the eyes in the photographic image S0 is detected by the detection unit 1, and the position of both eyes and the distance D between the eyes are output as information Q to the trimming unit 10 (S120). In the trimming unit 10, the photographic image S0 is trimmed to obtain a trimmed image S1a including only the left eye and a trimmed image S1b including only the right eye (S125). The trimmed image S1 is gray-converted by the gray converter 12 to become a grayscale image S2 (S130). The grayscale image S2 is subjected to smoothing processing and hole filling processing by the preprocessing unit 14, and further binarized by the binarizing unit 20 to become a binary image S4 (S135, S140). In the voting unit 30, the coordinates of each pixel of the binary image S4 are voted on the Hough space of the ring, and as a result, an integrated vote value W corresponding to the (X, Y) coordinate value indicating each circle center point is obtained. (S145). The center position candidate acquisition unit 35 first outputs the (X, Y) coordinate value corresponding to the largest integrated vote value to the collation unit 40 as the pupil center position candidate G (S150). The collation unit 40 collates the two center position candidates Ga and Gb from the center position candidate acquisition unit 35 based on the collation reference described above (S115), and the two center position candidates Ga and Gb use the collation reference. If the two are satisfied (S160: Yes), the two center position candidates Ga and Gb are output to the fine adjustment unit 45 as the center position, while the two center position candidates Ga and Gb do not satisfy the collation criteria (S160). : No), the center position candidate acquisition unit 35 is instructed to search for the next center position candidate (S150). The processing from step S150 to step S160 is repeated until the collation unit 40 determines that the center position candidate G from the center position candidate acquisition unit 35 satisfies the collation criteria.

  The fine adjustment unit 45 performs fine adjustment on the center position G output from the collation unit 40, obtains the final center position G ', and outputs it to the output unit 50 (S165).

  The output unit 50 attaches identification information “no face is included” to the photographic image S0 (S115: No) from the detection unit 1 and outputs and records it on a recording medium. The center position G ′ and the photographic image S0 corresponding to the center position G ′ are output in association with the recording medium and recorded.

  As described above, according to the pupil center detection apparatus of the present embodiment, the circle pupil is regarded as a set of a large number of concentric rings, and the coordinates of all the pixels of the binary image S4 of the trimmed image S1 are set in the Hough space of the ring. Vote, and add the vote value of the vote position that has the same (X coordinate value of the center of the circle, Y coordinate value of the center of the circle) among the voted vote positions, to obtain an integrated vote value, corresponding to the largest vote value Since the position indicated by the (X, Y) coordinate value is detected as the position of the center of the circle, a circle with a defect (for example, eyes in a meditation photo image or bangs partially covered the eyes) The center position of a pupil in a photographic image can be detected, and the center position of the pupil is accurately detected while being less susceptible to noise than conventional techniques using only edge pixels. be able to.

  Further, the pupil center position detection device of the present embodiment uses a feature that the luminance is smaller as it is closer to the center of the pupil, and for each voting position, the weight is increased as the luminance of the voting pixel is smaller. Since the value obtained by weighted addition of the number of votes of each pixel is used as the vote value of the vote position, the center position of the pupil can be detected more accurately.

  Furthermore, in the pupil center position detection apparatus of the present embodiment, a collation unit 40 is provided, and the collation unit 40 uses the difference between the center positions of the eyes of both eyes on the same face to be within a predetermined range. Determine whether the detected pupil center position is correct, and if it is not correct, recalculate it so that the center position of other circles such as eyelids near the eyes is the center position of the pupil. Since it can prevent detecting, the center position of a pupil can be detected more correctly.

  Furthermore, paying attention to the fact that only the skin color is near the eyes, the center position of the pupil can be determined with a high detection rate by using a trimmed image obtained by trimming the vicinity of the eyes for the outer frame. Can be detected.

  FIG. 19 is a block diagram showing a configuration of a pupil center position detection apparatus according to the second embodiment of the present invention. Note that the pupil center position detection device of the present embodiment is all other than that the voting unit 30 ′ described later is different from the voting unit 30 of the pupil center position detection device of the first embodiment shown in FIG. Is the same as the corresponding configuration of the pupil center position detecting device shown in FIG. 1, and therefore, in FIG. 19, the same configuration as the corresponding configuration of the pupil center position detecting device of the embodiment shown in FIG. The same reference numerals are given, and only the operation of the voting unit 30 ′ will be described.

  The voting unit 30 ′ in the pupil center position detection apparatus according to the second embodiment of the present invention shown in FIG. 19 first sets the coordinates of each pixel (pixel having a pixel value of 1) in the binarized image S4 as a ring. Vote on the Hough space (circle center point X coordinate, circle center point Y coordinate, radius r), and calculate the vote value of each vote position. Normally, when one vote position is voted by a pixel, the vote value at each vote position is obtained by adding 1 to the vote value as if it was voted once. When one vote is voted for a certain pixel, instead of adding 1 to the vote value, the brightness value of the voted pixel is referred to, and the smaller the brightness value, the higher the weight is added. The voting value at each voting position is obtained. Here, the voting unit 30 ′ uses the weighted voting value by using the weighting coefficient table shown in FIG. 17 as the integrated voting value W of each voting position, and (X , Y, r) are output to the center position candidate acquisition unit 35 in association with the coordinate values.

  The pupil center position detection device including the voting unit 30 ′ as shown in FIG. 19 has lower detection accuracy of the pupil center position than the pupil center position detection device shown in FIG. Since the adjustment unit 45 is provided, the center position of the pupil can be accurately detected. Further, since the vote value itself at each vote position is used as an integrated vote value, the amount of calculation is small and high-speed processing can be achieved.

  The preferred embodiment of the present invention has been described above. However, the circle center position detecting method and apparatus of the present invention and the program therefor are not limited to the present embodiment, and various methods can be used without departing from the gist of the present invention. You can change and increase / decrease.

  For example, the pupil center position detection apparatus of the embodiment shown in FIGS. 1 and 19 obtains a grayscale image S2 by performing gray conversion on an image in the vicinity of the eyes (trimming image S1) obtained by trimming the photographic image S0. The scale image S2 is processed from the preprocessing unit 14 to detect the pupil center position, but the trimming image S1 is processed from the preprocessing unit 14 without performing gray conversion. The pupil center position may be detected.

  In addition, the pupil center position detection apparatus of the embodiment shown in FIGS. 1 and 9 performs voting to the Hough space after binarizing the grayscale image S2 in order to speed up the processing and save memory. However, the voting to the Hough space may be performed directly using the gray scale image S2 without binarization.

  Further, in the pupil center position detection apparatus of the embodiment shown in FIGS. 1 and 9, the voting unit 30 and the voting unit 30 ′ pay attention to the fact that the pupil is closer to the center, and the luminance is lower. In order to increase the weight, the number of votes of each pixel is weighted and added to improve detection accuracy. Furthermore, the pupil uses the fact that the chromaticity is lower than that of the skin portion, the frame portion of the color glasses, and the color-dyed hair portion. For example, as shown in FIG. The number of votes of each pixel may be weighted and added so that the weight (different from the weight according to the luminance described above) becomes smaller. By doing so, it is possible to reduce the influence of noise in the colored portion other than the pupil, and the detection accuracy can be further increased. Also, the weighting method is not limited to the method shown in FIG. 20A. For example, an addition value corresponding to the chromaticity of the pixel as shown in FIG. You may make it add to the number of votes of.

  Further, a device for removing spectacle frames and hair noise may be further added. For example, when the number of votes of each vote position (X, Y, r) on the voted Hough space is projected on the XY plane and the distribution is viewed, coordinates corresponding to the center position of the pupil ( Focusing on the fact that the number of times becomes a clear peak at the point (X0, Y0), and when the peak part of the number of times is not a point and has a continuously extending distribution as shown in FIG. This part can be determined as a spectacle frame or hair noise and excluded from the candidate for the center position of the pupil.

  In the case of a face photo image, the pupil may be displayed in red. The circle center position detection method and apparatus of the present invention can also be applied to the detection of the center position of a pupil (red eye) displayed in red. Further, the center position of the pupil is detected without gray-converting the color face photo image, and the center position of the pupil is detected after gray-converting the color face photo image, and the two detected center positions are detected. You may apply to determining whether the pupil in a face photograph image has red eyes by comparing a difference. Of course, for example, when it is determined that the eye is red-eyed, processing such as red-eye correction may be performed.

  In the pupil center position detection apparatus of the embodiment shown in FIG. 1, the trimming image obtained by trimming the vicinity of the eyes as the outer frame usually has no circle other than the pupil and the collation by the collation unit 40 is performed. Therefore, the voting unit 30 determines the voting position where the coordinate value of the center point of the ring, that is, the (X, Y) coordinate value in the ring Hough space (X, Y, r) is the same among the voting positions. When the integrated vote value corresponding to each (X, Y) coordinate value is calculated by adding the vote values of each other, the range of the radius r of the vote position to be added is not limited. If the radius range of the circle (here pupil) is known, only the vote values at the voting positions within the radius range may be added to obtain the integrated vote value. By doing so, it is possible to prevent erroneous detection of the center position of a circle having a radius outside this radius range as the center position of the circle to be detected. In the pupil center position of this embodiment, for example, based on the distance D between both eyes, the radius range of the pupil can be defined to obtain the integrated vote value.

  Further, in the pupil center position detection device of the embodiment shown in FIGS. 1 and 19, the detection unit 1 detects whether or not a face is included, but a digital photograph including a face such as an ID photo When only the image is targeted, the eye position may be directly detected without detecting the face.

  Further, the face detection method and the eye position detection method are not limited to the methods used in the detection unit 1, and a conventionally known method may be used.

  In addition, instead of automatic detection, a position near the eye may be designated by the operator, and trimming may be performed based on the designated position.

  The pupil center position detection apparatus of the embodiment shown in FIGS. 1 and 19 is provided with a fine adjustment unit 45 to increase the accuracy of the detected center position of the pupil, However, fine adjustment is not always necessary.

  In addition, when performing fine adjustment, the mask operation is performed several times on the preprocessed grayscale image instead of the binary image, and as a result, the position of the pixel having the maximum value is obtained. Then, the center position of the pupil may be finely adjusted. Note that the mask used for the mask calculation is not limited to the 9 × 9 size used in the fine adjustment unit 45 of the pupil detection device of the present embodiment, and other size masks such as 5 × 5 are used. May be.

  Further, the output method of the detected center position of the pupil is not limited to being recorded on the recording medium in association with the photographic image as in the pupil center position detecting device of the present embodiment, but also according to the center position of the pupil. For example, it may be output to an image processing system that performs image processing or output to a device that trims a photographic image of a face based on the center position of the pupil. Good.

  Furthermore, the present invention is not limited to the pupil, and can be applied to detection of the center position of any circle.

The block diagram which shows the structure of the pupil center position detection apparatus used as the 1st Embodiment of this invention. The block diagram which shows the structure of the detection part 1 of the pupil center position detection apparatus shown in FIG. Diagram for explaining the center position of eyes (A) is a diagram showing a horizontal edge detection filter, (b) is a diagram showing a vertical edge detection filter Diagram for explaining calculation of gradient vector (A) is a figure which shows a person's face, (b) is a figure which shows the gradient vector of eyes and mouth vicinity of the person's face shown to (a). (A) is a diagram showing a histogram of the magnitude of a gradient vector before normalization, (b) is a diagram showing a histogram of the magnitude of a gradient vector after normalization, and (c) is a magnitude of a gradient vector obtained by quinarization. The figure which shows the histogram of the length, (d) is a figure which shows the histogram of the magnitude | size of the quinary gradient vector after normalization The figure which shows the example of the sample image known to be a face used for learning of 1st reference data The figure which shows the example of the sample image known to be a face used for learning of 2nd reference data Illustration for explaining face rotation Flow chart showing learning method of reference data Diagram showing how to derive a classifier The figure for demonstrating the stepwise deformation | transformation of an identification object image The flowchart which shows the process of the detection part 1. The figure for demonstrating the position which the trimming part 10 trims The figure for demonstrating how to obtain | require a binarization threshold value A diagram for explaining the weighting of voting values The flowchart which shows the process of the pupil center position detection apparatus of embodiment shown in FIG. The block diagram which shows the structure of the pupil center position detection apparatus used as the 2nd Embodiment of this invention. The figure for demonstrating the addition method of the number of votes according to the chromaticity of a pixel The figure for demonstrating the judgment of the noise based on distribution with respect to XY plane of the number of votes

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Detection part 10 Trimming part 12 Gray conversion part 14 Pre-processing part 18 Binarization threshold value calculation part 20 Binarization part 30,30 'Voting part 35 Center position candidate acquisition part 40 Collation part 45 Fine adjustment part 50 Output part S0 Photograph Image S1 Trimmed image S2 Grayscale image S3 Preprocessed image S4 Binary image

Claims (27)

In a circle center position detection method for detecting a center position of a circle included in the photographic image from a photographic image,
The coordinates of each pixel having a value other than 0 among the pixels constituting the photographic image are voted on an annular Hough space (X coordinate of the circle center point, Y coordinate of the circle center point, radial coordinate r). ,
The number of times that each vote position (X, Y, r) on the voted Hough space has been voted is obtained ,
When the circle is a pupil included in the photograph, the voting value at the voting position is obtained by weighted addition of the number of votes of each pixel so that the weight becomes smaller as the chromaticity of the voted pixel increases. Get
The voting value at the voting position is the first integrated voting value,
A circle center position detecting method, wherein a position indicated by the (X, Y) coordinate value of the voting position having the largest first integrated voting value is the center of the circle. In a circle center position detection method for detecting a center position of a circle included in the photographic image from a photographic image,
The coordinates of each pixel having a value other than 0 among the pixels constituting the photographic image are voted on an annular Hough space (X coordinate of the circle center point, Y coordinate of the circle center point, radial coordinate r). The number of times that each voted position (X, Y, r) in the Hough space that has been voted has been voted for ,
When the circle is a pupil included in the photograph, the voting value at the voting position is obtained by weighted addition of the number of votes of each pixel so that the weight becomes smaller as the chromaticity of the voted pixel increases. Get
Of each of the voting positions, the voting value of the voting position having the same (X, Y) coordinate value is added to obtain a second integrated voting value,
A circle center position detection method, wherein a position indicated by the (X, Y) coordinate value corresponding to the second integrated vote value is the center of the circle.   Of the voting positions having the same (X, Y) coordinate value, only the voting values of the voting positions within the radius range of the circle are added to obtain the second integrated voting value. The circle center position detecting method according to claim 2, wherein: Converting said photographic image is a color image into a gray-scale image, the circle center position detection method according to any one of claims 1 to 3, characterized in that said voting by using the gray-scale image . 5. The circle center position detection method according to claim 4, wherein the gray scale photographic image is binarized with a predetermined threshold value to obtain a binary image, and the voting is performed using the binary image. As the weight as low brightness of the pixel in which the vote is increased, either the 5 values obtained by weighting and adding the voting count of each of the pixels from claim 1, characterized in that said voting value The circle center position detection method according to 1. The circle center position detecting method according to any one of claims 1 to 6 , wherein a trimmed image obtained by trimming a face photographic image with the vicinity of eyes as an outer frame is used as the photographic image. Detecting the eye position from the face photo image,
The circle center position detection method according to claim 7, wherein the trimming is performed based on the detected eye position. A circle center position detection device for detecting a center position of a circle included in a photographic image from a photographic image,
The coordinates of each pixel having a value other than 0 among the pixels constituting the photographic image are voted on an annular Hough space (X coordinate of the circle center point, Y coordinate of the circle center point, radial coordinate r). The voting frequency of each vote position (X, Y, r) on the voted Hough space is obtained , and when the circle is a pupil included in the photograph, the chromaticity of the voted pixel Voting value acquisition means that obtains a voting value at the voting position by weighting and adding the voting times of each pixel so that the weight becomes smaller as the value increases, and the voting value at the voting position is a first integrated voting value; ,
Center position determining means for determining the position indicated by the (X, Y) coordinate value of the voting position having the largest first integrated voting value as the center of the circle. Circle center position detection device. A circle center position detection device for detecting a center position of a circle included in a photographic image from a photographic image,
The coordinates of each pixel having a value other than 0 among the pixels constituting the photographic image are voted on an annular Hough space (X coordinate of the circle center point, Y coordinate of the circle center point, radial coordinate r). The voting frequency of each vote position (X, Y, r) on the voted Hough space is obtained , and when the circle is a pupil included in the photograph, the chromaticity of the voted pixel Voting value acquisition means for obtaining a voting value at the voting position by weighted addition of the number of votes of each pixel so that the weight becomes smaller as the
Among the voting positions, voting value integrating means for adding the voting values at the voting positions having the same (X, Y) coordinate value to obtain a second integrated voting value;
Center position determination means having center position determining means with the position indicated by the (X, Y) coordinate value corresponding to the second integrated vote value being the largest as the center of the circle apparatus. The voting value integration means adds only the voting values of the voting positions within the radius range of the circle out of the voting positions having the same (X, Y) coordinate value, and 11. The circle center position detecting device according to claim 10 , wherein an integrated vote value is obtained. Image conversion means for obtaining a grayscale photographic image by converting the photographic image that is a color image into gray,
The circle center position detection device according to claim 9 , wherein the vote value acquisition unit performs the vote using the grayscale photographic image. Binarization means for binarizing the grayscale photographic image with a predetermined threshold to obtain a binarized image;
The circle center position detection apparatus according to claim 12 , wherein the vote value acquisition unit performs the vote using the binarized image. The voting value acquisition means uses, as the voting value, a value obtained by weighting and adding the number of votes of each pixel so that the weight increases as the luminance of the voted pixel decreases. The circle center position detection device according to any one of claims 9 to 13 . The circle according to any one of claims 9 to 14 , further comprising trimming means that uses a trimmed image obtained by trimming a face photo image with the vicinity of the eyes as an outer frame, as the photo image. Center position detection device. The circle according to claim 15 , further comprising eye position detecting means for detecting the position of the eyes from the face photograph image and providing information indicating the detected eye positions to the trimming means. Center position detection device. The difference between the center position of the left pupil and the center position of the right pupil determined using the cropped image of the left eye and the cropped image of the right eye is within a predetermined range according to the distance between the left eye and the right eye. It is determined whether or not the center position of the pupil is correct with reference to the above, and the integrated vote value corresponding to the center position obtained last time until it is determined that the center position is correct And a collating unit that obtains the next largest integrated vote value and obtains the position indicated by the (X, Y) coordinate value corresponding to the next largest integrated vote value as the center of the pupil. The circle center position detecting device according to claim 15 or 16 . The collation means obtains the next largest integrated vote value from each of the integrated vote values corresponding to the (X, Y) coordinate values indicating a position more than a predetermined distance from the center position obtained last time. The circle center position detecting device according to claim 17, wherein: Performs filling processing to the photographic image, any one of the photographic images bore filling process has been claim 9, further comprising a pre-processing means for subjecting the voting value obtaining means 18 The center position detection device of description. A program for causing a computer to execute circle center position detection processing for detecting a center position of a circle included in the photographic image from a photographic image,
In the circle center position detection processing, the coordinates of each pixel having a value other than 0 among the pixels constituting the photographic image are converted into an annular Hough space (X coordinate of the circle center point, Y coordinate of the circle center point, radius) Voting on directional coordinates r), and determining the number of votes for each vote position (X, Y, r) on the voted Hough space, and when the circle is a pupil included in the photo, The voting value of each voting position is obtained by weighting and adding the number of voting times of each pixel so that the weight becomes smaller as the chromaticity of the voted pixel becomes larger, and the voting value at the voting position is obtained as a first integrated vote. Voting value acquisition processing as a value,
And a center position determining process for determining a position indicated by the (X, Y) coordinate value of the voting position having the largest first integrated voting value as the center of the circle. A program for causing a computer to execute circle center position detection processing for detecting a center position of a circle included in the photographic image from a photographic image,
In the circle center position detection processing, the coordinates of each pixel having a value other than 0 among the pixels constituting the photographic image are converted into an annular Hough space (X coordinate of the circle center point, Y coordinate of the circle center point, radius) Voting on directional coordinates r), and determining the number of votes for each vote position (X, Y, r) on the voted Hough space, and when the circle is a pupil included in the photo, Voting value acquisition processing for obtaining a voting value at the voting position by weighted addition of the number of voting times of each pixel so that the weight becomes smaller as the chromaticity of the pixel that has voted increases ,
A voting value integration process of adding the voting values at the voting positions having the same (X, Y) coordinate value among the voting positions to obtain a second integrated voting value;
And a center position determination process in which the position indicated by the (X, Y) coordinate value corresponding to the second integrated vote value is the center of the circle. The voting value integration processing adds only the voting values of the voting positions within the radius range of the circle among the voting positions having the same (X, Y) coordinate value, and adds the second voting value. The program according to claim 21 , wherein an integrated vote value is obtained. The circle center position detection process includes an image conversion process for obtaining a grayscale photographic image by performing gray conversion on the photographic image that is a color image,
The voting value acquisition processing, claim 20 to 22 set forth in any one program which is characterized in that the voting by using the gray-scale image. The circle center position detection process includes a binarization process that binarizes the grayscale photographic image with a predetermined threshold to obtain a binarized image;
The program according to claim 23, wherein the vote value acquisition process performs the vote using the binarized image. The voting value acquisition process uses, as the voting value, a value obtained by weighting and adding the voting count of each pixel so that the weight increases as the luminance of the voted pixel decreases. Item 25. The program according to any one of Items 20 to 24 . The circle center position detecting process, the vicinity of the eyes in the outer frame, a claim 20, further comprising a trimming process for trimming image obtained by trimming the face image and the photographic image 25 of The program according to any one of the above. The trimming process includes an eye position detection process for detecting the position of the eye from the face photograph image, and the trimming is performed based on the eye position detected by the eye detection process. Item 26. The program according to item 26 .

Images