JP2007206963A - Image processor, image processing method, program and storage medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve detection accuracy in detection of a pupil from an image. <P>SOLUTION: This image processor comprises an image input part 100 inputting an image; an eye area detection part 101 detecting an eye area from the image inputted by the image input part; and a pupil area extraction part 102 for extracting a pupil area based on the luminance distribution of the image in the eye area detected by the eye area detection part. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a technique for detecting and extracting a target object, in particular, a face and eyes from a still image or a moving image.

  Conventionally, in the field of image recognition, a technique for detecting a recognition target from an image including the recognition target and a background has been devised. In particular, a technique for detecting a face or eyes as a specific recognition target is a very useful technique in various fields ranging from consumer products such as digital cameras to public systems such as security, including post-detection processing. Furthermore, the technique for detecting the pupil in the eye is used as an aid for increasing the accuracy of detecting the eye, or is used for detecting the line-of-sight direction.

As a technique for detecting these pupils, for example, a line-of-sight detection device disclosed in JP-A-08-327887 (Patent Document 1) is known. In this device, the horizontal luminance level distribution across the black eye is smoothed from the image near the eye, and is differentiated to obtain an absolute value, and a threshold value is set and an intermediate point at a location that matches the threshold value is determined as the pupil. The horizontal center position.
Japanese Patent Laid-Open No. 08-327887

  However, in such conventional pupil detection technology, when the resolution of the image is low or when the image is taken with high sensitivity, noise or the like that does not exist in the original subject is included in the luminance level distribution. Therefore, there is a problem that the pupil may be erroneously detected.

  Therefore, the present invention has been made in view of the above-described problems, and an object of the present invention is to improve detection accuracy when a pupil is detected from an image.

  In order to solve the above-described problems and achieve the object, an image processing apparatus according to the present invention includes an image input unit that inputs an image, and an eye region detection that detects an eye region from the image input by the image input unit. And pupil area extracting means for extracting a pupil area based on the luminance distribution of the image in the eye area detected by the eye area detecting means.

  The image processing method according to the present invention is detected in the image input step of inputting an image, the eye region detection step of detecting an eye region from the image input in the image input step, and the eye region detection step. A pupil region extracting step of extracting a pupil region based on the luminance distribution of the image in the eye region.

  A program according to the present invention causes a computer to execute the above image processing method.

  A storage medium according to the present invention stores the above program.

  According to the present invention, it is possible to improve detection accuracy when detecting a pupil from an image.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of the invention will be described in detail with reference to the accompanying drawings.

(First embodiment)
FIG. 1 is a diagram showing a configuration of an image processing apparatus according to the first embodiment of the present invention.

  In FIG. 1, reference numeral 100 denotes an image input unit such as an image pickup unit including an image pickup element such as a lens and a CCD or a CMOS sensor, a scanner, or an input unit for reading an image file. Reference numeral 101 denotes an eye area detection unit that extracts an eye area from the input image. Reference numeral 102 denotes a pupil region extraction unit that extracts a pupil region from the eye region detected by the eye region detection unit 101. Reference numeral 103 denotes a memory that holds the result selected by the pupil region extraction unit 102 as a pupil region candidate. A memory 104 holds an image input by the image input unit 100, a detection result of the eye region detected by the eye region detection unit 101, and a pupil region extraction result extracted by the pupil region extraction unit 102. . Reference numeral 105 denotes a system control unit that controls the entire system, and controls an eye region detection operation instruction, a pupil region extraction operation instruction, and the like. A display unit 106 displays the image input by the image input unit 100, the detection result of the eye region detected by the eye region detection unit 101, and the extraction result of the pupil region extracted by the pupil region extraction unit 102. is there. A recording unit 107 records the image input by the image input unit 100, the detection result of the eye region detected by the eye region detection unit 101, and the extraction result of the pupil region extracted by the pupil region extraction unit 102. is there.

  The pupil extraction operation of the first embodiment will be described with reference to FIGS.

  FIG. 2 is a flowchart showing a main routine of the pupil extraction operation.

  The system control unit 105 captures an image by photographing a subject with the image input unit 100 or reads an image (step S201), and detects an eye region with the eye region detection unit 101 (step S202). In the eye area detection unit 101, the face area detection unit (not shown) or the like detects the face area after detecting the face area, or detects the eye area directly from the input image. However, any other method may be used as long as the eye region is detected. When the eye area is detected, the pupil area extracting unit 102 extracts the pupil area (step S203). When a pupil region is extracted by the pupil region extraction unit 102 (step S203), display is performed on the display unit 106 (step S204), and recording is performed on the recording unit 107 (step S204).

  FIG. 3 is a flowchart showing the pupil region extraction processing routine in step S203 of FIG.

  The outline of the pupil region extraction process will be described with reference to FIG.

  When the system control unit 105 inputs the eye region image detected by the eye region detection unit 101 to the pupil region extraction unit 102, the pupil region extraction unit 102 first creates a horizontal line connecting the assumed positions of the left and right ends of the pupil. An approximately vertical line is selected (step S301). Next, a candidate is selected from the selected pupil left end right end candidate group, which is a pair of the left end and the right end of the pupil (step S302). Finally, if there are a plurality of candidate left and right edge pairs, the left and right edges of the pupil are selected (step S303), and the area sandwiched between them is extracted as the pupil area (step S304).

  Below, the detail of each selection method is demonstrated.

  FIG. 4 is a flowchart showing a pupil left end / right end candidate group selection processing routine in step S301 of FIG.

  In the pupil region extraction unit 102, the system control unit 105 calculates a value obtained by adding the luminances of the pixels in the eye region detected by the eye region detection unit 101 in the vertical direction for each vertical line (step S401). ). Then, a graph of horizontal (left-right) distribution of the calculated luminance sum is created (step S402). For the obtained luminance sum in the horizontal direction, a smoothing and non-smoothing distribution is prepared (step S403). This is because, for example, when the input image is rough, the noise can be reduced by smoothing the distribution.

  Next, the difference between the luminance sum of one vertical line and the luminance sum of the adjacent vertical line is calculated (step S404), and the average value of all the difference values that have increased, and the difference decreased. An average value of all the difference values being calculated is calculated (step S405). Therefore, a vertical line in which the difference from the luminance sum of the adjacent vertical lines has decreased and has changed to be equal to or greater than the average value of the decreased difference values is set as a pupil left end candidate line. Further, a vertical line in which the difference from the luminance sum of the adjacent vertical lines has increased and has changed to be equal to or greater than the average value of the increased difference values is set as a pupil right end candidate line. Then, they are selected as a pupil left end right end candidate group (step S406). In other words, where the difference from the brightness sum of the adjacent vertical line is greatly reduced, it is from the whitish area to the dark area, and where the difference from the brightness sum of the adjacent vertical line is greatly increased is from the dark area. It is thought that it has changed to a whitish area. Therefore, the pupil left end right end candidate group can be selected as described above.

  FIG. 5 is a schematic diagram showing a method for selecting a candidate group for the left and right pupil ends.

  In this example, in the luminance sum distribution, there are two vertical lines that are equal to or greater than the average value of the difference values in which the difference from the adjacent vertical line is decreasing, and the vertical lines that are equal to or greater than the average value of the increasing difference values. Is elected. In addition, if there are continuous lines that decrease more than the average value of the difference values that are decreasing, or continuous lines that increase more than the average value of the difference values that are increasing, respectively, the left edge of the pupil It is considered that this is a series of areas of the right end of the pupil. For this reason, one of the vertical lines continuously decreasing or increasing may be selected as the pupil left end candidate and the pupil right end candidate. In this way, if vertical lines that greatly change from white to black and black to white are selected as candidates corresponding to the left and right pupils of the pupil, it is highly likely that the correct left and right pupil candidates can be extracted. It is done.

  FIG. 6 is a flowchart showing the pupil left / right end candidate pair selection processing routine in step S302 of FIG.

  The system control unit 105 prepares the pupil region extraction unit 102 to smooth or not smooth the vertical line luminance sum distribution (step S601). However, if the vertical line luminance sum distribution is smoothed in the pupil left end / right end candidate group selection processing step S301 in FIG. 3, the smoothing is not performed in step S601. That is, three types of luminance sum distribution data, which are smoothed in the pupil left end right end candidate group selection processing step S301, those smoothed in the pupil left and right end candidate pair selection processing step S302, and those which are not smoothed, are obtained thereafter. Will be used for processing.

  Next, the maximum value and the minimum value of the luminance sum distribution are calculated (step S602). When the calculated number of minimum values is one (step S603), the left pupil candidate that first exists in the pupil left end candidate group by scanning to the outer corner of the eye from the minimum value is the left-right end candidate pair of pupils. The left end side is set (step S604). Further, in the right pupil candidate group, the right pupil candidate that first exists after scanning toward the top of the eye from the minimum value is set as the right end of the left and right pupil candidate pair (step S605). This is because when the number of local minimum values is one, the black region is only the pupil region, and it is considered that the left end of the pupil is located on the left side and the right end of the pupil is located on the right side across the local minimum value.

  When the number of calculated local minimum values is two (step S603), the left pupil candidate that first exists in the direction of the outer corner of the eye from the lower left corner group in the pupil left end candidate group. Is the left end side of the candidate pair of left and right pupil ends (step S606). Furthermore, in the right pupil candidate group, the right pupil candidate that is first present in the direction of the head of the eye after the first minimum value after scanning from the outer corner of the eye is the right end side of the left and right pupil candidate pair. (Step S607). This means that when the number of local minimums is 2, there is a high-luminance small area where a light source called catchlight is reflected near the center of the pupil, and the local minimum represents two pupil areas sandwiching the catchlight. It is because it is thought that it is. This is because it is considered that the left end of the pupil is on the left side of the minimum value on the left side of the catchlight and the right end of the pupil is on the right side of the minimum value on the right side of the catchlight.

  When the calculated number of local minimum values is other than 1 or 2 (step S603), the eye corners are compared with the local minimum value next to the first local maximum value after scanning from the outer corner of the pupil left end candidate group. The pupil left end candidate that first exists in the direction of the side is defined as the left end side of the left and right pupil candidate pair (step S608). Further, in the right pupil candidate group, the right pupil candidate that first exists in the direction of the head of the eye from the minimum next to the first maximum after scanning from the outer corner of the eye, It is set as the right end side of the right and left pupil candidate pair (step S609). In addition, in the left pupil candidate group, the left pupil candidate that first exists in the direction of the outer corner of the eye after the minimum following the first maximum after scanning from the top of the eye, The left and right pupil candidate pairs are set to the left end side (step S610). Further, in the right pupil candidate group, the right pupil candidate first present in the direction of the head side from the minimum value next to the first maximum value scanned from the top of the eye is the right end side of the left and right pupil candidate pair. (Step S611).

  That is, when the number of local minimum values is other than one or two, there is a black region in addition to the pupil region, and the left and right pupil candidate pairs may differ depending on the scanning direction. Scan from the side and select candidate left and right edge pairs for each.

  As described above, the selection method of the left and right pupil candidate pairs is changed according to the number of minimum values of the luminance sum distribution so that the right and left pupil right pair is not removed.

  7 to 14 are schematic diagrams showing a method of selecting the left and right pupil candidate pairs in step S302 of FIG.

  The example of FIG. 7 shows a case where the minimum value of the luminance sum distribution is one. In this example, the minimum value 701 is a vertical line portion near the center of the pupil, and the left pupil candidate 702 that first exists in the left direction from there is selected as the left end of the left and right pupil candidate pair. Also, the right pupil candidate 703 that first exists in the right direction of the minimum value 701 is selected as the right end of the left and right pupil candidate pair.

  The example of FIG. 8 shows a case where the minimum value of the luminance sum distribution is two. In this example, the minimum value 801 and the minimum value 802 are vertical line portions of the pupil region sandwiching the catchlight portion near the center of the pupil, and the left pupil candidate 803 that first exists in the left direction from the minimum value 801 is the left and right ends of the pupil. Selected at the left end of the candidate pair. Also, the right pupil candidate 804 present first in the right direction from the minimum value 802 is selected as the right end of the left and right pupil candidate pair.

  In either case of FIG. 7 or FIG. 8, the result of scanning from the corner of the eye is the same as the result of scanning from the head of the eye, so that only the scanning from the corner of the eye needs to be performed.

  The example of FIGS. 9 and 10 shows a case where the minimum value of the luminance sum distribution is not one or two. The minimum value exists in the vertical line portion of the pupil region sandwiching the catch light portion near the center of the pupil, the portion outside the pupil region, the noise portion, and the like.

  FIG. 9 shows the result of scanning from the corner of the eye. When scanning from the outer corner of the eye, the maximum value 901 present first is the vertical line portion near the white eye region on the left side of the pupil, and the next smallest local value 902 is the vertical line portion of the pupil region on the outer corner side of the catch light. Become. Further, the next maximum value is the vertical line portion of the catchlight portion, and the next minimum value 903 is the vertical line portion of the pupil region on the eye side of the catchlight. The pupil left end candidate 904 that first exists in the left direction from the minimum value 902 that exists next to the first existing maximum value 901 is selected as the left end of the left and right end candidate pair of the pupil, and the right direction from the next existing minimum value 903 The right pupil candidate 905 that exists first is selected as the right end of the left and right pupil candidate pair.

  FIG. 10 shows the result of scanning from the eye side. When scanning from the front of the eye, the maximum value 1001 existing first is a vertical line portion near the noise edge near the white eye region on the right side of the pupil, and the next minimum value 1002 is a vertical line portion near the noise. . The next maximum value is the vertical line portion near the left side of the noise edge near the white eye region on the right side of the pupil, and the next minimum value 1003 is the vertical line portion of the pupil region on the eye side from the catchlight. Become. The pupil right end candidate 1004 that exists first in the right direction from the minimum value 1002 that exists next to the maximum value 1001 that exists first is the right edge of the noise, and the pupil that first exists in the left direction from the minimum value 1003 that exists next. The left end candidate 1005 is the right end of the catch light. That is, the selected pupil left and right edge candidate pair indicates the wrong place.

  As shown in FIGS. 9 and 10, even when there is noise in the eye area, the right pupil right and left ends are more likely to be selected by performing the scan from the corner of the eye and the scan from the top of the eye. .

  The example of FIGS. 11 and 12 shows a case where the luminance sum distribution has a minimum value other than one or two and the pupil is closer to the left side in the eye. The minimum value exists in the vertical line portion of the pupil region sandwiching the catch light portion near the center of the pupil, the noise portion, and the like.

  FIG. 11 shows the result of scanning from the corner of the eye. When scanning from the outer corner of the eye, the maximum value 1101 existing first is the vertical line portion of the catchlight portion, and the next minimum value 1102 is the vertical line portion of the pupil region on the eye side of the catchlight. The next maximum value is the vertical line portion near the white eye region on the right side of the pupil, and the next minimum value 1103 is the vertical line portion near the noise near the white eye region on the right side of the pupil. The left pupil candidate 1104 that exists first in the left direction from the minimum value 1102 that exists next to the maximum value 1101 that exists first is the right end of the catchlight, and first exists in the right direction from the minimum value 1103 that exists next. Because there is no right pupil candidate to be selected, it is not selected.

  FIG. 12 shows the result of scanning from the eye side. When scanning from the front of the eye, the maximum value 1201 that exists first is the vertical line portion near the white eye region on the right side of the pupil, and the next minimum value 1202 is the vertical line portion of the pupil region on the eye side from the catchlight. Become. The next maximum value is the vertical line portion of the catchlight portion, and the next minimum value 1203 is the vertical line portion of the pupil region on the outer corner of the catchlight. The right pupil candidate 1204 that exists first in the right direction from the minimum value 1202 that exists next to the maximum value 1201 that exists first is selected as the right end of the left and right candidate pair of pupils, and the left value from the next existing minimum value 1202 The left pupil candidate 1205 that exists first is selected as the left end of the left and right pupil candidate pair.

  As shown in FIGS. 11 and 12, even when the pupil is on the left side in the eye, the right and left ends of the right pupil are selected by performing the scan from the outer corner of the eye and the scan from the front of the eye. The possibility increases. Similarly, when the pupil is on the right side of the eye, the possibility that the right and left ends of the pupil are selected in the same manner is increased. Furthermore, in FIG. 11 and FIG. 12, even when there is no noise in the eye area, a candidate pair of left and right ends of the pupil can be selected by the same selection method as in the case of two minimum values (FIG. 8).

  The example of FIGS. 13 and 14 shows a case where the luminance sum distribution is smoothed when there is noise in the eye region (FIGS. 9 and 10). As is apparent from the figure, the right and left edge candidate pair can be selected by the same selection method as in the case where the luminance sum distribution in the noise portion is smooth and there are two minimum values (FIG. 8). it can.

  When the luminance sum distribution is smoothed in step S403 of FIG. 4, the subsequent processing steps are also performed on the smoothed luminance sum distribution. That is, in the step of selecting the pupil left end right end candidate group, the smoothed luminance sum distribution is used for the calculation. On the other hand, when the luminance sum distribution is smoothed in step S601 of FIG. 6, the luminance sum distribution that has not been smoothed is used in the calculation in the previous step of selecting the left and right pupil candidate groups. Is done. In the step of calculating the maximum value and the minimum value of the luminance sum distribution for selecting the left and right pupil candidate pairs, a smoothed luminance sum distribution is used. Thus, the smoothing of the luminance sum distribution is performed before the step of selecting the pupil left end right end candidate group, or between the step of selecting the pupil left end right end candidate group and the step of selecting the pupil left and right end candidate pair. You can choose what to do. As a result, it is possible to increase the possibility of selecting the right and left pupil candidate pairs without being affected by noise or the like. In addition, even when the luminance sum distribution is not smoothed, a plurality of left and right pupil candidate pairs can be selected, and the possibility of selecting the right and left pupil candidate pairs can be further increased.

  FIG. 15 is a flowchart showing the pupil left and right edge selection processing routine in step S303 of FIG.

  In the pupil region extraction unit 102, the system control unit 105 measures the luminance distribution in the region sandwiched between the left and right pupil candidate pairs, and sets a luminance threshold value that is binarized into black and white (step S1501). Examples of the threshold value determination method include an average value of the luminance distribution and a value that maximizes the inter-class variance ratio by the “Otsu method”. However, other methods may be used as long as the threshold value can be determined. .

  Next, the pixel having the maximum luminance is selected within the region sandwiched between the left and right pupil candidate pairs (step S1502). If it is a correct pair of left and right pupil candidates, it is considered that there is a part corresponding to the catchlight near the center, so the area for selecting the pixel with the highest brightness is further near the center of the area sandwiched between the left and right pupil candidate pairs. It may be limited. Then, the circularity of an area equal to or higher than the luminance threshold including the pixel with the maximum luminance is calculated (step S1503). Finally, for the area between the candidate pair of left and right pupils selected under each condition, the circularity of the area above the luminance threshold that contains the pixel with the highest luminance is compared, and the one with the highest circularity is compared with the pupil. The left and right ends are set (step S1504). For example, if the region above the luminance threshold including the pixel with the maximum luminance is a catchlight portion, the circularity is greater than when other portions are selected. Details of the circularity calculation processing method will be described below.

  FIG. 16 is a flowchart of the circularity calculation processing routine for the region sandwiched between the left and right edge candidate pairs.

  In the pupil region extraction unit 102, the system control unit 105 obtains the distance from the pixel having the maximum luminance to the pixel that first shifts to the region below the luminance threshold toward the outside (step S1601). Next, the distance is determined by 360 ° in a rotation direction by 45 ° (step S1602). The rotation angle and distance are plotted on a graph with the rotation direction on the X axis and the distance obtained on the Y axis (step S1603), and an approximate straight line (straight line represented by y = b) of the plotted points is obtained (step S1603). S1604). Finally, the least square error between the plotted points and the approximate straight line is calculated (step S1605). If it is closer to a circle, the distance from the pixel with the maximum luminance to the pixel that first shifts to the region below the luminance threshold toward the outside becomes more constant, so the least square error between the plotted point and the approximate line becomes smaller. . Accordingly, it is assumed that the smaller the least square error, the greater the circularity (step S1606).

  17 and 18 are schematic diagrams showing a method for selecting the left and right ends of the pupil.

  The example in FIG. 17 shows a case where the right and left pupil candidate pair is the right and left pupils, and the pixel with the maximum luminance exists in the catchlight portion. In this case, the catchlight portion is an area that is equal to or higher than the luminance threshold including pixels with the maximum luminance, and the pupil portion outside the catchlight portion is an area that is equal to or lower than the luminance threshold. Therefore, the distance from the pixel having the maximum luminance to the pixel that first shifts to the area below the luminance threshold toward the outside is the distance to the edge of the catchlight. In other words, since the catchlight portion is likely to have a nearly circular shape, the distance is likely to be constant, and the least square error between the points plotted on the graph of rotation angle and distance and its approximate line is small. Is likely to be. In this example, the least square error is zero.

  The example of FIG. 18 shows a case where the left and right edge candidate pair is a white-eye portion on the right side of the pupil due to the influence of noise on the right side of the pupil, and a pixel with the maximum luminance exists in the white-eye portion. When the circularity is calculated in the same manner as in the example of FIG. 17, the distance from the pixel having the maximum luminance to the pixel that first shifts to the region below the luminance threshold value varies outwardly, and is plotted in the graph of the rotation angle and the distance And the least square error between the approximate line and the approximate straight line have a certain size.

  As shown in FIG. 17 and FIG. 18, when the right and left pupil candidate pair is the right and left right pupil pair, the transition from the pixel with the highest luminance to the region below the luminance threshold value is made for the first time as compared with the incorrect case. The distance to the target pixel is constant. Therefore, since the least square error between the point plotted on the graph of the rotation angle and distance when the distance is viewed in the rotation direction and the approximate straight line becomes small, the left and right ends of the pupil can be correctly selected.

(Second Embodiment)
In the second embodiment, as shown in the first embodiment, from the pixel having the maximum luminance within the region sandwiched between the left and right pupil candidate pairs to the pixel that first shifts to the region below the luminance threshold toward the outside. The distance is not calculated. Instead, the degree of circularity is obtained by tracking the boundary line between a region having a maximum luminance pixel equal to or higher than a certain luminance threshold and a region existing outside the luminance threshold equal to or lower than the luminance threshold.

  FIG. 19 is a flowchart of the pupil left / right end selection processing routine of the second embodiment.

  1 to 14 are the same as those in the first embodiment.

  The pupil left and right edge selection processing will be described in detail with reference to FIG.

In the pupil region extraction unit 102, the system control unit 105 scans from the upper left to the lower right within the region sandwiched between the left and right pupil candidate pairs. Then, a pixel that first shifts to a region having a luminance threshold value equal to or higher than a certain luminance threshold is obtained (step S1901), and that pixel is set as a target pixel (step S1902). Next, a pixel having a luminance threshold value or more is searched counterclockwise around the target pixel (step S1903), and a distance from the target pixel to the searched pixel is obtained (step S1904). At this time, if the searched pixel is adjacent to the target pixel in the vertical and horizontal directions, the distance is 1, and if the pixel is diagonal, the distance is √2. If the searched pixel is not the same pixel as the first target pixel (step S1905), the process returns to step S1902, the searched pixel is set as the target pixel (step S1902), and again exceeds the luminance threshold value counterclockwise around the target pixel. Are searched for (step S1903). Then, the distance from the target pixel to the searched pixel is obtained (step S1904). When the searched pixel becomes the same pixel as the first pixel of interest (step S1905), a total distance L from the first pixel to the same pixel is obtained (step S1906). Then, the total number of pixels S in a region having a maximum luminance pixel equal to or higher than a certain luminance threshold is obtained (step S1907). Finally, a case where L ² / 4πS is close to 1 is assumed to have a large circularity (step S1908).

  As described above, according to the first and second embodiments described above, a plurality of pupil candidates are extracted based on the characteristics and positional relationship of the eyes and pupils, and finally a region that seems to be a pupil is narrowed down to one. Yes. For this reason, it is possible to accurately detect the pupil even when the luminance level distribution of the image includes noise or the like that is not present in the original subject.

(Other embodiments)
The object of each embodiment is also achieved by the following method. That is, a storage medium (or recording medium) in which a program code of software that realizes the functions of the above-described embodiments is recorded is supplied to the system or apparatus. Then, the computer (or CPU or MPU) of the system or apparatus reads and executes the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiment, and the storage medium storing the program code constitutes the present invention. Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but the present invention includes the following cases. That is, based on the instruction of the program code, an operating system (OS) running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

  Furthermore, the following cases are also included in the present invention. That is, the program code read from the storage medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer. Thereafter, based on the instruction of the program code, the CPU or the like provided in the function expansion card or function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

  When the present invention is applied to the above storage medium, the storage medium stores program codes corresponding to the procedure described above.

1 is a diagram illustrating a configuration of an image processing apparatus according to a first embodiment of the present invention. It is a flowchart which shows the main routine of pupil extraction operation | movement. It is a flowchart which shows the pupil area extraction process routine in step S203 of FIG. It is a flowchart which shows the pupil left end right end candidate group selection process routine in step S301 of FIG. It is a schematic diagram showing the method of selecting a pupil left end right end candidate group. It is a flowchart which shows the pupil left-right end candidate pair selection process routine in step S302 of FIG. It is a schematic diagram showing the method of selecting the pupil left-right end candidate pair in step S302 of FIG. It is a schematic diagram showing the method of selecting the pupil left-right end candidate pair in step S302 of FIG. It is a schematic diagram showing the method of selecting the pupil left-right end candidate pair in step S302 of FIG. It is a schematic diagram showing the method of selecting the pupil left-right end candidate pair in step S302 of FIG. It is a schematic diagram showing the method of selecting the pupil left-right end candidate pair in step S302 of FIG. It is a schematic diagram showing the method of selecting the pupil left-right end candidate pair in step S302 of FIG. It is a schematic diagram showing the method of selecting the pupil left-right end candidate pair in step S302 of FIG. It is a schematic diagram showing the method of selecting the pupil left-right end candidate pair in step S302 of FIG. It is a flowchart which shows the pupil left-right end selection process routine in step S303 of FIG. It is a flowchart of the circularity calculation processing routine of the area | region pinched by the pupil right-and-left end candidate pair. It is a schematic diagram showing the method of selecting a pupil right-and-left end. It is a schematic diagram showing the method of selecting a pupil right-and-left end. It is a flowchart of the pupil left and right end selection processing routine of the second embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Image input part 101 Face area detection part 102 Intermediate result memory 103 Eye area extraction part 104 Memory 105 System control part 106 Display part 107 Recording part

Claims (20)

An image input means for inputting an image;
Eye area detecting means for detecting an eye area from the image input by the image input means;
A pupil region extraction unit that extracts a pupil region based on a luminance distribution of an image in the eye region detected by the eye region detection unit;
An image processing apparatus comprising:   The image processing apparatus according to claim 1, wherein the pupil region extracting unit extracts a pupil region based on a projected component distribution of the image in the eye region.   The image processing apparatus according to claim 1, wherein the pupil region extracting unit selects left and right edge candidates of the pupil based on a projected component distribution of the image in the eye region.   The image processing apparatus according to claim 3, wherein the left and right edge candidates of the pupil are selected based on a difference distribution in the left and right direction of the luminance summation in the vertical direction of the image in the eye region.   The pupil region extracting means selects a candidate pair at the left and right ends of the pupil with the left end and the right end of the pupil as a pair from the left and right end candidates of the pupil, and extracts the pupil region based on the selected candidate pair. The image processing apparatus according to claim 3.   The pupil region extracting means determines the candidate pair of the left and right edges of the pupil, the position of the maximum and minimum values of the horizontal distribution of the vertical luminance sum of the image in the eye region, and the position of the left and right edge candidates of the pupil. The image processing apparatus according to claim 5, wherein selection is performed based on   The pupil region extraction means selects a candidate pair at the left and right ends of the pupil by scanning the luminance distribution of the image in the eye region from the left side of the eye region, from the right side, or from both the left and right sides. The image processing apparatus according to claim 6.   The image processing apparatus according to claim 7, wherein the scan is performed after the luminance distribution of the image in the eye region is smoothed.   The pupil region extraction means sets a luminance threshold value within a candidate pair at the left and right ends of the pupil, and determines a pupil region based on a circularity of a region having a luminance equal to or higher than the luminance threshold with a maximum luminance position in the candidate pair as a center. The image processing apparatus according to claim 5, wherein the image processing apparatus is extracted. An image input process for inputting an image;
An eye region detection step of detecting an eye region from the image input in the image input step;
A pupil region extraction step of extracting a pupil region based on the luminance distribution of the image in the eye region detected in the eye region detection step;
An image processing method comprising:   The image processing method according to claim 10, wherein in the pupil region extracting step, a pupil region is extracted based on a projected component distribution of the image in the eye region.   The image processing method according to claim 10, wherein in the pupil region extracting step, left and right edge candidates of the pupil are selected based on a projected component distribution of the image in the eye region.   The image processing method according to claim 12, wherein the left and right edge candidates of the pupil are selected based on a difference distribution in a horizontal direction of a vertical luminance sum of an image in the eye region.   In the pupil region extracting step, from the left and right edge candidates of the pupil, selecting a candidate pair of left and right ends of the pupil with a left end and a right end of the pupil as a pair, and extracting a pupil region based on the selected candidate pair The image processing method according to claim 12, wherein:   In the pupil region extracting step, the left and right edge candidate pairs of the pupil are determined as the maximum and minimum positions of the horizontal distribution of the vertical luminance sum of the image in the eye region, and the left and right edge candidate positions of the pupil. The image processing method according to claim 14, wherein selection is performed based on the image.   In the pupil region extraction step, candidate pairs at the left and right ends of the pupil are selected by scanning the luminance distribution of the image in the eye region from the left side of the eye region, from the right side, or from both the left and right sides. The image processing method according to claim 15.   The image processing method according to claim 16, wherein the scan is performed after the luminance distribution of the image in the eye region is smoothed.   In the pupil region extraction step, a luminance threshold is set in the candidate pair at the left and right ends of the pupil, and the pupil region is determined based on the circularity of a region having a luminance equal to or higher than the luminance threshold with the maximum luminance position in the candidate pair as a center. The image processing method according to claim 14, wherein extraction is performed.   A program for causing a computer to execute the image processing method according to any one of claims 10 to 18.   A computer-readable storage medium storing the program according to claim 19.

Images