JP2009176047A - Center-of-eye measuring instrument, center-of-eye measuring program, and center-of-eye measuring method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a center-of-eye measuring instrument capable of rapidly measuring the center of the eye even with a low-speed processor, and to provide a center-of-eye measuring program and a center-of-eye measuring method. <P>SOLUTION: An eye region constituted of the pupil and the iris is made to correspond to the coordinate of an orthogonal coordinate system and detected, and then, a right boundary coordinate and a left boundary coordinate positioned in boundaries with other regions are detected. Besides, a plurality of boundary coordinates positioned in the upper right from the right boundary coordinate is detected and also a plurality of boundary coordinates positioned in the upper left from the left boundary coordinate is detected, thereby calculating the center coordinate of the eye region through the use of the plurality of detected boundary coordinates. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a technique for measuring the center of an eye region composed of a pupil and an iris.

  Conventionally, when measuring the line of sight of a person, a point light source of near-infrared light or visible light is installed at a certain distance (50 [cm] to 100 [cm]) from the person, and reflected light on the eyeball surface is photographed. A camera for this purpose was installed, and a reflected image and pupil of a point light source on the surface of the cornea were photographed with this camera, and a human gaze method was calculated from the direction.

For example, in Patent Document 1, when calculating the center of a captured pupil image, first, a region darker than the periphery is connected and segmented, and then the shape of each region is examined, and regions similar in shape to the pupil are obtained. List as candidates. Next, from the enumerated candidate regions, a region closest to the shape of the pupil is set as a pupil region, and edge detection is performed on the obtained pupil region to obtain the contour coordinates of the pupil. Finally, a technique for correcting the contour coordinates of the obtained pupil in consideration of corneal surface reflection and obtaining the center of the pupil by elliptic approximation is disclosed.
JP 2003-79777 A TW RIDLER, 1 other, "Picture Thresholding Using an Iterative Selection Method", IEEE TRANSACTIONS ON SYSTEMS, MAN AND CYBERNETICS, August 1978, VOL. SMC-8, No. 8, p.630-632 N. CHERNOV, 1 other, “Least Squares Fitting of Circles”, Journal of Mathematical Imaging and Vision, 2005, VOL. 23, p.239-252

  However, the method disclosed in Patent Document 1 has a high calculation load. For example, a processor with a relatively low calculation speed such as a DSP (abbreviation of Digital Signal Processor) cannot perform real-time processing. There was a problem that the technique of the above method could not be applied.

  The present invention has been made in view of the above, and it is an object of the present invention to provide an eye center measuring device, an eye center measuring program, and an eye center measuring method capable of measuring the eye center at high speed even with a low-speed processor. .

  According to the first aspect of the present invention, there is provided a video storage unit that stores at least a video of which an eye has been photographed, and the video region is read out from the video storage unit, and an eye region composed of a pupil and an iris is defined in an orthogonal coordinate system. Eye area detecting means for detecting the corresponding to the coordinates, and the first boundary coordinates and the second boundary coordinates which are the eye area and located at the boundary with the other area, and the first boundary coordinates Boundary detection means for further detecting a third boundary coordinate located on the upper right side and further detecting a fourth boundary coordinate located on the upper left side with respect to the second boundary coordinate; and And a center calculating means for calculating center coordinates of the eye area using the fourth boundary coordinates.

  In the present invention, after detecting the eye area composed of the pupil and the iris in association with the coordinates of the orthogonal coordinate system, the first boundary coordinates and the second boundary coordinates located at the boundary with the other area , And further detecting a third boundary coordinate located on the upper right side of the first boundary coordinate, and further detecting a fourth boundary coordinate located on the upper left side of the second boundary coordinate, Since the center coordinates of the eye region are calculated using the boundary coordinates to the fourth boundary coordinates, it is possible to provide an eye center measuring device capable of measuring the center of the eye at high speed even with a low speed processor.

  According to the second aspect of the present invention, the first boundary coordinate is a boundary coordinate located on the right side of the inner coordinate located inside the eye region, and the second boundary coordinate is the inner coordinate. The gist is that the boundary coordinates are located on the left side of the line.

  The gist of the present invention described in claim 3 is that the inner coordinates are estimated center coordinates estimated based on the eye area detected by the eye area detecting means.

  The gist of the present invention described in claim 4 is that the inner coordinates are inner coordinates located below the estimated central coordinates estimated based on the eye area detected by the eye area detecting means. To do.

  The gist of the present invention described in claim 5 is that the inner coordinates are boundary coordinates positioned below estimated central coordinates estimated based on the eye area detected by the eye area detecting means. To do.

  According to a sixth aspect of the present invention, the video is composed of a plurality of pixels having luminance values, and the eye area detecting means includes a plurality of areas where luminance values lower than a predetermined threshold value are continuous. And extracting either the region having the maximum number of pixels included in the region or the second largest region as the eye region based on the vertical positional relationship.

  According to the seventh aspect of the present invention, the center calculating unit fits a plurality of circles having different sizes to the first boundary coordinate to the fourth boundary coordinate, and uses the most suitable circle. The gist is to calculate the center coordinates.

  The present invention according to claim 8, wherein the most suitable circle is a sum of squares of distances from a circumferential coordinate on a circumference of a desired circle to each of the first boundary coordinate to the fourth boundary coordinate. The gist is that it is a circle that minimizes the objective function representing.

  The gist of the present invention described in claim 9 is to cause a computer to function as each means described in any one of claims 1 to 8.

  According to a tenth aspect of the present invention, there is provided a step of accumulating at least an image in which an eye is photographed in an image accumulating unit, an image obtained by reading the image from the image accumulating unit, and an eye region composed of a pupil and an iris being orthogonal Detecting in association with the coordinates of the system, and detecting the first boundary coordinates and the second boundary coordinates which are located in the boundary between the eye area and the other area, from the first boundary coordinates Further detecting a third boundary coordinate located at the upper right and further detecting a fourth boundary coordinate located at the upper left of the second boundary coordinate; and the first to fourth boundary coordinates. And a step of calculating center coordinates of the eye region using the boundary coordinates.

  According to the present invention, it is possible to provide an eye center measuring apparatus, an eye center measuring program, and an eye center measuring method capable of measuring the eye center at high speed even with a low-speed processor.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<About the block configuration and overall processing of the eye center measuring device>
FIG. 1 is a block diagram showing a block configuration of the eye center measuring apparatus according to the present embodiment. This eye center measuring device includes a video input unit 11, a frame extraction unit 12, a threshold calculation unit 13, an eye region detection unit 14, a center estimation unit 15, a boundary detection unit 16, a center calculation unit 17, The video storage unit 21 and the data storage unit 22 are provided.

  This eye center measuring apparatus may be configured as dedicated hardware, or may be configured using a general-purpose computer. Further, the processing of each unit may be executed by a computer program. Further, the computer program may be recorded on a readable recording medium. The processing results processed by each unit are accumulated in the video accumulation unit 21 or stored in the data storage unit 22.

  FIG. 2 is a flowchart showing the overall processing of the eye center measuring apparatus in the present embodiment. First, the video input unit 11 receives an input of a video in which eyes and eyebrows are photographed (hereinafter also referred to as “input image”) and stores the input in the video storage unit 21 (S101).

  Note that the input image handled in the present embodiment is a moving image composed of a plurality of frames, and each frame is a still image composed of a plurality of pixels. Here, assuming that the width of the input image is width and the height is height, each frame is composed of width × height pixels. Each pixel is data (luminance value) expressing a color in the RGB color model, and is composed of an x coordinate value, a y coordinate value, an R value, a G value, and a B value in the frame, and the x coordinate value is 1 or more and width or less. The y-coordinate value is an integer value between 1 and height, and the R value, G value, and B value are integer values between 0 and 255, respectively. Further, the coordinate value of the upper left pixel in each frame is (1, 1), the x axis is directed to the right, and the y axis is directed downward.

  Next, the frame extraction unit 12 reads the video stored in the video storage unit 21 and extracts one frame (hereinafter also referred to as “still image”) from the video composed of a plurality of frames ( S102).

  Subsequently, the threshold value calculation unit 13 calculates a threshold value for detecting the eyebrows and the eyes from the extracted still image, and stores it in the data storage unit 22 (S103).

  The threshold value calculated here is an integer value of 0 or more and 255 or less representing the luminance value, and the pixels included in the eyebrow part and the eye part are values less than the calculated threshold value. For the calculation of the threshold value, the ISODATA method disclosed in Non-Patent Document 1 can be used, or a value designated in advance may be used. Note that the center of the eye can be measured for various eye colors such as black and blue by appropriately changing the threshold value.

  Then, the eye region detection unit 14 reads the calculated threshold value from the data storage unit 22, extracts the eyebrows and the eyes from the still image extracted by the frame extraction unit 12, and based on the vertical positional relationship, After specifying the part, the eye area composed of the pupil and the iris is detected in association with the coordinates of the orthogonal coordinate system, and stored in the data storage unit 22 (S104). Details of processing in the eye area detection unit 14 will be described later.

  Subsequently, the center estimation unit 15 reads the coordinates constituting the eye area from the data storage unit 22, estimates the center coordinates of the eye area (hereinafter may be referred to as “estimated center coordinates”), and the data storage unit 22 (S105). Details of the processing in the center estimation unit 15 will be described later.

  Next, the boundary detection unit 16 reads the coordinates constituting the eye area from the data storage unit 22 and reads the estimated center coordinates from the data storage unit 22, and is the coordinates constituting the eye area, The right initial boundary coordinate and the left initial boundary coordinate located at the boundary are detected, and a plurality of boundary coordinates (hereinafter sometimes referred to as “upper right boundary coordinate group”) located further to the upper right than the right initial boundary coordinate are further detected. In addition, a plurality of boundary coordinates (hereinafter also referred to as “upper left boundary coordinate group”) located at the upper left of the left initial boundary coordinates are further detected and stored in the data storage unit 22 (S106). Details of processing in the boundary detection unit 16 will be described later.

  Finally, the center calculation unit 17 reads the detected right initial boundary coordinates, left initial boundary coordinates, and a plurality of boundary coordinates from the data storage unit 22, and calculates the center coordinates of the eye area (S107). Details of processing in the center calculation unit 17 will be described later.

<About the processing of the eye area detection unit>
FIG. 3 is an explanatory diagram for explaining processing of the eye region detection unit. First, the eye region detection unit 14 extracts the eyebrows and the eyes from the still image extracted by the frame extraction unit 12 using the threshold calculated by the threshold calculation unit 13 (S201). Next, the eye area detection unit 14 specifies a part corresponding to the eye part among the extracted eyebrow part and eye part (S202), and detects the eye area (S203).

  Specifically, first, all the pixels of the still image are classified into pixels having a luminance value higher than the threshold and pixels lower than the threshold. Furthermore, among pixels below the threshold, pixels adjacent in the vertical direction or the horizontal direction are assumed to be the same segment, and one or more segments are extracted.

  Next, for example, when a still image obtained by photographing the vicinity of the eyes is used, it can be assumed that the large segments are the eyebrows and the eyes, so the upper segment is the eyebrow and the lower segment is the eye . More specifically, among the extracted segments, the segment having the maximum number of pixels included and the second largest segment are set as candidates for the eye region. The average of the y-coordinate values of all the pixels included in each of the two candidate segments is calculated to calculate the centroid y-coordinate value, and the segment with the large centroid-y coordinate value, that is, the segment located below is selected as the eye region A segment. Then, a set of pixels included in the detected eye area segment is output as eye area information.

<About the processing of the center estimation unit>
The center estimation unit 15 estimates the center coordinates of the eye area detected by the eye area detection unit 14. Specifically, the center of gravity of the aforementioned eye area information is set as the estimated center coordinates. That is, a function center (S) for obtaining the center of gravity of a set of pixels is defined as shown in Expression (1), and a set of pixels representing eye area information is defined as _Seye . At this time, a coordinate value obtained by center (S _eye ) is output as an estimated center coordinate.

Here, S is a set of pixels, and x _S and y _S are the x coordinate and y coordinate of S, respectively.

  In the following, the subsequent processing will be described using the estimated central coordinates estimated based on the eye area detected by the eye area detection unit 14, but this is an example of the inner coordinates located inside the eye area. Thus, the same effect can be obtained even when the internal coordinates of an arbitrary place inside the eye area are used. For example, internal coordinates positioned below the estimated center coordinates, or boundary coordinates positioned below the estimated center coordinates (corresponding to center-lower boundary coordinates described later) may be used.

<About the processing of the boundary detection unit>
The boundary detection unit 16 detects a plurality of boundary coordinates through three steps. In the first step, the lower center boundary coordinates located below the estimated center coordinates are obtained, and in the second step, the right initial boundary coordinates located on the right side of the lower center boundary coordinates and the left side of the lower center boundary coordinates. In the third step, a plurality of boundary coordinates (upper right boundary coordinate group) positioned on the upper right side of the right initial boundary coordinate and a plurality of boundary coordinates positioned on the upper left side of the left initial boundary coordinate are obtained in the third step. (Upper left boundary coordinate group).

  FIG. 4 is an explanatory diagram for explaining the processing of the first step. The boundary detection unit 16 fixes x with respect to the estimated center coordinates (x, y) estimated by the center estimation unit 15, and sets the point where y is maximum in the eye region as the lower center boundary coordinates (S301). ).

  Alternatively, with respect to the estimated center coordinates (x, y), x is fixed and y is incremented by 1 to search the lower center boundary coordinates, and (x, y) is within the eye area, and (x, y + 1) ) Is outside the eye area, the (x, y) may be used as the lower center boundary coordinate.

  FIG. 5 is an explanatory diagram for explaining the processing of the second step. The boundary detection unit 16 fixes y with respect to the lower center boundary coordinate (x, y), increases x by 1 to search the right initial boundary coordinate, and lower the center lower boundary coordinate (x, y). On the other hand, y is fixed, x is decreased by 1 and the left initial boundary coordinate is searched (S401).

  Specifically, if x is found where (x, y) is in the eye area and (x + 1, y) is outside the eye area, that (x, y) is set as the right initial boundary coordinate. Similarly, if x is found where (x, y) is within the eye region and (x-1, y) is outside the eye region, the (x, y) left initial boundary coordinate is set.

  FIG. 6 is an explanatory diagram for explaining the processing of the third step. The boundary detection unit 16 increases y of the right initial boundary coordinate, detects x by increasing x, and increases y of the left initial boundary coordinate, and then decreases x to decrease the boundary coordinate. Detect (S501). Then, the process of S501 is repeated to detect a plurality of boundary coordinates (S502). More specific description will be given below.

First, a method for calculating the upper right boundary coordinate group will be described. The boundary detection unit 16 fixes x with respect to the right initial boundary coordinates (x, y), increases y, and sets the coordinates of the eye area that is found first as the upper inner coordinates of the right initial boundary coordinates. That is, the function for calculating the upper inner coordinates of the right initial boundary coordinates of the coordinates (x, y) is a function shown in Expression (2).

Next, the boundary coordinates on the right side of the upper inner coordinates (x, y) obtained by the above procedure are searched. Specifically, with respect to the upper inner coordinates (x, y), y is fixed, x is incremented by 1 and the boundary coordinates are searched, and the coordinates on the right of (x, y) are outside the eye area. If a certain x is found, that (x, y) is set as the boundary coordinate. That is, the function for calculating the boundary coordinates as the upper right boundary coordinate group is a function shown in Expression (3).

And the boundary detection part 16 can obtain the upper right boundary coordinate group by searching the upper inner coordinates of the detected boundary coordinates and repeatedly detecting the upper right boundary coordinates in the same procedure. That is, the upper right boundary coordinate group {r _k } can be expressed by Expression (4) using Expression (2) and Expression (3), where R is the right initial boundary coordinate. However, the calculation of the boundary coordinates ends when upper (r _k ) no longer exists.

The calculation of the upper left boundary coordinate group is similarly performed. First, as shown in Expression (5), a function for calculating boundary coordinates as an upper left boundary coordinate group is defined.

Expression (5) searches for boundary coordinates by fixing y with respect to coordinates (x, y) and decreasing x by 1 and the coordinates on the left of (x, y) are outside the eye area. If x is found, the (x, y) is set as the boundary coordinates. That is, the upper left boundary coordinate group {l _k } can be expressed by Expression (6) using Expression (2) and Expression (5), where L is the left initial boundary coordinate. However, the calculation of the boundary coordinates ends when upper (l _k ) no longer exists.

Finally, the boundary detection unit 16 outputs {r _k } and {l _k } calculated by the above procedure together as a plurality of boundary coordinates.

<About the processing of the center calculation unit>
FIG. 7 is an explanatory diagram for explaining the processing of the center calculation unit. The center calculation unit 17 fits a plurality of circles having different sizes to the plurality of boundary coordinates detected by the boundary detection unit 16, and uses the center coordinates of the most suitable circle obtained by the fitting as the center coordinates of the eye region. Calculate (S601).

For fitting a circle to a plurality of boundary coordinates, the Last Squares Fitting algorithm described in Non-Patent Document 2 can be used. Specifically, circle fitting is performed by minimizing the objective function F. A plurality of boundary coordinates are B = {(x _i , y _i )} _{i = 1, 2,..., N} , the center coordinates of the circle obtained by fitting are (a, b), and the radius is R. The objective function F (a, b, R) representing the sum of the squares of the distances from the coordinates (x _i , y _i ) on the circumference of each to the boundary coordinates is a function shown in Expression (7).

However, in the present embodiment, the objective function F (a, b, R) is represented by Expression (8) and Expression (9).

Then, a, b, and R that minimize the objective function F (a, b, R) are obtained. When partial differentiation of F (a, b, R) is performed for B, C, and D, Expression (10) is obtained.

  The simultaneous equations shown in equation (10) can be solved by Cholesky decomposition to obtain B, C, and D, and a, b, and R can be obtained from equation (11). The center calculation unit 17 outputs (a, b) obtained here as the center coordinates of the eye region.

  According to the present embodiment, after detecting the eye region composed of the pupil and the iris in association with the coordinates of the orthogonal coordinate system, the right boundary coordinate and the left boundary coordinate located at the boundary with the other region are detected. , Further detecting a plurality of boundary coordinates located at the upper right than the right boundary coordinates, further detecting a plurality of boundary coordinates located at the upper left than the left boundary coordinates, and using the detected plurality of boundary coordinates, the eye region Therefore, it is possible to provide an eye center measuring apparatus capable of measuring the eye center at high speed even with a low speed processor.

It is a block diagram which shows the block configuration of the eye center measuring apparatus in this Embodiment. It is a flow which shows the whole process of the eye center measuring apparatus in this Embodiment. It is explanatory drawing for demonstrating the process of an eye area | region detection part. It is explanatory drawing for demonstrating the process of the 1st step in a boundary detection part. It is explanatory drawing for demonstrating the process of the 2nd step in a boundary detection part. It is explanatory drawing for demonstrating the process of the 3rd step in a boundary detection part. It is explanatory drawing for demonstrating the process of a center calculation part.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 ... Video input part 12 ... Frame extraction part 13 ... Threshold calculation part 14 ... Eye area | region detection part 15 ... Center estimation part 16 ... Boundary detection part 17 ... Center calculation part 21 ... Video storage part 22 ... Data storage part

Claims (10)

Video storage means for storing at least the video of which the eye was shot;
Eye region detection means for reading out the image from the image storage unit and detecting an eye region composed of a pupil and an iris in association with coordinates of an orthogonal coordinate system;
A first boundary coordinate and a second boundary coordinate, which are the eye region and located at a boundary with another region, are detected, and a third boundary coordinate positioned at the upper right of the first boundary coordinate is further detected. Boundary detecting means for detecting, and further detecting a fourth boundary coordinate located at the upper left of the second boundary coordinate;
Center calculating means for calculating center coordinates of the eye region using the first boundary coordinates to the fourth boundary coordinates;
An eye center measuring apparatus characterized by comprising:   The first boundary coordinate is a boundary coordinate located on the right side of the internal coordinate located inside the eye area, and the second boundary coordinate is a boundary coordinate located on the left side of the internal coordinate. The eye center measuring apparatus according to claim 1.   3. The eye center measuring device according to claim 2, wherein the inner coordinates are estimated center coordinates estimated based on the eye area detected by the eye area detecting means.   3. The eye center measurement according to claim 2, wherein the inner coordinates are inner coordinates located below estimated central coordinates estimated based on the eye area detected by the eye area detecting means. apparatus.   3. The eye center measurement according to claim 2, wherein the inner coordinates are boundary coordinates located below estimated center coordinates estimated based on the eye area detected by the eye area detecting means. apparatus. The image is composed of a plurality of pixels having luminance values,
The eye area detecting means includes
A plurality of regions having continuous luminance values lower than a predetermined threshold are extracted, and either the region maximizing the number of pixels included in the region or the second largest region is determined based on the vertical positional relationship. The eye center measuring device according to claim 1, wherein the eye center measuring device is detected as a region. The center calculating means includes
7. A plurality of circles having different sizes are fitted to the first boundary coordinate to the fourth boundary coordinate, respectively, and the center coordinate is calculated using the most suitable circle. The eye center measuring apparatus according to any one of the preceding claims.   The most suitable circle is a circle that minimizes an objective function that represents the sum of squares of the distances from the circumferential coordinates on the circumference of the circle to be obtained to each of the first boundary coordinates to the fourth boundary coordinates. The eye center measuring apparatus according to claim 7, wherein the eye center measuring apparatus is provided.   9. An eye center measurement program that causes a computer to function as each means according to claim 1. Storing at least an image of which the eye is photographed in an image storage means;
Reading the video from the video storage means and detecting an eye region composed of a pupil and an iris in association with coordinates of an orthogonal coordinate system;
A first boundary coordinate and a second boundary coordinate, which are the eye region and located at a boundary with another region, are detected, and a third boundary coordinate positioned at the upper right of the first boundary coordinate is further detected. Detecting and further detecting a fourth boundary coordinate located at the upper left of the second boundary coordinate;
Calculating center coordinates of the eye region using the first boundary coordinates to the fourth boundary coordinates;
A method for measuring an eye center, comprising:

Images