JP2016224691A - Sight line calibration program, sight line calibration device, and sight line calibration method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a sight line calibration program, a sight line calibration device, and a sight line calibration method for improving sight line detection accuracy.SOLUTION: A sight line calibration program causes a computer to execute processing of: detecting a user's sight line; detecting height of the user's eyes; and calibrating the detected sight line according to the detected height of the eyes, using a sight line calibration model corresponding to the height of eyes.SELECTED DRAWING: Figure 9

Description

  The present disclosure relates to a line-of-sight calibration program, a line-of-sight calibration apparatus, and a line-of-sight calibration method.

  In line-of-sight detection, an error for each individual appears between the line of sight detected by the computer and the actual line of sight due to individual differences in eyeball shape. Therefore, a technique for correcting the line of sight by performing calibration is disclosed (for example, see Patent Document 1).

JP 2006-285715 A

  However, it is difficult to use a technique for performing calibration for each individual in an apparatus used by an unspecified number. On the other hand, when calibration is not performed, the line-of-sight detection accuracy decreases.

  In one aspect, an object of the present invention is to provide a gaze calibration program, a gaze calibration apparatus, and a gaze calibration method that can improve gaze detection accuracy.

  In one aspect, the line-of-sight calibration program uses, on a computer, a process for detecting a user's line of sight, a process for detecting the eye height of the user, and a line-of-sight calibration model corresponding to the eye height. And a process of calibrating the detected line of sight according to the detected eye height.

  The line-of-sight detection accuracy can be improved.

(A) And (b) is a figure showing a gaze angle change. It is the schematic of a gaze recognition apparatus. It is a figure explaining an error. (A) And (b) is a figure which illustrates about an error. It is a figure for demonstrating the gaze angle change in case a visual object makes | forms a perpendicular surface. It is a figure for demonstrating the gaze angle change in case a visual object makes a horizontal surface. It is a figure for demonstrating the gaze angle change in case a visual object inclines. 1 is a block diagram illustrating a configuration of a line-of-sight calibration apparatus according to Embodiment 1. FIG. It is a flowchart showing an example of operation | movement of a visual line calibration apparatus. (A) is a figure which illustrates the table of the correction coefficient stored in the database, (b) is an image figure of a line-of-sight calibration. It is an image figure of the visual line calibration which concerns on the modification 1. FIG. 10 is an example of a flowchart according to Modification 2. This is an example in which the optical axis of the wide-angle camera is directed obliquely upward from the horizontal direction from a position lower than the average face position of the user. 6 is a block diagram illustrating a configuration of a line-of-sight calibration apparatus according to Embodiment 2. FIG. (A)-(d) is a wide-angle image, (e) is a table showing the correspondence of the gravity center position of a face outline, the difference of a y-axis direction, and the distance of the narrow-angle camera 30 and a face. It is a block diagram for demonstrating the hardware constitutions of a control part.

  Prior to the description of the embodiments, an outline of a line-of-sight detection device including an imaging device will be described. The line-of-sight detection device is a device that recognizes a user's line-of-sight direction in a non-contact manner using an imaging device. For example, as illustrated in FIG. 1A, a reference point is a corneal reflection image (Purquieni image) generated in the user's eyeball (for example, pupil or iris) with respect to the infrared LED irradiated from the direction facing the user. And By detecting a change in the relative position (pixel) between this reference point and a point of interest such as the center of gravity of the pupil, the change in the line-of-sight angle can be calculated. For example, in the example of FIG. 1B, it is detected that the line of sight is directed downward when FIG. 1A is used as the reference position, and the angle can be calculated.

  FIG. 2 is a schematic diagram of the line-of-sight detection device 200. As illustrated in FIG. 2, the line-of-sight detection device 200 includes, for example, a wide angle camera 201 and a narrow angle camera 202. The wide-angle camera 201 has a wide-angle field of view and images the entire face of the user. Since the position of the wide-angle camera 201 is fixed, the wide-angle field of view is also fixed. The narrow-angle camera 202 has a narrow-angle field of view by a zoom mechanism and images a face part (eye region). The narrow-angle camera 202 is movable in the imaging direction and follows the eye area. The eye region is at least a part of the face including at least one eye.

  In the line-of-sight detection device 200, the wide-angle camera 201 detects the face contour position, the face contour barycentric position (x, y), and the like, thereby determining the imaging direction in which the narrow-angle camera 202 can detect the eye region. By moving the narrow-angle camera 202 in this imaging direction, the eye region can be imaged with high pixels. Thereby, high gaze detection resolution can be obtained. In other words, the line-of-sight detection device 200 can detect the line of sight by interlocking the wide-angle camera 201 and the narrow-angle camera 202. If the line of sight can be detected, the visual position in the visual target can be detected.

  However, an error may occur between the visual line detected by the visual line detection device 200 and the actual visual line. Therefore, it is conceivable to calibrate the line of sight. For example, as illustrated in FIG. 3, consider a case where the user looks at the upper end y1 and the lower end y2 of the visual target in order. The calibration in this case is, for example, an operation for obtaining a correlation between a change amount (pixel) of the target point with respect to a reference point in the pupil and a change amount (degree) of the line-of-sight angle. This correlation is hereinafter referred to as a calibration model. When the line of sight is detected, the viewing position (x, y) on the viewing plane can be calculated by considering the distance to the user based on the calibration model. However, it is difficult for an apparatus used by an unspecified majority to use a technique for performing calibration for each individual. On the other hand, the error increases when calibration is not performed.

  FIG. 4A and FIG. 4B are diagrams illustrating errors. First, as illustrated in FIG. 4A, when the plane viewed by the user is a vertical plane, the amount of change in the line-of-sight angle when viewing the visual object from the upper end y1 to the lower end y2 of the visual object is Different due to height difference (eye height difference). For example, there is a difference between the change amount α of the gaze angle of the tall user and the change amount β of the gaze angle of the short user. In particular, as illustrated in FIG. 4B, when the object to be viewed is a horizontal plane, the line-of-sight angle caused by the difference in eye height with respect to the same distance from y1 (back end) to y2 (front end) The amount of change is larger than the amount of line-of-sight angle change with respect to the vertical plane. In the following embodiments, a line-of-sight calibration apparatus, a line-of-sight calibration method, and a line-of-sight calibration program that can improve the line-of-sight detection accuracy will be described.

  First, an outline of the line-of-sight calibration according to the first embodiment will be described. FIG. 5 is a diagram for explaining a change in the line-of-sight angle when the visual target forms a vertical plane. As illustrated in FIG. 5, the height of the visual object is set to h. Specifically, the height h is the distance from the floor to the lower end of the visual target. Let H be the vertical width (line-of-sight change width) of the visual target. Let D be the distance from the user to the visual target. Specifically, the distance D is a distance from the tip of the user's eyes on the side of the visual object to the visual object. Let Y be the height of the user's eyes. Specifically, the height Y is the distance from the floor to the tip of the user's eyes on the visual target side. A user's line-of-sight angle change amount with respect to the line-of-sight change width H is θ. Of the line-of-sight angle change amount θ, the lower side than the horizontal is θ1, and the upper side is θ2.

The radian value of the line-of-sight angle change amount θ1 can be expressed by the following formula (1).
θ1 (rad) = arctan (Y−h) / D (1)
The angle value of the above formula (1) can be expressed by the following formula (2).
θ1 (deg) = θ1 (rad) × 180 / π (2)
The radian value of the line-of-sight angle change amount θ2 can be expressed by the following formula (3).
θ2 (rad) = arctan (h + H−Y) / D (3)
The angle value of the above formula (3) can be expressed by the following formula (4).
θ2 (deg) = θ2 (deg) × 180 / π (4)
The line-of-sight angle change amount θ can be expressed by the following formula (5).
θ = θ1 + θ2 (5)

  As an example, it is assumed that the height h = 100 cm, the line-of-sight change width H = 100 cm, the distance D = 200 cm, and the height Y = 170 cm. In this case, θ = θ1 + θ2 = 19.29 + 8.531 = 27.82. If the height Y = 150 cm, then θ = θ1 + θ2 = 11.31 + 16.7 = 28.01 degrees. Thus, a difference occurs in the line-of-sight angle change amount θ according to the height difference of the user.

  FIG. 6 is a diagram for explaining a change in the line-of-sight angle when the visual target is a horizontal plane. As illustrated in FIG. 6, the height of the object to be viewed is h. Specifically, the height h is the distance from the floor to the visual object. Let H be the width in the depth direction (line-of-sight change width) of the visual target viewed from the user. Let D be the distance from the user to the visual target. Specifically, the distance D is a distance from the tip of the user's eyes on the visual object side to the visual object in the horizontal direction. Let Y be the height of the user's eyes. Specifically, the height Y is the distance from the floor to the tip of the user's eyes on the visual target side. A user's line-of-sight angle change amount with respect to the line-of-sight change width H is θ. A change amount of the user's line-of-sight angle in the range from the vertically lower side to the far end of the visual object is defined as θ1. A change amount of the user's line-of-sight angle in the range from the vertically lower side to the front end of the visual object is defined as θ2.

The radian value of the line-of-sight angle change amount θ1 can be expressed by the following formula (6).
θ1 (rad) = arctan ((H + D) / (Y−h)) (6)
The angle value of the above formula (6) can be expressed by the following formula (7).
θ1 (deg) = θ1 (rad) × 180 / π (7)
The radian value of the line-of-sight angle change amount θ2 can be expressed by the following formula (8).
θ2 (rad) = arctan (D / (Y−h)) (8)
The angle value of the above formula (8) can be expressed by the following formula (9).
θ2 (deg) = θ2 (deg) × 180 / π (9)
The line-of-sight angle change amount θ can be expressed by the following formula (10).
θ = θ1-θ2 (10)

  As an example, assume that height h = 70 cm, line-of-sight change width H = 100 cm, distance D = 70 cm, and height Y = 170 cm. In this case, θ = θ1−θ2 = 59.53−34.99 = 24.54 degrees. If the height Y is 140 cm, then θ = θ1−θ2 = 67.62−45 = 22.22 degrees. Thus, a difference occurs in the line-of-sight angle change amount θ according to the height difference of the user.

  FIG. 7 is a diagram for explaining a change in the line-of-sight angle when the visual target is inclined with respect to the horizontal direction and the vertical direction while facing the user. As illustrated in FIG. 7, the height of the visual target is h. Specifically, the height h is the distance from the floor to the lower end of the visual target. Let H be the width of the visual target in the depth direction (gaze change width). The angle formed by the visual target with the horizontal plane is defined as θ3. Let d be the difference in height of the visual object relative to the horizontal plane. Let D be the distance from the user to the visual target. Specifically, the distance D is a distance from the front end of the user's eyes to the front side of the visual target in the horizontal direction. Let Y be the height of the user's eyes. Specifically, the height Y is the distance from the floor to the tip of the user's eyes on the visual target side. A user's line-of-sight angle change amount with respect to the line-of-sight change width H is θ. A change amount of the user's line-of-sight angle in the range from the vertically lower side to the far end of the visual object is defined as θ1. A change amount of the user's line-of-sight angle in the range from the vertically lower side to the front end of the visual object is defined as θ2.

The radian value of the line-of-sight angle change amount θ1 can be expressed by the following formula (11).
θ1 (rad) = arctan ((H × cos θ3) + D) / (Y−h−d) (11)
The angle value of the above formula (11) can be expressed by the following formula (12).
θ1 (deg) = θ1 (rad) × 180 / π (12)
The radian value of the line-of-sight angle change amount θ2 can be expressed by the following formula (13).
θ2 (rad) = arctan (D / (Y−h)) (13)
The angle value of the above formula (13) can be expressed by the following formula (14).
θ2 (deg) = θ2 (deg) × 180 / π (14)
The line-of-sight angle change amount θ can be expressed by the following formula (15).
θ = θ1-θ2 (15)
Also in the case of FIG. 7, a difference occurs in the line-of-sight angle change amount θ according to the height difference of the user.

  In this embodiment, the user's standing position is determined, and the distance D described with reference to FIGS. 5 to 7 is a known fixed value. In this case, an initial line-of-sight calibration model at the reference eye height is created in advance. Specifically, for a representative (for example, eye height = 170 cm), an initial calibration model is used to calculate a correlation between a change amount (pixel) of a target point with respect to a reference point in the pupil and a line-of-sight angle with respect to a visual target. Create as. The initial calibration model may be a map or table that holds each change amount and each line-of-sight angle, or may be a mathematical formula for calculating the line-of-sight angle from the amount of change. In the example of FIG. 5, the line-of-sight angle change amount (angle) from the upper end to the lower end of the visual target is used. In the example of FIG. 6 and FIG. 7, the line-of-sight angle change amount (angle) from the back end to the front end of the visual target is used.

  In this embodiment, the line of sight is further calibrated using the difference between the height of the eye of the user who is the line of sight detection target and the eye of the representative and the initial calibration model. For example, the initial calibration model is calibrated using the eye height difference, and the line of sight is calibrated using the obtained calibration model. Furthermore, the visual position (x, y) in the visual target is calculated using the obtained line of sight. According to this embodiment, it is not necessary to create a calibration model for each individual as long as the eye height difference with respect to the representative can be detected. In addition, the line-of-sight detection accuracy is improved as compared with the case where a calibration model is created for each individual. Hereinafter, specific examples will be described.

  FIG. 8 is a block diagram illustrating the configuration of the line-of-sight calibration apparatus 100 according to the first embodiment. As illustrated in FIG. 8, the line-of-sight calibration apparatus 100 includes a monitor 10, a wide-angle camera 20, a narrow-angle camera 30, an infrared LED 40, a control unit 50, and the like. The control unit 50 includes a face contour detection unit 51, a height difference calculation unit 52, a narrow-angle camera control unit 53, an eye area detection unit 54, a line-of-sight conversion unit 55, a visual position calculation unit 56, an application control unit 57, a database 58, and the like. Is provided.

  The monitor 10 is a display device such as an LCD and is a user's visual target. The wide-angle camera 20 has a wide-angle visual field, and images the entire face of the user who stops in front of the monitor 10. Since the position of the wide-angle camera 20 is fixed, the wide-angle field of view is also fixed. As an example of the wide-angle camera 20, a USB camera (MCM: 4302) manufactured by Microvision can be used. Note that the pixel is, for example, VGA (640 × 480 pixels). As an example, the lens angle of view is V / H / D = 23/30 / 37.5 degrees.

  The narrow-angle camera 30 has a narrow-angle field of view by a zoom mechanism, and acquires an image of a part (eye region) of the user's face that is framed in the field of view of the wide-angle camera 20. The narrow-angle camera 30 is movable in the imaging direction, and can follow the eye area by changing the imaging direction. The narrow-angle camera 30 is, for example, a PTZ (Pan Tilt Zoom) camera, and an SNC-Ep580 manufactured by SONY (registered trademark) can be used as an example. Note that the angle of view is variable. The F value is 1.6 (wide) to 3.5 (tele). The infrared LED 40 irradiates the user with infrared light.

  Hereinafter, the operation of the line-of-sight calibration apparatus 100 will be described with reference to the flowchart of FIG. The flowchart in FIG. 9 represents an example of the operation of the line-of-sight calibration apparatus 100. The face contour detection unit 51 captures a wide-angle image from the wide-angle camera 20 (step S1). Next, the face contour detection unit 51 detects a face contour region (step S2). Further, the face contour detection unit 51 calculates any width (vertical or horizontal) of the face contour region, and calculates attention points such as the center of gravity of the face contour region and the center of both eyes.

  Next, the height difference calculation unit 52 calculates the difference between the eye height of the representative model and the eye height of the user using the calculated attention point (step S3). Specifically, since the distance D is a fixed value and the position of the wide-angle camera 20 is also fixed, the height of the user's eyes can be calculated from the position of the point of interest in the wide-angle image. The height difference calculation unit 52 calculates the difference in eye height by obtaining the difference between the eye height and the eye height of the representative model.

  Next, the narrow-angle camera control unit 53 controls the angle, magnification, and the like of the narrow-angle camera 30 so that the user's face enters the narrow-angle visual field with a predetermined size (step S4). Thereby, the narrow-angle camera 30 can capture an area of the user's eyes. Next, the eye area detection unit 54 captures a narrow-angle image from the narrow-angle camera 30 (step S5). Next, the eye area detection unit 54 detects an eye area in the narrow-angle image (step S6). Specifically, the eye area detection unit 54 detects an outline frame including both eyes as an eye area. Alternatively, the eye region detection unit 54 estimates a contour frame including both eyes as an eye region from the coordinates received from the face contour detection unit 51.

  Next, the line-of-sight conversion unit 55 detects the inside of the eye in the eye region detected by the eye region detection unit 54 (step S7). Next, the line-of-sight conversion unit 55 calculates a relative change between the pupil centroid and the corneal reflection centroid (step S8). Next, the line-of-sight conversion unit 55 refers to the initial calibration model stored in the database 58 and obtains the line-of-sight angle from the calculated relative change (step S9).

  Next, the line-of-sight conversion unit 55 calibrates the obtained line-of-sight angle using the eye height difference calculated in step S3 (step S10). FIG. 10A is a diagram exemplifying a correction coefficient table stored in the database 58, and corresponds to a change in line-of-sight angle when the visual target in FIG. 6 forms a horizontal plane. As illustrated in FIG. 10A, the correction coefficient in the case of the representative eye height is set to 1, and the correction coefficient is set stepwise according to the eye height. For example, the line-of-sight conversion unit 55 multiplies the line-of-sight angle obtained in step S9 by a correction coefficient. FIG. 10B is an image diagram of line-of-sight calibration according to the present embodiment. As illustrated in FIG. 10B, in this embodiment, the line of sight is calibrated using the initial calibration model, the eye height difference, and the correction coefficient table.

  Next, the visual position calculation unit 56 calculates the visual position (x, y) on the monitor 10 from the line-of-sight angle obtained in Step S10, the distance D, and the eye height calculated in Step S3 (Step S10). S11). The application control unit 57 controls the monitor 10 based on the visual position calculated by the visual position calculation unit 56 (step S12). For example, the application control unit 57 changes the display content of the monitor 10. Specifically, the application control unit 57 performs detailed information expansion, display switching, scrolling, and the like on the monitor 10.

  According to the present embodiment, by using the initial calibration model created in advance for the representative, the labor of creating a calibration model for each individual can be saved. Furthermore, the line-of-sight detection accuracy can be improved by calibrating the line of sight using the eye height difference and the initial calibration model. As a result, the detection accuracy of the visual position in the visual object can be improved.

(Modification 1)
In the above example, the table illustrated in FIG. 10A is used, but a mathematical formula for calculating a correction coefficient may be used. For example, it is possible to use a mathematical formula in which the correction coefficient in the case of the eye height of the representative is 1, and the correction coefficient changes according to the eye height difference. For example, the line-of-sight conversion unit 55 multiplies the line-of-sight angle obtained in step S9 by the eye height difference obtained in step S3 and the correction coefficient obtained from the above formula. FIG. 11 is an image diagram of line-of-sight calibration according to the first modification. As illustrated in FIG. 11, in this modification, the line of sight is calibrated using an initial calibration model, an eye height difference, and a mathematical expression for calculating a correction coefficient. According to the first modification, the capacity of the table or the like can be reduced.

(Modification 2)
In the above example, only one type of reference eye height is prepared. However, a plurality of reference eye heights may be prepared in stages. FIG. 12 illustrates a flowchart executed in this case. The difference from the flowchart of FIG. 9 is that step S13, step S19 and step S20 are executed instead of step S3, step S9 and step S10. Hereinafter, step S13, step S19, and step S20 will be described.

  After execution of step S2, the height difference calculation unit 52 calculates the height of the user's eyes using the calculated attention point (step S13). Specifically, since the distance D is a fixed value and the position of the wide-angle camera 20 is also fixed, the height of the user's eyes is calculated from the position of the point of interest in the wide-angle image.

  After execution of step S8, the line-of-sight conversion unit 55 selects a model corresponding to the eye height obtained in step S13 from a plurality of calibration models stored in the database 58 (step S19). For example, the line-of-sight conversion unit 55 selects a calibration model having a reference eye height closest to the eye height obtained in step S13. Next, the line-of-sight conversion unit 55 obtains a line-of-sight angle from the relative change calculated in step S8 using the selected calibration model (step S20). According to the second modification, since a plurality of calibration models can be used, the line-of-sight detection accuracy is improved.

  In the first embodiment, it is assumed that the distance D is fixed, but the distance D may be calculated using a wide-angle image acquired by the wide-angle camera 20. When the distance D is calculated using a wide-angle image, it is preferable that the movement of the user in the front-rear direction with respect to the narrow-angle camera 30 appears in the wide-angle image. Therefore, it is preferable that the height at which the wide-angle camera 20 is installed is deviated from the average face position of the user. For example, the optical axis of the wide-angle camera 20 is oriented obliquely downward from the horizontal direction from a position higher than the user's average face position, or from a position lower than the user's average face position from the horizontal direction. Also, it is preferable that the angle is directed obliquely upward (tilt angle). When the user's body axis is deviated from vertical, it is preferable that the user body is oriented obliquely upward or obliquely downward with respect to the body axis. In order to detect the eye region, the wide-angle camera 20 is preferably directed obliquely upward (tilt angle) with respect to the user from a position lower than the average face position of the user.

  FIG. 13 is an example in which the optical axis of the wide-angle camera 20 is directed obliquely upward from the horizontal direction from a position lower than the average face position of the user. Note that the movement of the user in the front-rear direction with respect to the wide-angle camera 20 is reversed in the wide-angle image depending on whether the optical axis of the wide-angle camera 20 is obliquely upward or downward in the horizontal direction.

  FIG. 14 is a block diagram illustrating the configuration of the line-of-sight calibration apparatus 100a according to the second embodiment. As illustrated in FIG. 14, the point of difference between the line-of-sight calibration apparatus 100 a and the line-of-sight calibration apparatus 100 is that the control unit 50 further includes a distance calculation unit 59.

  The distance calculation unit 59 calculates the movement direction and the movement amount of the attention point in the wide-angle image by comparing the attention point detected by the face contour detection unit 51 with the previous frame (for example, for each frame). For example, the distance calculation unit 59 calculates the moving direction and the moving amount of the attention point in the wide-angle image by detecting the change in the coordinates (x, y) of the attention point in the wide-angle image for each frame.

  Further, the distance calculation unit 59 calculates the movement direction and movement amount of the user in the front-rear direction by comparing the contour width and the point of interest detected by the face contour detection unit 51 for each frame. The front-rear direction is a direction in which the user approaches the wide-angle camera 20 and a direction away from the wide-angle camera 20. 15A to 15D, assuming that the body axis is oriented in the vertical direction, the optical axis of the wide-angle camera 20 is inclined from a position lower than the average face position of the user from the horizontal direction. It is a wide-angle image (still image) when set upward (10 °). 15A to 15D are still images when the distance between the wide-angle camera 20 and the face is 1.5 m, 2 m, 2.5 m, and 3 m.

  As illustrated in FIGS. 15A to 15D, when the user leaves the wide-angle camera 20, the face position moves downward. Therefore, for example, the amount of change in the vertical direction (y-axis) of the attention point (center of gravity, center of both eyes, etc.) of the face outline can be associated with the distance between the wide-angle camera 20 and the face. Since the position of the wide-angle camera 20 is fixed, the distance D can be obtained using the distance between the wide-angle camera 20 and the face. Further, the distance D in the initial frame or the frame at a predetermined time can be used as the distance at the initial standing position of the user, and the distance D can be corrected using the obtained amount of movement in the front-rear direction.

  In addition, while the vertical coordinate of the attention point of the face outline can be used as a relative variation amount of the distance, it can also be used as an absolute amount of the distance. For example, in the case where the same person is assumed to be used many times, or when the target is focused on a user who is almost close to the height, the position of the attention point can be converted into the absolute amount of the distance. . When the wide-angle camera 20 is directed obliquely downward from the horizontal direction, the face position moves upward when the user leaves the wide-angle camera 20.

  FIG. 15E shows a correspondence relationship between the center of gravity position (x, y) of the face contour, the difference (pixel) in the y-axis direction, and the distance D when the wide-angle image is 640 pixels × 480 pixels. It is a table to represent. If this table is created in advance, the distance D can be acquired by detecting the change in the center of gravity of the face contour. What is necessary is just to acquire by the interpolation etc. between the numerical values of FIG.15 (e).

  If the center of gravity moves in the x-axis direction and does not move in the y-axis direction, the distance between the wide-angle camera 20 and the face is the same, and the user moves in the horizontal direction. Even if the position of the center of gravity moves in the y-axis direction, the distance does not change when the user squats down. Therefore, the amount of change in the width of the face contour region may be detected as an auxiliary. For example, even if the position of the center of gravity moves in the y-axis direction, it can be determined that the distance between the wide-angle camera 20 and the face has not changed as long as the amount of change in the width of the face contour is a predetermined amount or less. It is also possible to calculate the distance only from the pixel width of the face outline. However, since there are unstable elements such as individual differences in face size, face shadows caused by ambient light, etc., and changes in the boundary of the outline when the face is directed diagonally, It is preferable to use a point of interest. If the relationship between the position of the wide-angle camera 20 and the position of the narrow-angle camera 30 is obtained in advance, the distance between the narrow-angle camera 30 and the face is calculated by acquiring the distance between the wide-angle camera 20 and the face. Can do.

  If the distance D can be calculated, it is possible to specify how the angle to the line of sight changes even if the position of the user moves in the front-rear and left-right directions. Therefore, the line of sight can be corrected using the calculated distance D. In this case, the line-of-sight detection accuracy is improved.

(Other examples)
FIG. 16 is a block diagram for explaining another hardware configuration of the control unit 50. Referring to FIG. 16, the control unit 50 includes a CPU 101, a RAM 102, a storage device 103, an interface 104, and the like. Each of these devices is connected by a bus or the like. A CPU (Central Processing Unit) 101 is a central processing unit. The CPU 101 includes one or more cores. A RAM (Random Access Memory) 102 is a volatile memory that temporarily stores programs executed by the CPU 101, data processed by the CPU 101, and the like. The storage device 103 is a nonvolatile storage device. As the storage device 103, for example, a ROM (Read Only Memory), a solid state drive (SSD) such as a flash memory, a hard disk driven by a hard disk drive, or the like can be used. When the CPU 101 executes the imaging program, the control unit 50 is realized in the line-of-sight calibration apparatus 100.

  In each of the above examples, a plurality of cameras such as the wide-angle camera 20 and the narrow-angle camera 30 are used, but the present invention is not limited to this. For example, by using a high-quality camera, the functions of the wide-angle camera 20 and the narrow-angle camera 30 can be performed by a single camera. Moreover, in each said example, although the height of a user's eyes is calculated from the wide-angle image which the wide-angle camera 20 acquires, it is not restricted to it. For example, the eye height may be acquired from the face position by providing a time of flight (TOF) type infrared ranging sensor such as Kinect. If an image processing algorithm for detecting the body or face cannot be used, the height may be estimated according to the degree to which fixed features such as background wall patterns and houseplants are hidden. . In each of the above examples, the line-of-sight conversion unit 55 functions as an example of a line-of-sight detection unit, the height difference calculation unit 52 functions as an example of a height detection unit, and the line-of-sight conversion unit 55 calibrates the line of sight. Functions as an example.

  Although the embodiments of the present invention have been described in detail above, the present invention is not limited to such specific embodiments, and various modifications and changes can be made within the scope of the gist of the present invention described in the claims. It can be changed.

10 Monitor 20 Wide-angle camera 30 Narrow-angle camera 40 Infrared LED
DESCRIPTION OF SYMBOLS 50 Control part 51 Face contour detection part 52 Height difference calculation part 53 Narrow-angle camera control part 54 Eye area | region detection part 55 Gaze conversion part 56 Visual position calculation part 57 Application control part 58 Database 100 Gaze calibration apparatus

Claims (10)

On the computer,
Processing to detect the user's line of sight;
Detecting the eye height of the user;
A line-of-sight calibration model using a line-of-sight calibration model corresponding to the eye height, and performing a process of calibrating the detected line of sight according to the detected eye height. program.   In the process of calibrating the line of sight, the line of sight is calibrated using a line-of-sight calibration model at a reference eye height and the difference between the detected eye height and the reference eye height. The line-of-sight calibration program according to claim 1.   In the process of calibrating the line of sight, a calibration model at a reference eye height provided in a plurality of stages is selected according to the detected eye height, and the selected calibration is selected. The line-of-sight calibration program according to claim 1, wherein the line-of-sight is calibrated using a model.   The line-of-sight calibration program according to any one of claims 1 to 3, wherein an image of the user acquired by a camera is used when detecting the eye height of the user.   The process of detecting the height of the user's eyes includes a process of detecting the face outline of the user from the user's image and a process of detecting the height of the user's eyes based on the face outline. The line-of-sight calibration program according to claim 4.   6. The process for detecting a line of sight is a process for detecting a difference amount of a point of interest from a reference point in an eyeball in a user image acquired by a camera. The eye-gaze calibration program described in 1.   7. The line-of-sight calibration model is a correlation between an attention point of a predetermined user's pupil acquired in advance and a line-of-sight angle change amount with respect to a visual target. The line-of-sight calibration program according to one item. In the computer,
Based on the amount of movement of the point of interest in the image of the user acquired by the camera, a process of acquiring the distance between the visual target and the user;
The line-of-sight calibration program according to claim 1, wherein a process of reflecting the acquired distance on the line-of-sight calibration is executed. A line-of-sight detector that detects the line of sight of the user;
A height detector for detecting the height of the eyes of the user;
A line of sight characterized by comprising: a processing unit that calibrates the detected line of sight according to the detected eye height of the user using a line of sight calibration model corresponding to the height of the eye. Calibration device. The line-of-sight detection unit detects the user's line of sight,
A height detector detects the eye height of the user;
A processing unit calibrates the detected line of sight according to the detected eye height of the user using a line-of-sight calibration model corresponding to the eye height, Method.

Images