JP6756127B2 - Line-of-sight estimation device and line-of-sight estimation method - Google Patents

Description

The present invention relates to the estimation of the line of sight.

A line-of-sight estimation device that estimates a person's line of sight is expected to have a wide range of applications such as medical tests and a man-machine interface (line-of-sight input).

In the line-of-sight estimation device, it is effective to accurately grasp the position of the person's gaze in order to estimate the line of sight.

For example, Patent Document 1 discloses a line-of-sight detection method in which a subject gazes at a known point in the coordinate system, acquires data on the coordinate points of the position of the pupil of the subject, and determines the line of sight.

Further, in relation to the present invention, Patent Document 2 sets the first accuracy mode when the gazing point is included in the specific area, and the detection accuracy is higher than when the gazing point is included in the object display area. Disclose an electronic device that is set to the second precision mode.

Further, Patent Document 3 discloses a line-of-sight detection device that corrects a line-of-sight position using correction information stored in the storage member when it is determined that the user is a registered user.

Further, Patent Document 4 discloses an electronic book device that calculates a reader's gaze point from an imaged reader's face and switches the displayed display information page.

Further, Patent Document 5 and the like disclose a method of estimating the direction of the line of sight from the positional relationship between the inner corner of the eye, the outer corner of the eye and the pupil.

Further, Patent Document 6 discloses a method of extracting an image by calculating a local feature amount.

Japanese Unexamined Patent Publication No. 2005-185431 International Publication No. 2014/08424 Japanese Unexamined Patent Publication No. 2014-221795 Japanese Unexamined Patent Publication No. 2000-20534 Japanese Unexamined Patent Publication No. 2008-136789 Japanese Unexamined Patent Publication No. 2012-104078

As described above, the method disclosed in Patent Document 1 requires a person to gaze at a known point in the coordinate system. However, the line of sight of a person is easy to move. Therefore, even if a person tries to gaze at a known point in the coordinate system, it is often difficult for the person to keep gazing at that point for a certain period of time. Therefore, in the method disclosed in Patent Document 1, it is possible that the subject did not gaze at one point when estimating the line of sight. In that case, the gaze estimation will be inaccurate.

An object of the present invention is to provide a line-of-sight estimation device or the like capable of more accurate line-of-sight estimation.

The line-of-sight estimation device of the present invention includes a contact position specifying unit, an imaging unit, and a processing unit. The contact position specifying unit specifies a contact position, which is a position where a person makes contact with an object. The imaging unit captures a portion including the human eye at the time of the contact. The processing unit estimates that a straight line passing through the contact position and parallel to the direction of the line of sight of the eye derived from the image is the line of sight of the eye.

The line-of-sight estimation device and the like of the present invention can estimate the line of sight more accurately.

It is a conceptual diagram which shows the structural example of the line-of-sight estimation apparatus of 1st Embodiment. It is an image diagram showing an eye. It is an image diagram showing an example of the positional relationship between the eyes, the line of sight, and the gaze position. It is an image diagram showing a composite device. It is a conceptual diagram which shows the processing flow example of the processing performed by the line-of-sight estimation apparatus of 1st Embodiment. It is a conceptual diagram which shows the structural example of the line-of-sight estimation apparatus of 2nd Embodiment. It is an image diagram which shows the state of the correction by the derived inclination. It is a conceptual diagram which shows the processing flow example of the processing performed by the line-of-sight estimation apparatus of 2nd Embodiment. It is a conceptual diagram which shows the minimum structure of the line-of-sight estimation apparatus of this invention.

<First Embodiment>
The first embodiment is an embodiment of a line-of-sight estimation device capable of estimating a line of sight based on a captured image of a portion including a human eye and a contact position of the person.
[Configuration and operation]
FIG. 1 is a conceptual diagram showing a configuration of a line-of-sight estimation device 101a, which is an example of the line-of-sight estimation device of the first embodiment.

The line-of-sight estimation device 101a is a device for estimating the line-of-sight of a person from an image including the imaged person's eyes.

The line-of-sight estimation device 101a includes a contact position specifying unit 151a, an imaging unit 121a, and a processing unit 131a.

The contact position specifying unit 151a identifies a position (hereinafter, referred to as "contact position") in which a person (hereinafter referred to as "contact position"), which is not shown, makes contact with an object (hereinafter, "object A") which is not shown.

The contact in this case is performed by, for example, a finger or a stick, a pointer, or other object suitable for contact with a narrow area.

For example, if the object A is a touch panel that derives the position where another object is in contact, the contact position can be specified by the contact. Alternatively, the contact position may be specified by imaging the contact portion or the like.

The contact position specifying unit 151a sends information including the identified information indicating the contact position to the processing unit 131a.

The imaging unit 121a images the portion including the eyes of the person at the time immediately after the person comes into contact with the object. The image pickup unit 121a is, for example, a commercially available camera device for smartphones or a digital camera. The image capturing unit 121a sends the captured image to the processing unit 131a.

The processing unit 131a extracts the image of the human eye from the image sent by the imaging unit 121a.

The processing unit 131a can extract the image of the eye by, for example, the following method.

First, the processing unit 131a incorporates an eye image identification program trained by a sufficient amount of Har-like features of the eye image. Then, the local feature amount is calculated for the image sent by the imaging unit 121a, and the region having a high score is specified as an eye image. For example, Patent Document 6 discloses the above method performed by the processing unit 131a.

The processing unit 131a may extract an image of a human face prior to extracting an image of the eyes. In that case, the extraction of the face image can be performed by the same method as the above-mentioned extraction of the eye image. That is, first, the processing unit 131a incorporates a face image identification program trained by the Har-like feature amount of a sufficient amount of face images. Then, the local feature amount is calculated for the image including the face sent by the imaging unit 121a, and the region having a high score is specified as the face image.

By specifying the face image prior to specifying the eye image, it is possible to reduce the probability of erroneously recognizing an image of a portion other than the face that resembles the shape of the eye as an eye image.

Prior to the above-mentioned extraction of the eye image, the processing unit 131a may perform a process for optimizing the brightness and contrast in the target image. By this processing, it is possible to optimize the brightness and contrast of the face region in the captured image and improve the accuracy of the eye image to be extracted next. Improving the accuracy of the eye image leads to the improvement of the derivation accuracy of the direction of the line of sight, which will be described next.

Next, the processing unit 131a derives the gaze direction (gaze direction) of the person from the extracted eye image. The direction of the line of sight can be derived by, for example, the method disclosed in Patent Document 5 from the positional relationship between the inner corner of the eye 211a, the outer corner of the eye 221a and the pupil 231a of the eye 201a shown in FIG.

Then, the processing unit 131a estimates the line of sight from the derived direction of the line of sight and the contact position.

FIG. 3 is an image diagram showing the relationship between the eyes, the line of sight, and the gaze position.

In FIG. 3, it is assumed that the eyes 201a gaze at the surface 4a, which is a flat surface of the object 5a, which is an example of the object A described above. Further, in FIG. 3, the case where the eyes 201a are located at the positions 9i and 9j, which are two different positions on the line 8c parallel to the surface 4a, is shown. Further, in the eye 201a, the positional relationship between the inner corner of the eye, the outer corner of the eye, and the pupil is the same at the position 9i and the position 9j.

Then, the line of sight 6a of the eye 201a at the position 9i and the line of sight 6b of the eye 201a at the position 9j are in the same direction, and are deviated by the distance 2a on the line 8c. And this deviation is also maintained on the surface 4a.

That is, the position 9k, which is the gazing point of the eye 201a at the position 9i, and the position 9l, which is the gazing point of the eye 201a at the position 9j, are separated by a distance of 2a on a line parallel to the line 8c.

That is, in order to accurately estimate the line of sight of the eye 201a, it is not enough to determine the direction of the line of sight from the positional relationship between the inner corner of the eye, the outer corner of the eye, and the pupil, and it is possible to derive the position of the eye 201a, which is the starting point of the line of sight. There is a need. Then, once the position of the gazing point on the surface 4a is determined, the line of sight of the eye 201a can be derived. As described above, the line-of-sight estimation device 101a shown in FIG. 1 sets the position of the gazing point on the surface 4a as the contact position, so that the position of the eye 201a can be determined. Further, as described above, since the direction of the line of sight is estimated, the position of the eyes and the direction of the line of sight are determined, and the line of sight can be estimated.

The line-of-sight estimation device 101a shown in FIG. 1 can be realized by, for example, the composite device 161a shown in FIG.

The composite device 161a includes a housing 181a, a touch panel 152a, and a camera 122a.

The touch panel 152a corresponds to the contact position specifying portion 151a shown in FIG.

The camera 122a corresponds to the imaging unit 121a shown in FIG.

The processing unit 131a shown in FIG. 1 is built in the housing 181a. The processing unit 131a shown in FIG. 1 is, for example, a central processing unit built into the composite device 161a.

[Processing flow]
FIG. 5 is a conceptual diagram showing an example of a processing flow of processing performed by the line-of-sight estimation device 101a shown in FIG. Hereinafter, in each process of the figure showing the process flow, the description to the left of the colon symbol represents the subject of the process. The description to the right of the colon symbol indicates the processing content performed by the processing subject. Further, the reference numerals in the following description are the reference numerals of each configuration shown in FIG.

First, the contact position specifying unit 151a determines whether or not there has been a predetermined contact with the object A as a process of S101. The predetermined contact is, for example, the above-mentioned contact with a human finger or the like.

When the contact position specifying unit 151a determines that a predetermined contact with the object A has been made by the process of S101, the process of S102 and S103 is performed.

On the other hand, if the contact position specifying unit 151a does not determine that there has been a predetermined contact with the object A by the process of S101, the process of S101 is performed again.

When processing S102, the contact position specifying unit 151a notifies the imaging unit 121a that there has been a specific contact with the object A. The contact position specifying unit 151a promptly gives the notification after determining yes in S101.

Then, upon receiving the notification by the processing of S102, the imaging unit 121a images the position where it is assumed that the portion including the eyes of the person who has made the specific contact can be imaged as the processing of S104. The image capturing unit 121a sends information including the captured image to the processing unit 131a.

Next, the processing unit 131a extracts an eye image from the image sent from the imaging unit 121a as the processing of S105. The method for extracting the eye image is as described above.

Then, the processing unit 131a estimates the direction of the line of sight using the eye image extracted by the processing of S105 as the processing of S106. The method of estimating the direction of the line of sight is as described above.

On the other hand, when the process of S103 is performed, the contact position specifying unit 151a specifies the contact position where a predetermined contact with the object A has occurred. Then, the contact position specifying unit 151a sends information representing the specified contact position to the processing unit 131a.

Then, the processing unit 131a performs the processing of S107 after the processing of both S103 and S106. That is, the processing unit 131a derives the position of the eye 201a by the contact position specified by the processing of S103 and the direction of the line of sight derived by the processing of S106 as the processing of S107. For example, with reference to FIG. 3, it is assumed that the position 9k is the contact position specified by the process of S103. Then, the line of sight passes through the position 9k and is determined to be the line of sight 6a which is a straight line parallel to the direction of the line of sight estimated by the process of S106.
Then, the line-of-sight estimation device 101a ends the process shown in FIG.

FIG. 5 shows a case where the processing unit 131a estimates the direction of the line of sight from the image of one eye extracted from the image captured by the image capturing unit 121a. However, the processing unit 131a may estimate the direction of the line of sight from the direction of the line of sight estimated for each of the eyes extracted from the image captured by the imaging unit 121a. For example, the processing unit 131a may set the average of the directions of the line of sight estimated for each of the eyes as the direction of the line of sight.

In the line-of-sight estimation device 101a shown in FIG. 1, the contact position specifying unit identifies a position where a person makes contact with a finger or the like. Further, in the line-of-sight estimation device 101a, the imaging unit 121a images the region including the human eye immediately after the contact. Then, the processing unit 131a extracts an eye image from the captured image. Then, the processing unit 131a estimates the gaze position in the image from the image of the eye. Here, in general, when a person touches (touches) an object with a finger or the like, the person has a property of gazing at the touch position of the object. Therefore, at the moment or immediately after the touch, there is a high probability that the person is gazing at the position of the object touched by the person. Therefore, the line-of-sight estimation device 101a can estimate the line of sight more accurately by deriving the position of the eye 201a from the contact position and the direction of the line of sight.

The above effect is particularly remarkable when a touch panel of a mobile terminal provided with a touch panel such as a smartphone is used for the position detection unit. With the spread of smartphones and the like, users and the like have been cultivated a habit of carefully watching the touch position when touching the touch panel of a mobile terminal. This is because the touch panel of a mobile terminal having a limited size requires the user to accurately touch a small area. Therefore, by using the touch panel of the mobile terminal, the probability that a person accurately gazes at the touch position is improved. Further, usually, when operating the touch panel of the mobile terminal, a person operates the touch panel while keeping the distance between the eyes and the mobile terminal substantially constant. Even if the direction of the line of sight is constant, the gaze position cannot be determined unless the distance between the eye and the object A is constant. In the case of the touch panel of a mobile terminal, since the positions of the eyes and the touch panel are almost constant, there is a high probability that the gaze position can be specified more accurately. Further, when operating the touch panel of the mobile terminal, the person keeps the relative angle between the touch panel and his / her face substantially constant. Therefore, the imaging unit 121a always images the portion including the person's eyes from a constant angle. Therefore, the accuracy of eye extraction by the processing unit 131a is improved. For the above reasons, when the contact position specifying unit 151a is provided with the touch panel of the mobile terminal, the probability that the line of sight is estimated more accurately by the line of sight estimation device 101a is further improved.
[effect]
In the line-of-sight estimation device of the first embodiment, the contact position specifying unit identifies a position where a person makes contact with a finger or the like. In addition, the line-of-sight estimation device of the first embodiment images the region including the human eye immediately after the contact. Then, the line-of-sight estimation device of the first embodiment extracts an image of the eye from the captured image. Then, the line-of-sight estimation device of the first embodiment estimates the gaze position in the image from the image of the eye. Here, in general, when a person touches (touches) an object with a finger or the like, the person has a property of gazing at the touch position of the object. Therefore, at the moment or immediately after the touch, there is a high probability that the person is gazing at the position of the object touched by the person. Therefore, the line-of-sight estimation device of the first embodiment can estimate the line of sight more accurately by deriving the line of sight from the contact position and the direction of the line of sight.
<Second embodiment>
The second embodiment is an embodiment relating to a line-of-sight estimation device that extracts an image of an eye using information on the inclination of an imaging unit.
[Configuration and operation]
FIG. 6 is a conceptual diagram showing the configuration of the line-of-sight estimation device 101b, which is an example of the line-of-sight estimation device of the second embodiment.

The line-of-sight estimation device 101b includes a tilt derivation unit 171a connected to the processing unit 131a in addition to the configuration of the line-of-sight estimation device 101a shown in FIG.

The description of the line-of-sight estimation device 101b is the same as the description of the line-of-sight estimation device 101a shown in FIG. 1, except for the following. Therefore, in the following, a portion different from the line-of-sight estimation device 101a shown in FIG. 1 will be described. If the description of the line-of-sight estimation device 101a shown in FIG. 1 and the following description are inconsistent, the following description takes precedence.

The tilt deriving unit 171a derives the tilt of the imaging unit 121a. The inclination is the inclination of the actual imaging direction by the imaging unit 121a with respect to the direction assumed as the imaging direction of the image of the portion including the eyes used for extracting the image of the eyes.

The tilt deriving unit 171a is, for example, a gyro fixed to the imaging unit 121a.

When the tilt deriving unit 171a is fixed to the imaging unit 121a, the tilt of the tilt deriving unit 171a is equal to the tilt of the imaging unit 121a. Therefore, the inclination deriving unit 171a can derive the inclination of the imaging unit 121a by deriving its own inclination.

The inclination deriving unit 171a sends information representing the derived inclination to the processing unit 131a.

When extracting the image of the eye, the processing unit 131a performs extraction using the inclination received from the inclination deriving unit 171a.

FIG. 7 is an image diagram showing a state of correction performed by the inclination derived by the inclination deriving unit 171a.

FIG. 7A is a diagram showing an image of the eye 201a imaged vertically from the front direction toward the drawing. Further, FIG. 7B is a diagram showing the positions 9a to 9d on the line 8a shown in FIG. 7A on the line 8a drawn horizontally. The direction of the arrow 7a shown in FIG. 7B is the imaging direction.

FIG. 7C is a diagram showing an eye 201a imaged from a direction inclined downward toward the drawing by an inclination angle θ. The tilt angle θ is the tilt of the imaging unit 121a shown in FIG. Further, FIG. 7 (d) is a diagram showing the positions 9e to 9h on the line 8b shown in FIG. 7 (c) on the line 8b drawn horizontally. The direction parallel to the direction of the arrow 7b shown in FIG. 7B is the imaging direction. The angle formed by the arrow 7a shown in FIG. 7B and the arrow 7b shown in FIG. 7D is the inclination angle θ. The distance between any two positions 9e to 9h corresponds to the distance between the two positions 8a to 8d shown in FIG. 7B, cosθ. Is the distance multiplied by.

Therefore, as shown in FIGS. 7 (e) and 7 (f), the distance between the positions in which the image pickup unit 121a is tilted is multiplied by 1 / cos θ, as shown in FIGS. 7 (e) and 7 (f). It is possible to obtain the same image of the eyes 201a as the image of the eyes 201a represented by.

It is assumed that the extraction of the eye image performed by the processing unit 131a is performed on the eye image captured from a fixed direction. Therefore, the correction of the inclination improves the accuracy of extracting the eye image by the processing unit 131a.
[Processing flow]
FIG. 8 is a conceptual diagram showing an example of a processing flow of processing performed by the line-of-sight estimation device 101b shown in FIG.

The description of each process represented by FIG. 8 is the same as the description of each process represented by FIG. 5 above, except for the following description. However, if the following description and the description of each process shown in FIG. 5 are inconsistent, the following description takes precedence.

When the contact position specifying unit 151a determines that a predetermined contact with the object A has been made by the processing of S101, the contact position specifying unit 151a performs the processing of S102a and S103.

On the other hand, if the contact position specifying unit 151a does not determine that there has been a predetermined contact with the object A by the process of S101, the process of S101 is performed again.
When the contact position specifying unit 151a performs the processing of S102a, the contact position specifying unit 151a informs the imaging unit 121a and the tilt deriving unit 171a that the contact has occurred.

Then, the imaging unit 121a performs the processing of S104, and the tilt deriving unit 171a performs the processing of S103a.

The processing of S104 is as described above.

When the processing of S103a is performed, the tilt deriving unit 171a derives the tilt of the imaging unit 121a. Then, the inclination deriving unit 171a sends information including the derived inclination to the processing unit 131a.

After the processing of S104 and the processing of S103a, the processing unit 131a performs the processing of S105a. That is, the processing unit 131a uses the inclination of the imaging unit 121a derived by the tilt deriving unit 171a by the processing of S103a as the processing of S105a, and extracts the eye image from the image captured by the imaging unit 121a by the processing of S104. Do. At that time, the processing unit 131a corrects the image of the portion including the human eye by, for example, tilting the imaging unit 121a, and extracts the image of the eye from the corrected image of the portion including the human eye. ..

Then, the processing unit 131a performs the processing of S106. The description of the processing after S106 is as described above.
[effect]
The line-of-sight estimation device of the second embodiment extracts an image of the eyes from the image captured by the imaging unit using the inclination of the imaging unit. Therefore, the line-of-sight estimation device of the second embodiment can improve the extraction accuracy of the eye image. Improving the extraction accuracy of the eye image leads to the improvement of the derivation accuracy in the direction of the line of sight. Then, the improvement of the derivation accuracy in the direction of the line of sight leads to the improvement of the derivation accuracy of the gaze position. Therefore, the line-of-sight estimation device of the second embodiment can more accurately estimate the line-of-sight to be derived. Therefore, the line-of-sight estimation device of the second embodiment can estimate the line-of-sight with higher accuracy.

FIG. 9 is a conceptual diagram showing the configuration of the line-of-sight estimation device 102x, which is the minimum configuration of the line-of-sight estimation device of the present invention.

The line-of-sight estimation device 102x includes a contact position specifying unit 151x, an imaging unit 121x, and a processing unit 131x.

The contact position specifying unit 151x specifies a contact position, which is a position where a person makes contact with an object.

The imaging unit 121x images the portion including the human eye at the time of the contact.

The processing unit 131x estimates that a straight line passing through the contact position and parallel to the direction of the line of sight of the eye derived from the image is the line of sight of the eye.

Generally, when a person touches (touches) an object with a finger or the like, the person has a property of gazing at the touch position of the object. Therefore, at the time of the touch, there is a high probability that the person is gazing at the position of the object touched by the person. Therefore, the line-of-sight estimation device 102x estimates the line-of-sight more accurately by estimating the line-of-sight from the direction of the line-of-sight derived from the image of the portion including the human eye at the time of touch and the contact position. Can be done.

As described above, the line-of-sight estimation device 102x exhibits the effects described in the section of [Effects of the Invention] by the above configuration.

Although the present invention has been described above with reference to preferred embodiments, the present invention is not necessarily limited to the above embodiments, and can be variously modified and implemented within the scope of the technical idea.

In addition, some or all of the above embodiments may be described as in the following appendix, but are not limited to the following.

(Appendix 1)
A contact position specifying part that specifies the contact position, which is the position where a person makes contact with a predetermined object,
An imaging unit that captures an image of a portion including the human eye at the time of contact,
A processing unit that estimates a straight line that passes through the contact position and is parallel to the direction of the line of sight of the eye derived from the image as the line of sight of the eye.
A line-of-sight estimation device.

(Appendix 2)
The line-of-sight estimation device according to Appendix 1, wherein the processing unit derives the direction of the line of sight from the image of the person's eyes extracted from the image of the portion including the person's eyes.

(Appendix 3)
The line-of-sight estimation device according to Appendix 2, wherein the processing unit extracts an image of the human eye by calculating a local feature amount.

(Appendix 4)
The line-of-sight estimation device according to Appendix 2 or 3, wherein the processing unit extracts an image of the person's eyes from an image of the person's face extracted from an image of a portion including the person's eyes.

(Appendix 5)
A tilt deriving unit for deriving the tilt of the imaging unit is further provided, and the processing unit extracts an image of the human eye using the tilt of the imaging unit derived by the tilt deriving unit. The line-of-sight estimation device according to any one of Appendix 4.

(Appendix 6)
The line-of-sight estimation device according to Appendix 5, wherein the tilt derivation unit includes a gyro.

(Appendix 7)
The processing unit corrects the image of the portion including the human eye by the tilt of the imaging unit derived by the tilt derivation unit, and the corrected image of the portion including the human eye is used to obtain the image of the eye. The line-of-sight estimation device according to Appendix 5 or Appendix 6, which performs extraction.

(Appendix 8)
The line-of-sight estimation device according to any one of Supplementary note 1 to Supplementary note 7, wherein the contact position specifying portion includes a touch panel.

(Appendix 9)
The line-of-sight estimation device according to Appendix 8, wherein the touch panel is a touch panel of a mobile terminal.

(Appendix 10)
The line-of-sight estimation device according to any one of Supplementary note 1 to Supplementary note 9, wherein the contact is performed by the finger of the person or an operation object operated by the person.

(Appendix 11)
The line-of-sight estimation device according to Appendix 10, wherein the operation object is a stick or a pen.

(Appendix 12)
The line-of-sight estimation device according to any one of Supplementary note 1 to Supplementary note 11, wherein the imaging unit includes a camera.

(Appendix B1)
A gaze point estimation device comprising the line-of-sight estimation device according to any one of Supplementary Note 1 to Supplementary Note 13, and estimating a gaze point.

(Appendix C1)
Identify the contact position, which is the position where a person made contact with an object,
An image of the part including the human eye at the time of the contact is taken.
The line of sight of the person is estimated from the direction of the line of sight derived from the image and the contact position.
Line-of-sight estimation method.

(Appendix D1)
Identify the contact position, which is the position where a person made contact with an object,
An image of the part including the human eye at the time of the contact is taken.
The line of sight of the person is estimated from the direction of the line of sight derived from the image and the contact position.
A line-of-sight estimation program that causes a computer to perform processing.

4a Surface 5a Object 6a, 6b Line of sight 7a, 7b Arrow 8a, 8b, 8c Line 9a, 9b, 9c, 9d, 9e, 9f, 9g, 9h, 9i, 9j, 9k, 9l
Position 101a, 101b, 102x Line-of-sight estimation device 121a, 121x Imaging unit 122a Camera 131a, 131x Processing unit 151a, 151x Contact position identification unit 152a Touch panel 161a Composite device 171a Tilt detection unit 181a Housing 201a Eyes 211a Pupils 221a

Claims (8)

A contact position specifying part that specifies the contact position, which is the position where a person makes contact with a predetermined object,
An imaging unit that captures an image of a portion including the human eye at the time of contact,
A processing unit that estimates a straight line that passes through the contact position and is parallel to the direction of the line of sight of the eye derived from the image as the line of sight of the eye.
A tilt derivation unit that derives the tilt of the imaging unit,
With
The processing unit derives the direction of the line of sight from the image of the human eye extracted from the image of the portion including the human eye.
The processing unit extracts an image of the human eye using the inclination of the imaging unit derived by the inclination derivation unit .
Line-of-sight estimation device. The processing unit corrects the image of the portion including the human eye by the tilt of the imaging unit derived by the tilt derivation unit, and the corrected image of the portion including the human eye is used to obtain the image of the eye. the extraction, gaze estimation device according to claim 1. The line-of-sight estimation device according to claim 1 or 2 , wherein the processing unit extracts an image of the human eye by calculating a local feature amount. The method according to any one of claims 1 to 3, wherein the processing unit extracts an image of the person's eyes from an image of the person's face extracted from an image of a portion including the person's eyes. Line-of-sight estimation device. The line-of-sight estimation device according to any one of claims 1 to 4 , wherein the contact position specifying unit includes a touch panel. The line-of-sight estimation device according to claim 5 , wherein the touch panel is a touch panel of a mobile terminal. The line-of-sight estimation device according to any one of claims 1 to 6 , wherein the contact is performed by the finger of the person or an operation object operated by the person. Identify the contact position, which is the position where a person made contact with an object,
An image of the part including the human eye at the time of the contact is taken.
The direction of the sight line derived from the image, from said contact position, to estimate the person's line of sight,
Derivation of the direction of the line of sight is performed from the image of the human eye extracted from the image of the portion including the human eye.
Derivation of the inclination of the imaging unit, which is the part for imaging.
It is a line-of-sight estimation method
An image of the human eye is extracted using the inclination of the imaging unit.
Line-of-sight estimation method.

Images