JP2021026310A - Sight line measurement apparatus and sight line measurement system - Google Patents

Abstract

To provide a sight line measurement apparatus which can measure the sight line of a user even when a user is at a lateral recumbent position and there is no sufficient space near the user.SOLUTION: A sight line measurement apparatus comprises: a display device which displays a video on a display region provided on the upper side for a user being at a lateral recumbent position; a mirror which is arranged between the user and the display region and reflects the video toward the user; a sensor which captures an image of an eyeball of the user being at the lateral recumbent position through the mirror; an estimation unit which estimates the position coordinates of the gazing point of the user on the video on the basis of the captured image of the eyeball; and an output unit which outputs the estimated position coordinates.SELECTED DRAWING: Figure 1

Description

The present invention relates to a line-of-sight measurement device and a line-of-sight measurement system.

Severly disabled children (persons) who have difficulty moving their bodies may use a gaze measuring device to operate the screen on the bed. The line-of-sight measurement sensor used in the line-of-sight measurement device includes a wearable type (glasses type) and a non-wearable type (fixed monitor type).

The wearable line-of-sight measurement sensor is a spectacle-type device that captures the optical features of the eyeball with a camera fixed to the spectacles and estimates the direction of the line of sight (for example, Patent Document 1). This method has high measurement accuracy because the positional relationship between the camera and the eyeball does not change easily, and there are many devices for basic research, but it is very expensive. In addition, it is difficult to lie in a lateral decubitus position while wearing the same as general eyeglasses.

Many non-wearable line-of-sight measurement sensors capture the line of sight by a sensor fixed to a monitor (for example, Patent Document 2). In recent years, inexpensive games for games have become widespread. This sensor is usually optimized for a distance of 50 to 70 cm from the user's eyeball to the sensor (monitor) and cannot be used in a narrow space. That is, there is often not enough space on the bedside, and when the user is in the lateral decubitus position, there is a problem that it is difficult to secure a sufficient distance from the user to the sensor.

Further, when the sensor recognizes both eyes of the user, higher line-of-sight detection accuracy can be obtained as compared with the case where only one eye is recognized. However, when the user is in the lateral decubitus position, the eyeball on the pillow is hidden by the pillow, and the sensor estimates the line of sight from only one eyeball, so that there is a problem that it is difficult to obtain high accuracy.

Japanese Unexamined Patent Publication No. 2017-012746 Japanese Unexamined Patent Publication No. 2006-285531

The line-of-sight measuring device according to the embodiment of the present disclosure provides a line-of-sight measuring device capable of measuring the user's line of sight even when the user is in the lateral decubitus position and there is not enough space around the user. The purpose.

The line-of-sight measuring device according to the embodiment of the present disclosure is arranged between a display device that displays an image in a display area provided on the upper side of a user in a lateral decubitus position and the user and the display area, and directs the image to the user. A mirror that reflects the image, a sensor that captures an image of the user's eyeball in the lateral decubitus position through the mirror, and an estimation unit that estimates the position coordinates of the user's gazing point on the image based on the captured eyeball image. It is characterized by having an output unit that outputs the estimated position coordinates.

In addition, it further has a light source that irradiates the user with infrared rays through a mirror, the sensor acquires the infrared reflection pattern and the position of the pupil, and the estimation unit reflects the infrared rays acquired from the image of the captured eyeball. It is preferable to estimate the position coordinates of the user's gazing point on the image based on the pattern and the position of the pupil.

Further, the mirror is preferably a surface reflector.

Further, the display device is preferably a projector.

Further, it is preferable to further have a projector mirror arranged on the video output side of the projector.

Further, the line-of-sight measurement system preferably includes a line-of-sight measurement device and a pillow that supports the user's head and is arranged at a predetermined position from the sensor.

Further, it is preferable that the pillow is cut out around the eyeball on the side that comes into contact with the pillow so that the sensor can capture images of the two eyeballs of the user in the lateral decubitus position.

In addition, it is preferable to have a concave shape for accommodating the user's ears so that the position of the user's eyeball can be maintained at a predetermined position on the pillow.

According to the line-of-sight measuring device according to the embodiment of the present disclosure, the line-of-sight of the user can be measured even when the user is in the lateral decubitus position and there is not enough space around the user.

FIG. 5 is a side view of the line-of-sight measuring device according to the first embodiment of the present disclosure as viewed from the user's head direction. FIG. 5 is a side view of the line-of-sight measuring device according to the second embodiment of the present disclosure as viewed from the user's head direction. FIG. 5 is a side view of the line-of-sight measuring device according to the second embodiment of the present disclosure as viewed from the back side of the user. It is the figure which looked down on the actual arrangement example of the line-of-sight measurement apparatus which concerns on Example 2 of this disclosure from above. It is the figure which looked up from the lower side with the actual arrangement example of the line-of-sight measurement apparatus which concerns on Example 2 of this disclosure. It is a side view of the pillow which comprises the line-of-sight measurement system which concerns on embodiment of this disclosure. It is a drawing of the pillow which constitutes the line-of-sight measurement system which concerns on embodiment of this disclosure, (a) is a plan view, (b) is a perspective view. It is a figure which shows the measurement example of the gaze point measured by using the line-of-sight measurement system which concerns on embodiment of this disclosure.

Hereinafter, the line-of-sight measurement device and the line-of-sight measurement system according to the present invention will be described with reference to the drawings. However, it should be noted that the technical scope of the present invention is not limited to those embodiments and extends to the inventions described in the claims and their equivalents.

[Example 1]
First, the line-of-sight measuring device according to the first embodiment of the present disclosure will be described. FIG. 1 shows a side view of the line-of-sight measuring device according to the first embodiment of the present disclosure as viewed from the user's head direction. The line-of-sight measuring device 101 according to the first embodiment of the present disclosure includes a display device 1, a mirror 2, a sensor 3, an estimation unit 4, and an output unit 5. The estimation unit 4 and the output unit 5 can be realized by executing a program stored in a memory (not shown) by a CPU built in the computer 10. The computer 10 can be placed on a table 50 provided above the user 20.

It is assumed that the user 20 who uses the line-of-sight measuring device 101 according to the first embodiment of the present disclosure lies on the bed 40 in a lateral decubitus position. Further, it is assumed that the user 20 puts his head on the pillow 30.

The display device 1 displays an image in a display area 13 provided on the upper side of the user 20 in the lateral decubitus position. As the display device 1, a liquid crystal display device, an organic EL display device, or the like can be used. The display device 1 displays an image in the display area 13 based on the image signal from the computer 10. The display area 13 of the display device 1 is preferably arranged so as to be substantially parallel to the horizontal plane of the bed 40 on which the user 20 lies. The display device 1 is fixed by a fixture (not shown) so that it can be freely positioned at a desired position.

The mirror 2 is arranged between the user 20 and the display area 13 of the display device 1, and reflects the image toward the user 20. Further, the sensor 3 captures an image of the eyeballs (21, 22) of the user 20 in the lateral decubitus position through the mirror 2. The image data captured by the sensor 3 is input to the computer 10. The mirror 2 is preferably a surface reflector. A sputtering mirror can be used as the surface reflector. When the mirror 2 is composed of a normal mirror, the optical path length may have an error depending on the thickness of the glass. However, by using a surface reflector, the optical path length can be measured accurately, and the position of the gazing point can be measured. The accuracy of the coordinates can be improved.

When the user 20 is in the lateral decubitus position and the mirror 2 is not used, it is conceivable to arrange the display device at the position shown in 1a of FIG. However, it is necessary to secure a distance L0 between the sensor 3a arranged in the vicinity of the display device 1a and the eyeballs (21,22) of the user 20 of about 50 to 70 cm, and a sufficient space is provided around the bed 40. If not, it is difficult to secure this distance L0.

Therefore, in the line-of-sight measuring device 101 according to the first embodiment of the present disclosure, an image is displayed in the display area 13 above the user 20 in the lateral decubitus position, and a mirror 2 is arranged between the user 20 and the display area 13. ing. Since the sensor 3 needs to be arranged on the user 20 side from the display area 13, by arranging the display area 13 on the upper side of the user 20, the sensor 3 and the eyeballs (21, 22) of the user 20 and the sensor 3 are arranged via the mirror 2. It is easy to secure the required distance of 50 to 70 cm as the distance L1 between them. If the sensor 3 is separated from the eyeballs (21, 22) of the user 20 by a predetermined distance, the display device 1 may be further separated upward. Further, the longitudinal direction of the sensor 3 is preferably parallel to the line connecting the eyeballs 21 and 22 of the user 20. In other words, the longitudinal direction of the sensor 3 is preferably orthogonal to the longitudinal direction of the bed 40.

The estimation unit 4 estimates the position coordinates of the gazing point of the user 20 on the image based on the captured image of the eyeball (21, 22). When the sensor 3 is a visible light camera, the gazing point can be estimated from the positional relationship between the two, with the inner corner of the eyeball (21, 22) as the reference point and the iris as the moving point. Alternatively, the user 20 may further have a light source 6 that irradiates the user 20 with infrared rays via the mirror 2, and an infrared camera may be used as the sensor 3. In this case, the sensor 3 acquires the infrared reflection pattern and the position of the pupil, and the estimation unit 4 acquires the infrared reflection pattern and the position of the pupil obtained from the image of the captured eyeball, and the estimation unit 4 of the user 20 on the image. The position coordinates of the gazing point can be estimated.

The output unit 5 outputs the estimated position coordinates of the gazing point. For example, the output unit 5 can display the estimated position coordinates of the gazing point on the screen of the display device 1. The user 20 can operate various software operated by the computer 10 by stopping or moving the gazing point.

As described above, according to the line-of-sight measuring device according to the first embodiment, the line-of-sight of the user can be measured even when the user is in the lateral decubitus position and there is not enough space around the user. ..

[Example 2]
Next, the line-of-sight measuring device according to the second embodiment will be described. The line-of-sight measuring device 102 according to the second embodiment is different from the line-of-sight measuring device 101 according to the first embodiment in that the display device is the projector 11. Since the other configurations of the line-of-sight measuring device 102 according to the second embodiment are the same as the configurations of the line-of-sight measuring device 101 according to the first embodiment, detailed description thereof will be omitted.

FIG. 2 shows a side view of the line-of-sight measuring device according to the second embodiment of the present disclosure as viewed from the user's head direction, and FIG. 3 shows the line-of-sight measuring device according to the second embodiment of the present disclosure from the back side of the user. The side view seen is shown. In FIG. 3, the computer is not shown. Further, FIG. 4 shows a bird's-eye view of an actual arrangement example of the line-of-sight measurement device according to the second embodiment of the present disclosure, and FIG. 5 shows an actual arrangement of the line-of-sight measurement device according to the second embodiment of the present disclosure. An example is shown looking up from below. The projector 11 can be placed on a table 50 provided above the user 20.

The light output from the projector 11 may be projected directly onto the display area 13. Alternatively, the projector mirror 12 may be arranged on the video output side of the projector 11, and the light output from the projector 11 may be projected onto the display area 13 via the projector mirror 12. Since the user 20 sees the image of the projector 11 through the mirror 2, it is necessary to display the image inverted in advance in the display area 13. However, when the projector mirror 12 is used, the inverted image is projected on the display area 13, so that the projector 11 does not need to project the inverted image in advance.

According to the line-of-sight measurement device according to the second embodiment, since it is not necessary to arrange the display device on the upper side of the user 20 by using the projector 11 as the display device, the risk of the display device falling on the user 20 side is eliminated. can do.

Next, the line-of-sight measurement system according to the embodiment of the present disclosure will be described. The line-of-sight measurement system according to the embodiment of the present disclosure includes the above-mentioned line-of-sight measurement device (101, 102) and a pillow 30 that supports the head of the user 20 and is arranged at a predetermined position from the sensor 3. FIG. 6 shows a side view of a pillow constituting the line-of-sight measurement system according to the embodiment of the present disclosure. When the user 20 is in the lateral decubitus position, the eyeball 22 on the pillow 30 side of the eyeballs (21, 22) may be hidden by the pillow 30 because the head of the user 20 sinks into the pillow 30. .. In this case, it is possible to estimate the position coordinates of the gazing point only with the eyeball 21 which is not hidden by the pillow 30, but the accuracy is lowered. Therefore, the pillow constituting the line-of-sight measurement system according to the embodiment of the present disclosure is on the side that comes into contact with the pillow 30 so that the sensor 3 can capture images of the two eyeballs (21, 22) of the user 20 in the lateral decubitus position. It is characterized in that the periphery of the eyeball 22 is cut out.

FIG. 7 (a) shows a plan view of a pillow constituting the line-of-sight measurement system according to the embodiment of the present disclosure, and FIG. 7 (b) shows a perspective view. The overall size of the pillow 30 is, for example, 27 cm in length and 50 cm in width. Notches 31 are provided on both sides of the pillow 30. The angle formed by the notch 31 is preferably about 60 °. As a result of using the pillow 30 having such a shape, the pressure on the head could be uniformly and reduced. The pillow 30 shown in FIGS. 7 (a) and 7 (b) is provided with notches on both the left and right sides, but the present invention is not limited to such an example, and the eyeball is used when the user 20 views an image. It may be provided only on the side facing the pillow.

Further, it is preferable to have a concave shape for accommodating the ears of the user 20 so that the position of the eyeballs (21, 22) of the user 20 can be maintained at a predetermined position on the pillow 30. For example, as shown in FIGS. 7A and 7B, it is preferable to provide the recess 32 in the substantially central portion of the pillow 30. The width of the central portion of the pillow 30 is about 25 cm, which is large enough to support the head of the user 20. Further, the recess 32 has a diameter r of about 10 cm and can accommodate the ears of the user 20. By providing the recess 32, the user 20 can naturally fix the head near the center of the pillow 30, and the distance between the sensor 3 and the eyeballs (21, 22) can be kept constant.

Next, a measurement example of the gazing point measured by using the line-of-sight measurement system according to the embodiment of the present disclosure will be described. FIG. 8 shows a measurement example of the gazing point measured by using the line-of-sight measurement system according to the embodiment of the present disclosure. Tobii EyeX was used for the sensor 3. The upper left corner of the screen is the origin of the X and Y axes. The nine points indicated by black circles represent the positions of the points displayed on the screen, and the positions indicated by the square dots are the estimated positions of the gazing points. From the measurement results, it can be seen that the position of the gazing point is accurately estimated by the line-of-sight measurement system according to the embodiment of the present disclosure.

As described above, according to the line-of-sight measuring device according to the embodiment of the present disclosure, the line-of-sight of the user can be measured even when the user is in the lateral decubitus position and there is not enough space around the user. A line-of-sight measuring device can be provided.

1 Display device 2 Mirror 3 Sensor 4 Estimator 5 Output unit 6 Light source 10 Computer 11 Projector 12 Projector mirror 13 Display area 20 Users 21 and 22 Eyeballs 30 Pillows 40 Beds 50 units

Claims (8)

A display device that displays images in the display area provided on the upper side for the user in the lateral position,
A mirror that is placed between the user and the display area and reflects the image toward the user.
A sensor that captures an image of the user's eyeball in the lateral decubitus position through the mirror,
An estimation unit that estimates the position coordinates of the user's gazing point on the image based on the captured eyeball image, and
An output unit that outputs the estimated position coordinates,
A line-of-sight measuring device characterized by having. Further having a light source that irradiates the user with infrared rays through the mirror.
The sensor acquires the infrared reflection pattern and the position of the pupil.
The estimation unit estimates the position coordinates of the user's gazing point on the image based on the infrared reflection pattern and the position of the pupil acquired from the image of the captured eyeball.
The line-of-sight measuring device according to claim 1. The line-of-sight measuring device according to claim 1 or 2, wherein the mirror is a surface reflector. The line-of-sight measuring device according to any one of claims 1 to 3, wherein the display device is a projector. The line-of-sight measuring device according to claim 4, further comprising a projector mirror 12 arranged on the video output side of the projector. The line-of-sight measuring device according to any one of claims 1 to 5.
A pillow that supports the user's head and is placed in a predetermined position from the sensor,
Line-of-sight measurement system with. The line-of-sight measurement system according to claim 6, wherein the pillow is cut out around the eyeball on the side that comes into contact with the pillow so that the sensor can capture an image of two eyeballs of a user in a lateral decubitus position. The line-of-sight measurement system according to claim 6 or 7, which has a concave shape for accommodating the user's ears so that the position of the user's eyeball can be maintained at a predetermined position on the pillow.