JP2012055418A - View line detection device and view line detection method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a view line detection device detecting a view line of a test subject to peripheral environment without detecting an angle of an eyeball to a head.SOLUTION: This view line detection device 100 detecting the view line of the test subject A to the peripheral environment B includes: a reflection light detection means 106 detecting reflection light reflected from the eyeball a of the test subject A; and a viewed point detection means 108 detecting a viewed point by the test subject A based on the reflection light and light of the peripheral environment B. The view line detection device 100 can further include a peripheral environment light detection means detecting the light of the peripheral environment.

Description

  The present invention relates to a line-of-sight detection device and a line-of-sight detection method for detecting a line of sight to a surrounding environment of a subject.

Gaze detection is an important and indispensable technology for building modern and next-generation information environments. The application range of gaze detection technology is wide, such as user interface using gaze information, ubiquitous / ambient environment, human behavior recognition / understanding and communication analysis. Therefore, many technologies including commercial products have been put into practical use (Patent Document 1).

JP-A-5-146408

  However, in the system described in Patent Document 1, in order to estimate the line of sight after directly irradiating the eyeball with infrared illumination and measuring the angle of the eyeball with respect to the head, a camera, a posture sensor, or the like is attached to the head of the subject. It must be a burden on the subject. It is also necessary for the subject to calibrate by moving the line of sight to a specific position at the start of use.

  The present invention has been made in view of the above problems, and an object thereof is to provide a visual line detection device and a visual line detection method that can detect the visual line of a subject without measuring the angle of the eyeball with respect to the head. It is in.

In order to solve the above-described problem, a characteristic configuration of a visual line detection device according to the present invention is a visual line detection device that detects a visual line toward a surrounding environment of a subject, and detects reflected light reflected from the eyeball of the subject. The apparatus includes a reflected light detection unit and a subject detection unit that detects a subject by the subject based on the reflected light and the ambient light.

As described in the section of the background art, the conventional gaze detection device detects the angle of the eyeball with respect to the head of the subject. As a result, a camera, a posture sensor, etc. must be mounted on the subject's head, which places a burden on the subject. It is also necessary for the subject to calibrate by moving the line of sight to a specific position at the start of use. On the other hand, according to the line-of-sight detection device of the present invention, the subject's viewpoint is detected based on the reflected light and the light in the surrounding environment. Therefore, it is possible to detect the line of sight of the subject to the surrounding environment without detecting the angle of the eyeball with respect to the head. As a result, it is possible to realize a line-of-sight detection apparatus that can be applied to human visual and intelligence development understanding, such as detecting the line of sight of infants and determining the presence or absence of developmental disorders.

According to a preferred embodiment of the eye gaze detection apparatus according to the present invention, the visual line detection device further includes a projection unit that projects light onto the surrounding environment, and the viewpoint detection unit collates the reflected light with the projection light projected onto the surrounding environment. Then, the subject by the subject is detected. Since the projection light can be applied to a wide area, it is possible to detect the line of sight in a wide area environment. Moreover, since the light projected onto the surrounding environment is active, the brightness of the light can be easily controlled, bright reflected light can be detected, and stable line of sight detection is possible.

According to a preferred embodiment of the line-of-sight detection apparatus according to the present invention, the visual line detection device further includes a projection light detection unit that detects the projection light, and the subject detection unit collates the reflected light with the detected projection light to determine the subject. Detect the subject by. Therefore, even when light is projected onto an uneven surrounding environment, the line-of-sight detection device can detect light in the surrounding environment of the subject. As a result, the subject's line of sight can be detected with high accuracy without being influenced by the shape of the surrounding environment.

  According to a preferred embodiment of the visual line detection device according to the present invention, the visual line detection device further includes display means for displaying light in the surrounding environment, and the subject detection means collates the reflected light with the display light displayed on the display means. Then, the subject by the subject is detected. Therefore, since the shape of the surrounding environment is always the display area of the display means, the shape of the surrounding environment can always be recognized. As a result, the subject's line of sight can be detected with high accuracy. Further, since the light displayed on the display means is active, it is easy to control the brightness of the light, bright reflected light can be detected, and stable line-of-sight detection is possible.

  According to a preferred embodiment of the eye gaze detection apparatus according to the present invention, the reflected light detection means further includes eyeball tracking means for tracking the eyeball of the subject. Therefore, even if the subject is a person whose behavior is unstable or a person who is not seated, the reflected light from the eyeball of the subject can be accurately detected. As a result, stable line-of-sight detection becomes possible, and the subject's line of sight can be detected with high accuracy.

According to a preferred embodiment of the line-of-sight detection device according to the present invention, the apparatus further comprises a first recording means for recording a plurality of reflected light data and a second recording means for recording a plurality of optical data of the surrounding environment. The detection means detects the subject by the subject by comparing the plurality of reflected light data with the plurality of light data of the surrounding environment. Therefore, even if the reflected light cannot be detected at a desired timing, the reflected light detected at another timing can be collated with the light in the surrounding environment. Even when the reflected light cannot be detected at a desired timing, the data that failed to be detected can be estimated by extrapolation or interpolation of data that has been successfully detected at another timing. As a result, the line of sight of the subject can be detected with higher accuracy.

According to a preferred embodiment of the visual line detection device according to the present invention, the plurality of optical data indicate a plurality of gray code patterns. As a result, the subject's viewpoint can be detected with a simple pattern with high accuracy.

According to a preferred embodiment of the visual line detection device according to the present invention, the reflected light is light reflected from the peripheral portion of the pupil of the subject. Therefore, the light corresponding to the center of the pupil can be estimated by extrapolation or interpolation from the light reflected from the peripheral portion of the subject's pupil. As a result, stable line-of-sight detection is possible.

According to a preferred embodiment of the visual line detection device according to the present invention, the light in the surrounding environment is infrared light. Since infrared light is invisible, it cannot be recognized by the subject. Accordingly, the subject's line of sight can be detected without being noticed by the subject, and the subject's behavior can be prevented from becoming unstable due to being confused by the light in the surrounding environment. As a result, stable line-of-sight detection is possible.

In order to solve the above-described problem, a characteristic configuration of a gaze detection method according to the present invention is a gaze detection method for detecting a gaze to the surrounding environment of the subject, and detects reflected light reflected from the eyeball of the subject. A reflected light detection step and a subject detection step of detecting a subject by the subject based on the reflected light and the ambient light.

According to the visual line detection method according to the present invention, the same effects as those of the visual line detection device according to the present invention described above can be obtained. That is, according to the line-of-sight detection method of the present invention, the subject's viewpoint is detected based on the reflected light and the ambient light. Therefore, it is possible to detect the line of sight of the subject to the surrounding environment without detecting the angle of the eyeball with respect to the head. As a result, it is possible to realize a line-of-sight detection apparatus that can be applied to human visual and intelligence development understanding, such as detecting the line of sight of infants and determining the presence or absence of developmental disorders.

According to a preferred embodiment of the eye gaze detection method according to the present invention, it further includes a projection step of projecting light onto the surrounding environment, and the subject detection step collates the reflected light with the projection light projected onto the surrounding environment. Then, the subject is detected by the subject.

According to a preferred embodiment of the eye gaze detection method according to the present invention, the method further includes a display step of displaying light of the surrounding environment, and the subject detection step is a display displayed by executing the reflected light and the display step. The subject is detected by comparing with the light.

It is a schematic diagram which shows the gaze detection apparatus which concerns on Embodiment 1 of this invention. It is a flowchart which shows operation | movement of the gaze detection apparatus which concerns on Embodiment 1 of this invention. It is a photograph which shows the image detected by each component of the gaze detection apparatus which concerns on Embodiment 1 of this invention. It is a schematic diagram which shows the gaze detection apparatus which concerns on Embodiment 2 of this invention. It is a flowchart which shows operation | movement of the gaze detection apparatus which concerns on Embodiment 2 of this invention. 4 shows a gray code pattern used in line of sight detection according to an embodiment of the present invention. It is a photograph which shows the image detected by each component of the gaze detection apparatus based on the Example of this invention.

  With reference to FIGS. 1-7, embodiment regarding the gaze detection apparatus and gaze detection method of this invention is described. The present invention is not intended to be limited to the configurations described in the embodiments and drawings described below, and includes configurations equivalent to those configurations.

[Embodiment 1]
[Projection-type gaze detection device]
FIG. 1 is a schematic diagram showing a visual line detection device 100 according to Embodiment 1 of the present invention. The line-of-sight detection apparatus 100 is a projection-type line-of-sight detection apparatus. The line-of-sight detection device 100 detects the line of sight of the subject A toward the surrounding environment B. The line-of-sight detection apparatus 100 includes a projection unit 102, a projection light detection unit 104, a reflected light detection unit 106, and a subject detection unit 108.

  The projection unit 102 has a light source and projects light onto the surrounding environment B. The projection unit 102 is, for example, a projector. The projecting unit 102 projects, for example, visible light indicating a scenery or visible light indicating a gray code pattern onto the surrounding environment B. In the embodiment of the present invention, the surrounding environment B is a space around the subject A, and may be a space indicated by XY coordinates or a space indicated by XYZ coordinates. When the surrounding environment B is a room wall, the projection unit 102 projects light onto the room wall. When the surrounding environment B is a mountain rock wall, the projection means 102 projects light onto the uneven rock wall. Further, if the projection unit 102 is a 3D image device, a realistic 3D image formed of a dot array can be projected in a space that does not exist except for air.

The projection light detection unit 104 detects projection light. The projection light detection unit 104 is, for example, a digital camera or a CCD camera, and can detect light in the visible light region.

  The reflected light detection means 106 detects the reflected light reflected from the eyeball a of the subject A. The reflected light detection means 106 is, for example, a digital camera, a CCD camera, or a pan / tilt / zoom camera, and can detect light in the visible light region. The reflected light detection means 106 detects light reflected from the eyeball a of the subject A who is stationary or the eyeball a of the subject A who is moving.

  The subject detection unit 108 collates the reflected light detected by the reflected light detection unit 106 with the projection light detected by the projection light detection unit 104 to detect the subject b by the subject A. For example, the subject detection unit 108 associates digital data indicating reflected light with digital data indicating reflected light, and detects the subject b included in the surrounding environment B from the associated digital data. In addition, in this specification, the viewpoint b by the subject A indicates a point, a region, or a part that is seen by the subject A's eyes.

The reflected light detection unit 106 may include an eyeball tracking unit 110. The subject detection unit 108 may include a first recording unit 112 and a second recording unit 114. Details of the eye tracking unit 110, the first recording unit 112, and the second recording unit 114 will be described later.

[Gaze detection method]
FIG. 2 is a flowchart showing the operation of the visual line detection device 100. FIG. 3 is a photograph showing an image detected by each component of the line-of-sight detection device 100. The line-of-sight detection method will be described with reference to FIGS. As shown below, the line-of-sight detection according to Embodiment 1 of the present invention is realized by executing steps 202 to 216 by the line-of-sight detection apparatus 100.

Step 202: The projection means 102 projects light onto the surrounding environment B by driving the visual line detection device 100. FIG. 3A1 shows a projection image projected by the projection unit 102 onto the room wall. After projecting light onto the ambient environment B, operation proceeds to step 204.

Step 204: The projection light detection means 104 detects the projection light projected on the surrounding environment B. For example, the projection light detection unit 104 detects the projection light and performs image processing on the projection image indicated by the projection light. FIG. 3A2 shows the projection image data processed by the projection light detection unit 104. FIG. The projection image data is transmitted to the viewpoint detection unit 108, and the operation proceeds to Step 206.

Step 206: The subject A looks at the projection light projected on the surrounding environment B. The operation proceeds to step 208 while the subject A is still looking at the projection light.

Step 208: At least a part of the projection light projected on the surrounding environment B is reflected on the surface of the eyeball a of the subject A. FIG. 3 (b1) shows a projected image shown on the surface of the eyeball a of the subject A. The operation proceeds to step 210 in a state where the projection light is reflected on the surface of the eyeball a of the subject A.

Step 210: At least part of the projection light is reflected from the surface of the eyeball a of the subject A. With the projection light reflected from the surface of the eyeball a of the subject A, the operation proceeds to step 212.

Step 212: The reflected light detection means 106 detects the reflected light reflected from the eyeball a of the subject A. For example, the reflected light detection unit 106 detects the reflected light and performs image processing on the reflected image indicated by the reflected light. FIG. 3B2 shows the reflected image data processed by the reflected light detection means 106. The reflected image data is transmitted to the subject detection means 108, and the operation proceeds to step 214.

Step 214: The subject detection means 108 collates the reflected light and the projected light. For example, the subject detection unit 108 associates the reflected image data received from the reflected light detection unit 106 with the projection image data received from the projection light detection unit 104. After the association, operation proceeds to step 216.

Step 216: The subject b is detected from the associated digital data.

The line-of-sight detection device 100 and the line-of-sight detection method according to Embodiment 1 of the present invention have been described above with reference to FIGS. 1 to 3. According to the line-of-sight detection device of the present invention, the subject's viewpoint is detected based on the reflected light and the ambient light. Therefore, it is possible to detect the line of sight of the subject to the surrounding environment without detecting the angle of the eyeball with respect to the head. As a result, it is possible to realize a line-of-sight detection apparatus that can be applied to human visual and intelligence development understanding, such as detecting the line of sight of infants and determining the presence or absence of developmental disorders.

According to a preferred embodiment of the eye gaze detection apparatus according to the present invention, the visual line detection device further includes a projection unit that projects light onto the surrounding environment, and the viewpoint detection unit collates the reflected light with the projection light projected onto the surrounding environment. Then, the subject by the subject is detected. Since the projection light can be applied to a wide area, it is possible to detect the line of sight in a wide area environment. Moreover, since the light projected onto the surrounding environment is active, the brightness of the light can be easily controlled, bright reflected light can be detected, and stable line of sight detection is possible.

According to a preferred embodiment of the line-of-sight detection apparatus according to the present invention, the visual line detection device further includes a projection light detection unit that detects the projection light, and the subject detection unit collates the reflected light with the detected projection light to determine the subject. Detect the subject by. Therefore, even when light is projected onto a rough surrounding environment, the visual line detection device can detect the light in the surrounding environment of the subject. As a result, the subject's line of sight can be detected with high accuracy without being influenced by the shape of the surrounding environment.

As described with reference to FIG. 1, the reflected light detection unit 106 may include an eyeball tracking unit 110 that tracks the eyeball a of the subject A. The eyeball a of the subject A can be tracked by detecting the face area of the subject A with a digital camera and panning / tilting another camera having a zoom lens with a high magnification. For example, face recognition is performed with a normal camera to detect the position of the eye area, and then an enlarged image of the eye area is captured with a zoom camera. The eyeball tracking means 110 can be attached to the head of the subject A. However, if a configuration for photographing while panning, tilting, and zooming can be realized, the eyeball of the subject A can be obtained by installing it in the surrounding environment B without attaching the equipment to the subject. a can be tracked. In step 212, the eye tracking unit 110 tracks the eye A of the subject A, so that the reflected light detection accuracy of the reflected light detection unit 106 is increased.

As described above, according to the preferred embodiment of the eye gaze detection apparatus according to the present invention, the reflected light detection means further includes eyeball tracking means for tracking the eyeball of the subject. Therefore, even if the subject is a person whose behavior is unstable or a person who is not seated, the reflected light from the eyeball of the subject can be accurately detected. As a result, stable line-of-sight detection becomes possible, and the subject's line of sight can be detected with high accuracy.

  As described with reference to FIG. 1, the subject detection unit 108 includes a first recording unit 112 that records a plurality of optical data indicating the surrounding environment B, and a second recording unit 114 that records a plurality of reflected light data. May further be provided. The subject detection unit 108 detects the subject b by the subject A by collating a plurality of reflected light data with a plurality of light data indicating the surrounding environment B.

At time t ₁ , the projection unit 102 projects the projection light 1. The projection light detection unit 104 detects the first light data corresponding to the projection light 1 and records the first light data in the first recording unit 112. The reflected light detection means 106 detects first reflected light data corresponding to the projection light 1 and records the first reflected light data in the second recording means 114. At time t ₂ , the projection unit 102 projects the projection light 2. The projection light detection unit 104 detects the second light data corresponding to the projection light 2 and records the second light data in the first recording unit 112. The reflected light detection means 106 detects the second reflected light data corresponding to the projection light 2 and records the second reflected light data in the second recording means 114.

And from time t ₁ to time t _x, the first recording unit 112, the first x light data is recorded from the first optical data. The second recording means 114 records the xth reflected light data from the first reflected light data. The subject detection means 108 collates a plurality of reflected light data (first reflected light data to xth reflected light data) with a plurality of light data indicating the surrounding environment B (first light data to xth light data). Thus, the subject b by the subject A is detected.

Thus, according to the preferred embodiment of the visual line detection device according to the present invention, the first recording means for recording the plurality of reflected light data and the second recording means for recording the plurality of optical data of the surrounding environment are further provided. The subject detection unit detects the subject by the subject by comparing the plurality of reflected light data with the plurality of light data of the surrounding environment. Therefore, even if the reflected light cannot be detected at a desired timing, the reflected light detected at another timing can be collated with the light in the surrounding environment. In addition, data that has failed to be detected can be estimated by extrapolation or interpolation of data that has been successfully detected. As a result, the line of sight of the subject can be detected with higher accuracy.

  When the reflected light detection means 106 can detect the reflected light reflected from the eyeball a of the subject A, it is not essential that the reflected light detection means 106 includes the eyeball tracking means 110. When the subject detection unit 108 can collate a plurality of reflected light data with a plurality of light data indicating the surrounding environment B, the line-of-sight detection device 100 includes a first recording unit 112 and a second recording unit 114. That is not essential.

  Furthermore, when the subject detection unit 108 can detect the subject b of the subject A based on the reflected light and the light of the surrounding environment B, it is not essential that the line-of-sight detection device 100 includes the projection unit 102. If the subject detection unit 108 can detect the subject b of the subject A by collating the reflected light with the projection light projected on the surrounding environment B, the line-of-sight detection device 100 uses the projection light detection unit 104. It is not essential to prepare.

  In the line-of-sight detection device, “the reflected light detection means detects the reflected light reflected from the eyeball a of the subject A” and “the subject detection means detects the subject A based on the reflected light and the ambient light. The projection means and the projection light detection means are not indispensable when the subject b can be detected. The ambient environment light detection means can detect ambient environment light. The projection unit and the projection light detection unit function as ambient environment light detection unit.

Furthermore, the line-of-sight detection device of the present invention is not limited to a projection type equipped with a projection unit and a projection light detection unit. The line-of-sight detection device of the present invention may be a display type. Hereinafter, the display-type gaze detection apparatus will be described.

[Embodiment 2]
[Display-type eye-gaze detector]
FIG. 4 is a schematic diagram showing a visual line detection device 200 according to Embodiment 2 of the present invention. The line-of-sight detection apparatus 200 is a display-type line-of-sight detection apparatus. The line-of-sight detection device 200 detects the line of sight of the subject A toward the surrounding environment B, like the line-of-sight detection device 100. The line-of-sight detection apparatus 200 includes a display unit 202, a reflected light detection unit 106, and a subject detection unit 108. The reflected light detection unit 106 includes an eyeball tracking unit 110. The subject detection unit 108 includes a first recording unit 112 and a second recording unit 114. Since each component other than the display unit 202 has the same function as each component included in the visual line detection device 100 described in the first embodiment with reference to FIG. 1, detailed description thereof is omitted.

The display means 202 displays the ambient environment B light. The display means 202 is, for example, a display board that emits light or a monitor screen. The light of the surrounding environment B is transmitted to the viewpoint detection unit 108 as display image data displayed on the display unit 202. The reflected light detection means 106 detects the reflected light reflected from the eyeball a of the subject A and transmits it to the subject detection means 108 as reflected image data. The subject detection means 108 collates the reflected light detected by the reflected light detection means 106 with the display light displayed by the display means 202 to detect the subject b by the subject A.

[Gaze detection method]
FIG. 5 is a flowchart showing the operation of the visual line detection device 200. The line-of-sight detection method will be described with reference to FIG. As shown below, the line-of-sight detection according to the second embodiment of the present invention is realized by executing steps 502 to 508 and steps 210 to 216 by the line-of-sight detection device 200. Steps other than Step 502 to Step 508 are the same as the steps included in the visual line detection method described in Embodiment 1 with reference to FIG. 2, and thus detailed description thereof is omitted.

Step 502: The display means 202 displays the ambient environment B light on the display screen. After displaying light on the display screen, operation proceeds to step 504.
Step 504: The display means 202 transmits display image data indicating the light of the surrounding environment B to the viewpoint detection means 108, and the operation proceeds to step 506.
Step 506: Subject A looks at the display image displayed on display means 202. The operation proceeds to step 508 with subject A viewing the image.
Step 508: At least a part of the display image displayed on the display means 202 is reflected on the surface of the eyeball a of the subject A. In a state where the display image is reflected on the surface of the eyeball a of the subject A, the operation proceeds to step 210.
Step 210 and step 212 are sequentially executed. In step 214, the viewpoint detection unit 108 associates the reflected image data received from the reflected light detection unit 106 with the display image data received from the display unit 202. Further, step 216 is executed to detect the subject b of the subject A from the associated digital data.

The visual line detection device 200 and the visual line detection method according to Embodiment 2 of the present invention have been described above with reference to FIGS. 4 and 5. According to a preferred embodiment of the visual line detection device according to the present invention, the visual line detection device further includes display means for displaying light in the surrounding environment, and the subject detection means collates the reflected light with the display light displayed on the display means. Then, the subject by the subject is detected. Accordingly, since the shape of the surrounding environment is always the display area of the display means, the shape of the surrounding environment can always be recognized. As a result, the subject's line of sight can be detected with high accuracy. Further, since the light displayed on the display means is active, it is easy to control the brightness of the light, bright reflected light can be detected, and stable line-of-sight detection is possible.

In the line-of-sight detection device 200 according to the second embodiment of the present invention, the display unit 202 that displays the ambient environment light functions as an ambient environment light detection unit that can detect the ambient environment light. The display unit 202 transmits the ambient environment light to be displayed to the viewpoint detection unit.

  As described above, the line-of-sight detection device and line-of-sight detection method of the present invention have been described with reference to FIGS. 1 to 5, but the reflected light is not limited to the light reflected from the center of the pupil of the subject A. When the reflected light detection means 106 can detect the reflected light reflected from the eyeball a of the subject A, the reflected light can be light reflected from the periphery of the pupil of the subject A. Therefore, the light corresponding to the center of the pupil can be estimated by extrapolation or interpolation from the light reflected from the peripheral portion of the subject's pupil. As a result, stable line-of-sight detection is possible.

Moreover, when the light projected from the projection means 102 can be projected on the surrounding environment B, the light projected from the projection means 102 is not limited to visible light. It may be invisible light (such as infrared light). In addition, the projection light detection unit 104 and the reflected light detection unit 106 may be a camera that can detect invisible light when the projection light can be detected. For example, the projecting means 102 is a high brightness infrared projector. A high-intensity infrared projector can be realized by changing the light source to an infrared lamp based on a commercially available projector and slightly reconfiguring the optical system if necessary. By combining a plurality of infrared projectors, a configuration capable of covering a wider range can be realized. Since infrared light is invisible, it cannot be recognized by the subject. Accordingly, the subject's line of sight can be detected without being noticed by the subject, and the subject's behavior can be prevented from becoming unstable due to being confused by the light in the surrounding environment.

FIG. 6 shows a gray code pattern used in line-of-sight detection according to an embodiment of the present invention. Gray code patterns are often used in three-dimensional measurement using a projector. The gray code pattern represents one bit in one projection image. In this embodiment, a gray code pattern of 5 bits (5 in the X direction and 5 in the Y direction) is used in each of the X direction and the Y direction. The resolution in each of the X direction and the Y direction is 32.

At time t1, the gray code pattern from space x1 to space x5 is High (H), and the gray code pattern from space x5 to space x9 is Low (L). At time t2, the gray code patterns of space x1 to space x3 and space x5 to space x7 are High (H), and the gray code patterns of space x3 to space x5 and space x7 to space x9 are Low (L). At time t3, the Gray code patterns of the space x1 to the space x2, the space x3 to the space x4, the space x5 to the space x6, and the space x7 to the space x8 are High (H), and the space x2 to the space x3 and the space x4 to the space x5. The gray code patterns from the space x6 to the space x7 and from the space x8 to the space x9 are Low (L). Therefore, when the gray code pattern is HHL, the viewpoint of the subject A is between the space x2 and the space x3, and when the gray code pattern is LHH, the viewpoint of the subject A is between the space x5 and the space x6. It can be detected.

FIG. 7 is a photograph showing an image detected by each component of the visual line detection device 100 according to the embodiment of the present invention. An embodiment of the present invention will be described with reference to FIGS. A line-of-sight detection apparatus 100 according to an embodiment of the present invention includes a visible light projector (projection means 102), a first camera (projection light detection means 104), a second camera (reflection light detection means 106), and an information processing apparatus. (Viewee detection means 108, for example, CPU). The distance between the projector projection surface and the eyeball a of the subject A was about 3 m, and the second camera was installed so that the distance between the second camera and the eyeball a was about 60 cm.

The experimental environment is a normal light source environment, and the projection surface of the visible light projector is a normal room wall (FIG. 7A). A spatio-temporal pattern (gray code pattern) was projected onto the projector projection plane (the wall of the room) by a visible light projector. A striped pattern appears on the surface of eyeball a of subject A. FIG. 7B shows the surface of the eyeball a at time t1, the surface of the eyeball a at time t2, and the surface of the eyeball a at time t3. The first camera (projection light detection means 104) detects the projection light, and digitally processes the projection image indicated by the projection light (FIG. 7C). The projection image data is transmitted to the information processing apparatus (viewpoint detection means 108). The second camera (reflected light detection means 106) detects the reflected light and digitally processes the reflected image indicated by the reflected light (FIG. 7 (d)). The reflected image data is transmitted to the information processing apparatus (viewpoint detection means 108). The information processing apparatus (the subject detection unit 108) associates the reflected image data with the projection image data received from the projection light detection unit 104, and detects the subject b from the associated digital data.

Assuming that the image projected from the visible light projector is projected onto a target area of 4 m square, the image can be identified with a resolution of 12.5 cm. In this case, the gaze estimation angle is about 2.3 degrees, and the accuracy of the conventional method for estimating the gaze from the position of the pupil is 5 to 6 degrees. In the embodiment of the present invention, it was confirmed that a pattern can be detected stably by an image of at least about 85 frame / sec.

According to a preferred embodiment of the visual line detection device according to the present invention, the plurality of optical data indicate a plurality of gray code patterns. As a result, the subject's viewpoint can be detected with a simple pattern with high accuracy.

By applying the line-of-sight detection device and line-of-sight detection method of the present invention, it is possible to detect the line of sight of a customer in a store, and a product display is realized with higher economic efficiency. It is also possible to analyze the movement of the subject's line of sight in the living space, and to design a living environment that is easy for the subject to use. As an application to the user interface, a ubiquitous / ambient interface using gaze information in the living space can be developed. The line of sight is particularly important as information obtained from children, the elderly and persons with disabilities, and a non-wearing line-of-sight tracking interface can play a major role. For example, gaze information is important for diagnosing developmental disorders in children such as autism, but children can be diagnosed using gaze information obtained in a laboratory environment. By realizing the line-of-sight detection device according to the embodiment of the present invention, it is possible to obtain high-precision line-of-sight information of a non-wearing infant and to make a great contribution to the life science field.

A subject a eyeball B ambient environment b subject 100 gaze detection device 102 projection means 104 projection light detection means 106 reflected light detection means 108 subject detection means 110 eyeball tracking means 112 first recording means 114 second recording means 200 gaze detection apparatus 202 Display means

Claims (14)

A line-of-sight detection device that detects a line of sight to the surrounding environment of a subject,
Reflected light detection means for detecting reflected light reflected from the eyeball of the subject;
A line-of-sight detection device comprising: a subject detection means for detecting a subject by the subject based on the reflected light and the light of the surrounding environment.   The line-of-sight detection device according to claim 1, further comprising ambient environment light detection means for detecting light in the ambient environment. Further comprising projection means for projecting light onto the surrounding environment,
The line-of-sight detection device according to claim 1, wherein the subject detection unit detects the subject by the subject by comparing the reflected light with the projection light projected on the surrounding environment. A projection light detecting means for detecting the projection light;
The gaze detection device according to claim 3, wherein the subject detection unit detects the subject by the subject by comparing the reflected light with the detected projection light. Further comprising display means for displaying light in the surrounding environment,
The line-of-sight detection device according to claim 1, wherein the subject detection unit detects the subject by the subject by collating the reflected light with the display light displayed on the display unit.   The line-of-sight detection device according to claim 1, wherein the reflected light detection unit further includes an eyeball tracking unit that tracks the eyeball of the subject. A first recording means for recording a plurality of reflected light data; and a second recording means for recording a plurality of optical data of the surrounding environment,
The said viewpoint detection means detects the said viewpoint by the said test subject by collating the said several reflected light data and the said several optical data of the said surrounding environment, The said viewpoint by the test subject is detected. The visual line detection device according to one item. The line-of-sight detection device according to claim 7, wherein the plurality of optical data indicate a plurality of gray code patterns.   The line-of-sight detection device according to claim 1, wherein the reflected light is light reflected from a peripheral portion of the pupil of the subject. The line-of-sight detection device according to claim 1, wherein the light in the surrounding environment is infrared light. A gaze detection method for detecting a gaze to the surrounding environment of a subject,
A reflected light detection step of detecting reflected light reflected from the eyeball of the subject;
A line-of-sight detection method, comprising: a subject detection step of detecting a subject by the subject based on the reflected light and the light of the surrounding environment.   The line-of-sight detection device according to claim 11, further comprising a surrounding environment light detection step of detecting light in the surrounding environment. Further comprising a projecting step of projecting light onto the surrounding environment,
The gaze detection method according to claim 11, wherein the subject detection step performs the detection of the subject by the subject by collating the reflected light with the projection light projected on the surrounding environment. And further comprising a display step of displaying the ambient light.
The line-of-sight detection according to claim 11, wherein the subject detection step performs detection of the subject by the subject by collating the reflected light with display light displayed by executing the display step. Method.

Images