KR20170118618A - Eye capturing device - Google Patents

Abstract

The present invention relates to a binocular imaging device using a single camera in which an optical device is disposed so that a line of sight of both eyes can be photographed by a single camera, and an eye tracking device using the same.

Description

[0001] EYE CAPTURING DEVICE [0002]

The present invention relates to a camera for photographing an eye to track a line of sight, and is characterized in that both eyes are photographed by one camera or a front view of the eye and eyes are photographed by a single camera.

Eye tracking is a technology that analyzes images taken by a user's eyes with a camera and determines which direction the user is gazing at. These cameras are usually attached to glasses or included in a virtual reality head-mounted display (HMD).

As a gaze tracking device for a conventional HMD, one known in Patent Documents 1 and 2 has been proposed.

As shown in FIG. 7, the goggle-type wearer's gaze tracking device of Patent Document 1 includes a first camera 110 for photographing the direction of the eyes and a second camera 140 for photographing the user's eyes. The second camera includes an infrared light source 120 for illuminating the infrared light toward the user's eye and an infrared reflected visible light mirror 130 for reflecting the infrared light of the light source toward the eye. These mirrors are known as hot mirrors.

In addition, a camera for taking an eye can be installed on each of the right and left eyes, so that both eyes can be photographed separately.

As described above, in the gaze tracking apparatuses of Patent Documents 1 and 2, the eyes and the front are photographed with two cameras, and therefore, the following problems arise.

First, power consumption is doubled compared to using a single camera.

Second, it takes twice as much load on the communication channel (for example, USB) that transfers the camera image to the eye tracking device software than when using one camera.

Third, since the heat generated by the camera is twice as much as that of using one camera, operation error may occur due to a larger heat load inside the HMD due to the double heat generation and the load on the communication channel.

Fourth, since two cameras are used, manufacturing costs are more expensive and the circuit is more complicated than in the case of using one camera, thus contradicting the miniaturization trend of HMD.

Patent Document 1: Korean Patent No. 10-0949743 Patent Document 2: Korean Patent No. 10-1471488 Patent Document 3: PCT / US2014 / 038651 (Korean Patent Publication No. 19-2016-0111019)

The first embodiment of the present invention has been devised to solve the above-described problems, and it is an object of the present invention to provide a binocular photographing apparatus using a single camera in which a layout of an optical mechanism is arranged so that a single camera can photograph a line of sight of both eyes. have.

It is another object of the present invention to provide an apparatus for photographing an eye and a front with a single camera.

In order to achieve the above-mentioned object, the first embodiment is characterized in that it includes an optical module which reflects the infrared rays reflected from both eyes in front of one camera to the camera side. Such an optical module includes a mirror and a lens. The lens of the optical module is a convex lens having a short focal length, and the camera photographs an image formed on the convex lens, and the camera is installed away from the convex lens, so that heat generated from the camera is easily emitted to the outside.

The second embodiment includes an optical module that reflects infrared light reflected from the eyes in front of one camera and light reflected from an object in the gaze direction toward the camera. The optical module is characterized in that infrared rays reflected from the eye are reflected toward the camera side and visible light is blocked, so that external objects are not superimposed on the eye image. In addition, visible light or infrared rays reflected from an external object are reflected to the camera side, and the camera captures an infrared ray or visible light image of an external object.

The present invention has the following effects.

By reflecting the light reflected from both eyes or the light reflected from the eye and the sight direction object to a common point in front of the camera with an optical module (light collecting means) including a reflection mirror, in the first embodiment, . In the case of the second embodiment, one eye and an external object in the eye direction of the eye can be photographed with a single camera. The ocular imaging apparatus of the present invention with such a configuration has low power consumption, less load on the communication channel, less heat load, low cost, less circuit complexity, and light weight.

The light collecting means photographs the left and right eyeballs or objects in the eyeball and eye directions respectively in two regions of the image sensor divided by a straight line passing through the center of the long side of the rectangular image sensor, The light is collected so that the length of the quadrangle of this image sensor is parallel to the short side, and the wasted pixel area is eliminated, so that the eyeball can be enlarged as much as possible.

In the case of the first embodiment, it is easy to discharge the heat of a single camera to the outside by locating the single camera biased toward the case side of the hmd.

Unlike the other eye-tracking devices (for example, in Patent Document 3, in which a camera is installed in the eyeglass frame of the eye) in which the camera is mounted on the spectacle frame in a disoriented manner in the case of the second embodiment, The camera is not exposed, so it looks good. For example, you can not see the camera at all from the outside if you make a spectacle lens dark like a sunglass.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1A is a perspective view illustrating a schematic configuration of a binocular imaging apparatus using a single camera according to a preferred embodiment of the present invention; FIG.
1B shows a photographed eye image.
1C is another example of a photographed eye image.
1D is another example of a photographed eye image.
Figure 2a is a side view of Figure 1;
FIG. 2B is a view showing an eyeball taken with a convex relay lens. FIG.
Fig. 2c is a view showing the eyeball being shot with a concave relay lens; Fig.
Figure 3a is a front view of Figure 1;
FIG. 3B is a modification of FIG. 3A. FIG.
Figure 4 is a plan view of Figure 1;
FIG. 5 is a side view showing a schematic configuration of a binocular imaging apparatus using a single camera in which a 45-degree reflection mirror is omitted in FIG.
FIG. 6A is a perspective view showing a schematic configuration of a binocular image taking apparatus using a single camera in which a relay lens is omitted in FIG. 5; FIG.
Figure 6b shows the reflection of light to a single camera with two right prisms.
6C is a view showing a lens formed integrally with a right angle prism.
FIG. 6D is a view showing a lens formed integrally with a mirror. FIG.
FIG. 7 is a block diagram showing the configuration of a wearable line-of-sight tracking apparatus having a conventional goggle type.
Fig. 8 is a configuration diagram showing another conventional gaze tracking device. Fig.
9 is a perspective view showing one eye taken with two cameras
Fig. 10 shows a configuration for photographing an eyeball with four lenses
11 is a side view of Fig. 10
12 is a plan view
Fig. 13 is a view showing the image taken by the configuration of Fig. 10
14 is a perspective view of the configuration of the second embodiment
15A is a detailed view of the camera and the optical module of Fig. 14
Fig. 15B is a detailed view of the camera and the optical module of Fig. 14
Fig. 16 is a plan view of Fig. 14
17 is a side view of Fig. 14
FIG. 18 is a view showing a detachable 3d polarizing spectacle lens module
19 is a front view of a conventional ocular device
20 is a rear view of a conventional ocular imaging apparatus
21 is a side view of the conventional ocular imaging apparatus
Fig. 22 is a view showing a mirror image of the camera in the configuration of Fig. 16
23 is a side view of Fig. 22
Figure 24 shows a camera with a camera attached to the legs of a pair of glasses

Example 1

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.

The term " light gathering " used in the present invention collects the first light and the second light in both directions coming from both the first mirror and the second mirror, which are the first reflection mirrors, .

The binocular imaging apparatus using a single camera according to the present embodiment as shown in FIG. 1A relates to a form included in a head mount display (hmd) for a virtual reality. The binocular photographing apparatus includes first and second infrared visible visible light passing mirrors (first and second infrared visible visible light passing mirrors) for reflecting a first light (infrared ray) reflected from a first eyeball and a second light (infrared ray) reflected from a second eyeball to an intermediate point between two eyes a hot mirror, a light collecting means for advancing the light reflected by the first and second mirrors toward the camera, and a camera for photographing the light incident by the light collecting means.

That is to say, the gist of the present invention is that a two-way light that primarily reflects the first and second light beams from the first and second eyes through the first and second mirrors (a middle point between two eyes, Vertical line), and has a layout of optical paths for advancing the first and second lights in a single direction to a single camera. Therefore, all of the light in both eyes can be photographed with one camera.

The first and second mirrors are first and second hot mirrors for primary (first) reflection of reflected infrared rays illuminated in both directions. The hot mirror is an optical module that reflects infrared light and passes visible light.

Further, the light collecting means may be realized by a first optical module and a second optical module, wherein the first and second optical modules are formed by a mirror (for example, a mirror-like or V-shaped reflecting mirror of this embodiment) or a prism Lt; / RTI >

As shown in Figs. 1A, 2A, 3A, and 4, the binocular photographing apparatus 1 using a single camera according to the first embodiment includes a binocular photographing apparatus 1 using a single camera, A first infrared illuminating unit 30 for illuminating an infrared ray in a binocular direction facing the screen of the display, a second infrared illuminating unit 30 for illuminating the first infrared illuminating unit 30 by infrared rays of the first and second infrared illuminating units 30, A first and a second infrared mirrors 50 for reflecting the illuminated images in the binocular and a third mirror 50 for reflecting the infrared rays reflected through the first and second infrared reflecting mirrors 50 to the single camera 10, , And a four-reflection mirror (70).

Here, the third and fourth reflecting mirrors 70 correspond to the light collecting means.

When the binocular photographing apparatus 1 using the single camera according to the first embodiment is applied to a virtual reality (VR) HMD, an image processing unit for analyzing an image photographed by the single camera 10 and detecting an eyeball Not shown).

The single camera 10 is disposed near the center between the eyes (the same distance between two eyes, or the point where the distance is the shortest, or the vicinity thereof is the most preferable).

The single camera 10 includes a lens 11 for receiving a binocular image reflected through the first and second reflecting mirrors 70 and a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor) An infrared sensor 11 attached to the front surface of the lens 11 or the front surface of the image sensor 13 and transmitting infrared rays only through infrared rays. The infrared transmission filter may be omitted.

The single camera 10 photographs the light reflected from the eyes in different areas of the image sensor 13 as shown in FIG. 1B. Usually, the image sensor 13 has a rectangular shape whose width is longer than vertical length. For example, the number of pixels in the horizontal and vertical directions may be 640 * 480.

The method of dividing the rectangular area into two and photographing the left and right eyes in each area may be the same as the method shown in FIG. 1B or 1C or 1D according to the method of disposing the mirror and the camera. Among these three methods, the method of FIG. 1b is most preferable. This is because the wasted pixel area 13a, which is not used for photographing in Fig. 1C and Fig. 1D, does not exist in Fig. 1B. In other words, if the photograph is taken as shown in FIG. 1B, the eyeball can be enlarged and photographed without wasting pixels. In order to take such a photograph, the direction and position of the camera and the mirror are adjusted as shown in FIG. 1A to draw a straight line passing through the centers 640a and 640b of the long side 640 of the rectangle of the image sensor 13, (Arrow 480a) of the eyeball is parallel to the short side length 480 of the rectangle.

The first and second infrared illumination units 30 are for illuminating the infrared rays in both directions, and may be composed of an infrared LED (Light Emitting Diode) 30.

As shown in Figs. 2A and 4, the infrared LED 30 is disposed on the left and right sides of each of the binoculars to illuminate the eyeball.

In particular, the infrared LED 30 is preferably disposed along the periphery of the HMD lens 40 focusing the display.

The first and second infrared ray reflection mirrors 50 and 50 are formed of a hot mirror having a property of transmitting visible light and reflecting infrared light. The first and second infrared ray reflection mirrors 50 and 50 ' (Such as a funnel shape) 45 ° toward the center between both eyes on the outside of the HMD lens 40 so that the user's eye image illuminated by the infrared rays is refracted by 45 ° so as to be input to the third and fourth reflection mirrors 70 do.

In addition, as shown in FIG. 2A, the first and second infrared ray reflecting mirrors 50 have a bell shape that widens toward the front as viewed from the side.

The third and fourth reflecting mirrors 70 serve as a relay for reflecting the infrared rays reflected by the first and second infrared reflecting mirrors 50 to be input to the single camera 10 again.

That is, as shown in the front view of FIG. 3A, the third and fourth reflecting mirrors 70 are arranged in a mountain shape (that is, vertically inverted V-shaped) perpendicular to the first and second infrared reflecting mirrors 50 .

The mountain-shaped reflection mirror 70 is arranged so as to be inclined in a direction to be widened from the top to the bottom.

2A, when a single camera 10 is large, a space in which a single camera 10 is installed between two eyes is narrow and is placed close to the user's forehead or forehead, And is arranged in a direction perpendicular to the reflection mirror 70. [

The distance between the mount mirror 70 and the single camera 10 is increased so that the image relay means such as the convex relay lens 80 or the concave relay lens 80f is disposed on the mount mirror 70 It is preferable.

The convex relay lens 80 of FIG. 3B shows a modification in which the convex relay lens 80 is disposed between the first and second infrared mirrors 50 and 70.

In this case, if the focal length of the lens constituting the image relay means is longer than the lens diameter (that is, if it is a telephoto lens), only a part of the pupil may be photographed. In order to stably track the eyeball in the photographed image, the pupil and its surrounding whiteness region must also be photographed on a single screen. Therefore, the convex relay lens 80 or the concave relay lens 80f constituting the image relay means must have a focal length of It is preferable to make them substantially equal or shorter. It is preferable that the lens 11 of the single camera 10 uses a telephoto lens having a long focal length so as to enlarge a full-screen enlarged image of a convex or concave relay lens of a small diameter of the remote image relay means.

2B shows a state in which an eyeball is photographed using a convex relay lens 80, and FIG. 2C shows a state in which an eyeball is photographed using a concave relay lens 80f (a mirror is not shown in the drawing).

2B, an image 80b is formed on an image plane (a portion where the light reflected from the object is focused on the lens and focuses) through the convex relay lens 80. [ The lens 11 of the single camera 10 photographs the eye image 80b on the image sensor 13. [ It is preferable that the single camera 10 uses a telephoto lens with a narrow viewing angle since the small eye image 80b formed by the convex relay lens 80 should be enlarged to fill the screen.

2C, the image relay means is constituted by a concave relay lens 80f, and the light reflected from the eye forms an image 80g on the image plane 80e between the eyeball and the concave relay lens 80f. The camera 10 photographs the image 80g by the concave relay lens 80f.

The image relay means may include a first image forming means for concealing an image of the first eyeball and a second image forming means for concealing an image of the second eyeball. The first and second image- It may be further limited to the lens 80 or the first and second concave relay lenses 80f.

6A, when the micro-camera 10 is installed close to the reflecting mirror 70, the lens 11 of the micro-camera 10 captures both the pupil and the whitish on one screen It is preferable to use a lens (approximately a standard lens) having a viewing angle sufficient to allow the image to be displayed.

The image relay means transmits the light reflected by the mountain-shaped reflection mirror 70 so as to form an image close to the single camera 10, thereby allowing the single camera 10 to photograph the eyes of both eyes.

Further, the single camera 10 and the mount mirror / image relay means are disposed at right angles to each other, so that a 45 degree reflection mirror 90 is further provided on the optical path to reflect the light to the single camera 10 Respectively.

The mirror-type reflective mirror 70 / the image relay means is disposed inside the first and second infrared-reflective mirrors 50.

On the other hand, if the single camera 10 is arranged on a line-like upper side as the mirror-type mirror 70 / image relay means as shown in Fig. 5, the 45 degree reflection mirror 90 can be omitted.

As shown in FIG. 5, when the camera 10 is positioned at the side of the case 1a of the hmd, the heat of the camera 10 can be easily discharged to the outside through the case 1a.

On the other hand, when the camera is small, the single camera 10 may be positioned directly on the reflecting mirror 70 as shown in FIG. 6A to directly track the line of sight of both eyes.

FIG. 6B shows a modified example in which the mirror-shaped reflective mirror 70 is replaced with two right-angled prisms 70a. The first and second lights incident on the left and right sides are totally reflected toward the camera 10 in the right prism 70a.

6C is a modification of the light collecting means 70b in which a convex or concave Fresnel lens is formed on the surface through which the light ray of the right angle prism passes and the relay lens is integrally formed in the prism.

6D is a modification of the light collecting means 70c in which the mirror surface is formed as a convex or concave curved surface and the concave lens or the convex lens is integrally formed in the mirror.

The image processing unit (not shown) can detect the center of the pupil using the binocular image of the user photographed by the single camera 10, and extract the center coordinate value of the detected pupil.

As described above, when the optical apparatus according to the present embodiment is used, if the camera is small, the camera can be installed between the two eyes as shown in FIG. 6A. However, the smaller the camera, the lower the resolution and the slower the shooting speed. To solve this problem, when a high-resolution or high-speed camera is used, the camera becomes large and the space between the two eyes becomes narrow, making it difficult to install the camera. Especially, if you install the camera in hmd, you may obscure the view if the camera is big. When the camera is large, it is preferable to use the relay lenses 80 and 80f so that the camera can be photographed at a high resolution instead of being installed above the hmd which is far from the eyes and has a large space as shown in Fig. 5 or Fig. 2A . If the relay lens 80 is omitted in the configuration of FIG. 5, the third and fourth mirrors 70 and 70 must be large in order to photograph the entire eyeball (the pupil and its peripheral area). However, . However, if the relay lens is made very small, the mirror can be made very small. Even if the relay lens is very small, if the camera lens is made into a telephoto lens, the eye image formed on the small relay lens can be enlarged and enlarged on the image sensor screen.

Using such a relay lens type imaging device, a high-speed camera with a large size can be easily installed inside the hmd to capture the eyeball and calculate the gaze direction, thereby providing a technique called foveated rendering used in virtual reality hmd It can be easily implemented. Forbidden rendering is a technique to reduce the burden on the graphics card by creating a graphic image of the virtual world of the portion of the virtual reality hmd wearer's head facing the high resolution and creating the surrounding image with low resolution.

Usually, the eye tracking device performs position adjustment (calibration) immediately after wearing the eye tracking device to track the eyeball. This position adjustment means to obtain the correlation between the image of the eyeball, the eye direction calculated from the image, and the mouse cursor coordinates corresponding to the eye direction. For example, a blinking mark or mark is sequentially output to a specific point on the screen (for example, four vertices of a square), and an eyeball is photographed when an hmd wearer with a gaze tracker looks at the mark, And coordinates the coordinates of the eyeball detected in the image with the positions of the marks displayed on the screen.

However, it is inconvenient to wear Hmd for a long time, so you can wear it again after removing the hmd. In this case, it may be different from the originally worn position. This will change the relative position between the camera and the eye, and you will have to perform the calibration (calibration) again for eye tracking. In order to overcome these inconveniences, we have proposed a three-dimensional (3-D) relative position calculation between the eyeball and the camera by analyzing the stereoscopic images obtained by photographing one eyeball in stereo with two cameras. ellipse in 3D space) ( https://www.researchgate.net/publication/220810955_Calibration-free_eye_tracking_by_reconstruction_of_the_pupil_ellipse_in_3D_space ).

FIG. 9 shows an eye tracking apparatus according to the present invention, in which two cameras photograph one eye reflected through a hot mirror in stereo.

The eye tracking apparatus of the present invention shown in FIG. 1A can be modified as shown in FIG. 10 so that both eyes can be photographed in stereo. In Fig. 1A, two relay lenses 80 are provided on the left and right sides (i.e., two relay lenses are provided in two rows and one row, which are referred to as first and second imaging means) . In contrast, in FIG. 10, four relay lenses are provided in a matrix of two rows and two columns (this is referred to as first, second, third and fourth imaging means). FIG. 11 is a side view of such a device, Is a plan view. Fig. 13 shows an example of an image photographed with relay lenses of 2 rows and 2 columns. When stereo matching the left two images L1 and L2 in the captured image, the three-dimensional relative position between the left eye and the camera can be obtained. When the right two images R1 and R2 are stereo-matched, Dimensional relative position can be used to position (calibrate) the eye tracking.

Example 2

The present embodiment relates to a configuration in which a camera and optical modules 501a and 501b are provided on one or both of the nose receivers 500a and 500b of ordinary glasses as shown in FIG. 14 and the eyes and the eyes are simultaneously photographed. In this case, the spectacle lens includes the spectacles display 502 to output the augmented reality image. The user can analyze the image of the eye with the camera and move the mouse cursor 503 of the computer screen displayed on the glasses-type display. Also, the virtual object 504 of the augmented reality can be manipulated by recognizing the shape of the hand and the gesture motion in the image of the wearer's hand in the sight line direction photographed by the camera. Light incident from the eye direction is incident on the camera through the first optical module, and light reflected from the eye travels to the camera through the second optical module. The optical module includes optical elements such as mirrors, prisms, diffraction plates, holograms, and lenses, and changes the direction of light by reflecting, diffracting, or refracting light.

Conventional glasses-type displays have a problem in that a camera for taking a forward image and a camera for taking an eye are installed (that is, one or more cameras are installed), and the apparatus is heavy and consumes a large amount of power. By using the configuration of this embodiment, it is possible to solve such a problem by photographing the eyes and the front with one camera.

15A is an enlarged view of the camera and optical module 501a of Fig. This configuration is configured to be bilaterally symmetrical to the left and right nose glasses of the glasses. In FIG. 15A, the camera 600 is provided with third and fourth mirrors 601 and 602 in the form of a V-shape. The first and second mirrors are installed on the upper side 603 and the lower side 604 of the third and fourth mirrors 601 and 602, respectively. The first and second mirrors may be formed of convex mirrors or concave mirrors to increase the viewing angle of the camera. A concave relay lens or a convex relay lens (first and second image forming means) may be added to the first and second mirrors in the same manner as in Embodiment 1.

The first and second mirrors may be divided into two convex mirrors or two concave mirrors 601a, 601b, 602a, and 602b as shown in FIG. 15B so that an external object in the eye direction and the sight direction can be photographed in stereo. Alternatively, a concave relay lens or a convex relay lens may be arranged side by side (in one row or two rows or two rows and one row) in front of the first and second mirrors so that an object in the eye and a sight line direction can be photographed in stereo, (First, second, third, and fourth imaging means) may be added.

The camera reflects the upper image reflected by the upper third mirror 601 of the v-shaped mirror and the upper first mirror 603 and the lower fourth mirror 602 and the lower second mirror 604 of the v- The lower image is taken on one screen. For example, shooting two images on one screen means shooting one screen by dividing one screen vertically or horizontally as shown in FIG. In this case, the third mirror 601 and the upper first mirror 603 correspond to the first optical module, and the lower fourth mirror 602 and the lower second mirror 604 correspond to the second optical module .

The light reflected from the eyeball 605 is reflected by the spectacle lens 505 and then sequentially reflected on the lower second mirror 604 and the lower third mirror 602 of the v-shaped mirror, and then reaches the camera. The light reflected from a forward object (for example, the wearer's hand) 606 passes through the spectacle lens 505 and then passes through the upper first mirror 603 and the upper third mirror 603 of the v- And reaches the camera. It is also possible to omit some of these mirrors by adjusting the shooting direction of the camera (optical axis, straight line perpendicular to the camera lens). The first mirror 603 and the third mirror 601 may be integrated into one mirror, or the second mirror 604 and the fourth mirror 602 may be integrated into a single mirror.

Instead of such a mirror, a reflection type prism, a diffraction element, a hologram, or the like may be used. If the distance between the camera and the mirror is too long, a relay lens (image relay means) may be added on the optical path as shown in FIGS. 2B and 2C of the first embodiment. An infrared light such as an infrared LED is applied to the eyeball 605 so that the infrared ray is reflected from the eye and then reflected by the eyeglass lens 505. A region 505a near the eye of the eyeglass lens (Hot mirror coating) is preferably applied to the infrared ray-visible region. Also, in the vicinity of the nose 505b of the spectacle lens (i.e., reflected or reflected from the forward object) so that the light reflected from the forward object 606 (e.g., the wearer's hand) can reach the camera through the spectacle lens 505 It is preferable not to apply a hot mirror coating to the lens region through which the emitted light passes to reach the camera. In addition, a visible light cutoff filter 608 is installed between the lower fourth mirror 602 of the v-shaped mirror and the camera lens so that the objects 607 on the front outside the spectacle lens are not superimposed on the eyeball in the field of view of the camera shooting the eyeball . The reason for shooting the eyeball through the lower fourth mirror rather than the upper third mirror is that the risk of covering the eyeball by the eyelid is low when the eyeball is taken from the bottom.

Fig. 16 is a plan view of the configuration of Fig. 15A and Fig. 17 is a front view of the configuration of Fig. 15A.

Instead of applying a hot mirror coating to the spectacle lens, a small hot mirror coated lens module as shown in Figure 18 may be sandwiched between the existing eyeglasses and the wearer's eye. Such a removable lens module is used in a conventional polarizing type 3d glasses module. A camera and an optical module may be included in or near the nose support of such a hot mirror coated lens module. In this case, the user wears his or her existing glasses and wears these hot mirror coated lens modules to shoot and track the eyes.

The configuration of this embodiment is characterized in that a hot mirror coating is applied to a spectacle lens and an eye image reflected on the coated surface is photographed by a camera. On the other hand, the eyeglass-type eye-taking apparatus of Patent Document 3 is a method of photographing the eyeball with the camera but not directly photographing the eyeball reflected on the lens but directly photographing the eyeball. FIG. 19 is a front view of an eyeball photographing device posted on the homepage of Tobii, the applicant of Patent Document 3, and FIG. 20 is a view 2b of Patent Document 3. FIG. Note that the camera is not visible in the front view, and that the camera 611 is visible in FIG. 20D (rear view), the side view of the device is as shown in FIG. That is, the spectacle lens 700 is the outermost and the camera 611 exists between the spectacle lens 700 and the eye 703. In order to directly photograph the eyeball with the camera (that is, to photograph the eyeball reflected directly on the lens, as opposed to photographing the eyeball reflected on the lens) as shown in FIG. 21, the camera should be protruded by a predetermined distance 702 from the eyeball slightly forward (away from the face skin) there is a problem. However, in the apparatus of the present embodiment, as shown in FIG. 22, the eyeball reflected on the spectacle lens is photographed, so that the camera is located outside the spectacle lens, so that the eyeball can be photographed at such a distance at least. FIG. 22 shows the effect that the camera 600 of FIG. 16 is reflected by the lens 505 and is located outside the lens. Reference numeral 801 in FIG. 22 denotes a mirror image in which the camera 600 is reflected by the spectacle lens, and 800 denotes a distance between the camera and the eye in the mirror image. This interval corresponds to 702 in Fig. That is, by shooting the reflected eyeballs on the hot mirror coated lens, the camera has close distance to close the eyeball, which has the advantage of shooting at a long distance (ie, less distortion of the eyeball in the captured image). Also, by installing these cameras on the nose pod rather than the eyeglass frame, the camera is virtually invisible from the outside, and the camera can be installed on the rimless eyeglasses. In contrast, the conventional ophthalmologic apparatus as shown in FIG. 19 must have a spectacle frame 610 (frame) to be installed thereon, and the spectacle frame protrudes forward from the skin. That is, the conventional eye-taking apparatus of Patent Document 3 can not be installed in the rimless eyeglasses. Figure 23 shows a side view of the device of this embodiment for comparison with Figure 21; 23, the lens of the spectacles of this embodiment does not need to protrude in an unsightly manner unlike the conventional apparatus of FIG.

Such a camera may be attached to the eyeglass leg 901 as shown in Fig. 24 to photograph the eye reflected on the spectacle lens. However, the eyeglass legs 901 are generally above the nose receiver 501a. Therefore, the camera attached to the eyeglass legs 901 is required to shoot while looking downward from above the eyes, so that there is a risk that the eyes are covered by the eyelids. However, when the camera is placed on the lower nose of the eyeglass legs, the eye can be taken from the bottom to the top so that there is less risk of covering the eyes with the eyelids.

It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present invention as defined by the following claims .

10: Single camera 30: 1st and 2nd infrared illumination units (Infrared LED)
40: 1st and 2nd hmd lenses 50: 1st and 2nd infrared reflection mirrors (hot mirror)
70: Third and fourth reflecting mirrors (mirror reflecting mirrors)
80: convex relay lens 80f: concave relay lens
90: 45 degree reflection mirror 13a: Wasted pixels
80c: image forming surface 11: camera lens
13: Image sensor 80b, 80g: eye image by relay lens
70a, 70b: prism 70c: convex reflection mirror
100: Glasses 110: Front camera
140: eyeball camera 120: infrared light
502: eyeglass type display 503: mouse cursor
50: spectacle lens 500a, 500b:
501a, 501b: camera and optical module 504: virtual object
606,607: Outside thing 600: Camera
603: First mirror 604: Second mirror
601: Third mirror 602: Fourth mirror
605: eyeball 608: visible light blocking filter
505a: ocular reflection light reflection surface 505b: external object reflection light transmission surface
610: eyeglass frame 611: camera
700: spectacle lens 702,800: gap between camera and eye
703: eyeball 801: mirror image of the camera
901: Glasses bridge 900: Glasses attached to the glasses legs

Claims (25)

A single camera for photographing a first light reflected by the first eyeball and a second light reflected by the second eyeball;
A first hot mirror for reflecting the first light;
A second hot mirror for reflecting the second light;
Light collecting means for collecting the first light and the second light reflected by the first hot mirror and the second hot mirror and advancing the light toward the single camera;
And a binocular image capturing unit for capturing the binocular image. The method according to claim 1,
Wherein the light collecting means collects light so that images of light reflected by the first and second eyes are photographed in different areas of the single camera, respectively. The method according to claim 1,
Wherein the light collecting means is disposed at a midpoint between the first eyeball and the second eyeball. The method according to claim 1,
The light collecting means includes a first optical module for advancing the first light reflected from the first hot mirror toward the single camera and a second optical module for advancing the second light reflected from the second hot mirror toward the single camera Wherein the binocular image capturing device comprises an optical module. 5. The method of claim 4,
Wherein the first and second optical modules include a mirror, a prism, a hologram, or a diffraction element. 6. The method of claim 5,
Wherein the optical module includes an optical element whose surface is of V-shape or mountain-like shape. 5. The method of claim 4,
Wherein the light collecting means includes an image relay means for imaging an ocular image, and the camera captures the imaged image. 8. The method of claim 7,
The image relay means includes a first imaging means for imaging the first eyeball and a second imaging means for concealing an image of the second eyeball, and the camera is configured to capture an image of the first and second imaging means Wherein the first and second images are displayed on the display unit. 8. The method of claim 7,
Wherein the image relay means includes a first imaging means and a third imaging means for capturing a stereo image of the first eyeball, a second imaging means and a fourth imaging means for capturing a stereo image of the second eyeball, , 2, 3, 4 imaging means for capturing a stereoscopic image. 9. The method of any one of claims 9 to 9,
Wherein the image forming means includes a convex lens, a concave lens, a convex mirror, or a concave mirror. 11. The method of claim 10,
The focal length of the convex lens, the concave lens, the convex mirror, or the concave mirror may be such that the entire eyeball region is included in the image of the image formed by the convex lens, the concave lens, the convex mirror, or the concave mirror, And the distance between the first and second cameras is equal to or greater than a predetermined distance. 3. The method of claim 2,
Wherein the light collecting means collects light so that the left and right eyes are respectively photographed in two regions of the image sensor divided by a straight line passing through the center of the long side of the rectangular image sensor. 8. The method of claim 7,
Wherein the single camera is located close to the case-mount display (hmd) case for easy heat dissipation.

A single camera for photographing a first light incident on the eye direction and a second light reflected on the eyeball;
A first optical module for advancing the first light toward the camera;
A hot mirror for reflecting the second light and transmitting visible light;
A second optical module for advancing the second light reflected by the hot mirror toward the camera;
And an image capturing unit for capturing an image of the eye. 15. The method of claim 14,
Wherein the first and second optical modules collect light so that the images of the first and second lights are respectively photographed in different areas of the single camera. 16. The apparatus according to claim 15, wherein the first and second optical modules or cameras are provided on the rim of the spectacles, the legs of the spectacles, the nose, or the vicinity thereof. 15. The apparatus according to claim 14, wherein the visual field of the camera for photographing the second light includes a visible light blocking filter. 15. The apparatus according to claim 14, wherein the first and second optical modules include a mirror, a prism, a hologram, or a diffractive element for changing the path of the first and second lights. 19. The method of claim 18,
Wherein the optical module includes an optical element whose surface is a V-shaped or mountain-shaped. 15. The apparatus according to claim 14, wherein the first and second optical modules include image relay means for imaging the images of the first and second lights, and the camera captures the image formed thereon, . 21. The method of claim 20,
The image relay means includes a first imaging means for imaging the first light and a second imaging means for imaging the second light, and the camera is configured to capture an image of the first and second imaging means The apparatus for measuring an eyeball using a single camera. 21. The method of claim 20,
Wherein the image relay means includes a first imaging means and a third imaging means for capturing a stereo image of the first light, a second imaging means and a fourth imaging means for capturing a stereo image of the second light, , 2, 3, 4 imaging means for imaging the stereoscopic image. 21. The container according to any one of claims 21 to 21,
Wherein the image forming means includes a convex lens, a concave lens, a convex mirror, or a concave mirror. 24. The method of claim 23,
The focal length of the convex lens, the concave lens, the convex mirror, or the concave mirror may be such that the entire eyeball region is included in the image of the image formed by the convex lens, the concave lens, the convex mirror, or the concave mirror, And the distance between the eyeball and the eyeball is the distance. 16. The method of claim 15,
Wherein the first optical module and the second optical module collect light in such a manner that the object and the eyeball in the visual line direction are respectively photographed in the two regions of the image sensor divided by the straight line passing through the center of the long side of the rectangular image sensor. Eye - taking device using camera.

Images