WO2018164104A1 - Dispositif de traitement d'image oculaire - Google Patents

Dispositif de traitement d'image oculaire Download PDF

Info

Publication number
WO2018164104A1
WO2018164104A1 PCT/JP2018/008522 JP2018008522W WO2018164104A1 WO 2018164104 A1 WO2018164104 A1 WO 2018164104A1 JP 2018008522 W JP2018008522 W JP 2018008522W WO 2018164104 A1 WO2018164104 A1 WO 2018164104A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
light source
pupil
period
corneal reflection
Prior art date
Application number
PCT/JP2018/008522
Other languages
English (en)
Japanese (ja)
Inventor
嘉伸 海老澤
Original Assignee
国立大学法人静岡大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 国立大学法人静岡大学 filed Critical 国立大学法人静岡大学
Priority to JP2019504593A priority Critical patent/JP6957048B2/ja
Publication of WO2018164104A1 publication Critical patent/WO2018164104A1/fr

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
    • A61B3/113—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for determining or recording eye movement

Definitions

  • the present invention relates to an eye image processing apparatus that detects a pupil from a human image.
  • a difference image is generated using a bright pupil image and a dark pupil image acquired by a video camera, and a pupil image is detected based on the difference image (Patent Document 1 below).
  • position correction between images based on the cornea reflection image in each image is performed. Such position correction improves the detection accuracy of the pupil image.
  • Non-illuminated images are acquired at the timing when the light source is turned off in order to improve the detection accuracy of pupil images in difference images even under the influence of ambient light such as sunlight.
  • the difference image based on the bright pupil image and the unilluminated image and the difference image based on the dark pupil image and the unilluminated image are created, and the difference image based on these two difference images is used.
  • An apparatus for detecting the pupil is also known (Patent Document 2 below).
  • One aspect of the present invention has been made in view of the above problems, and provides an eye image processing apparatus capable of improving the detection accuracy of a pupil image in an image even when ambient light is present.
  • the purpose is to provide.
  • an eye image processing apparatus has a camera that acquires an eye image by capturing an eye of a subject, and a distance from the center of the opening of the camera is the first.
  • a first light source arranged at a distance position, a second light source arranged at a second distance position where the distance from the center of the opening is larger than the first distance, the first light source and the second light source.
  • a lighting control unit that controls lighting of the light source, a bright pupil image that is an eye image acquired by the camera during the first period when the first light source is turned on, and a second light source that is turned on.
  • a calculation unit that calculates a position of the hole image the calculation unit detects the position of the cornea reflection image of the subject in the bright pupil image, the dark pupil image, and the background image, and based on the bright pupil image and the background image
  • a difference image based on the position of the corneal reflection image a difference bright pupil image is obtained, and a difference image based on the position of the corneal reflection image is calculated based on the dark pupil image and the background image.
  • the position of the pupil image of the subject is calculated by performing subtraction or division on the differential bright pupil image and the differential dark pupil image.
  • the pupil is relatively obtained by acquiring an image with the camera in accordance with the lighting timing of the first light source arranged at a position relatively close to the center of the opening of the camera. A bright pupil image was obtained, and the image was obtained with the camera in accordance with the lighting timing of the second light source disposed at a position relatively far from the center of the opening, so that the pupil appeared relatively dark. A dark pupil image is obtained.
  • a background image in which a corneal reflection image including the influence of ambient light is captured is obtained.
  • the difference bright pupil image obtained by subtracting the bright pupil image from the background image with reference to the corneal reflection image, and the difference from the dark pupil image and the background image using the cornea reflection image as a reference are obtained.
  • the position of the pupil image is calculated using the obtained difference dark pupil image.
  • the detection accuracy of a pupil image in an image can be improved even when ambient light is present.
  • FIG. 1 It is a figure which shows schematic structure of the pupil detection apparatus concerning 1st Embodiment. It is the top view which looked at the illuminating device of FIG. 1 from the outer side of the housing
  • a pupil detection device 1 includes a camera 2, an illumination device (light source) 3, and a control device 4.
  • the camera 2 includes a housing 5, an image sensor 6 such as a CCD or CMOS housed in the housing 5, and an objective lens 7 housed in the housing 5.
  • This camera 2 may be a high-speed camera in which the acquisition time interval of one frame of an image is very short, a so-called medium-speed camera, or a camera having a frame rate of about 60 Hz.
  • the housing 5 has a circular opening 8 formed on a surface facing the eyeball A of the observation subject.
  • the objective lens 7 is disposed between the opening 8 and the image sensor 6.
  • the optical axis L 0 of the objective lens 7 coincides with the central axis of the opening 8.
  • the imaging element 6 is fixed so that its light receiving surface intersects perpendicularly to the optical axis L0 of the objective lens 7.
  • the image sensor 6 generates eye image data by capturing an image of the eyeball A of the subject and outputs the eye image data to the control device 4.
  • the control device 4 controls the light emission intensity, lighting timing, and lighting period of the lighting device 3 and the imaging timing and imaging period of the camera 2. Further, the control device 4 executes difference processing, pupil detection processing, and corneal reflection detection processing based on the eye image data output from the image sensor 6.
  • the diameter of the opening 8 is smaller than the diameter of the objective lens 7 and is approximately the same as the effective diameter of the objective lens 7.
  • the illumination device 3 emits illumination light toward the subject's face.
  • the lighting device 3 includes a casing 9 and light sources 3A, 3B, and 3C embedded in the casing 9.
  • the casing 9 is attached to the housing 5 so as to cover the outside of the opening 8 along the edge of the opening 8.
  • the light sources 3A, 3B, 3C are all provided on the casing 9 so as to emit illumination light along the optical axis L0 of the objective lens 7.
  • the light source (first light source) 3A is a light source for irradiating illumination light for obtaining a bright pupil image toward the face of the subject.
  • a bright pupil image refers to an image in which the pupil of the subject appears relatively brighter than a dark pupil image described later.
  • the light source 3A includes, for example, a plurality of semiconductor light emitting elements (LEDs) whose output light has a center wavelength in the near-infrared region, and is disposed at a first distance D1 that is relatively close to the center of the opening 8. ing.
  • the light emitting elements constituting the light source 3 ⁇ / b> A are arranged in a ring shape at equal intervals along the edge of the opening 8 on the outside of the opening 8 on the casing 9.
  • the light source 3 ⁇ / b> A is preferably provided at a position as close as possible to the edge of the opening 8. As a result, as will be described later, in the image of the subject illuminated by the light source 3A, the pupil is projected brighter, and detection is easy even with a small pupil. In the image of the subject illuminated by the light source 3A, a cornea reflection image of the subject is also generated.
  • the light source (second light source, dark pupil light source) 3B is a light source for irradiating illumination light for obtaining a dark pupil image toward the face of the subject.
  • the dark pupil image refers to an image in which the pupil of the subject appears relatively dark compared to the above-described bright pupil image.
  • the light source 3B includes, for example, a plurality of semiconductor light emitting elements (LEDs) whose output light has a center wavelength in the near infrared region, and is disposed at a position of the second distance D2 that is relatively far from the center of the opening 8. ing. This second distance D2 is greater than the first distance D1.
  • the light emitting elements constituting the light source 3B are spaced apart from the light source 3A to the outside of the opening 8 on the casing 9 and arranged in a ring shape at equal intervals.
  • the light source (second light source, corneal reflection generation light source) 3C directs illumination light for conspicuously generating a corneal reflection image in the bright pupil image, the dark pupil image, and the background image toward the subject's face. It is a light source for irradiating.
  • the background image refers to an image mainly showing an image of the face of the subject illuminated by ambient light such as sunlight.
  • the light source 3C includes, for example, a plurality of semiconductor light emitting elements (LEDs) whose output light has a center wavelength in the near infrared region, and the distance from the center of the opening 8 is arranged at a third distance D3.
  • the third distance D3 is larger than the first distance D1 and smaller than the second distance D2.
  • the light emitting elements constituting the light source 3C are arranged in a ring shape on the casing 9 between the light source 3A and the light source 3B, spaced apart from the light sources 3A and 3B at equal intervals.
  • the pupil When the pupil is imaged by the camera 2 at the timing when the illumination light is emitted from the light source 3A to the eyeball A of the subject, a bright pupil image is acquired, and the illumination light is emitted from the light source 3B to the eyeball A of the subject.
  • the dark pupil image When the pupil is imaged by the camera 2 at the timing, the dark pupil image is acquired. This is due to the following properties. That is, when the illumination light to the eyeball A is incident from a position relatively distant from the optical axis L0 of the camera 2, the illumination light is incident from the pupil of the eyeball A, reflected inside the eyeball, and passed through the pupil again. Is difficult to reach the camera 2 in the direction of the illumination light, so that the pupil appears relatively dark.
  • the center wavelength of the output light of the light source 3A may be set to the same wavelength as the center wavelength of the output light of the light sources 3B and 3C, or may be set to a different wavelength.
  • the light source 3A which is a light source for acquiring a bright pupil image, uses a light source having an output light wavelength shorter than about 900 nm, such as 850 nm, for example, because the brightness of the light reflected from the retina is strong. desirable.
  • the light source 3B which is a light source for dark pupil acquisition, uses a light source with a wavelength of output light longer than about 900 nm, such as 940 nm, for example, in that the brightness of the light reflected back from the retina is weak. desirable.
  • the light source 3C only needs to have an intermediate effect between the light source 3A and the light source 3B for the illumination effect on the pupil of the subject.
  • the wavelength of 940 nm output light having the same wavelength as the light source 3B The light source is used.
  • the light source 3A having an output wavelength with a relatively short wavelength such as 850 nm is used, the light emission of the light source 3A is recognized by the subject when the amount of light is large, which affects driving when the subject is a car driver or the like.
  • a light source having a wavelength longer than about 900 nm is used as the light source 3A, which is the same as the light sources 3B and 3C in which light emission is difficult to be recognized by the subject.
  • the luminance difference of the pupil portion between the bright pupil image and the dark pupil image is less likely to occur due to the wavelength of the light source.
  • the light emission intensity (light emission power) of the light sources 3A and 3B is determined by the illuminance on the face of the subject who is a subject to be photographed when the light source 3A is emitted and the illuminance on the face when the light source 3B is emitted It is set to be substantially the same.
  • the light sources 3A, 3B, and 3C can control the lighting timing and the light emission amount of the illumination light independently by a control signal from the control device 4 described later.
  • the light sources 3A, 3B, and 3C are configured such that the lighting period can be controlled, and as a result, the light emission amount determined by the product of the light emission intensity and the lighting period is controlled.
  • control device 4 included in the pupil detection device 1 will be described with reference to FIGS. 3 and 4.
  • the control device 4 may be a computer that executes control of the image sensor 6 and the light sources 3A, 3B, and 3C, and detection of the pupil and corneal reflection of the subject.
  • the control device 4 may be constructed by a stationary or portable personal computer (PC), may be constructed by a workstation, or may be constructed by another type of computer.
  • the control device 4 may be constructed by combining a plurality of arbitrary types of computers. When a plurality of computers are used, these computers are connected via a communication network such as the Internet or an intranet.
  • the control device 4 includes a CPU (processor) 101, a main storage unit 102, an auxiliary storage unit 103, a communication control unit 104, an input device 105, and an output device 106.
  • the CPU 101 executes an operating system, application programs, and the like.
  • the main storage unit 102 includes a ROM and a RAM.
  • the auxiliary storage unit 103 is configured by a hard disk, a flash memory, or the like.
  • the communication control unit 104 includes a network card or a wireless communication module.
  • the input device 105 includes a keyboard and a mouse.
  • the output device 106 includes a display, a printer, and the like.
  • Each functional element of the control device 4 to be described later reads predetermined software on the CPU 101 or the main storage unit 102, operates the communication control unit 104, the input device 105, the output device 106, and the like under the control of the CPU 101, This is realized by reading and writing data in the main storage unit 102 or the auxiliary storage unit 103. Data and a database necessary for processing are stored in the main storage unit 102 or the auxiliary storage unit 103.
  • the control device 4 includes an image sensor driving unit 11, a lighting control unit (lighting control unit) 12, and a detection unit (calculation unit) 13 as functional components.
  • the image sensor driving unit 11 is a functional element that controls the shooting timing of the image sensor 6. Specifically, the imaging device 6 is controlled to repeatedly capture images at a predetermined frame rate and a predetermined exposure time, and sequentially acquire a bright pupil image, a background image, and a dark pupil image.
  • the lighting control unit 12 is a functional element that controls the lighting timing of the light sources 3 ⁇ / b> A, 3 ⁇ / b> B, and 3 ⁇ / b> C and the light emission amount within the exposure period of the imaging device 6 in synchronization with the imaging timing of the imaging device 6.
  • the lighting control unit 12 controls the light emission amounts of the light sources 3A, 3B, 3C by setting respective lighting periods. Specifically, the lighting control unit 12 turns on the light source 3A and the light source 3C continuously (or simultaneously) when taking a bright pupil image, and turns on only the light source 3C when taking a background image, thereby taking a dark pupil image. Sometimes, the light source 3B and the light source 3C are controlled to be continuously (or simultaneously) lit.
  • the detection unit 13 is a functional element that detects the pupil and corneal reflection in the eye image data using the eye image data output from the image sensor 6. The output destination of the information regarding the detected pupil and corneal reflection is not limited at all.
  • the control device 4 may display the result on a monitor as an image, graphic, or text, or may store the result in a storage device such as a memory or a database, or send it to another computer system via a communication network. May be.
  • the detection unit 13 includes an image acquisition unit 14, a difference image generation unit 16, a lighting time setting unit 17, a pupil detection unit 18, and a corneal reflection detection unit 19 as functional components.
  • the image acquisition unit 14 acquires, as eye image data, a bright pupil image, a background image, and a dark pupil image that are sequentially photographed from the image sensor 6 at a predetermined frame rate.
  • the difference image generation unit 16 generates a pupil detection image using the bright pupil image, the background image, and the dark pupil image. Specifically, the difference image generation unit 16 performs inter-pixel correspondence between the difference bright pupil image generated from the bright pupil image and the background image, and the difference dark pupil image generated from the dark pupil image and the background image.
  • a pupil detection image is generated by calculating the luminance subtraction or division.
  • the pupil detection unit 18 is a functional element that calculates the position of the pupil image using the pupil detection image.
  • the corneal reflection detection unit 19 is a functional element that calculates the position of the corneal reflection image using the bright pupil image, the background image, and the dark pupil image.
  • An example of processing performed by the pupil detection unit 18 and the corneal reflection detection unit 19 is as follows. First, the pupil detection unit 18 binarizes the pupil detection image, and performs isolated point removal, noise removal by morphological processing, and labeling. Then, the pupil detection unit 18 detects a pixel group having a shape most likely to be a pupil as a pupil.
  • the corneal reflection detection unit 19 binarizes the vicinity of the pupil of the bright pupil image with a threshold value higher than the pupil luminance, and obtains the center of the corneal reflection image as the center of gravity in consideration of the luminance.
  • the pupil luminance is given not by the area of the ellipse obtained as a result of the ellipse fitting, but by the luminance average of the pixels constituting the pupil obtained by binarization.
  • the corneal reflection detection unit 19 calculates the position of the corneal reflection image with respect to the background image and the dark pupil image.
  • the threshold used when obtaining the corneal reflection image from the background image is based on the threshold used when obtaining the corneal reflection image from the bright pupil image in the past and the threshold used when obtaining the corneal reflection image from the dark pupil image in the past. To be determined.
  • the lighting time setting unit 17 sets the lighting time of the lighting device 3 so that the degree of brightening the face of the subject within the exposure time of the camera 2 is equal between the acquired bright pupil image and dark pupil image.
  • the lighting time of the illumination device 3 is set so that the luminance of the corneal reflection image in the bright pupil image, the background image, and the dark pupil image is stabilized at the target value.
  • the degree to which the subject's face in the image is brightened (hereinafter simply referred to as “illumination intensity”) is not determined only by the lighting time of the illumination device 3, but the output of the light sources 3A, 3B, 3C.
  • the lighting time setting unit 17 sets the lighting time of each light source 3A, 3B, 3C in consideration of the wavelength of the output light of the light sources 3A, 3B, 3C and the emission intensity of the light sources 3A, 3B, 3C. .
  • the lighting time setting unit 17 controls the operation of the light sources 3A, 3B, and 3C via the lighting control unit 12 so that the lighting device 3 emits light for the set lighting time.
  • FIG. 5 is a timing chart showing the lighting timing of each light source controlled by the lighting time setting unit 17 and the lighting control unit 12, where (a) is the imaging timing of the camera 2 and (b) is the lighting of the light source 3A.
  • a shooting period (exposure period, first period) T 1 in which a bright pupil image is captured by the image sensor driving unit 11 and a shooting period (third period) T in which a background image is captured. 3.
  • An imaging period (second period) T 2 in which the dark pupil image is imaged and an imaging period T 3 in which the background image is imaged are set in this order, and then the imaging periods T 1 , T 3 , T 2 are set. , T 3 are periodically repeated.
  • the lighting control unit 12 the light source 3A in synchronization with the imaging period T 1, so 3C is lit, the lighting timing of the light emitting period T B of the light source 3A, and the lighting period T of the light source 3C overlapping in time it The lighting timing of CR is controlled.
  • the lighting control unit 12 so that only the light source 3C in synchronism with the shooting period T 3 is turned on, lighting timing of the light emitting period T CR of the light source 3C is controlled, the light source 3B in synchronism with shooting period T 2 as 3C is lit, the lighting timing of lighting period T D of the light source 3B, and the lighting timing of lighting period T CR source 3C overlapping in time it is controlled.
  • the light source 3B is turned off, for emphasis corneal reflected image in the bright pupil image light source 3C is lit for lighting period T CR to.
  • the light source 3C is lit for lighting period T CR to generate an image of corneal reflection to the image of the illuminated face by ambient light.
  • the light source 3A is turned off, the light source 3C for emphasis corneal reflected image in a dark pupil image Lights only during the lighting period TCR .
  • the lighting period T B and a lighting period T D is set so that the illumination intensity is equal .
  • the lighting period TCR at the time of acquisition of each image is set to be equal so that the brightness of the corneal reflection image can be balanced among the bright pupil image, the background image, and the dark pupil image.
  • the luminance component due to the ambient light or the light source 3C is canceled by the difference, and only the luminance component due to the light source 3A remains.
  • the luminance component due to the ambient light or the light source 3C is canceled by the difference, and only the luminance component due to the light source 3B remains.
  • the lighting time setting unit 17 sets the brightness value of the corneal reflection image in the bright pupil image, the background image, and the dark pupil image to be constant over time in the bright timing acquired in the previous timing frame.
  • pupil image to change the lighting period T CR frame of the next timing on the basis of the luminance values of the cornea reflection image of the background image, and the dark pupil image.
  • the “frame” refers to a continuous frame (for example, three consecutive frames) including one bright pupil image frame, one dark pupil image frame, and one background image. Accordingly, the lighting time setting unit 17 changes the lighting period T CR frame next timing.
  • the C i and brightness value of the peak of the cornea reflection image of the previous frame (i is a natural number)
  • the difference image generation unit 16 calculates a difference bright pupil image and a difference dark pupil image with respect to three consecutive bright pupil images, background images, and dark pupil images. That is, the difference image generation unit 16 aligns the bright pupil image and the background image with each other based on the position of the corneal reflection image, and then calculates a difference in luminance value between the images, thereby calculating the difference bright pupil image. To get. On the other hand, the difference image generation unit 16 aligns the dark pupil image and the background image with each other based on the position of the corneal reflection image, and then calculates the difference in luminance value between the images to calculate the difference darkness image. A pupil image is acquired.
  • the difference image generation unit 16 aligns the difference bright pupil image and the difference dark pupil image with each other based on the position of the corneal reflection image, the difference image generation unit 16 subtracts or divides the luminance value between these images.
  • the pupil detection image is acquired by the above.
  • the position (coordinates) of one image is shifted so that the positions of the corneal reflection images of the images coincide with each other, and then between one image and the other image. Subtract or divide by. By doing so, it is possible to obtain an image in which only the pupil portion of the subject appears by removing the cornea reflection image from the pupil detection image.
  • FIG. 6 is a graph showing a one-dimensional distribution of luminance in the vicinity of the pupil of the eye image data acquired by the image acquisition unit 14 and the difference image generation unit 16 of the control device 4, and (a) is a bright pupil image.
  • One-dimensional distribution part (b) shows the one-dimensional distribution of the dark pupil image
  • part (c) shows the one-dimensional distribution of the background image
  • part (d) shows the one-dimensional distribution of the pupil detection image.
  • These graphs show a one-dimensional distribution of eye image data on a straight line including corneal reflection. In this way, the light emission powers of the light sources 3A and 3B are adjusted so that the brightness values I D1 and I D2 of the bright pupil image and the dark pupil image are equal to those other than the pupil portion.
  • the bright pupil image has a luminance value I B is a relatively large value of the pupil portion
  • the luminance value I B of the pupil portion has a relatively small value I E in dark pupil image.
  • the luminance value saturation value It is set so as not to exceed I MAX . Therefore, in the pupil detection image, the luminance of the non-pupil portion is approximately zero appear only the luminance value I C at the pupil portion, noise and cornea reflection image due to ambient light is removed. As a result, the accuracy of the position of the pupil image calculated by the pupil detection unit 18 is improved.
  • an image is acquired by the camera 2 in accordance with the lighting timing of the light source 3A disposed at a position relatively close to the center of the opening 8 of the camera 2, so that the pupil is relatively A bright pupil image appears brightly, and the image is acquired by the camera 2 in accordance with the lighting timing of the light source 3B arranged at a position relatively far from the center of the opening 8 so that the pupil becomes relatively dark.
  • a captured dark pupil image is obtained.
  • the light sources 3A and 3B are turned off, and the camera 2 acquires an image in accordance with the timing when the light source 3C arranged at an intermediate position from the center of the opening 8 is turned on.
  • a background image including the corneal reflection image is obtained.
  • the difference bright pupil image obtained by subtracting the bright pupil image from the background image with reference to the corneal reflection image, and the difference from the dark pupil image and the background image using the cornea reflection image as a reference are obtained.
  • the position of the pupil image is calculated using the obtained difference dark pupil image.
  • the corneal reflection image can be stably generated in the bright pupil image, the dark pupil image, and the background image, and the position caused by the time difference in the acquisition timing between the bright pupil image and the dark pupil image and the background image The deviation can be corrected reliably. As a result, the detection accuracy of the position of the pupil image can be further improved.
  • an unilluminated image is acquired by turning off the illumination device, and the bright pupil image Create a differential bright pupil image based on the unilluminated image, create a differential dark pupil image based on the dark pupil image and the unilluminated image, and then use the differential bright pupil image and the differential dark pupil image to create the pupil. Detected.
  • this method when a medium-speed camera is used, if the subject's head moves, the head position shifts between the bright pupil image, the dark pupil image, and the unilluminated image. In some cases, the position of the pupil image cannot be obtained.
  • an unilluminated image is based on the position of the corneal reflection image detected in the differential bright pupil image and the differential dark pupil image. If the illuminating device is turned on, a position where a cornea reflection image is predicted to be generated is calculated using a constant velocity model, and the position between the bright pupil image and the dark pupil image and the unilluminated image is calculated based on the position. After the correction, the difference bright pupil image and the difference dark pupil image are acquired.
  • the constant velocity model cannot be applied, that is, when the head acceleration is large, such as when the head starts to move or just before the head stops moving, the position correction may not work well. is there.
  • the corneal reflection does not move only by the movement of the head but also moves by the rotation of the eyeball.
  • the constant velocity model is not established.
  • the time between these two sets may be long, and it may be difficult to predict the position in the pupil or corneal reflection image. As a result, the position of the pupil image may not be obtained accurately.
  • the present embodiment it is possible to correct the positional deviation due to the time difference in the acquisition timing between the bright pupil image and the dark pupil image and the background image.
  • the detection accuracy can be improved reliably.
  • the light emission amounts of the light sources 3A and 3B are controlled so that the detection unit 13 has the same illumination intensity between the bright pupil image and the dark pupil image, but this is disclosed in Japanese Patent Application Laid-Open No. 2016-093253.
  • the light emission amount of the light sources 3A and 3B may be controlled to be adjusted according to the brightness of the pupil.
  • the detection unit 13 may control the lighting period of each of the light sources 3A, 3B, and 3C as described below.
  • the detection unit 13 may control the lighting period as shown in FIG. FIG.
  • FIG. 7 is a timing chart showing another example of the lighting timing of each light source controlled by the lighting time setting unit 17 and the lighting control unit 12, in which (a) part is the imaging timing of the camera 2, and (b) part. Indicates the lighting timing of the light source 3A, (c) indicates the lighting timing of the light source 3B, and (d) indicates the lighting timing of the light source 3C.
  • the lighting intensity setting unit 17 uses the same illumination intensity so that the brightness of the face other than the pupil portion is equal between the bright pupil image and the dark pupil image by the lighting time setting unit 17 as in the control method shown in FIG. a lighting period T B and a lighting period T D is set to be.
  • the lighting period of the light source 3C is set so that T CR -T B.
  • the lighting time setting unit 17, the dark in a shooting period T 2 of the time pupil image acquisition, the lighting period of the light source 3C, the shooting period T from the illumination intensity of the light source 3C illumination intensity of the light source 3C is the shooting period T 3 2 is set to a value obtained by subtracting the illumination intensity of the light source 3B.
  • the lighting period of the light source 3C is set so that T CR -T D.
  • the lighting time setting unit 17, and the light emission amount of the light source 3A, 3C in the shooting period T 1, the light source 3B in shooting period T 2, the light emission amount of 3C, and the light emission amount of the light source 3C in shooting period T 3 Is adjusted so that the light emission amount of the light source 3C is uniform.
  • the brightness of the corneal reflection image in the bright pupil image, the background image, and the dark pupil image can be made equal.
  • the luminance of the corneal reflection image in the background image can be made sufficiently high to be detectable, and saturation of the luminance of the corneal reflection image in the bright pupil image and the dark pupil image can be prevented.
  • the configuration of the illumination device 3 shown in FIG. 2 is an example, and other various modifications are possible.
  • the light sources 3B and 3C may be arranged away from the opening 8. That is, in the configuration shown in FIG. 8, the light sources 3B and 3C form two concentric ring shapes, and the two link-shaped light sources 3B and 3C are symmetrical with respect to the opening 8 as a reference. Is arranged.
  • the distance from the center of the opening 8 of the light sources 3B and 3C is made larger than that of the light source 3A, and the distance from the center of the opening 8 of some light sources 3C is made smaller than that of the light source 3B.
  • the configuration shown in FIG. 9 is an example in which the light sources 3A, 3B, and 3C including the ultra-bright LED are arranged on a straight line.
  • the two light source groups of the light sources 3B and 3C arranged close to each other are arranged at positions that are symmetrical with respect to the opening 8 as a reference.
  • the distance from the center of the opening 8 of the light sources 3B and 3C is made larger than that of the light source 3A, and the distance from the center of the opening 8 of the light source 3C is made smaller than that of the light source 3B.
  • the positions of the light sources 3A and 3C that emit light when acquiring the bright pupil image are different from the positions of the light sources 3B and 3C that emit light when acquiring the dark pupil image, respectively.
  • the spectacle reflection image generated in the above image becomes difficult to cancel by subtraction or division. Therefore, it is desirable to keep the light source 3B as far from the opening 8 as possible.
  • the distance of the light source 3B be large in order to cause a difference in the brightness of the pupil image. Therefore, the effect of canceling the spectacle reflection image and the intensity of the pupil image are in a trade-off relationship.
  • the detection unit 13 may control the lighting period of the illumination device 3 as shown in FIG.
  • FIG. 10 is a timing chart showing another example of the lighting timing of each light source controlled by the lighting time setting unit 17 and the lighting control unit 12, wherein (a) part is the imaging timing of the camera 2, and (b) part. Indicates the lighting timing of the light source 3A, (c) indicates the lighting timing of the light source 3B, and (d) indicates the lighting timing of the light source 3C. That is, lighting time setting unit 17, the shooting period T 2 of the dark pupil image, it may be controlled so as to light for a moment at a much shorter period than the light source 3A lighting period T B.
  • a spectacle reflection image with a saturated luminance value can be generated at a position that matches the bright pupil image. By subtracting or dividing the bright pupil image and the dark pupil image, Reflection can be canceled out.
  • the illumination device 3 is shown in FIG. A configuration as shown in FIG.
  • the difference between the configuration example shown in FIG. 11 and the configuration example shown in FIG. 8 is that a light source 3D for spectacle reflection cancellation is added.
  • the light source 3D is disposed at a position where the distance from the center of the opening 8 is larger than the light source 3A and smaller than the light source 3C.
  • the difference between the configuration example shown in FIG. 12 and the configuration example shown in FIG. 9 is that a light source 3D for spectacle reflection cancellation is added.
  • the light source 3D is disposed at a position where the distance from the center of the opening 8 is larger than the light source 3A and smaller than the light source 3C.
  • the light source 3D similar to the lighting timing of the light source 3A shown in FIG. 10, the lighting time setting unit 17, so as to light for a moment at a much shorter period than the lighting period T B in the imaging period T 2 of the dark pupil image Be controlled.
  • the reason why the spectacle reflection is less likely to be a problem is that the light source 3A and the other light sources 3B and 3C are close to each other in terms of angle with respect to the opening 8. This is because the image is easily canceled.
  • the lighting device 3 shown in FIG. 8, FIG. 9, FIG. 11 or FIG. 12 should have its lateral width as small as possible when it is required to reduce the size of the device or to narrow the range of the light source. Is preferred.
  • the corneal reflection image produced by the light source that emits light when acquiring the dark pupil image or the background image is not two circular ones, but a horizontally long ellipse shape or a circular shape that overlaps horizontally.
  • Cheap These two circular images are synthesized images of the left and right light source groups. When the luminance of each image becomes unbalanced (for example, due to the effect of tears), the center of luminance of the entire image As a result, the accuracy of position detection of the cornea reflection image is affected.
  • the width of the illumination device 3 is set so that the corneal reflection image has an elliptical shape.
  • the corneal reflection detection unit 19 obtains the center of each corneal reflection image, These midpoints are calculated as the position of the corneal reflection image necessary for position correction.
  • the corneal reflection detection unit 19 performs the central corneal reflection image. Is calculated as the position of the corneal reflection image necessary for position correction.
  • the corneal reflection detection unit 19 calculates the position of the center of gravity of the corneal reflection image as the position of the corneal reflection image necessary for position correction. Also in this case, it is preferable that the lateral width of the illumination device 3 is set so that the size of the overlapped corneal reflection image becomes small. However, the detection accuracy of the position of the corneal reflection image does not have to be so high when the pupil image is small, and is sufficient to correct the position between the images, that is, the detection accuracy of the position where the pupil can be detected robustly. If there is enough.
  • FIG. 13 is a block diagram illustrating a functional configuration of the visual line detection device 201.
  • the line-of-sight detection device 201 includes two optical systems 202 a and 202 b each constituted by the light sources 3 ⁇ / b> A, 3 ⁇ / b> B, 3 ⁇ / b> C, and 3 ⁇ / b> D and the camera 2 including the imaging device 6, and the control device 4.
  • the light sources 3A, 3B, 3C, and 3D may be shared between the two optical systems 202a and 202b.
  • the detection unit 13 of the control device 4 further includes a visual line detection unit 20 as a functional element for detecting the visual line direction of the subject.
  • the image sensor driving unit 11 of the control device 4 controls the photographing timing of the image sensors 6 of the two optical systems 202a and 202b
  • the lighting control unit 12 is a light source 3A, 3B, 3C, of the two optical systems 202a and 202b. 3D lighting timing is controlled.
  • the two optical systems 202a and 202b can be controlled so that the light from the light source does not interfere with each other by shifting the photographing timing and lighting timing of each other. Further, in order to reduce the time difference of photographing by each camera 2 as much as possible, the photographing timings of the imaging elements 6 of the two optical systems 202a and 202b may be controlled to be synchronized.
  • the lighting periods of the light sources 3A, 3B, 3C, and 3D of the two optical systems 202a and 202b are set independently by the two optical systems 202a and 202b in the same manner as the setting procedure by the lighting time setting unit 17 described above.
  • the time difference between the eye image data obtained by the plurality of cameras 2 can be eliminated, and an error when obtaining the three-dimensional coordinates of the pupil by stereo matching from the plurality of pupil images is reduced.
  • This can reduce the error in line-of-sight detection. That is, if synchronization is established between the cameras 2 of each optical system, there is no difference in eye image data acquisition time between the two cameras, and errors due to the movement of the subject's head can be reduced.
  • the configuration of the illumination device 3 may be the configuration illustrated in FIG. 2 or may be a configuration in which some light sources are shared as illustrated in FIG.
  • the optical system 202a includes a light source 3A 1 that is a first light source, a light source 3B 1 that is a dark pupil light source, 3C 1 that is a corneal reflection generation light source, and a spectacle reflection canceling light source.
  • 3D 1 is provided around the opening 81 of the camera 2, as the optical system 202b, is the first light source 3A 2, the light source 3B 2, corneal reflection generating source is a dark pupil light source is a light source 3C 2, and 3D 2 is eyeglasses reflection cancellation light source, is provided on the periphery of the opening 8 camera 2.
  • the dark pupil light source 3B 12 and the light source 3C 12 which is a corneal reflection generating light source have two openings 8 1 and 8 2. It is provided in the middle.
  • the light source 3B 12 and the light source 3C 12 shared between the two optical systems 202a and 202b are controlled by the lighting time setting unit 17 so as to be turned on in synchronization with the photographing period of both cameras 2.
  • the light source 3B 12 and the light source 3C 12 need to be lit during the photographing period of the two cameras 2, they are arranged in a ring shape when the durability of the light emitting elements of the light source tends to be lowered.
  • the number of light emitting elements may be increased to about twice that of other light sources.
  • the configuration of the illuminating device 3 may be as shown in FIG. 15 in the case where the photographing timings of the cameras 2 of the two optical systems 202a and 202b are synchronized.
  • a dedicated light source optical system 202a a light source 3A 1 is a first light source and glasses reflection cancellation light source is provided around the opening 81 of the camera 2
  • a dedicated optical system 202b as the light source
  • the light source 3A 2 is a first light source and glasses reflection cancellation light source is provided around the opening 8 camera 2
  • a light source 3B 12 that is a light source (second light source) is provided between the two openings 8 1 and 8 2 .
  • FIG. 16 is a timing chart showing another example of the lighting timing of each light source controlled by the lighting time setting unit 17 and the lighting control unit 12, wherein (a) part is the imaging timing of the camera 2, and (b) part. Are the lighting timings of the light sources 3A 1 and 3A 2, and the part (c) shows the lighting timing of the light source 3B 12 .
  • the light sources 3A 1 , 3A 2 and the light source 3B 12 are turned on during the bright pupil image capturing period T 1 of each camera 2, and the light sources are emitted during the dark pupil image capturing period T 2 of each camera 2.
  • the light source 3A 1, 3A 2 is turned a very small amount of light emission. According to such control, the total light emission amount of the illumination device 3 is made the same when each image is acquired. At the same time, the luminance difference of the pupil portion between the bright pupil image and the dark pupil image appears due to the difference in position between the first light source and the second light source.
  • the illumination device 3 may have a configuration illustrated in FIG.
  • the optical system 202a as 202b shared light source, the dark pupil light source and the light source 3B 12 is a corneal reflection generating source, the two openings 81, 82 of the two outside It is divided and provided.
  • the difference between the distance from the respective openings 81 of the camera 2, 8 2 is increased between the first and second light sources, in particular, the opening from the opening 8 1 the distance to the light source 3B 12 disposed on the outside of part 82 is increased, the distance from the opening 8 2 to the light source 3B 12 arranged outside the opening 81 is increased.
  • the illuminating device 3 having the configuration shown in FIGS. 14, 15, and 17, since the bright pupil image, the dark pupil image, and the background image are illuminated with a symmetric light amount, the calculated corneal reflection image The position is stabilized.
  • the line-of-sight detection unit 20 of the control device 4 targets each of the position of the pupil image detected based on the pupil detection image and the difference dark pupil image and the difference bright pupil image for each of the two optical systems 202a and 202b.
  • the position of the detected cornea reflection image is acquired.
  • the line-of-sight detection unit 20 calculates the three-dimensional position of the subject's pupil using the positions of the pupil images obtained corresponding to the two optical systems 202a and 202b. Further, the line-of-sight detection unit 20 uses the calculated three-dimensional position of the pupil, the position of the pupil image obtained for each of the two optical systems 202a and 202b, and the position of the corneal reflection image, and the gaze direction and gaze point of the subject. Is detected.
  • the center of the pupil image on the pupil detection image coincides with the center of the pupil image at the time of acquiring the dark pupil image.
  • the line of sight is detected using the position of the pupil image and the position of the corneal reflection image detected on the dark pupil image.
  • the position of the differential dark pupil image is corrected with respect to the differential bright pupil image
  • the line of sight is detected using the position of the pupil image on the bright pupil image and the position of the corneal reflection image detected on the bright pupil image. Is done.
  • a method developed by the present inventors see International Publication No. 2012/020760
  • the line-of-sight detection device 201 in each of the two optical systems, it is possible to obtain a bright pupil image and a dark pupil image from which noise due to the influence of ambient light has been removed, and using these, the position of the pupil image Detection accuracy can be improved. Furthermore, by using the positions of the pupil images detected by the two optical systems, it is possible to improve the detection accuracy of the subject's gaze direction and gazing point.
  • the present invention is not limited to the embodiment described above.
  • the configuration of the above embodiment can be variously changed.
  • the detection unit 13 of the control device 4 detects the pupil position, corneal reflection position, or line-of-sight direction of the subject.
  • the posture may be detected. That is, the detection unit 13 calculates the center positions of the right and left nostrils of the subject on these images for three consecutive frames of the bright pupil image, the background image, and the dark pupil image.
  • the illumination device 3 since the illumination device 3 is turned on, it is possible to calculate the positions of the left and right pupil images based on the luminance by setting a window based on the position of the pupil image.
  • the detection unit 13 uses the position of the left and right pupil images on the respective frames and the center position of the left and right nostrils to obtain a normal vector indicating the face direction (face posture) of the subject person as a pinhole of the camera 2. It can be calculated using a model. As the calculation method of the center position of the nostril and the calculation method of the face direction, for example, the method described in International Publication No. 2010/010926 can be used. Further, the detection unit 13 may use the position correction method between the bright pupil image and the dark pupil image using the head posture disclosed in the publication. Since the position correction at the time of detection of the pupil image can be performed by the correction method, the window for detecting the corneal reflection image can be set to a smaller area. As a result, not only the accuracy of position detection of the corneal reflection image is improved, but also the image processing is speeded up.
  • the corneal reflection detection unit 19 of the detection unit 13 determines the position of the corneal reflection image on each eye image acquired by the cameras 2 1 and 2 2 corresponding to the two optical systems 202a and 202b.
  • a true corneal reflection image or a false corneal reflection image may be determined by the following processing.
  • the cameras 2 1 and 2 2 have been subjected to camera calibration in advance, and there are a plurality of images on each eye image of a frame synchronized (or close in time) by the two cameras 2 1 and 2 2 . It is assumed that the candidate position of the cornea reflection image is calculated.
  • 18 to 20 are conceptual diagrams for explaining the corneal reflection position authenticity determination processing by the corneal reflection detection unit 19.
  • the corneal reflection detection unit 19 is detected by one of the camera 2 1 and the candidate positions of the cornea reflection image detected on the obtained image of the eye by the eye image acquired by the other camera 2 2 2
  • a pair of candidate positions for the corneal reflection image is selected from the plurality of candidate positions for the corneal reflection image.
  • the corneal reflection detection unit 19, the one position of the pair of candidate positions specified camera 2 1 the criteria, and the other position of the pair of candidate positions specified camera 2 2 the criteria The distance between is calculated, and the authenticity of the pair of candidate positions is determined based on the distance.
  • the subject's cornea is a corneal sphere CB having a spherical surface
  • each of the cameras 2 1 and 2 2 has a lighting device at a position that can be regarded as coincident with the camera center (pinhole).
  • 3 1 and 3 2 are attached.
  • the center C of the corneal sphere CB exists in the direction of the corneal reflections R 1 and R 2 captured by the cameras 2 1 and 2 2 . Therefore, if the camera calibration is accurate, ideally, the straight lines L 1 and L 2 extending from the two cameras 2 1 and 2 2 toward the corneal reflections R 1 and R 2 are mutually at the center C of the corneal sphere CB. Has an intersection.
  • the corneal reflection detecting unit 19 for each of the selected candidate position in total per pair, the two straight lines L 1, whose distance between L 2 decreases linearly L 1, L 2 on the point (nearest point ) identify the G 1, G 2, and calculates the distance G 1 G 2 between recent their point G 1, G 2. Then, the corneal reflection detection unit 19 determines a pair of candidate positions where the distance G 1 G 2 is equal to or less than a preset threshold value as a pair of true corneal reflection image positions. Thereafter, the detection unit 13 performs processing for correcting the position between the eye images, detecting the pupil image, detecting the line of sight, and detecting the face posture, using a pair of true corneal reflection image positions.
  • the corneal reflection detection unit 19 uses the camera centers C 1 and C 2 of the two cameras 2 1 and 2 2 as a reference, as shown in FIG. between the camera center C 1 of a C 2 and the plane C 1 C 2 G 1 through the nearest point G 1, their camera center C 1, and C 2 and the plane C 1 C 2 G 2 through the nearest point G 2
  • a pair of true corneal reflection image positions may be determined by calculating an angle ⁇ formed by the above and determining whether the angle ⁇ is equal to or smaller than a preset threshold value.
  • the second light source includes a dark pupil light source and a corneal reflection generation light source that is closer to the dark pupil light source than the dark pupil light source.
  • the first light source and the corneal reflection generating light source are turned on.
  • the dark pupil light source and the corneal reflection generating light source are turned on.
  • the corneal reflection light source is turned on.
  • the light source for generation may be turned on.
  • the corneal reflection image can be stably generated in the bright pupil image, the dark pupil image, and the background image, and the position caused by the time difference in the acquisition timing between the bright pupil image and the dark pupil image and the background image The deviation can be corrected reliably.
  • the lighting control unit is configured so that the brightness of the corneal reflection image of the subject in the bright pupil image, the dark pupil image, and the background image is constant, in the first period, the second period, and the third period.
  • the light emission amount of the light source for generating corneal reflection may be adjusted. In this case, it is possible to correctly detect the cornea reflection image of the subject in the bright pupil image, the dark pupil image, and the background image even when the subject changes or the position of the subject changes, and as a result, the pupil The detection accuracy of the image position can be maintained.
  • the lighting control unit includes the light emission amounts of the first light source and the corneal reflection generation light source in the first period, the light emission amounts of the dark pupil light source and the corneal reflection generation light source in the second period, Adjusting the light emission amount of the light source for corneal reflection generation in the first period, the second period, and the third period so that the light emission amount of the light source for corneal reflection generation in the period is made uniform. Good.
  • the brightness of the corneal reflection image can be made equal among the bright pupil image, the dark pupil image, and the background image, and the position of the corneal reflection image in them can be detected correctly.
  • a bright pupil image and a dark pupil image from which noise due to the influence of ambient light is effectively removed can be obtained.
  • the lighting control unit may control the first light source to light up with a light emission amount smaller than that in the first period in the second period.
  • the spectacle reflection of the subject can be canceled by subtracting or dividing the difference bright pupil image and the difference dark pupil image.
  • the detection accuracy of the position of the pupil image can be further improved.
  • the apparatus further includes a spectacle reflection canceling light source that is closer to the opening center than the second light source and far from the first light source, and the lighting control unit includes the spectacle reflection canceling light source in the second period. It is good also as controlling to make it light with the light emission amount smaller than the light emission amount of the 1st light source.
  • the spectacle reflection of the subject can be canceled by subtracting or dividing the difference bright pupil image and the difference dark pupil image. As a result, the detection accuracy of the position of the pupil image can be further improved.
  • At least two or more optical systems including a camera, a first light source, and a second light source are provided, and the calculation unit targets pupil images obtained by each of the two or more optical systems, and pupil images And the position of the cornea reflection image may be calculated, and the line-of-sight direction of the subject may be calculated based on the position of the pupil image and the position of the cornea reflection image. Also in this case, the detection accuracy of the position of the pupil image by two or more optical systems can be improved, and as a result, the detection accuracy of the line-of-sight direction can also be improved.
  • the calculation unit includes a plurality of candidate positions of the corneal reflection image detected on the eye image acquired by one of the two or more optical systems, and the other optical system of the two or more optical systems.
  • the candidate position of the corneal reflection image is selected from the plurality of candidate positions of the corneal reflection image detected on the eye image acquired by the above, and the pair of the specified optical camera is used as a reference.
  • the authenticity of the pair of candidate positions may be determined based on the distance between one of the positions and the other position of the pair specified based on the camera of the other optical system. .
  • the calculation unit may further detect the position of the subject's nostril on the eye image and calculate the face posture of the subject based on the position of the pupil and the position of the nostril. In this way, the face posture of the subject can be calculated with high accuracy.
  • One aspect of the present invention is to use an eye image processing apparatus that detects a pupil from a human image, and can improve the detection accuracy of a pupil image in an image even when ambient light is present. It is.
  • SYMBOLS 1 Pupil detection apparatus, 2, 2 1 , 2 2 ... Camera, 3 ... Illumination device, 3A, 3A 1 , 3A 2 ... Light source (first light source), 3B, 3B 1 , 3B 12 , 3B 2 ... Light source (the second light source, the dark pupil light source), 3C, 3C 12 ... light source (second light source, the corneal reflection generator for the light source), 3D ... light source (light source for eyeglasses reflection cancellation), 4 ... control unit, 8,8 1 , 8 2 ... opening, 12 ... lighting control unit (lighting control part), 13 ... detection unit (calculation part), 17 ... lighting time setting part (lighting control part), 201 ... gaze detection device.
  • first light source first light source
  • Light source the second light source, the dark pupil light source
  • 3C, 3C 12 ... light source second light source, the corneal reflection generator for the light source

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Ophthalmology & Optometry (AREA)
  • Biomedical Technology (AREA)
  • Human Computer Interaction (AREA)
  • Medical Informatics (AREA)
  • Physics & Mathematics (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Eye Examination Apparatus (AREA)

Abstract

La présente invention concerne un dispositif de détection de pupille (1) muni : d'un appareil photographique (2) ; d'une source de lumière (3A) ; de sources de lumière (3B, 3C) à une distance plus grande d'un centre d'ouverture que la source de lumière (3A) ; d'une unité de commande d'éclairage (12) ; et d'une unité de détection (13) pour acquérir une image brillante de pupille acquise dans une période durant laquelle la source de lumière (3A) est éclairée, une image sombre de pupille acquise dans une période durant laquelle la source de lumière (3B) est éclairée, et une image de référence acquise dans une période durant laquelle la source de lumière (3C) est éclairée et calculer la position des images de pupille ; l'unité de détection (13) acquérant une image brillante de pupille de différence en calculant une image de différence qui utilise la position d'une image de réflexion cornéenne comme référence sur la base de l'image brillante de pupille et l'image de référence, et calculant une image de différence qui utilise la position de l'image de réflexion cornéenne comme référence sur la base de l'image sombre de pupille et de l'image de référence, puis calcule, après l'acquisition d'une image sombre de pupille de différence, la position de l'image de pupille en effectuant une soustraction ou une division sur l'image brillante de pupille de différence et l'image sombre de pupille de différence comme objets.
PCT/JP2018/008522 2017-03-08 2018-03-06 Dispositif de traitement d'image oculaire WO2018164104A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2019504593A JP6957048B2 (ja) 2017-03-08 2018-03-06 眼部画像処理装置

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2017044269 2017-03-08
JP2017-044269 2017-03-08

Publications (1)

Publication Number Publication Date
WO2018164104A1 true WO2018164104A1 (fr) 2018-09-13

Family

ID=63448380

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2018/008522 WO2018164104A1 (fr) 2017-03-08 2018-03-06 Dispositif de traitement d'image oculaire

Country Status (2)

Country Link
JP (1) JP6957048B2 (fr)
WO (1) WO2018164104A1 (fr)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112674713A (zh) * 2020-11-22 2021-04-20 泰州国安医疗用品有限公司 昏暗环境瞳孔辨识系统
WO2021087573A1 (fr) * 2019-11-07 2021-05-14 Seeing Machines Limited Suivi oculaire de pupille brillante à haute performance
EP3882686A1 (fr) * 2020-03-19 2021-09-22 Aptiv Technologies Limited Système optique et procédé de surveillance du regard d'un conducteur de véhicule
JP7431246B2 (ja) 2019-01-25 2024-02-14 マジック リープ， インコーポレイテッド 異なる露光時間を有する画像を使用した眼追跡

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008246004A (ja) * 2007-03-30 2008-10-16 National Univ Corp Shizuoka Univ 瞳孔検出方法
JP2016093253A (ja) * 2014-11-12 2016-05-26 国立大学法人静岡大学 瞳孔検出用光源装置の制御方法

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008246004A (ja) * 2007-03-30 2008-10-16 National Univ Corp Shizuoka Univ 瞳孔検出方法
JP2016093253A (ja) * 2014-11-12 2016-05-26 国立大学法人静岡大学 瞳孔検出用光源装置の制御方法

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7431246B2 (ja) 2019-01-25 2024-02-14 マジック リープ， インコーポレイテッド 異なる露光時間を有する画像を使用した眼追跡
WO2021087573A1 (fr) * 2019-11-07 2021-05-14 Seeing Machines Limited Suivi oculaire de pupille brillante à haute performance
EP3882686A1 (fr) * 2020-03-19 2021-09-22 Aptiv Technologies Limited Système optique et procédé de surveillance du regard d'un conducteur de véhicule
CN113496197A (zh) * 2020-03-19 2021-10-12 Aptiv技术有限公司 监测车辆驾驶员视线的光学系统和方法
US11436847B2 (en) 2020-03-19 2022-09-06 Aptiv Technologies Limited Optical system and method for monitoring a vehicle driver's eye gaze
CN113496197B (zh) * 2020-03-19 2024-04-12 Aptiv技术股份公司 监测车辆驾驶员视线的光学系统和方法
CN112674713A (zh) * 2020-11-22 2021-04-20 泰州国安医疗用品有限公司 昏暗环境瞳孔辨识系统
CN112674713B (zh) * 2020-11-22 2021-10-08 青岛大学附属医院 昏暗环境瞳孔辨识系统

Also Published As

Publication number Publication date
JP6957048B2 (ja) 2021-11-02
JPWO2018164104A1 (ja) 2020-01-23

Similar Documents

Publication Publication Date Title
JP6586523B2 (ja) 構造化光を用いた視線追跡
US10437327B2 (en) Eye tracking device and method for operating an eye tracking device
US10481403B2 (en) Contact lens with retinal camera
JP6159263B2 (ja) 照射特性を調整可能にして少なくとも１つの眼における少なくとも１つのパラメータを捕捉する光学的測定装置及び方法
US7819525B2 (en) Automatic direct gaze detection based on pupil symmetry
US20180160079A1 (en) Pupil detection device
US10896324B2 (en) Line-of-sight detection device and method for detecting line of sight
WO2018164104A1 (fr) Dispositif de traitement d'image oculaire
CN106175801B (zh) 眨眼测量方法、眨眼测量装置及眨眼测量程序
EP3075304B1 (fr) Dispositif d'assistance de détection de la ligne de visée et procédé d'assistance de détection de la ligne de visée
JP6201956B2 (ja) 視線検出装置および視線検出方法
JP6638354B2 (ja) 視線検出装置及び視線検出方法
CN109964230B (zh) 用于眼睛度量采集的方法和设备
US11675429B2 (en) Calibration, customization, and improved user experience for bionic lenses
JP2018099174A (ja) 瞳孔検出装置及び瞳孔検出方法
JP6900994B2 (ja) 視線検出装置及び視線検出方法
JP6555707B2 (ja) 瞳孔検出装置、瞳孔検出方法及び瞳孔検出プログラム
US10748301B2 (en) Corneal reflex position detection device, eye-gaze tracking device, and corneal reflex position detection method
US10123695B2 (en) Diagnosis supporting device and method of supporting diagnosis
JP2016028669A (ja) 瞳孔検出装置、および瞳孔検出方法
JP7228885B2 (ja) 瞳孔検出装置
JP6687195B2 (ja) 眼部画像処理装置
WO2021251146A1 (fr) Dispositif de détection de pupille
JP2017162233A (ja) 視線検出装置および視線検出方法
JP2016049258A (ja) 照明撮像装置及びそれを備えた視線検出装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18764544

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2019504593

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18764544

Country of ref document: EP

Kind code of ref document: A1