WO2022085276A1 - 情報処理システム、目状態測定システム、情報処理方法および非一時的なコンピュータ可読媒体 - Google Patents

情報処理システム、目状態測定システム、情報処理方法および非一時的なコンピュータ可読媒体 Download PDF

Info

Publication number
WO2022085276A1
WO2022085276A1 PCT/JP2021/029657 JP2021029657W WO2022085276A1 WO 2022085276 A1 WO2022085276 A1 WO 2022085276A1 JP 2021029657 W JP2021029657 W JP 2021029657W WO 2022085276 A1 WO2022085276 A1 WO 2022085276A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
subject
eye
information processing
angle
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2021/029657
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
尚司 谷内田
満晶 井上
智大 末石
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NEC Corp
University of Tokyo NUC
Original Assignee
NEC Corp
University of Tokyo NUC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by NEC Corp, University of Tokyo NUC filed Critical NEC Corp
Priority to JP2022556425A priority Critical patent/JP7640946B2/ja
Priority to US18/032,686 priority patent/US12243234B2/en
Priority to EP21882417.5A priority patent/EP4233690A4/en
Publication of WO2022085276A1 publication Critical patent/WO2022085276A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G06T7/0014—Biomedical image inspection using an image reference approach
    • G06T7/0016—Biomedical image inspection using an image reference approach involving temporal comparison
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A61B5/163—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state by tracking eye movement, gaze, or pupil change
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
    • A61B3/11—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for measuring interpupillary distance or diameter of pupils
    • A61B3/112—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for measuring interpupillary distance or diameter of pupils for measuring diameter of pupils
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
    • A61B3/113—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for determining or recording eye movement
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
    • A61B5/0077—Devices for viewing the surface of the body, e.g. camera, magnifying lens
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/66—Analysis of geometric attributes of image moments or centre of gravity
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/695—Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/90—Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Measuring devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor or mobility of a limb
    • A61B5/1103—Detecting muscular movement of the eye, e.g. eyelid movement
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30041—Eye; Retina; Ophthalmic

Definitions

  • the present disclosure relates to an information processing system, an eye condition measuring system, an information processing method and a non-temporary computer-readable medium, particularly an information processing system for evaluating a change in the eye condition of a subject, an eye condition measuring system, an information processing method and a non-transitory.
  • an information processing system for evaluating a change in the eye condition of a subject
  • an eye condition measuring system for evaluating a change in the eye condition of a subject
  • an eye condition measuring system an information processing method and a non-transitory.
  • temporary computer readable media Regarding temporary computer readable media.
  • Patent Document 1 discloses a method of calculating pupil motion from a change in pupil position detected based on an image of a driver's face.
  • the pupil position cannot be detected when the head moves and the eyeball is out of the field of view of the camera. Therefore, the subject needs to fix his head so that the eyeball does not come out of the field of view of the camera.
  • Patent Document 2 a method of detecting the amount of movement of the eyeball based on the difference between the reference position indicating the position of the blood vessel image in the white eye region of the reference image and the position of the blood vessel image in the white eye region of the detection image is used. It has been disclosed. In this method, since the eyeball image of the detection image is returned to the position of the reference image by the Affine transformation from the three points including the positions of the inner and outer corners of the eye, it is possible to prevent image blurring. However, even in this method, the subject is required to fix the head to some extent so that the eyeball does not come out of the field of view of the camera.
  • Patent Document 3 discloses an information processing system that captures a target area designated by an image captured by a wide-angle first camera with a narrow-angle second camera.
  • the above-mentioned Patent Document 3 does not disclose that the second camera only captures a human face or the like and captures an eye region which is a minute region. Further, the above-mentioned Patent Document 3 does not disclose a technique for measuring and evaluating a minute change in the state of the eye.
  • an object of the present disclosure is an information processing system, an eye condition measuring system, an information processing method, and a non-temporary method for appropriately evaluating a change in the eye condition of a subject in a relaxed state without fixing the head.
  • the purpose is to provide a computer-readable medium.
  • the information processing system includes a first acquisition means, a second acquisition means, a movement control means, and a state evaluation means.
  • the first acquisition means acquires image data related to the first image obtained by capturing the head of the subject at the first angle of view from the first image pickup means.
  • the second acquisition means acquires image data related to a second image obtained by capturing the subject's eye region with a second angle of view narrower than the first angle of view from the second image pickup means.
  • the movement control means moves the visual field range of the second image pickup means based on the position information of the head of the subject acquired based on the first image.
  • the state evaluation means evaluates the change in the eye state of the subject based on the time-series data of the second image.
  • the eye condition measuring system includes a first imaging means, a second imaging means, and an information processing device.
  • the first imaging means captures the head of the subject at the first angle of view.
  • the second imaging means captures the eye region of the subject with a second angle of view narrower than the first angle of view.
  • the information processing apparatus includes a first acquisition means, a second acquisition means, a movement control means, and a state evaluation means.
  • the first acquisition means acquires image data related to the first image obtained by capturing the head of the subject at the first angle of view from the first image pickup means.
  • the second acquisition means acquires image data related to a second image obtained by capturing the subject's eye region with a second angle of view narrower than the first angle of view from the second image pickup means.
  • the movement control means moves the visual field range of the second image pickup means based on the position information of the head of the subject acquired based on the first image.
  • the state evaluation means evaluates the change in the eye state of the subject based on the time-series data of the second image.
  • image data related to the first image obtained by capturing the head of the subject at the first angle of view is acquired from the first imaging means, and the subject's head is captured from the second imaging means.
  • Image data related to the second image obtained by capturing the eye region with a second angle of view narrower than the first angle of view is acquired, and based on the position information of the head of the subject acquired based on the first image. Then, the viewing range of the second image pickup means is moved, and the change in the state of the eyes of the subject is evaluated based on the time-series data of the second image.
  • the non-temporary computer-readable medium stores a program for causing a computer to execute a first acquisition process, a second acquisition process, a movement control process, and a state evaluation process.
  • the first acquisition process is a process of acquiring image data related to a first image obtained by capturing the head of a subject at a first angle of view from a first image pickup means.
  • the second acquisition process is a process of acquiring image data related to a second image obtained by capturing an eye region of the subject with a second angle of view narrower than the first angle of view from the second image pickup means.
  • the movement control process is a process of moving the visual field range of the second imaging means based on the position information of the head of the subject acquired based on the first image.
  • the state evaluation process is a process for evaluating a change in the eye state of the subject based on the time-series data of the second image.
  • an information processing system an eye condition measurement system, an information processing method, and a non-temporary computer-readable medium that suitably evaluates a subject's eye condition change in a relaxed state without fixing the head.
  • FIG. 3 is a system configuration diagram of an eye condition measurement system to which the information processing system according to the second embodiment can be applied. It is a block diagram which shows the functional structure of the information processing apparatus which concerns on Embodiment 2. It is a figure for demonstrating the movement control processing by the movement control unit which concerns on Embodiment 2. FIG. It is a figure for demonstrating the focal position control process by the movement control part which concerns on Embodiment 2. FIG. It is a figure for demonstrating the state evaluation process by the state evaluation part which concerns on Embodiment 2. FIG.
  • FIG. 2 It is a flowchart which shows the information processing procedure of the information processing apparatus which concerns on Embodiment 2. It is a system block diagram of the eye condition measurement system which concerns on Embodiment 3. FIG. It is a system block diagram of the eye condition measurement system which concerns on Embodiment 4. FIG. It is a block diagram which shows the functional structure of the information processing apparatus which concerns on Embodiment 4.
  • FIG. 1 is a block diagram showing a functional configuration of the information processing system 10 according to the first embodiment.
  • the information processing system 10 is a computer device that evaluates a change in the state of the eyes of a subject.
  • the information processing system 10 includes a first acquisition unit 11, a second acquisition unit 12, a movement control unit 13, and a state evaluation unit 15.
  • the first acquisition unit 11 is also called the first acquisition means.
  • the first acquisition unit 11 is communicably connected to the first image pickup unit 30, and image data (first image data) related to the first image obtained by capturing the head of the subject at the first angle of view from the first image pickup unit 30.
  • the first imaging unit 30 is a camera that images the head of the subject at the first angle of view.
  • the first image pickup unit 30 is also referred to as a first image pickup means.
  • the second acquisition unit 12 is also called the second acquisition means.
  • the second acquisition unit 12 is communicably connected to the second image pickup unit 40, and image data (second image data) related to the second image obtained by capturing the eye region of the subject at the second angle of view from the second image pickup unit 40.
  • image data second image data
  • the second imaging unit 40 is a camera that captures the eye region of the subject at the second angle of view.
  • the second image pickup unit 40 is also referred to as a second image pickup means.
  • the second angle of view is narrower than the first angle of view.
  • the eye region may be an eyeball or a peripheral region including the eyeball.
  • the movement control unit 13 is also called a movement control means.
  • the movement control unit 13 moves the visual field range of the second imaging unit 40 based on the position information of the head of the subject acquired based on the first image.
  • the field of view of the second image pickup unit 40 is a range in which the second image pickup unit 40 takes an image, and is also called a capture volume.
  • the field of view is defined based on the angle of view and the optical axis of the camera, and the larger the angle of view, the wider.
  • the state evaluation unit 15 is also called a state evaluation means.
  • the state evaluation unit 15 evaluates the change in the eye state of the subject based on the time-series data of the second image.
  • FIG. 2 is a block diagram showing a hardware configuration of the information processing system 10 according to the first embodiment.
  • the information processing system 10 has a processor 100, a ROM 101 (Read Only Memory), a RAM 102 (Random Access Memory), and an interface unit 103 (IF; Interface) as a main hardware configuration.
  • the processor 100, ROM 101, RAM 102, and the interface unit 103 are connected to each other via a data bus or the like.
  • the processor 100 has a function as an arithmetic unit that performs control processing, arithmetic processing, and the like.
  • the processor 100 may be a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (field-programmable gate array), a DSP (digital signal processor), an ASIC (application specific integrated circuit), or a combination thereof.
  • the ROM 101 has a function for storing a control program, an arithmetic program, and the like executed by the processor 100.
  • the RAM 102 has a function for temporarily storing processing data and the like.
  • the interface unit 103 inputs / outputs signals to and from the outside via wired or wireless. Further, the interface unit 103 accepts the operation of inputting data by the user and displays the information to the user. For example, the interface unit 103 communicates with the first image pickup unit 30 and the second image pickup unit 40.
  • the eye region is moved by moving the visual field range of the narrow-angle camera based on the position information of the head of the subject acquired based on the wide-angle first image. Is magnified and imaged to evaluate changes in eye condition. Therefore, even if the subject's head is not fixed, it is possible to prevent the eye region from being out of the field of view of the narrow-angle camera. Thereby, the information processing system 10 can suitably evaluate the change in the state of the eyes of the subject in a relaxed state without fixing the head.
  • FIG. 3 is a system configuration diagram of an eye condition measuring system 1a to which the information processing system according to the second embodiment (hereinafter referred to as an information processing apparatus) can be applied.
  • the left-right direction of the subject P is the X-axis direction
  • the front-back direction of the subject P is the Y-axis direction
  • the vertical direction of the subject P is the Z-axis direction.
  • the eye condition measurement system 1a is a computer system that measures and evaluates changes in the eye condition of the subject.
  • the eye condition measuring system 1a measures and evaluates the vibration state of the pupil derived from the saccade phenomenon.
  • the eye condition measuring system 1a includes an information processing device 10a, a first imaging unit 30a, a second imaging unit 40, a movable mirror 50, a driving unit 51, and a light source 61.
  • the distance between the first imaging unit 30a and the subject P in the Y-axis direction is D.
  • D is, for example, 2 ⁇ 0.5 [m].
  • the information processing device 10a corresponds to the information processing system 10 of FIG.
  • the information processing device 10a is connected to the first image pickup unit 30a, the second image pickup unit 40, and the drive unit 51.
  • the information processing apparatus 10a receives the first image data from the first image pickup unit 30a, and receives the control signal necessary for the eye region of the subject P to fall within the visual field range of the second image pickup unit 40, and the subject P. It generates a control signal necessary for focusing the second image pickup unit 40 in the eye region. Then, the information processing apparatus 10a transmits the control signal in the visual field range in the X-axis and Z-axis directions to the drive unit 51, and the control signal of the focal position to the second image pickup unit 40.
  • the first image pickup unit 30a is a camera having the same function as the first image pickup unit 30 in FIG.
  • the first imaging unit 30a has a wide-angle camera 31.
  • the wide-angle camera 31 captures at least the face of the subject P at the first angle of view and generates the first image data.
  • the focal length of the wide-angle camera 31 is preset so that the face of the subject P, which is separated by D [m], can be imaged.
  • the focal length of the wide-angle camera 31 may be less than 200 [mm], preferably less than 70 [mm], and is 28 [mm] in the second embodiment.
  • the focal length of the wide-angle camera 31 may be 12 [mm].
  • the frame rate of the wide-angle camera 31 is preset in order to be able to track the shaking of the eye region due to the shaking of the head of the subject P and to suppress an excessive increase in the amount of data.
  • the frame rate of the wide-angle camera 31 may be 120 [fps] or more and 1200 [fps] or less, preferably 240 [fps] or more and 1000 [fps] or less, and more preferably 480 [fps] or more and 1000. It may be [fps] or less, and is 500 [fps] in the second embodiment.
  • the wide-angle camera 31 may capture a subject at a frame rate equal to or higher than the frame rate of the second imaging unit 40 described later. This makes it easy to suitably control the visual field range of the second imaging unit 40 according to the shaking of the eye region.
  • the wide-angle camera 31 transmits the first image data to the information processing apparatus 10a in response to the generation of the first image data.
  • the second image pickup unit 40 is a camera that captures the eye area of the subject with a second angle of view narrower than the first angle of view and generates second image data.
  • the second imaging unit 40 may be a near-infrared camera in order to facilitate pupil detection, which will be described later.
  • the detection wavelength of the light receiving element of the second imaging unit 40 is referred to as a pupil detection wavelength
  • the pupil detection wavelength is, for example, a wavelength of 940 [nm].
  • the second imaging unit 40 has a telephoto lens and a liquid lens.
  • the liquid lens is a lens that determines a focal position (focus point), and may be controlled based on a control signal of the focal position by the information processing apparatus 10a.
  • the focal length of the second imaging unit 40 is set so that the eye region of the subject P, which is separated by D [m] from the movable mirror 50, can be imaged, and is longer than the focal length of the wide-angle camera 31.
  • the focal length of the second imaging unit 40 may be 100 [mm] or more, preferably 150 [mm] or more, and is 200 ⁇ 50 [mm] in the second embodiment.
  • the second imaging unit 40 controls the focal position based on the distance information in order to focus on the eye region.
  • the frame rate of the second imaging unit 40 is set in advance in order to observe a change in the state of the eyes, a saccade phenomenon in this example, and to suppress an excessive increase in the amount of data.
  • the frame rate of the second imaging unit 40 may be 120 [fps] or more and 1000 [fps] or less, preferably 240 [fps] or more and 1000 [fps] or less, and more preferably 500 [fps] or more and 1000 [fps]. ] Or less, which is 500 [fps] in the second embodiment.
  • the second image pickup unit 40 transmits the second image data to the information processing apparatus 10a in response to the generation of the second image data.
  • the movable mirror 50 is a pair of mirrors that move the optical axis of the second imaging unit 40.
  • the movable mirror 50 has a first movable mirror 50-1 and a second movable mirror 50-2.
  • first movable mirror 50-1 and the second movable mirror 50-2 are simply referred to as a movable mirror 50.
  • Each movable mirror 50 is fixedly connected to the drive unit 51 via a support portion (not shown) so as to have a predetermined tilt angle, and the tilt angle can be changed by rotating the support portion. It is composed.
  • the support portion of the connection destination is configured to rotate around the Z axis
  • the support portion of the connection destination rotates around the X axis. It is configured as follows.
  • the movable mirror 50 can move the optical axis of the second image pickup unit 40 in the X-axis direction and the Z-axis direction, that is, the visual field range of the second image pickup unit 40 can be moved in the X-axis direction and the Z-axis direction.
  • the movable mirror 50 is a galvano mirror having a relatively small mass and high responsiveness.
  • the eye condition measurement system 1a facilitates imaging of the eye region by linking the visual field range of the second imaging unit 40 with the minute and high-speed movement of the head of the subject P.
  • the drive unit 51 is also called a drive means.
  • the drive unit 51 is a drive motor that rotates each movable mirror 50 via a support unit.
  • the drive unit 51 has a first drive unit 51-1 corresponding to the first movable mirror 50-1, and a second drive unit 51-2 corresponding to the second movable mirror 50-2.
  • the first drive unit 51-1 and the second drive unit 51-2 may also be simply referred to as a drive unit 51.
  • the drive unit 51 rotates the movable mirror 50 according to the control signal of the visual field range in the X-axis and Z-axis directions of the information processing apparatus 10a.
  • the light source 61 is a light source that illuminates the face of the subject P.
  • the light source 61 is a light source having a wavelength region corresponding to the wavelength for detecting the pupil, and is a near-infrared light source of 940 [nm] in the second embodiment.
  • the second imaging unit 40 receives the light of the path of the eye region of the subject P ⁇ the first movable mirror 50-1 ⁇ the second movable mirror 50-2 ⁇ the second imaging unit 40, so that the subject P is exposed to the light. Image the eye area.
  • FIG. 4 is a block diagram showing a functional configuration of the information processing apparatus 10a according to the second embodiment.
  • the information processing device 10a of FIG. 4 has an element position estimation unit 14 and an output unit 16 in addition to the components of the information processing device 10 of FIG.
  • the first acquisition unit 11 is connected to the first image pickup unit 30a (wide-angle camera 31), and receives and acquires the first image data from the first image pickup unit 30a.
  • the first acquisition unit 11 may acquire the first image data at a frame rate equal to or higher than the frame rate at which the second acquisition unit 12 acquires the second image data.
  • the first acquisition unit 11 supplies the acquired first image data to the movement control unit 13.
  • the second acquisition unit 12 is connected to the second image pickup unit 40, and receives and acquires the second image data from the second image pickup unit 40.
  • the second acquisition unit 12 supplies the acquired second image data to the element position estimation unit 14. Further, the second acquisition unit 12 may supply the acquired second image data to the movement control unit 13.
  • the movement control unit 13 generates information on changes in the position of the subject P's face in the X-axis and Z-axis directions based on the time-series data of the first image.
  • the movement control unit 13 calculates the rotation amount of each of the first movable mirror 50-1 and the second movable mirror 50-2 based on the change information.
  • the movement control unit 13 may use the second image data in addition to the change information in calculating the rotation amount.
  • the movement control unit 13 generates a control signal of the visual field range related to the X-axis and the Z-axis direction based on each rotation amount, and sends the control signal of the visual field range related to the Z-axis direction to the first drive unit 51-1.
  • a control signal in the visual field range in the axial direction is transmitted to the second drive unit 51-2. Further, the movement control unit 13 performs focus position control processing based on the time-series data of the first image and the second image, generates information on the change in the face position of the subject P in the Y-axis direction, and controls the focus position. To generate. Then, the movement control unit 13 controls the liquid lens by transmitting the control signal to the second image pickup unit 40. In this way, the second imaging unit 40 moves the visual field range and the focal position (focus point) based on the change information of the position of the face.
  • the element position estimation unit 14 is also called an element position estimation means.
  • the element position estimation unit 14 estimates the position of the eye element in the second image.
  • the position of the eye element estimated by the element position estimation unit 14 includes the position of the center of gravity of the pupil of the subject P and the positions of the outer and inner corners of the eye.
  • the positions of the outer and inner corners of the eyes may be either one, or may be the positions of any other fixed points instead.
  • the element position estimation unit 14 supplies the position information of the eye element to the state evaluation unit 15.
  • the state evaluation unit 15 generates difference information on the positions of the eye elements in the second image, and evaluates the vibration state of the pupil of the subject P based on the difference information.
  • the vibration state of the pupil may be at least one of the vibration amount, the vibration direction, the vibration frequency and the vibration duration of the pupil.
  • the state evaluation unit 15 supplies information regarding the vibration state of the pupil to the output unit 16.
  • the output unit 16 is also called an output means.
  • the output unit 16 outputs information regarding the vibration state of the pupil as an evaluation result.
  • the output unit 16 may have a display unit (not shown) for displaying the evaluation result. Further, the output unit 16 may have a transmission unit (not shown) for transmitting the evaluation result to an external device (not shown).
  • FIG. 5 is a diagram for explaining the movement control process by the movement control unit 13 according to the second embodiment.
  • the second image IMG_b is shown in this figure.
  • the eye region including the inner corner Ib and the outer corner of the eye of the subject P is imaged.
  • the movement control unit 13 calculates the position coordinates (referred to as eye position coordinates) of the eye area in the first image.
  • the eye position coordinates are the position coordinates of the center of gravity of the eye region when the eye region in the first image is approximated to a substantially elliptical region, but the coordinates are not limited to this and are not limited to this. It may be the position coordinate range of the eye area.
  • the movement control unit 13 virtually projects the eye area in the first image onto the second image IMG_b, and the image area (projection area) in the second image corresponding to the eye position coordinates in the first image. ) Calculate the position coordinates of the center of gravity VC of VA.
  • the movement control unit 13 determines the center of gravity VC of the projection area VA based on the difference information of the eye position coordinates of the first image between the previous shooting timing and the current shooting timing and the second image at the previous shooting timing.
  • the position coordinates of may be calculated.
  • the difference information of the eye position coordinates is an example of the above-mentioned change information of the face position.
  • the movement control unit 13 calculates the amount of rotation of the first movable mirror 50-1 and the second movable mirror 50-2 so that the center of gravity VC of the projection region VA is arranged at the center C of the second image.
  • the movement control unit 13 can move the visual field range of the second image pickup unit 40 in the X-axis and Z-axis directions so that the eye region A is always arranged in the center of the second image IMG_b.
  • the projection region VA and the eye region A are substantially elliptical regions symmetric with respect to the major axis and the minor axis, and the outer corners of the eyes and the inner corners of the eyes located at both ends of each of the projection region VA and the eye region A.
  • the midpoint between Ib and Ib coincides with its center of gravity.
  • the number of pixels in the width direction of the eye region included in the second image IMG_b is set to be within a predetermined range with respect to the number of pixels in the width direction of the second image IMG_b.
  • the number of pixels in the width direction of the second image IMG_b is xb
  • the number of pixels in the height direction is zb.
  • xb ⁇ zb 640 ⁇ 480.
  • the number of pixels from the inner corner of the eye Ib to the outer corner of the eye Ob of the subject P imaged in the second image IMG_b is x1. That is, x1 / xb is maintained within a predetermined range.
  • x1 / xb may be 0.5 or more and less than 1, preferably 0.8 or more and less than 1.
  • the information processing apparatus 10a can accurately detect the pupil from the second image IMG_b.
  • the movement control unit 13 moves and controls the first and second movable mirrors 50-1 and 50-2 so that the eye region A is arranged in the center of the second image IMG_b, it is in the Y-axis direction.
  • the amount of rotation varies depending on the distance D between the camera and the subject P. Therefore, the movement control unit 13 needs to adjust the focus of the second image IMG_b.
  • FIG. 6 is a diagram for explaining the focus position control process by the movement control unit 13 according to the second embodiment.
  • This figure shows a schematic view of the upper surface of the subject P, the first image pickup unit 30a and the second image pickup unit 40 on the upper side, and the second image of the first image IMG_a and the second image pickup unit 40 of the first image pickup unit 30a on the lower side. Shows IMG_b.
  • the movement control unit 13 calculates the position coordinates (xai, sai) of the inner corner Ia of the subject P in the IMG_a in the first image. Then, the movement control unit 13 acquires the second image IMG_b by controlling the rotation of the movable mirror 50. The angle of the movable mirror 50 at this time is expressed as G (xg, zg). The movement control unit 13 calculates the position coordinates (xbi, zbi) of the inner corner Ib of the second image IMG_b.
  • the movement control unit 13 has the position coordinates (xai, sai) of the inner corner Ia of the IMG_a in the first image, the position coordinates (xbi, zbi) of the inner corner Ib of the second image IMG_b, and the angle G (xg, zg) of the movable mirror 50. Is used to calculate the distance D by the stereo imaging method. Then, the movement control unit 13 determines the focal position of the second imaging unit 40 based on the distance D, and generates control information for moving the focus to the focal position.
  • the movement control unit 13 treats the first image pickup unit 30a and the second image pickup unit 40 like a stereo camera, and the focus of the second image pickup unit 40 having a narrow depth of field is focused at the eye position of the subject P. Adjust to do.
  • the movement control unit 13 uses the position coordinates of the inner corners of the eyes Ia and Ib, but instead of these, the position coordinates of the outer corners of the eyes Oa and Ob may be used.
  • FIG. 7 is a diagram for explaining the state evaluation process by the state evaluation unit 15 according to the second embodiment.
  • the second image IMG_b is also shown in this figure.
  • the state evaluation unit 15 calculates the relative position of the center of gravity G of the pupil, which is the moving point, from the reference point for the second image IMG_b at each imaging timing.
  • the reference point is preferably a fixed point whose position does not change substantially depending on the degree of eyelid opening or the movement of the line of sight.
  • the state evaluation unit 15 can use the position of the corneal reflex image as a reference point, but in the second embodiment, a point based on at least one position of the outer corner of the eye Ob and the inner corner of the eye Ib is used. That is, the state evaluation unit 15 calculates the relative position based on the position information of at least one of the outer corner of the eye Ob and the inner corner of the eye Ib and the position information of the center of gravity G of the pupil.
  • the state evaluation unit 15 calculates the relative position between the straight line L connecting the position of the outer corner of the eye Ob and the position of the inner corner of the eye Ib and the center of gravity G of the pupil.
  • the state evaluation unit 15 sets the relative position of the center of gravity G of the pupil as the distance in the X-axis and Z-axis directions from the midpoint of the outer corner of the eye Ob and the inner corner of the eye Ib (which may coincide with the center C of the second image IMG_b) to the center of gravity G. Calculate ⁇ x and ⁇ z.
  • the state evaluation unit 15 may calculate a value obtained by normalizing the distances ⁇ x and ⁇ z in the X - axis and Z-axis directions at the distance x1 as the relative position of the center of gravity G of the pupil in the second image IMG_b.
  • the state evaluation unit 15 may use only one of the positions of the outer corner of the eye and the inner corner of the eye Ib as a reference point. Then, the state evaluation unit 15 evaluates the vibration state of the pupil based on the difference information between the adjacent imaging timings of the relative position of the center of gravity G of the pupil.
  • the state evaluation unit 15 uses at least one position of the outer corner of the eye and the inner corner of the eye Ib as a reference point. Therefore, the eye condition measurement system 1a does not need to be provided with an infrared light source for forming a corneal reflex image as compared with the case of using a corneal reflex image, and the system configuration is simplified. Further, the state evaluation unit 15 can perform high-speed processing and miniaturize the apparatus because the calculation for the relative position calculation is simple and the calculation amount is reduced. Therefore, although the information processing device 10a has been described as an independent computer device in the second embodiment, the information processing device 10a can be mounted in the first image pickup unit 30a or in the second image pickup unit 40.
  • FIG. 8 is a flowchart showing an information processing procedure of the information processing apparatus 10a according to the second embodiment.
  • the movement control unit 13 detects the face of the subject P from the first image (step S11). For example, the movement control unit 13 may detect the face of the subject P by using a trained convolutional neural network (CNN) that inputs the first image. Then, the movement control unit 13 may generate a normalized image obtained by cutting out an image region corresponding to the face of the subject P.
  • CNN convolutional neural network
  • the movement control unit 13 calculates the eye position coordinates in the first image (step S12).
  • the movement control unit 13 may detect the eye region of the subject P from the generated normalized image by template matching, and calculate the position coordinates of the center of gravity of the eye region in the first image as the eye position coordinates.
  • the amount of rotation of the movable mirror 50 may be calculated based on the second image.
  • the movement control unit 13 generates a control signal in the visual field range in the X-axis and Z-axis directions based on the calculated rotation amount.
  • the movement control unit 13 calculates the movement amount of the focal position of the second imaging unit 40 by the method shown in FIG. 6, and generates a control signal of the focal position based on the movement amount.
  • the movement control unit 13 controls the movement of the visual field range and the focal position of the second imaging unit 40 (step S15). Specifically, the movement control unit 13 transmits a control signal in the visual field range in the X-axis and Z-axis directions to the drive unit 51. Further, the movement control unit 13 transmits a control signal of the focal position to the liquid lens.
  • step S16 the movement control unit 13 determines whether or not to end a series of shooting. If the movement control unit 13 does not end (No in step S16), the process returns to step S10, and if it ends (Yes in step S16), the process proceeds to step S17.
  • step S17 the second acquisition unit 12 acquires the second image data from the second image pickup unit 40.
  • the information processing apparatus 10a repeats the processes of steps S18 to 19 for the number of frames of the acquired second image.
  • the element position estimation unit 14 detects the eye region from the second image by, for example, template matching, and estimates the position coordinates of the eye elements (outer corners of the eyes, inner corners of the eyes, and the center of gravity of the pupil).
  • the element position estimation unit 14 estimates the position coordinates of the center of gravity of the pupil by detecting the image area of the pupil by, for example, binarization, edge detection, and Hough transform, and calculating the position coordinates of the center of gravity of the image area. good.
  • step S19 the state evaluation unit 15 evaluates the vibration state by calculating the relative position of the center of gravity of the pupil by the method shown in FIG. 7 and generating the difference information of the relative position.
  • step S20 the output unit 16 outputs information on the evaluated vibration state.
  • steps S17 to 19 may be executed in parallel with the processes of steps S10 to 16. Further, the specific processing of each step is not limited to the above description.
  • the information processing apparatus 10a of the second embodiment can exert the same effect as that of the first embodiment.
  • it is necessary to strictly suppress the vibration of the head so it is necessary to firmly fix the head. Therefore, since it is difficult for the subject to fix the head immediately after the head surgery due to a brain disease, it was not possible to inspect the eye condition such as the saccade phenomenon.
  • the evaluation value evaluated by the information processing system 10a can be used as a judgment material for the transition stage of rehabilitation of a brain disease, and the medical staff can reduce the burden on the subject who is a patient and recover the medical condition.
  • the degree of can be easily diagnosed.
  • the company can quantitatively measure the effect of the advertisement.
  • the third embodiment is characterized in that the visual field range of the first imaging unit moves according to the movement of the head of the subject, and the visual field range of the second imaging unit moves accordingly.
  • FIG. 9 is a system configuration diagram of the eye condition measurement system 1b according to the third embodiment.
  • the eye condition measurement system 1b according to the third embodiment has basically the same configuration and function as the eye condition measurement system 1a according to the second embodiment.
  • the eye condition measurement system 1b has a first image pickup unit 30b and an information processing apparatus 10b in place of the first image pickup unit 30a and the information processing device 10a, and further has a light source 63, a long pass filter 70, and a half mirror 71. It is different from the eye condition measurement system 1a.
  • the first imaging unit 30b has a tracking camera 32 whose visual field range moves according to the movement of the head of the subject P.
  • the visual field range of the tracking camera 32 is moved according to the movement of the eye region of the subject P so that the eye region of the subject P is imaged in the center of the first image.
  • the field of view of the tracking camera 32 is moved by the rotation of the movable mirror 50 by the drive unit 51 according to the control signal of the field of view of the information processing apparatus 10b, similarly to the second image pickup unit 40. That is, the movable mirror 50 moves the optical axis of the first imaging unit 30b in addition to the optical axis of the second imaging unit 40.
  • the field of view of the second image pickup unit 40 moves in conjunction with the movement of the field of view of the tracking camera 32.
  • the tracking wavelength is smaller than the pupil detection wavelength which is the detection wavelength of the light receiving element of the second imaging unit 40.
  • the tracking wavelength is 850 [nm].
  • the angle of view, focal length, and frame rate of the tracking camera 32 are the same as those of the wide-angle camera 31 of the second embodiment.
  • the light source 63 is a light source that illuminates the eye area of the subject P.
  • the light source 63 is a light source having a wavelength region corresponding to a tracking wavelength, and is a near-infrared light source of 850 [nm] in the third embodiment.
  • the half mirror 71 is a half mirror that reflects a part of the light of 850 [nm] incident from the light source 63 toward the long pass filter 70. Further, the half mirror 71 transmits a part of the light of 850 [nm] incident from the long pass filter 70 toward the tracking camera 32.
  • the half mirror 71 may be a beam splitter having an arbitrary transmission and reflection ratio instead of the half mirror.
  • the long pass filter 70 is an optical filter that transmits light having a wavelength for detecting the pupil and reflects the wavelength for tracking.
  • the long-pass filter 70 is arranged between the second image pickup unit 40 and the movable mirror 50, and transmits light having a wavelength for detecting a pupil incident from the movable mirror 50 toward the second image pickup unit 40. Further, the long pass filter 70 reflects the light of the tracking wavelength incident from the movable mirror 50 toward the half mirror 71.
  • the tracking camera 32 of the first imaging unit 30b is the eye region (or face) of the subject P ⁇ the first movable mirror 50-1 ⁇ the second movable mirror 50-2 ⁇ the long path filter 70 ⁇ the half mirror 71 ⁇ the tracking camera.
  • the eye region (or face) of the subject P is imaged at the first angle of view.
  • the second imaging unit 40 receives light from the eye region of the subject P ⁇ the first movable mirror 50-1 ⁇ the second movable mirror 50-2 ⁇ the long path filter 70 ⁇ the second imaging unit 40.
  • the eye area of subject P is imaged at the second angle of view.
  • the information processing device 10b has basically the same configuration and function as the information processing device 10a, but the movement control processing in the X-axis and Z-axis directions by the movement control unit 13 is performed in the second embodiment (FIGS. 5 and 8). It is different from the process shown in step S14).
  • the movement control unit 13 first calculates the eye position coordinates in the first image. Then, the movement control unit 13 determines the first movable mirror 50-1 and the second movable mirror 50- so that the center of gravity of the eye region in the first image is arranged at the center of the first image at the next shooting timing. Calculate the amount of rotation of 2.
  • the tracking camera 32 of the first image pickup unit 30b has a frame rate equal to or higher than the frame rate of the second image pickup unit 40, the eye region of the subject P is the center of the image even in the second image captured by the second image pickup unit 40. It will be reflected in.
  • the information processing apparatus 10b can more accurately follow the movement of the head of the subject P and move the visual field range of the second imaging unit 40. Thereby, the information processing apparatus 10b can suitably evaluate the change in the eye state of the subject P in a relaxed state without fixing the head.
  • Embodiment 4 of the present disclosure will be described with reference to FIGS. 10 to 11.
  • the fourth embodiment is characterized in that the position of the field of view of the camera having the narrowest angle is roughly adjusted by the image having the widest angle, and the position of the field of view is finely adjusted by the image having an angle of view of an intermediate size.
  • FIG. 10 is a system configuration diagram of the eye condition measurement system 1c according to the fourth embodiment.
  • the eye condition measurement system 1c according to the fourth embodiment has basically the same configuration and function as the eye condition measurement system 1b according to the third embodiment.
  • the eye condition measurement system 1c is different from the eye condition measurement system 1b in that it has a third image pickup unit 33 and an information processing device 10c instead of the information processing device 10b.
  • the third image pickup unit 33 is a camera that captures at least the face of the subject P at the third angle of view and generates the third image data related to the third image.
  • the third angle of view is larger than the first angle of view of the tracking camera 32 and larger than the second angle of view of the second imaging unit 40.
  • the third image pickup unit 33 transmits the third image data to the information processing apparatus 10c in response to the generation of the third image data.
  • the information processing device 10c has basically the same function as the information processing device 10b, but is in that the movement control of the viewing range of the tracking camera 32 and the second imaging unit 40 is roughly adjusted based on the third image data. It is different from the information processing device 10b.
  • FIG. 11 is a block diagram showing a functional configuration of the information processing apparatus 10c according to the fourth embodiment.
  • the information processing device 10c has a third acquisition unit 17 in addition to the configuration of the information processing device 10b.
  • the third acquisition unit 17 receives and acquires the third image data from the third image pickup unit 33, and supplies the third image data to the movement control unit 13.
  • the movement control unit 13 detects the face of the subject P from the third image, calculates the eye position coordinates in the third image, and based on the eye position coordinates, the movable mirror 50 such that the eye area fits in the first image. Calculate the amount of rotation of. Then, the movement control unit 13 transmits a control signal in the visual field range based on the rotation amount to the drive unit 51, and roughly adjusts the tilt angle of the movable mirror 50.
  • Such a rough adjustment process may be executed before step S10 in FIG. Further, in the rough adjustment process, in order to suppress an increase in the processing time, for example, when the face of the subject P is not detected in the first image in step S11 of FIG. 8, or the eyes of the subject P in the first image in step S12. It may be executed only when the area is not detected.
  • the movement control process using the first image having a first angle of view narrower than the third image is a process corresponding to steps S10 to 15 in FIG. 8, but can also be called a fine adjustment process.
  • the first angle of view of the fourth embodiment may be any angle of view necessary for imaging at least the eye region of the subject P, and may be narrower than the first angle of view of the second to third embodiments.
  • the face detection process corresponding to step S11 in FIG. 8 may be omitted. As a result, it is possible to suppress an increase in a series of processing times due to the addition of the rough adjustment processing.
  • the information processing apparatus 10c of the fourth embodiment has the second imaging unit 40 so that the eye region of the subject P does not deviate from the visual field range of the second imaging unit 40 even if the head of the subject P moves significantly.
  • the field of view can be moved.
  • the subject P can move freely even during imaging, so that the information processing apparatus 10c can evaluate the change in the state of the eyes in a more relaxed state.
  • the state of the eye to be evaluated is the amount of contraction of the pupil
  • the state evaluation unit 15 may evaluate the amount of contraction of the pupil of the subject P based on the time-series data of the second image. Then, the state evaluation unit 15 may evaluate the degree of interest when viewing an image such as an advertisement image based on the amount of contraction of the pupil.
  • the present disclosure has been described as a hardware configuration, but the present disclosure is not limited to this.
  • the present disclosure can also realize various processes related to the state evaluation method by causing a processor to execute a computer program.
  • Non-temporary computer-readable media include various types of tangible storage mediums.
  • Examples of non-temporary computer-readable media include magnetic recording media (eg, flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (eg, magneto-optical disks), CD-ROMs (ReadOnlyMemory), CD-Rs, Includes CD-R / W, semiconductor memory (eg, mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (RandomAccessMemory)).
  • the program may also be supplied to the computer by various types of transient computer readable medium.
  • Examples of temporary computer readable media include electrical, optical, and electromagnetic waves.
  • the temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.
  • the computer is composed of a computer system including a personal computer, a word processor, and the like.
  • the computer is not limited to this, and can be configured by a LAN (local area network) server, a computer (personal computer) communication host, a computer system connected on the Internet, or the like. It is also possible to distribute the functions to each device on the network and configure the computer in the entire network.
  • (Appendix 1) From the first imaging unit, the first acquisition unit that acquires image data related to the first image obtained by capturing the head of the subject at the first angle of view, and A second acquisition unit that acquires image data related to a second image obtained by capturing the subject's eye region with a second angle of view narrower than the first angle of view from the second image pickup unit.
  • a movement control unit that moves the visual field range of the second imaging unit based on the position information of the head of the subject acquired based on the first image.
  • An information processing system including a state evaluation unit that evaluates a state change of the eyes of the subject based on the time-series data of the second image.
  • the movement control unit generates information on changes in the position of the head of the subject based on the time-series data of the first image, and moves the visual field range of the second imaging unit based on the change information.
  • the information processing system according to 1. (Appendix 3) The information processing system according to Appendix 2, wherein the first acquisition unit acquires image data related to the first image at a frame rate equal to or higher than the frame rate at which the second acquisition unit acquires image data related to the second image. .. (Appendix 4) It has an element position estimation unit that estimates the position of the center of gravity of the pupil of the subject in the second image and the position of at least one of the outer and inner corners of the eye.
  • the state evaluation unit is described in any one of Supplementary note 1 to 3 for evaluating the vibration state of the pupil of the subject based on the position information of at least one of the outer corner of the eye and the inner corner of the eye and the position information of the center of gravity of the pupil.
  • Information processing system (Appendix 5)
  • the state evaluation unit calculates the relative position of the center of gravity of the pupil with respect to the straight line connecting the position of the outer corner of the eye and the position of the inner corner of the eye, and evaluates the vibration state of the pupil of the subject based on the relative position.
  • Information processing system Appendix 5
  • the information processing device is A first acquisition unit that acquires image data related to the first image from the first image pickup unit, and A second acquisition unit that acquires image data related to the second image from the second image pickup unit, and A movement control unit that moves the visual field range of the second imaging unit based on the position information of the head of the subject acquired based on the first image.
  • An eye condition measurement system including a condition evaluation unit that evaluates a change in the eye condition of the subject based on the time-series data of the second image.
  • a movable mirror that moves the optical axis of the second imaging unit Further equipped with a drive unit for rotating the movable mirror, The movement control unit generates change information of the position of the head of the subject based on the time-series data of the first image, and calculates the rotation amount of the movable mirror based on the change information.
  • the described eye condition measurement system (Appendix 9) The eye condition measuring system according to Appendix 8, wherein the movable mirror moves the optical axes of the first imaging unit and the second imaging unit. (Appendix 10) The eye condition measuring system according to any one of Supplementary note 7 to 9, wherein the first imaging unit captures a subject at a frame rate equal to or higher than the frame rate of the second imaging unit.
  • An information processing method including a state evaluation stage for evaluating a state change of the eyes of the subject based on the time series data of the second image.
  • the information processing system according to the present disclosure can be used to evaluate changes in the eye condition of the subject.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Animal Behavior & Ethology (AREA)
  • Molecular Biology (AREA)
  • Veterinary Medicine (AREA)
  • Biophysics (AREA)
  • Public Health (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Surgery (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Ophthalmology & Optometry (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Pathology (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Geometry (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Child & Adolescent Psychology (AREA)
  • Developmental Disabilities (AREA)
  • Educational Technology (AREA)
  • Hospice & Palliative Care (AREA)
  • Psychiatry (AREA)
  • Psychology (AREA)
  • Social Psychology (AREA)
  • Eye Examination Apparatus (AREA)
PCT/JP2021/029657 2020-10-20 2021-08-11 情報処理システム、目状態測定システム、情報処理方法および非一時的なコンピュータ可読媒体 Ceased WO2022085276A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2022556425A JP7640946B2 (ja) 2020-10-20 2021-08-11 情報処理システム、目状態測定システム、情報処理方法およびプログラム
US18/032,686 US12243234B2 (en) 2020-10-20 2021-08-11 Information processing system, eye state measurement system, information processing method, and non-transitory computer readable medium
EP21882417.5A EP4233690A4 (en) 2020-10-20 2021-08-11 INFORMATION PROCESSING SYSTEM, EYE CONDITION MEASURING SYSTEM, INFORMATION PROCESSING METHOD, AND NON-TRANSITORY COMPUTER-READABLE MEDIUM

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2020-176066 2020-10-20
JP2020176066 2020-10-20

Publications (1)

Publication Number Publication Date
WO2022085276A1 true WO2022085276A1 (ja) 2022-04-28

Family

ID=81291237

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/029657 Ceased WO2022085276A1 (ja) 2020-10-20 2021-08-11 情報処理システム、目状態測定システム、情報処理方法および非一時的なコンピュータ可読媒体

Country Status (4)

Country Link
US (1) US12243234B2 (https=)
EP (1) EP4233690A4 (https=)
JP (1) JP7640946B2 (https=)
WO (1) WO2022085276A1 (https=)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023079862A1 (ja) * 2021-11-05 2023-05-11 パナソニックＩｐマネジメント株式会社 撮像システム、処理装置、および撮像システムにおいてコンピュータによって実行される方法
WO2026083751A1 (ja) * 2024-10-15 2026-04-23 ソニーグループ株式会社 情報処理装置、情報処理方法、及びプログラム

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20240004976A1 (en) * 2022-06-29 2024-01-04 Mikhail Belenkii Secure personal communication device

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH1040386A (ja) * 1996-07-25 1998-02-13 Oki Electric Ind Co Ltd 虹彩認識システム
JP2005323905A (ja) * 2004-05-17 2005-11-24 Nippon Hoso Kyokai <Nhk> 眼球運動測定装置、及び眼球運動測定プログラム
WO2012001755A1 (ja) 2010-07-02 2012-01-05 株式会社ソニー・コンピュータエンタテインメント 情報処理システム、情報処理装置および情報処理方法
WO2016195066A1 (ja) 2015-06-05 2016-12-08 聖 星野 眼球の運動を検出する方法、そのプログラム、そのプログラムの記憶媒体、及び、眼球の運動を検出する装置
JP2018088647A (ja) * 2016-11-29 2018-06-07 株式会社デンソー 眼球追跡装置及び撮像システム
US20190121105A1 (en) * 2017-10-20 2019-04-25 Centraled Technology Co., Ltd. Dynamic zoom lens for multiple-in-one optical system title
JP2019195377A (ja) 2018-05-07 2019-11-14 オムロン株式会社 データ処理装置、モニタリングシステム、覚醒システム、データ処理方法、及びデータ処理プログラム
WO2020188629A1 (ja) * 2019-03-15 2020-09-24 オムロン株式会社 覚醒状態判定装置、覚醒システム、及び覚醒状態判定方法
JP2020176066A (ja) 2019-04-16 2020-10-29 学校法人麻布獣医学園 口臭抑制用組成物

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN100433043C (zh) 2006-04-18 2008-11-12 南京大学 自动跟踪无侵害虹膜图像采集装置
JP6354979B2 (ja) * 2014-03-31 2018-07-11 株式会社ニデック 眼底撮影装置
JP6530239B2 (ja) 2015-05-28 2019-06-12 浜松ホトニクス株式会社 両眼計測装置、両眼計測方法、及び両眼計測プログラム
JP7196606B2 (ja) * 2016-03-04 2022-12-27 株式会社ニデック 眼科情報処理装置および眼科情報処理プログラム
US11544359B2 (en) 2016-11-08 2023-01-03 Proprius Technolgies S.A.R.L Unique patterns extracted from involuntary eye motions to identify individuals
JP7147130B2 (ja) 2016-12-26 2022-10-05 スズキ株式会社 ハイブリッド車両の駆動制御装置
US11200713B2 (en) * 2018-10-05 2021-12-14 Amitabha Gupta Systems and methods for enhancing vision
US10948986B2 (en) * 2019-04-09 2021-03-16 Fotonation Limited System for performing eye detection and/or tracking

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH1040386A (ja) * 1996-07-25 1998-02-13 Oki Electric Ind Co Ltd 虹彩認識システム
JP2005323905A (ja) * 2004-05-17 2005-11-24 Nippon Hoso Kyokai <Nhk> 眼球運動測定装置、及び眼球運動測定プログラム
WO2012001755A1 (ja) 2010-07-02 2012-01-05 株式会社ソニー・コンピュータエンタテインメント 情報処理システム、情報処理装置および情報処理方法
WO2016195066A1 (ja) 2015-06-05 2016-12-08 聖 星野 眼球の運動を検出する方法、そのプログラム、そのプログラムの記憶媒体、及び、眼球の運動を検出する装置
JP2018088647A (ja) * 2016-11-29 2018-06-07 株式会社デンソー 眼球追跡装置及び撮像システム
US20190121105A1 (en) * 2017-10-20 2019-04-25 Centraled Technology Co., Ltd. Dynamic zoom lens for multiple-in-one optical system title
JP2019195377A (ja) 2018-05-07 2019-11-14 オムロン株式会社 データ処理装置、モニタリングシステム、覚醒システム、データ処理方法、及びデータ処理プログラム
WO2020188629A1 (ja) * 2019-03-15 2020-09-24 オムロン株式会社 覚醒状態判定装置、覚醒システム、及び覚醒状態判定方法
JP2020176066A (ja) 2019-04-16 2020-10-29 学校法人麻布獣医学園 口臭抑制用組成物

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4233690A4

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023079862A1 (ja) * 2021-11-05 2023-05-11 パナソニックＩｐマネジメント株式会社 撮像システム、処理装置、および撮像システムにおいてコンピュータによって実行される方法
WO2026083751A1 (ja) * 2024-10-15 2026-04-23 ソニーグループ株式会社 情報処理装置、情報処理方法、及びプログラム

Also Published As

Publication number Publication date
US20230386038A1 (en) 2023-11-30
EP4233690A1 (en) 2023-08-30
JPWO2022085276A1 (https=) 2022-04-28
EP4233690A4 (en) 2024-08-28
JP7640946B2 (ja) 2025-03-06
US12243234B2 (en) 2025-03-04

Similar Documents

Publication Publication Date Title
JP5368765B2 (ja) 撮影制御装置、撮影装置、撮影制御方法、プログラム、記憶媒体
JP5512682B2 (ja) 眼球運動、特に眼底の運動の測定装置および測定方法
JP5828070B2 (ja) 撮像装置および撮像方法
JP7640946B2 (ja) 情報処理システム、目状態測定システム、情報処理方法およびプログラム
EP2752151A1 (en) Fundus observation apparatus
US20060098087A1 (en) Housing device for head-worn image recording and method for control of the housing device
US10984236B2 (en) System and apparatus for gaze tracking
CN100482146C (zh) 球形目标的运动跟踪系统
JP7221587B2 (ja) 眼科装置
JP2023029537A (ja) 眼科装置
CN111989030B (zh) 图像处理方法、程序及图像处理装置
WO2022024104A1 (en) Eye tracking systems and methods
JPH0449943A (ja) 眼球運動分析装置
JP4568024B2 (ja) 眼球運動測定装置、及び眼球運動測定プログラム
JP2018015021A (ja) 眼科装置
US20240388800A1 (en) Medical visualisation system and method for video stabilisation in such a system
Irie et al. A laser-based eye-tracking system
Kim et al. High-speed lensless eye tracker for microsaccade measurement
JP6930841B2 (ja) 眼科装置
JPH05199995A (ja) 瞳孔撮影装置と視線検出装置
US8967803B2 (en) Image capturing apparatus and auto-focusing method thereof
JP2022044645A (ja) 眼科装置
JP6754096B2 (ja) 眼科用観察システムおよび眼科用観察制御プログラム
Sarès et al. Analyzing head roll and eye torsion by means of offline image processing
JP6632285B2 (ja) 眼科撮影装置及びその制御方法、並びに、プログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21882417

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022556425

Country of ref document: JP

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 18032686

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2021882417

Country of ref document: EP

Effective date: 20230522

WWG Wipo information: grant in national office

Ref document number: 18032686

Country of ref document: US