WO2023079862A1 - 撮像システム、処理装置、および撮像システムにおいてコンピュータによって実行される方法 - Google Patents

撮像システム、処理装置、および撮像システムにおいてコンピュータによって実行される方法 Download PDF

Info

Publication number
WO2023079862A1
WO2023079862A1 PCT/JP2022/035983 JP2022035983W WO2023079862A1 WO 2023079862 A1 WO2023079862 A1 WO 2023079862A1 JP 2022035983 W JP2022035983 W JP 2022035983W WO 2023079862 A1 WO2023079862 A1 WO 2023079862A1
Authority
WO
WIPO (PCT)
Prior art keywords
imaging device
living body
image data
orientation
imaging
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/035983
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
貴真 安藤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Panasonic Intellectual Property Management Co Ltd
Original Assignee
Panasonic Intellectual Property Management Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Panasonic Intellectual Property Management Co Ltd filed Critical Panasonic Intellectual Property Management Co Ltd
Priority to JP2023557890A priority Critical patent/JPWO2023079862A1/ja
Priority to CN202280067568.8A priority patent/CN118076301A/zh
Publication of WO2023079862A1 publication Critical patent/WO2023079862A1/ja
Priority to US18/624,249 priority patent/US20240260832A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/61—Control of cameras or camera modules based on recognised objects
    • H04N23/611—Control of cameras or camera modules based on recognised objects where the recognised objects include parts of the human body
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B10/00—Instruments for taking body samples for diagnostic purposes; Other methods or instruments for diagnosis, e.g. for vaccination diagnosis, sex determination or ovulation-period determination; Throat striking implements
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/0033—Features or image-related aspects of imaging apparatus, e.g. for MRI, optical tomography or impedance tomography apparatus; Arrangements of imaging apparatus in a room
    • A61B5/004—Features or image-related aspects of imaging apparatus, e.g. for MRI, optical tomography or impedance tomography apparatus; Arrangements of imaging apparatus in a room adapted for image acquisition of a particular organ or body part
    • A61B5/0042—Features or image-related aspects of imaging apparatus, e.g. for MRI, optical tomography or impedance tomography apparatus; Arrangements of imaging apparatus in a room adapted for image acquisition of a particular organ or body part for the brain
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
    • A61B5/0077—Devices for viewing the surface of the body, e.g. camera, magnifying lens
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/02—Detecting, measuring or recording for evaluating the cardiovascular system, e.g. pulse, heart rate, blood pressure or blood flow
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/02—Detecting, measuring or recording for evaluating the cardiovascular system, e.g. pulse, heart rate, blood pressure or blood flow
    • A61B5/026—Measuring blood flow
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/02—Detecting, measuring or recording for evaluating the cardiovascular system, e.g. pulse, heart rate, blood pressure or blood flow
    • A61B5/026—Measuring blood flow
    • A61B5/0261—Measuring blood flow using optical means, e.g. infrared light
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B15/00—Special procedures for taking photographs; Apparatus therefor
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B17/00—Details of cameras or camera bodies; Accessories therefor
    • G03B17/56—Accessories
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/695—Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B2562/00—Details of sensors; Constructional details of sensor housings or probes; Accessories for sensors
    • A61B2562/02—Details of sensors specially adapted for in-vivo measurements
    • A61B2562/0233—Special features of optical sensors or probes classified in A61B5/00
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B2576/00—Medical imaging apparatus involving image processing or analysis
    • A61B2576/02—Medical imaging apparatus involving image processing or analysis specially adapted for a particular organ or body part
    • A61B2576/026—Medical imaging apparatus involving image processing or analysis specially adapted for a particular organ or body part for the brain

Definitions

  • the present disclosure relates to an imaging system, a processing device, and a computer-implemented method in an imaging system.
  • the reflected light generated by irradiating the subject area of the living body with light includes components that pass through the surface and inside of the subject area. By detecting such reflected light, it is possible to acquire biological information of the subject, such as surface information and/or internal information.
  • Patent Literatures 1 and 2 disclose devices for acquiring internal information of a subject.
  • the present disclosure provides an imaging system capable of stably acquiring biometric information of a subject in a non-contact manner in an environment in which a living body moves.
  • An imaging system includes a first imaging device having a first field of view, a second imaging device having a second field of view narrower than the first field of view, and changing the orientation of the second imaging device.
  • the first imaging device images a living body to generate first image data
  • the second imaging device images a subject part of the living body to generate a second image data is generated
  • the second image data is sent to a processing device that generates data indicative of biometric information of the subject based on the second image data
  • the electrically powered device converts the first image data into
  • the direction of the second imaging device is changed based on the position of the living body in the image based on the subject, and the state in which the subject is included in the second field of view is maintained.
  • an imaging system capable of stably acquiring biometric information of a subject area in a non-contact manner in an environment where a living body moves.
  • FIG. 1A is a block diagram that schematically illustrates the configuration of an imaging system according to an exemplary embodiment of the present disclosure
  • FIG. FIG. 1B is a diagram schematically showing control circuits and signal processing circuits included in the first light source, the second light source, the second imaging device, and the processing device in the imaging system of FIG. 1A.
  • FIG. 2 is a flow chart schematically showing an example of correction operation performed by the processing device when the living body moves.
  • FIG. 3A is a diagram for explaining the operation of the electric device.
  • FIG. 3B is a diagram for explaining the operation of the electric device;
  • FIG. 3C is a diagram for explaining the deviation amount Q1 and the deviation amount Q2 in the first image.
  • FIG. 3D is a diagram for explaining the first rotation amount and the second rotation amount.
  • FIG. 4A is a perspective view schematically showing a first example of an electric device that supports an imaging device
  • FIG. 4B is a perspective view schematically showing a second example of the electrically powered device that supports the imaging device.
  • FIG. 4C is a perspective view schematically showing a third example of a motorized device that supports an imaging device;
  • FIG. 5 is a diagram schematically showing an example of imaging a living body by an imaging system according to a modification of this embodiment.
  • FIG. 6A is a diagram showing a comparative example in which cerebral blood flow information of a subject after movement is acquired with the orientation of the imaging device fixed.
  • FIG. 6B is a diagram showing an example in which cerebral blood flow information of the subject after movement is acquired after changing the direction of the imaging device according to the movement of the living body.
  • FIG. 6A is a diagram showing a comparative example in which cerebral blood flow information of a subject after movement is acquired with the orientation of the imaging device fixed.
  • FIG. 6B is a diagram showing an example in which
  • FIG. 7 is a diagram showing an example of the configuration of the second imaging device.
  • FIG. 8A is a diagram showing an example of the operation of emitting the first optical pulse and the second optical pulse.
  • FIG. 8B is a diagram showing another example of the operation of emitting the first optical pulse and the second optical pulse.
  • FIG. 9A is a diagram schematically showing an example of temporal changes in surface reflection components and internal scattering components included in a reflected light pulse when the light pulse has an impulse waveform.
  • FIG. 9B is a diagram schematically showing an example of temporal changes in the surface reflection component and the internal scattering component included in the reflected light pulse when the light pulse has a rectangular waveform.
  • FIG. 9C is a flow chart outlining the operation of the processor with respect to the first light source, the second light source, and the second imaging device.
  • the subject In an environment in which a living body moves, such as an example of obtaining surface blood flow information on the forehead and/or cerebral blood flow information of a person who is working or driving a vehicle, the subject It may be required to obtain the biometric information of the body. In a configuration in which the orientation of an imaging device that acquires biological information is fixed, it may not be possible to stably acquire biological information of a subject after movement.
  • An imaging system includes a first imaging device having a relatively wide field of view for acquiring position information of a living body, and a relatively narrow field of view for acquiring biological information of a subject part of the living body. and a second imaging device.
  • the orientation of the second imaging device can be changed based on the positional information of the living body acquired by the first imaging device so that the subject part of the living body after movement can be imaged.
  • the following describes an imaging system, a processing device, and a computer-implemented method in an imaging system according to embodiments of the present disclosure.
  • the imaging system includes a first imaging device having a first field of view, a second imaging device having a second field of view narrower than the first field of view, and changing the orientation of the second imaging device.
  • a motorized device capable of The first imaging device images a living body to generate first image data.
  • a said 2nd imaging device images the to-be-tested part of the said living body, and produces
  • the second image data is sent to a processing device that generates data representing biological information of the subject based on the second image data.
  • the electric device changes the orientation of the second imaging device based on the position of the living body in the image based on the first image data, and maintains a state in which the subject is included in the second field of view.
  • the imaging system according to the second item is the imaging system according to the first item, wherein the electric device can change the orientation of the first imaging device.
  • the electric device synchronously changes orientations of the first imaging device and the second imaging device based on the position of the living body in the image based on the first image data.
  • the relative position of the second field of view and the subject area relationship can be known.
  • the imaging system according to the third item is the imaging system according to the second item, wherein the imaging system includes the processing device.
  • the processing device can generate data indicating the biometric information of the subject.
  • the imaging system according to the fourth item is the imaging system according to the third item, wherein the image based on the first image data includes the face of the living body.
  • the processing device causes the motorized device to change the orientation of the first imaging device so that a specific position of the image based on the first image data is included in the facial region of the living body.
  • the processing device causes the electric device to change the orientation of the first imaging device
  • the imaging system based on the first image data
  • the electric device further changes the orientation of the first imaging device so as to reduce the amount of deviation between the specific position of the image and the specific position of the face of the living body.
  • An imaging system is the imaging system according to any one of the second to fifth items, wherein the subject part includes the forehead part of the living body.
  • the processing device causes the motorized device to change the orientation of the second imaging device such that the second field of view includes the forehead and eyebrows of the living subject.
  • the edge portion of the eyebrow is used as a feature point in the correction that matches the position of the subject before the living body moves and the position of the subject after the living body moves by image processing-based tracking. can be used as
  • An imaging system is the imaging system according to any one of the second to sixth items, wherein the processing device causes the electric device to cause the second field of view to include the subject portion. 2. After changing the orientation of the imaging device, determine a pixel region of a portion corresponding to the subject portion in the image based on the second image data.
  • the imaging system according to the eighth item is the imaging system according to any one of the seventh items, wherein the pixel region is the subject part in the image based on the second image data before the living body moves. matches the pixel area of the portion corresponding to .
  • the imaging system according to the ninth item is the imaging system according to any one of the first to eighth items, wherein the biological information is cerebral blood flow information of the biological body.
  • the imaging system according to the tenth item is the imaging system according to any one of the first to ninth items, comprising at least one light source that emits a light pulse for irradiating the subject part of the living body.
  • this imaging system it is possible to irradiate the subject area of the living body and acquire the biometric information of the subject area.
  • the processing device related to the eleventh item is the processing device used in the imaging system.
  • the imaging system includes a first imaging device having a first field of view, a second imaging device having a second field of view narrower than the first field of view, and a motorized device capable of changing the orientation of the second imaging device.
  • the processing device comprises a processor and a memory storing a computer program executed by the processor.
  • the computer program causes the processor to cause the first imaging device to image a living body to generate first image data, and instructs the electrically powered device to determine the position of the living body in an image based on the first image data.
  • this processing device it is possible to stably acquire biometric information of the subject area in a non-contact manner in an environment where the living body moves.
  • the processing device is the processing device according to the eleventh item, wherein the electric device can change the orientation of the first imaging device.
  • Changing the orientation of the second imaging device based on the position of the living body in the image based on the first image data may include changing the orientation of the second imaging device based on the position of the living body in the image based on the first image data.
  • the method includes synchronously changing the orientation of the imaging device and the second imaging device.
  • the method according to the thirteenth item is a computer-implemented method in the imaging system.
  • the imaging system includes a first imaging device having a first field of view, a second imaging device having a second field of view narrower than the first field of view, and a motorized device capable of changing the orientation of the second imaging device. And prepare.
  • the method includes causing the first image capturing device to capture an image of a living body to generate first image data, and causing the electrically powered device to perform the second image based on the position of the living body in an image based on the first image data. changing the orientation of an imaging device to maintain a state in which the subject portion of the living body is included in the second field of view; and generating data indicating biometric information of the subject based on the second image data.
  • the method according to the 14th item is the method according to the 13th item, wherein the electric device can change the orientation of the first imaging device.
  • Changing the orientation of the second imaging device based on the position of the living body in the image based on the first image data may include changing the orientation of the second imaging device based on the position of the living body in the image based on the first image data.
  • the method includes synchronously changing the orientation of the imaging device and the second imaging device.
  • all or part of a circuit, unit, device, member or section, or all or part of a functional block in a block diagram is, for example, a semiconductor device, a semiconductor integrated circuit (IC), or an LSI (large scale integration). ) may be performed by one or more electronic circuits.
  • An LSI or IC may be integrated on one chip, or may be configured by combining a plurality of chips.
  • functional blocks other than memory elements may be integrated into one chip.
  • LSIs or ICs may be called system LSIs, VLSIs (very large scale integration), or ULSIs (ultra large scale integration) depending on the degree of integration.
  • a FIpld Programmable Gate Array which is programmed after the LSI is manufactured, or a reconfigurable logic device that can reconfigure the connection relationships inside the LSI or set up the circuit partitions inside the LSI can also be used for the same purpose.
  • FPGA FIpld Programmable Gate Array
  • circuits, units, devices, members or parts can be executed by software processing.
  • the software is recorded on one or more non-transitory storage media, such as ROMs, optical discs, hard disk drives, etc., such that when the software is executed by a processor, the functions specified in the software are performed. It is executed by processors and peripherals.
  • a system or apparatus may include one or more non-transitory storage media on which software is recorded, a processor, and required hardware devices such as interfaces.
  • light refers not only to visible light (having a wavelength of about 400 nm to about 700 nm), but also to electromagnetic waves including ultraviolet rays (having a wavelength of about 10 nm to about 400 nm) and infrared rays (having a wavelength of about 700 nm to about 1 mm). means.
  • FIG. 1A is a block diagram that schematically illustrates the configuration of an imaging system according to an exemplary embodiment of the present disclosure
  • FIG. FIG. 1A shows the head and torso of a human assuming that the living body 10 is a human.
  • the living body 10 is illuminated with illumination light or ambient light such as sunlight.
  • the living body 10 is not always stationary, such as when working or driving a vehicle, but may move.
  • the living body 10 is not limited to humans, and may be animals, for example.
  • a region surrounded by a dotted line shown in FIG. 1A represents the subject 11 of the living body 10 .
  • the imaging system 100 shown in FIG. 1A includes a first light source 20a, a second light source 20b, a first imaging device 30a, a second imaging device 30b, an electric device 40, and a processing device 50.
  • the processing device 50 comprises control circuitry 52 , signal processing circuitry 54 and memory 56 .
  • the first light source 20a and the second light source 20b are also referred to as "light source 20" without distinction.
  • the first imaging device 30a and the second imaging device 30b are also referred to as "imaging device 30" without distinction.
  • FIG. 1B is a diagram schematically showing the control circuit 52 and the signal processing circuit 54 included in the first light source 20a, the second light source 20b, the second imaging device 30b, and the processing device 50 in the imaging system 100 of FIG. 1A. is.
  • FIG. 1B shows an enlarged view of the subject 11 of the living body 10 .
  • the light source 20 emits light pulses for irradiating the subject 11 of the living body 10 .
  • the first imaging device 30a has a relatively wide first field of view 12a, and acquires the position information of the living body 10 from the reflected light generated by the above ambient light being reflected by the living body 10.
  • the second imaging device 30b has a relatively narrow second field of view 12b, and acquires biological information of the subject 11 from the reflected light pulse generated by the light pulse being reflected by the subject 11 of the living body 10.
  • the second field of view 12b is located inside the first field of view 12a.
  • the area surrounded by the dashed-dotted line indicates the first field of view 12a
  • the area surrounded by the dashed line indicates the second field of view 12b.
  • the electric device 40 supports the first imaging device 30 a and the second imaging device 30 b and changes the orientation of the imaging device 30 in response to a signal from the processing device 50 based on the position information of the living body 10 .
  • the living body 10 is included in the first field of view 12a and the subject 11 of the living body 10 is in the second field of view 12a even after the living body 10 moves.
  • the state contained in the field of view 12b is maintained.
  • the biological information may be, for example, cerebral blood flow information of the living body 10, or blood flow information of the face or scalp.
  • the first light source 20a emits a first light pulse Ip1 for irradiating the test site 11, as shown in FIG. 1B.
  • the first light pulse Ip1 has a first wavelength.
  • the second light source 20b emits a second light pulse Ip2 for illuminating the subject 11, as shown in FIG. 1B.
  • the second light pulse Ip2 has a second wavelength that is longer than the first wavelength.
  • the number of first light sources 20a is one, but it may be plural.
  • the number of second light sources 20b is one, but may be plural. Depending on the application, it is not necessary to use both the first light source 20a and the second light source 20b, and either one may be used.
  • each of the first optical pulse I p1 and the second optical pulse I p2 is also referred to as “optical pulse I p ” without distinction.
  • the light pulse Ip includes a rising portion and a falling portion.
  • the rising portion is the portion of the optical pulse Ip from when the intensity starts to increase until when the increase ends.
  • the trailing portion is the portion of the optical pulse Ip from when the intensity starts to decrease until the decrease ends.
  • the surface reflection component I1 includes three components: a direct reflection component, a diffuse reflection component, and a diffuse reflection component.
  • a direct reflection component is a reflection component for which the angle of incidence is equal to the angle of reflection.
  • the diffuse reflection component is a component that diffuses and reflects due to the uneven shape of the surface.
  • the scattered reflection component is the component that is scattered and reflected by the internal tissue near the surface.
  • the scattered reflection component is a component that is scattered and reflected inside the epidermis.
  • the surface reflection component I1 reflected on the surface of the test portion 11 includes these three components.
  • Internally scattered component I2 is described as not including components scattered and reflected by internal tissue near the surface.
  • the surface reflection component I1 and the internal scattering component I2 are reflected or scattered, the direction of travel of these components changes, and a portion of the surface reflection component I1 and a portion of the internal scattering component I2 are reflected or scattered as a reflected light pulse. 2 reaches the imaging device 30b.
  • the surface reflection component I1 reflects surface information of the living body 10, for example, blood flow information of the face and scalp.
  • facial appearance, skin blood flow, heart rate, or perspiration amount of the living body 10 can be known from the blood flow information of the face and scalp.
  • the cerebral blood flow, blood pressure, blood oxygen saturation, or heart rate of the living body 10 can be known from the cerebral blood flow information.
  • Detecting the surface reflection component I1 " may be interpreted as "detecting a portion of the surface reflection component I1 ".
  • 'Detecting the internal scatter component I2 ' may be interpreted as 'detecting a portion of the internal scatter component I2 '.
  • a method for detecting the internally scattered component I2 from the reflected light pulse will be described later.
  • Each of the first wavelength of the first optical pulse I p1 and the second wavelength of the second optical pulse I p2 may be any wavelength included in the wavelength range of 650 nm or more and 950 nm or less, for example.
  • This wavelength range is included in the red to near-infrared wavelength range.
  • the above wavelength range is called the "window of the body" and has the property of being relatively difficult to be absorbed by moisture and skin in the body.
  • detection sensitivity can be increased by using light in the above wavelength range.
  • the light used is believed to be absorbed primarily by oxygenated hemoglobin (HbO 2 ) and deoxygenated hemoglobin (Hb).
  • HbO 2 oxygenated hemoglobin
  • Hb deoxygenated hemoglobin
  • changes in blood flow result in changes in the concentration of oxygenated hemoglobin and deoxygenated hemoglobin.
  • the degree of light absorption also changes. Therefore, when the blood flow changes, the amount of detected light also changes with time.
  • Oxygenated hemoglobin and deoxygenated hemoglobin differ in the wavelength dependence of light absorption. When the wavelength is 650 nm or more and shorter than 805 nm, the light absorption coefficient of deoxygenated hemoglobin is greater than that of oxygenated hemoglobin. At a wavelength of 805 nm, the light absorption coefficient of deoxygenated hemoglobin and the light absorption coefficient of oxygenated hemoglobin are equal. When the wavelength is longer than 805 nm and 950 nm or less, the light absorption coefficient of oxygenated hemoglobin is greater than that of deoxygenated hemoglobin.
  • the first wavelength of the first optical pulse Ip1 is set to be 650 nm or more and shorter than 805 nm
  • the second wavelength of the second optical pulse Ip2 is set to be longer than 805 nm and 950 nm or less
  • the concentration of oxygenated hemoglobin contained in the blood inside the subject 11 and the deoxygenation of The concentration of hemoglobin can be determined. More detailed internal information of the subject 11 can be acquired by irradiating two light pulses having different wavelengths.
  • the light source 20 can be designed in consideration of the influence on the user's retina.
  • the light source 20 is a laser light source such as a laser diode, and can satisfy class 1 of laser safety standards established in various countries. If Class 1 is satisfied, the test area 11 is illuminated with light of such low intensity that the accessible emission limit (AEL) is below 1 mW. Note that the light source 20 itself does not need to satisfy Class 1.
  • a diffuser plate or neutral density filter may be placed in front of the light source 20 to diffuse or attenuate the light so that class 1 laser safety standards are met.
  • the first imaging device 30a acquires the position information of the living body 10 from the reflected light generated by the reflection of the environmental light from the living body 10 .
  • the first imaging device 30 a images the living body 10 to generate first image data, and sends the first image data to the processing device 50 .
  • the first image data does not need to be imaged data, and may be raw data of a plurality of pixel values of a plurality of pixels distributed two-dimensionally. A plurality of pixels and a plurality of pixel values correspond one-to-one.
  • the position information of the living body is reflected in the first image data.
  • An image based on the first image data is called a "first image".
  • the first imaging device 30a can follow the living body 10 while the living body 10 exists in the first field of view 12a.
  • the first imaging device 30a can be, for example, a monochrome camera or an RGB camera.
  • the second imaging device 30b acquires biometric information of the test site 11 of the living body 10 from the reflected light pulse generated by the light pulse IP being reflected by the test site 11 of the living body 10 .
  • the second imaging device 30 b images the subject 11 of the living body 10 to generate second image data, and sends the second image data to the processing device 50 .
  • the second image data does not need to be imaged data, and may be raw data of a plurality of pixel values of a plurality of pixels distributed two-dimensionally. A plurality of pixels and a plurality of pixel values correspond one-to-one.
  • Biological information of the subject 11 of the living body 10 is reflected in the second image data.
  • An image based on the second image data is called a "second image".
  • the number of pixels of the test part 11 included in the second image can be made larger than the number of pixels of the test part 11 included in the first image. . Therefore, noise can be reduced by averaging multiple pixel values of multiple pixels in the second image, and the SN ratio of imaging can be improved.
  • the second imaging device 30b can have a plurality of pixels arranged two-dimensionally on the imaging surface. Each pixel may comprise a photoelectric conversion element, eg a photodiode, and one or more charge storages.
  • the second imaging device 30b can be any image sensor, such as a CCD image sensor or a CMOS image sensor, for example. The details of the configuration of the second imaging device 30b will be described later.
  • the second imaging device 30b detects at least a part of the rise period component of the reflected light pulse generated by the light pulse Ip being reflected by the subject 11, and outputs a signal corresponding to the intensity thereof. Surface information of the test portion 11 is reflected in the signal. Alternatively, the second imaging device 30b detects at least a part of the falling period component of the reflected light pulse generated by the light pulse Ip being reflected by the test area 11, and outputs a signal corresponding to the intensity thereof. . Internal information of the subject 11 is reflected in the signal.
  • the “rising period” of the reflected light pulse refers to the period from when the intensity of the reflected light pulse starts increasing to when it ends increasing on the imaging surface of the second imaging device 30b.
  • the “falling period” of the reflected light pulse refers to the period from when the intensity of the reflected light pulse starts decreasing to when it ends decreasing on the imaging surface of the second imaging device 30b. More precisely, the “rising period” means the period from when the intensity of the reflected light pulse exceeds a preset lower limit to when it reaches a preset upper limit.
  • the “falling period” means a period from when the intensity of the reflected light pulse falls below a preset upper limit to when it reaches a preset lower limit.
  • the upper limit value can be set to a value that is, for example, 90% of the peak value of the intensity of the reflected light pulse
  • the lower limit value can be set to a value that is, for example, 10% of the peak value.
  • the second imaging device 30b can be equipped with an electronic shutter.
  • the electronic shutter is a circuit that controls imaging timing.
  • the electronic shutter controls one signal accumulation period during which the received light is converted into an effective electrical signal and accumulated, and a period during which the signal accumulation is stopped.
  • the signal accumulation period is also called an "exposure period”.
  • the width of the exposure period is also called “shutter width”.
  • the time from the end of one exposure period to the start of the next exposure period is also called a "non-exposure period”.
  • the second imaging device 30b can adjust the exposure period and the non-exposure period in the range of sub-nanoseconds, eg, 30 ps to 1 ns, using the electronic shutter.
  • a conventional TOF (Time-of-Flight) camera whose purpose is to measure distance detects all of the light that is emitted from the light source 20, reflected by the subject, and returned.
  • Conventional TOF cameras require the shutter width to be greater than the light pulse width.
  • the shutter width need not be greater than the pulse width of the reflected light pulse.
  • the shutter width can be set to a value of 1 ns or more and 30 ns or less, for example. According to the imaging system 100 of this embodiment, the shutter width can be reduced, so that the influence of dark current contained in the detection signal can be reduced.
  • the motorized device 40 supports the imaging device 30 and can change the orientation of the imaging device 30 by panning and/or tilting rotation by a motor.
  • a pan rotation can move the field of view of the imaging device 30 in the horizontal direction
  • a tilt rotation can move the field of view of the imaging device in the vertical direction.
  • the operation of changing the orientation of the imaging device 30 by pan rotation is called “pan correction”
  • tilt correction the operation of changing the orientation of the imaging device 30 by tilt rotation
  • the electric device 40 changes the orientation of the imaging device 30 in response to the signal from the processing device 50, following the movement of the living body 10 in the first image.
  • the living body 10 is included in the first visual field 12a and the subject 11 of the living body 10 is included in the second visual field 12b even after the living body 10 moves vertically and horizontally. can be maintained.
  • the electric device 40 is obtained by, for example, synchronously changing the orientation of the first imaging device 30a and the orientation of the second imaging device 30b. In this case, the relative positional relationship between the first field of view 12a and the second field of view 12b does not depend on the orientation of the imaging device 30.
  • the electric device 40 may change the orientation of the second imaging device 30b without changing the orientation of the first imaging device 30a.
  • the electric device 40 includes at least one motor selected from the group consisting of, for example, a DC motor, a brushless DC motor, a PM motor, a stepping motor, an induction motor, a servo motor, an ultrasonic motor, an AC motor, and an in-wheel motor. obtain.
  • the electric device 40 may include a pan rotation motor and a tilt rotation motor separately.
  • the electric device 40 may rotate the imaging device 30 in the roll direction with a motor.
  • Roll direction means the direction about the axis of rotation perpendicular to the axis of rotation for pan rotation and the axis of rotation for tilt rotation.
  • the subject 11 of the living body 10 is included in the second field of view 12b. can be maintained.
  • a detailed configuration of the electric device 40 will be described later.
  • a control circuit 52 included in the processing device 50 controls the operation of the light source 20 , the imaging device 30 and the signal processing circuit 54 .
  • the control circuit 52 adjusts the time difference between the emission timing of the light pulse Ip from the light source 20 and the shutter timing of the second imaging device 30b.
  • the time difference is also called "phase difference”.
  • the “emission timing” of the light source 20 is the timing at which the light pulse emitted from the light source 20 starts rising.
  • “Shutter timing” is the timing to start exposure.
  • the control circuit 52 may adjust the phase difference by changing the emission timing, or may adjust the phase difference by changing the shutter timing.
  • the control circuit 52 may be configured to remove the offset component from the signal detected by each pixel of the second imaging device 30b.
  • the offset component is a signal component due to environmental light such as sunlight or illumination light, or disturbance light.
  • a signal processing circuit 54 included in the processing device 50 generates and outputs data indicating the position information of the living body 10 based on the first image data. From the data, the positions of the living body 10 and its test portion 11 in the first image can be specified. The signal processing circuit 54 generates and outputs data indicating biological information of the subject 11 of the living body 10 based on the second image data. Surface information and/or internal information of the subject 11 is reflected in the data. A method for calculating the amount of change from the initial value of each concentration of HbO 2 and Hb in the blood of the brain as internal information will be described later in detail.
  • the signal processing circuit 54 can estimate the psychological state and/or physical state of the living body 10 based on the surface information and/or internal information of the subject 11 .
  • the signal processing circuit 54 may generate and output data indicating the psychological state and/or physical state of the living body 10 .
  • a psychological state can be, for example, a mood, an emotion, a state of health, or a temperature sensation.
  • Moods can include, for example, moods such as pleasant or unpleasant.
  • Emotions may include, for example, feelings of relief, anxiety, sadness, or resentment.
  • a state of health may include, for example, a state of well-being or fatigue.
  • Temperature sensations may include, for example, sensations of hot, cold, or muggy.
  • the psychological state may also include indexes representing the degree of brain activity, such as interest, proficiency, proficiency, and concentration.
  • the physical condition can be, for example, the degree of fatigue, drowsiness, or drunkenness.
  • the control circuit 52 can be, for example, a combination processor and memory or an integrated circuit such as a microcontroller containing a processor and memory.
  • the control circuit 52 executes a computer program recorded in the memory 56 by the processor, for example, to adjust the emission timing and the shutter timing, and cause the signal processing circuit 54 to perform signal processing.
  • the signal processing circuit 54 includes, for example, a digital signal processor (DSP), a programmable logic device (PLD) such as a field programmable gate array (FPGA), or a central processing unit (CPU) or image processing arithmetic processor (GPU) and a computer program. It can be realized by a combination of The signal processing circuit 54 executes signal processing by the processor executing a computer program recorded in the memory 56 .
  • DSP digital signal processor
  • PLD programmable logic device
  • FPGA field programmable gate array
  • CPU central processing unit
  • GPU image processing arithmetic processor
  • the signal processing circuit 54 and the control circuit 52 may be one integrated circuit or separate individual circuits. At least one of the signal processing circuitry 54, control circuitry 52, and memory 56 may be components of an external device, such as a remotely located server. In this case, an external device such as a server exchanges data with the rest of the components via wireless or wired communication.
  • an external device such as a server exchanges data with the rest of the components via wireless or wired communication.
  • control circuit 52 the operation of the control circuit 52 and the operation of the signal processing circuit 54 are collectively described as the operation of the processing device 50 .
  • the imaging system 100 includes a first imaging optical system that forms a two-dimensional image of the living body 10 on the imaging surface of the first imaging device 30a, and a two-dimensional image of the subject 11 on the imaging surface of the second imaging device 30b. and a second imaging optical system that forms a .
  • the optical axis of the first imaging optical system is substantially orthogonal to the imaging surface of the first imaging device 30a.
  • the optical axis of the second imaging optical system is substantially orthogonal to the imaging plane of the second imaging device 30b.
  • Each of the first and second imaging optical systems may include a zoom lens. When the focal length is changed using the zoom lens of the first optical system, the resolution of the two-dimensional image of the living body 10 captured by the first imaging device 30a changes.
  • the resolution of the two-dimensional image of the living body 10 captured by the second imaging device 30b changes. Therefore, even if the living body 10 is far away, it is possible to enlarge a desired measurement area and observe it in detail.
  • the imaging system 100 emits light in the wavelength band emitted from the light source 20, or light in the wavelength band emitted from the light source 20 and light in the vicinity of the wavelength band, between the subject 11 and the second imaging device 30b.
  • a band-pass filter can be constituted by a multilayer filter or an absorption filter, for example. Considering the temperature change of the light source 20 and the band shift due to oblique incidence on the filter, the bandwidth of the band-pass filter may have a width of about 20 nm or more and 100 nm or less.
  • the imaging system 100 When acquiring internal information, the imaging system 100 includes a first polarizing plate between the test part 11 and the light source 20, and a second polarizing plate between the test part 11 and the second imaging device 30b. good too.
  • the polarization direction of the first polarizing plate and the polarization direction of the second polarizing plate may have a crossed Nicols relationship.
  • FIG. 2 is a flow chart schematically showing an example of correction operation performed by the processing device 50 when the living body 10 moves.
  • the processing device 50 executes the operations of steps S101 to S108 shown in FIG. 3A and 3B are diagrams for explaining the operation of the electric device 40.
  • FIG. 3A and 3B are diagrams for explaining the operation of the electric device 40.
  • step S101 the processing device 50 causes the first imaging device 30a to image the living body 10 to generate and output first image data.
  • the first image shows an object existing inside the first field of view 12a.
  • the first image includes the face of living body 10 .
  • step S102 the processing device 50 extracts the face of the living body 10 from the first image by machine learning processing based on the first image data, and calculates the amount of deviation between the center of the extracted face and the center of the first image. do.
  • the processing device 50 has a cascade classifier trained on human faces.
  • the classifier reads the first image data, encloses the face portion of the living body 10 in the first image with a rectangular frame, and outputs the coordinates of the frame in the first image.
  • the thick rectangular frame shown in FIG. 3A corresponds to the rectangular frame in the first image.
  • the white double-headed arrow shown in FIG. 3A represents the amount of deviation between the center of the face in the first field of view 12a and the center of the first field of view 12a, and the difference between the center of the face in the first image and the center of the first image. It corresponds to the amount of deviation.
  • step S103 the processing device 50 determines whether the amount of deviation between the center of the face in the first image and the center of the first image is equal to or less than a predetermined threshold.
  • the predetermined threshold may be, for example, 1/2 or less of the width of the face extracted by machine learning processing. If the amount of displacement is less than half the width of the face, the center of the first image can be included in the region of the extracted face, and the face can be placed approximately at the center of the first image. . If the determination in step S103 is No, the processing device 50 performs the operation of step S104. If the determination in step S103 is Yes, the processing device 50 performs the operation of step S106.
  • step S104 the processing device 50 estimates the amount of pan rotation and/or tilt rotation of the electric device based on the amount of deviation.
  • the rotation angle ⁇ to be corrected to some extent can be calculated based on the distance L and the amount of deviation between the center of the face in the first field of view 12a and the center of the first field of view 12a.
  • the distance L is the distance between the center of the imaging surface of the first imaging device 30a and the center of the first field of view 12a.
  • the amount of deviation between the center of the face in the first field of view 12a and the center of the first field of view 12a is associated with the number of pixels of the amount of deviation between the center of the face in the first image and the center of the first image and the actual distance. can be known by
  • the processing device 50 may estimate the amount of pan rotation and/or tilt rotation of the electric device as follows.
  • the processing device 50 determines the focal length f of the optical lens provided in the first imaging device 30a and the deviation between the center of the face in the first field of view 12a formed on the first imaging device 30a and the center of the first field of view 12a.
  • a rotation angle ⁇ to be corrected is calculated based on the amount h.
  • the amount of deviation h between the center of the face in the first field of view 12a formed on the first imaging device 30a and the center of the first field of view 12a is the deviation between the center of the face in the first image and the center of the first image. It can be known by associating the number of pixels of the quantity with the pixel size.
  • step S105 as shown in FIG. 3B, the processing device 50 pans and/or tilts the motorized device 40 by the estimated amount of rotation to change the orientation of the first imaging device 30a and the direction of the second imaging device 30b. Change orientation synchronously.
  • the processing device 50 repeats the operations from steps S101 to S105 until the amount of deviation falls below the threshold.
  • the processing device 50 causes the electric device 40 to synchronously change the orientation of the first imaging device 30a and the orientation of the second imaging device 30b, and then instructs the electric device 40 to reduce the amount of deviation. Synchronize and repeat the action of further changing the direction.
  • the center of the first image can be included within the extracted face region if the threshold is less than or equal to 1/2 the width of the face. Therefore, it can be said that the processing device 50 causes the electric device 40 to change the orientation of the first imaging device 30a so that the center of the first image is included in the facial region of the living body 10 .
  • the reason for repeatedly correcting the amount of deviation is due to various factors such as changes in occlusion caused by the three-dimensional shape of the subject 11, changes in motor torque, and divergence between the rotation axis of the motor and the optical axis of the imaging device. This is because the calculated rotation angle ⁇ may not be able to correct the deviation amount at once.
  • the subject 11 When the amount of deviation is equal to or less than the threshold, the subject 11 can be contained inside the second field of view 12b. Even in that case, since the size of the subject 11 is smaller than the size of the face of the living body 10, the following problems may occur. That is, even if the displacement amount is equal to or less than the threshold value, the position of the subject 11 in the second visual field 12b is different before and after the living body 10 moves, as shown in FIGS. 1A and 3B. may not be tracked accurately.
  • image processing-based tracking based on the second image data is performed.
  • image processing-based tracking is described below.
  • step S106 the processing device 50 causes the second imaging device 30b to image the subject 11 to generate and output second image data.
  • the second image shows an object existing inside the second field of view 12b.
  • the second image includes the forehead portion of the living body 10 .
  • Step S107 the processing device 50 corrects body motion of the living body 10 by image processing-based tracking based on the second image data.
  • Body motion correction by image processing-based tracking is a process of suppressing displacement of an image region corresponding to the subject 11 in the second image before and after movement of the living body 10 to a predetermined threshold or less.
  • the predetermined threshold may be 10 pixels, or 3 pixels, for example.
  • Such body motion correction makes it possible to more accurately acquire biological information of the subject 11 before and after the living body 10 moves.
  • tracking correction based on feature points of a 2D image such as the KLT algorithm
  • tracking correction by 3D matching based on an ICP algorithm that utilizes a 3D model based on ranging is applied.
  • three-dimensional rotation correction by three-dimensional affine transformation is also performed in addition to horizontal and vertical deviation correction.
  • ranging for example, the technology disclosed in International Publication No. 2021/145090 can be used.
  • International Publication No. 2021/145090 can be used.
  • the entire disclosure of Japanese Patent Application No. 2020-005761 is incorporated herein by reference.
  • the imaging system 100 by changing the orientation of the imaging device 30 with the electric device 40, at least part of the forehead can be included in the second field of view 12b.
  • the forehead By including the forehead, light pulses emitted from the light source 20 can irradiate the brain through the forehead, and cerebral blood flow information can be obtained from reflected pulsed light generated by the light irradiation.
  • the eyebrows may be included in the second field of view.
  • the edges of the eyebrows can be used as feature points during tracking correction, and the accuracy of tracking correction based on feature points of 2D images or 3D matching can be improved. become.
  • the nose may be included in the second field of view by changing the orientation of the imaging device 30 with the motorized device 40 . By including the nose, it is possible to increase the variation in unevenness of feature points in three-dimensional matching, and to improve the accuracy of tracking correction.
  • the processing device 50 determines a pixel region corresponding to the part to be inspected in the second image based on the result of correcting the ten body movements of the living body.
  • the pixel region corresponds to the pixel region of the portion corresponding to the subject 11 in the second image before the living body 10 moves.
  • both pixel regions match means that the positional deviation between the two pixel regions is 10 pixels or less.
  • the processing device 50 generates and outputs data indicating biological information of the subject 11 from the determined pixel region.
  • the amount of deviation between the center of the face and the center of the first image is used.
  • a position other than the center of the face may be set as the specific position of the face
  • a position other than the center of the first image may be set as the specific position of the first image
  • the amount of deviation may be defined by the specific position of the face and the specific position of the first image.
  • a specific location on the face can be, for example, the location of the eyes or the nose.
  • the specific positions of the first image are, for example, two virtual vertical lines that divide the first image into three equal parts in the horizontal direction, and two virtual horizontal lines that divide the first image into three equal parts in the vertical direction. can be any one of the four pixels each closest to the intersection of .
  • the specific position of the first image may be determined so as to compensate for the deviation between the center of the first field of view 12a and the center of the second field of view 12b. Such a shift in the center of the field of view may occur due to the different installation positions of the first imaging device 30a and the second imaging device 30b. Even if the center of the first image is aligned with the center of the face of the living body 10 due to the deviation of the center of the field of view, the part to be inspected 11 may protrude from the second field of view 12b, and the measurement accuracy may decrease.
  • the shift amount of the center of the field of view can be estimated by pre-calibration.
  • the shift amount of the center of the field of view can be estimated by, for example, the following method. The method obtains a first image and a second image by photographing the same object by the first imaging device 30a and the second imaging device 30b, respectively, and extracts the object in the first image and the second image. It is to compare the coordinates of the positions. Another position shifted from the center of the first image based on the estimated deviation amount of the center of the field of view is determined as the specific position of the first image.
  • the displacement of the center of the field of view can be compensated for and the subject 11 can be placed inside the second field of view 12b. Furthermore, the center of the subject 11 can be aligned with the center of the second image.
  • the second image data is generated and output when the amount of deviation between the center of the face and the center of the first image is equal to or less than the threshold.
  • the generation and output of the second image data may be performed at any timing regardless of whether the amount of deviation is equal to or less than the threshold.
  • the processing device 50 causes the electric device 40 to synchronously change the orientations of the first imaging device 30a and the second imaging device 30b.
  • the processing device 50 may cause the electric device 40 to change the orientation of the second imaging device 30b without changing the orientation of the first imaging device 30a.
  • the processing device 50 may calculate a movement vector of the living body 10 from position information before and after movement of the living body 10 based on the first image data, and change the orientation of the second imaging device 30b by the movement vector. .
  • FIG. 4A is a perspective view schematically showing a first example of the electric device 40 that supports the imaging device 30.
  • the electric device 40 shown in FIG. 4A supports the first imaging device 30a and the second imaging device 30b, and synchronously changes the orientation of the first imaging device 30a and the orientation of the second imaging device 30b.
  • a light source 20 is attached to the second imaging device 30b.
  • the electric device 40 shown in FIG. 4A includes a first electric mechanism 42a and a second electric mechanism 42b for performing pan correction and tilt correction on the imaging device 30, respectively.
  • the first imaging device 30a has a first lens 32a with a relatively wide field of view
  • the second imaging device 30b has a second lens 32b with a relatively narrow field of view.
  • a first field of view 12a and a second field of view 12b shown in FIG. 1A are defined by a first lens 32a and a second lens 32b, respectively.
  • the first imaging device 30a and the second imaging device 30b are arranged such that the first lens 32a and the second lens 32b are close to each other.
  • Such an arrangement allows the center of the field of view of the first lens 32a and the center of the field of view of the second lens 32b to be close to each other.
  • the center position of the face in the first image can be corrected, and at the same time, the center position of the subject 11 in the second image can also be corrected. can.
  • the distance between the optical axes of the first lens 32a and the second lens 32b can be, for example, 80 mm or less.
  • the center of the first lens 32a or the center of the second lens 32b is used as a reference, and the center of the first visual field 12a
  • the deviation angle from the center of the second field of view 12b can be suppressed to 10° or less.
  • the distance between the optical axes of the first lens 32a and the second lens 32b is, for example, 40 mm or less, the deviation angle can be suppressed to 5° or less.
  • the distance between the optical axes of the first lens 32a and the second lens 32b is, for example, 20 mm or less, the deviation angle can be suppressed to 3° or less.
  • FIG. 4B is a perspective view schematically showing a second example of the electric device 40 that supports the imaging device 30.
  • the electric device 40 shown in FIG. 4B includes a first electric mechanism 42a and a second electric mechanism 42b for performing pan correction and tilt correction, respectively, on the first imaging device 30a.
  • the electric device 40 shown in FIG. 4B further includes a third electric mechanism 42c and a fourth electric mechanism 42d for performing pan correction and tilt correction, respectively, on the second imaging device 30b.
  • the electric device 40 shown in FIG. 4B can individually change the orientation of the first imaging device 30a and the orientation of the second imaging device 30b. Therefore, it is possible to change the orientation of the second imaging device 30b without changing the orientation of the first imaging device 30a.
  • the optical axis of the first lens 32a is brought closer to the rotation shafts of the first motorized mechanism 42a and the second motorized mechanism 42b, and the optical axis of the second lens 32b is moved closer to the third motorized mechanism 42c. and the fourth electric mechanism 42d can be designed to be close to the rotating shaft.
  • FIG. 4C is a perspective view schematically showing a third example of the electric device 40 that supports the imaging device 30.
  • the electric device 40 shown in FIG. 4C has an arm structure capable of changing the orientation of the imaging device 30 in six axial directions.
  • the six axial directions include the front-back direction, the up-down direction, the left-right direction, the pan direction, the tilt direction, and the roll direction.
  • the electric device 40 shown in FIG. 4C makes it possible to more accurately correct the positional relationship between the second imaging device 30b and the part 11 to be inspected.
  • the second imaging device 30b can also be moved in the distance direction, and even when the part 11 to be inspected approaches the second imaging device 30b or moves away from the second imaging device 30b, the second imaging device 30b and the subject can move. It becomes possible to keep the distance from the detection unit 11 constant. As a result, even if the living body 10 moves with a higher degree of freedom, it is possible to stably acquire the biological information of the subject 11 .
  • FIG. 5 is a diagram schematically showing an example of imaging the living body 10 by the imaging system according to the modified example of this embodiment.
  • the imaging system 110 shown in FIG. 5 includes a display 60 in addition to the configuration of the imaging system 100 shown in FIG. 1A.
  • the configuration other than the imaging device 30 is omitted.
  • the living body 10 views the display 60 from the front, right side, or left side of the display 60 as viewed from the living body 10 .
  • the display 60 is arranged near the imaging device 30 but not between the living body 10 and the imaging device 30 .
  • Nearby means that the minimum distance between the first imaging device 30a and the second imaging device 30b, whichever is closer to the display 60, and the display 60 is 50 cm or less.
  • the imaging device 30 is behind the display 60 and at a position higher than the display 60 .
  • the imaging device 30 can be arranged, for example, on the top, bottom, left, or right of the display 60 when viewed from the living body 10 .
  • the display 60 can be, for example, a desktop PC monitor, a notebook PC monitor, or a test equipment monitor.
  • pan correction and/or tilt correction are performed on the imaging device 30 regardless of which direction the living body 10 is viewing the display 60, so that the imaging device 30 is always facing the living body 10.
  • the biological information of the subject 11 can be obtained.
  • the angle formed by the optical axis of the second imaging device 30b and the face of the subject 11 is always kept constant.
  • the incident intensity when the light pulse emitted from the light source 20 is incident on the face of the subject 11 depends on the incident angle. Therefore, keeping the angle between the optical axis of the second imaging device 30b and the face of the subject 11 constant is effective for stably acquiring biological information of the subject 11 .
  • a function of detecting the orientation of the face of the living body 10 may be added to the imaging device 30 .
  • the orientation of the face of the living body 10 is the orientation of the face with respect to the imaging device 30 or the display 60 .
  • the processing device 50 detects the orientation of the face based on the first image data and/or the second image data, and when the face of the living body 10 faces the imaging device 30 or the display 60, the subject 11 biometric information may be generated and output.
  • the processing device 50 may further utilize the generated biological information, for example, to estimate the psychological state and/or physical state of the living body 10 . That is, the processing device 50 may determine whether or not to generate and output biometric information, or utilize the biometric information, based on the detected orientation of the face.
  • the processing device 50 may limit generation and output of biometric information when the amount of deviation between the specific position of the face of the living body 10 and the specific position of the first image exceeds a certain threshold.
  • a restriction makes it possible to exclude noise data different from data of biological information desired to be obtained when the living body 10 looks away or leaves the seat.
  • a method of detecting the orientation of the face for example, a method of estimating the orientation of the face by landmark detection that detects feature points such as the eyes, nose, mouth, and outline of the face, or a method of estimating the orientation of the face from three-dimensional data of the face. A method of estimating may be used.
  • Example 10 Next, an example of the imaging system 100 according to this embodiment will be described together with a comparative example.
  • the cerebral blood flow information of the subject 11 after movement was acquired.
  • the cerebral blood flow information of the subject 11 after movement was acquired with the orientation of the imaging device 30 fixed.
  • a phantom model imitating a human head as the living body 10 was irradiated with near-infrared light pulses.
  • the absorption and scattering coefficients of the phantom model are equal to the absorption and scattering coefficients of the human head, respectively.
  • the imaging system 100 was moved by the drive stage to change the relative positions of the imaging device 30 and the phantom model.
  • the drive stage can move imaging system 100 in the X and/or Y directions.
  • the X and Y directions are the horizontal and vertical directions of the first image, respectively.
  • the amount of movement of the living body 10 was ⁇ 10 mm, ⁇ 20 mm, ⁇ 30 mm, ⁇ 60 mm, and ⁇ 90 mm in the X direction, and ⁇ 10 mm, ⁇ 20 mm, and ⁇ 30 mm in the Y direction. Although the amount of movement of the living body 10 may be larger, the movement of the living body 10 is set so that the embodiment in which the imaging apparatus 30 is pan-corrected and/or tilt-corrected can be compared with the comparative example in which such corrections are not made. The amount was set to a range in which the test area 11 is included in the second visual field 12b. The orientation of the first imaging device 30a and the orientation of the second imaging device 30b were synchronously changed by the motorized device 40 shown in FIG. 4A.
  • FIG. 6A is a diagram showing a comparative example in which cerebral blood flow information of the subject 11 after movement is acquired with the orientation of the imaging device 30 fixed.
  • FIG. 6B is a diagram showing an example in which cerebral blood flow information of the subject 11 after movement is acquired after changing the orientation of the imaging device 30 according to the movement of the living body 10 .
  • the horizontal axis represents the movement amount (mm) of the living body 10
  • the vertical axis represents the signal change amount from the initial value obtained from the second image data.
  • “base” on the horizontal axis represents the initial state before movement.
  • the number of measurements was 3 in the comparative example and 7 in the example.
  • the size of the bars shown in FIGS. 6A and 6B represents the average absolute value of the measured signal variation.
  • the error bar represents the range from the minimum to the maximum absolute value of the measured signal variation. Since there is no change in cerebral blood flow before and after the living body 10 moves, the signal change amount may be zero.
  • Factors that caused the signal value to fluctuate include an increase in tracking correction error due to three-dimensional matching and an increase in the illuminance distribution error of the illuminance light pulse in the ROI due to the large movement.
  • the absolute value of the signal change amount was small overall, and decreased to about 1/4 to 1/2 compared to the comparative example shown in FIG. 6A. Even when the movement amount of the living body 10 was 90 mm, a significant improvement was seen from the comparative example shown in FIG. 6A.
  • the imaging system 100 has the following effects. Not only can the subject 11 of the living body 10 after movement be included in the second field of view 12b, but also the accuracy of tracking correction by 3D matching can be improved, and the illuminance distribution error of the illuminance light pulse can be reduced. As a result, biometric information can be stably acquired even if the living body 10 moves.
  • the imaging device 30 is pan-corrected and/or tilt-corrected so that it can follow the living body 10 moving in the X direction and/or the Y direction. If the imaging device 30 is further corrected so that it can follow the living body 10 moving in the Z direction perpendicular to the X and Y directions as well, it is possible to obtain biological information more stably. .
  • This matter includes the configuration of the second imaging device 30b, the operation of emitting the first light pulse Ip1 and the second light pulse Ip2 , the method of detecting the internal scattering component I2 , and the concentration of HbO2 and Hb in blood. This is calculation of the amount of change from the initial value.
  • FIG. 7 is a diagram showing an example of the configuration of the second imaging device 30b.
  • Pixel 201 includes one photodiode, not shown. Although eight pixels arranged in two rows and four columns are shown in FIG. 7, more pixels may actually be arranged.
  • Each pixel 201 includes a first floating diffusion layer 204 and a second floating diffusion layer 206 .
  • the wavelength of the first optical pulse Ip1 is 650 nm or more and shorter than 805 nm
  • the wavelength of the second optical pulse Ip2 is longer than 805 nm and 950 nm or less.
  • the first floating diffusion layer 204 accumulates charges generated by receiving the first reflected light pulse from the first light pulse Ip1 .
  • the second floating diffusion layer 206 accumulates charges generated by receiving the second reflected light pulse from the second light pulse Ip2 .
  • the signals accumulated in the first floating diffusion layer 204 and the second floating diffusion layer 206 are treated as if they were two pixel signals of a general CMOS image sensor, and are output from the second imaging device 30b.
  • Each pixel 201 has two signal detection circuits.
  • Each signal detection circuit includes a source follower transistor 309 , a row select transistor 308 and a reset transistor 310 .
  • Each transistor is, for example, a field effect transistor formed on a semiconductor substrate, but is not limited to this.
  • one of the input and output terminals of source follower transistor 309 is connected to one of the input and output terminals of row select transistor 308 .
  • the one of the input and output terminals of source follower transistor 309 is typically the source.
  • the one of the input and output terminals of row select transistor 308 is typically the drain.
  • the gate which is the control terminal of the source follower transistor 309, is connected to the photodiode. Signal charges of holes or electrons generated by the photodiode are accumulated in a floating diffusion layer, which is a charge accumulation part between the photodiode and the source follower transistor 309 .
  • the first floating diffusion layer 204 and the second floating diffusion layer 206 are connected to photodiodes.
  • a switch may be provided between the photodiode and each of the first floating diffusion layer 204 and the second floating diffusion layer 206 . This switch switches the conduction state between the photodiode and each of the first floating diffusion layer 204 and the second floating diffusion layer 206 in response to the signal accumulation pulse from the processing device 50 . This controls the start and stop of signal charge accumulation in each of the first floating diffusion layer 204 and the second floating diffusion layer 206 .
  • the electronic shutter in this embodiment has a mechanism for such exposure control.
  • the signal charges accumulated in the first floating diffusion layer 204 and the second floating diffusion layer 206 are read out by turning on the gate of the row selection transistor 308 by the row selection circuit 302 .
  • the current flowing from the source follower power supply 305 to the source follower transistor 309 and the source follower load 306 is amplified according to the signal potential of the first floating diffusion layer 204 and the second floating diffusion layer 206 .
  • An analog signal based on this current read out from the vertical signal line 304 is converted into digital signal data by an analog-digital (AD) conversion circuit 307 connected for each column. This digital signal data is read column by column by the column selection circuit 303 and output from the second imaging device 30b.
  • AD analog-digital
  • the row selection circuit 302 and column selection circuit 303 After reading one row, the row selection circuit 302 and column selection circuit 303 read out the next row, and so on, to read the signal charge information of the floating diffusion layers of all the rows. After reading all the signal charges, the processing device 50 resets all the floating diffusion layers by turning on the gate of the reset transistor 310 . This completes imaging of one frame. Similarly, by repeating the high-speed imaging of the frames, the imaging of a series of frames by the second imaging device 30b is completed.
  • the second imaging device 30b may be another type of imaging device.
  • the second imaging device 30b may be, for example, a CCD type, a single photon counting device, or an amplified image sensor such as EMCCD or ICCD.
  • FIG. 8A is a diagram showing an example of the operation of emitting the first optical pulse Ip1 and the second optical pulse Ip2 .
  • the emission of the first optical pulse Ip1 and the emission of the second optical pulse Ip2 may be alternately switched multiple times. As a result, it is possible to reduce the time difference between acquisition timings of the detection images by the two kinds of wavelengths, and to use the first optical pulse Ip1 and the second optical pulse Ip2 almost simultaneously even when the subject 11 moves. imaging is possible.
  • FIG. 8B is a diagram showing another example of the operation of emitting the first optical pulse Ip1 and the second optical pulse Ip2 .
  • the emission of the first optical pulse Ip1 and the emission of the second optical pulse Ip2 may be switched for each frame.
  • detection of the reflected light pulse by the first light pulse Ip1 and detection of the reflected light pulse by the second light pulse Ip2 can be switched for each frame.
  • each pixel 201 may have a single charge reservoir. With such a configuration, the number of charge storage units in each pixel 201 can be reduced, so the size of each pixel 201 can be increased, and the sensitivity can be improved.
  • FIG. 9A is a diagram schematically showing an example of temporal changes of the surface reflection component I1 and the internal scattering component I2 included in the reflected light pulse when the light pulse Ip has an impulse waveform.
  • FIG. 9B is a diagram schematically showing an example of temporal changes of the surface reflection component I1 and the internal scattering component I2 included in the reflected light pulse when the light pulse Ip has a rectangular waveform.
  • the diagram on the left side of each diagram shows an example of the waveform of the light pulse Ip emitted from the light source 20, and the diagram on the right side shows an example of the waveforms of the surface reflection component I1 and the internal scattering component I2 included in the reflected light pulse. show.
  • the surface reflection component I1 has a waveform similar to that of the light pulse Ip
  • the internal scattering component I2 is the surface reflection component I2. It has an impulse response waveform that lags behind component I1 . This is because the internal scattering component I 2 corresponds to a combination of light rays that have passed through various paths within the subject 11 .
  • the surface reflection component I1 has a waveform similar to that of the light pulse Ip
  • the internal scattering component I2 is It has a waveform in which a plurality of impulse response waveforms are superimposed.
  • the inventors confirmed that the superimposition of a plurality of impulse response waveforms can amplify the light amount of the internal scattering component I2 detected by the imaging device 30, compared to the case where the light pulse Ip has an impulse waveform.
  • the internally scattered component I2 can be effectively detected.
  • 9B represents an example of a shutter open period during which the electronic shutter of the imaging device 30 is opened. If the pulse width of the rectangular pulse is on the order of 1 ns to 10 ns, the light source 20 can be driven with a low voltage. Therefore, it is possible to reduce the size and cost of the imaging system 100 in this embodiment.
  • Patent Literature 2 discloses an example of such a streak camera. These streak cameras use ultrashort light pulses with femtosecond or picosecond pulse widths to measure at the desired spatial resolution.
  • the surface reflection component I1 and the internal scattering component I2 can be detected separately. Therefore, the light pulse emitted from the light source 20 does not have to be an ultrashort light pulse, and the pulse width can be arbitrarily selected.
  • the amount of light of the internal scattering component I2 is extremely small, which is approximately one to several ten thousandths of the amount of light of the surface reflection component I1 . can be small. Furthermore, considering the laser safety standards, the amount of light that can be emitted is extremely small. Therefore, detection of the internal scatter component I2 becomes very difficult. Even in such a case, if the light source 20 emits a light pulse Ip with a relatively large pulse width, it is possible to increase the integrated amount of the internal scattering component I2 with a time delay. As a result, the amount of detected light can be increased and the SN ratio can be improved.
  • the light source 20 can emit a light pulse Ip with a pulse width of 3 ns or more, for example.
  • the light source 20 may emit a light pulse Ip with a pulse width of 5 ns or more, or 10 ns or more.
  • the light source 20 can emit an optical pulse Ip with a pulse width of 50 ns or less, for example.
  • the light source 20 may emit an optical pulse Ip with a pulse width of 30 ns or less, or even 20 ns or less. If the pulse width of the rectangular pulse is several ns to several tens of ns, the light source 20 can be driven at a low voltage. Therefore, it is possible to reduce the cost of the imaging system 100 in this embodiment.
  • the irradiation pattern of the light source 20 may be, for example, a pattern having a uniform intensity distribution within the irradiation area.
  • the imaging system 100 according to the present embodiment differs from the conventional apparatus disclosed in Patent Document 1, for example.
  • the detector and the light source are separated by about 3 cm, and the surface reflection component is spatially separated from the internal scattering component, so the irradiation pattern should have a discrete intensity distribution. I can't help it.
  • the surface reflection component I1 can be temporally separated from the internal scattering component I2 and reduced. Therefore, the light source 20 having an irradiation pattern having a uniform intensity distribution can be used.
  • An irradiation pattern having a uniform intensity distribution may be formed by diffusing the light emitted from the light source 20 with a diffusion plate.
  • the internal scattering component I2 can be detected even just below the irradiation point of the subject 11 .
  • the internal scattering component I2 can be detected even just below the irradiation point of the subject 11 .
  • FIG. 9C is a flowchart outlining the operation of the processing device 50 regarding the first light source 20a, the second light source 20b, and the second imaging device 30b.
  • the processing device 50 causes the second imaging device 30b to detect at least part of the fall period components of each of the first and second reflected light pulses by performing the operation schematically shown in FIG. 9C.
  • step S201 the processing device 50 causes the first light source 20a to emit the first light pulse Ip1 for a predetermined time. At this time, the electronic shutter of the second imaging device 30b is in a state of stopping exposure. The processing device 50 causes the electronic shutter to stop exposure until the surface reflection component I1 of the first reflected light pulse reaches the second imaging device 30b.
  • step S202 the processing device 50 causes the electronic shutter to start exposure at the timing when the internal scattering component I2 of the first reflected light pulse reaches the second imaging device 30b.
  • step S203 the processing device 50 causes the electronic shutter to stop exposure after a predetermined time has elapsed.
  • Signal charges are accumulated in the first floating diffusion layer 204 shown in FIG. 7 by steps S102 and S103.
  • the signal charges are called "first signal charges”.
  • step S204 the processing device 50 causes the second light source 20b to emit the second light pulse Ip2 for a predetermined time. At this time, the electronic shutter of the second imaging device 30b is in a state of stopping exposure. The processing device 50 causes the electronic shutter to stop exposure until the surface reflection component I1 of the second reflected light pulse reaches the second imaging device 30b.
  • step S205 the processing device 50 causes the electronic shutter to start exposure at the timing when the internal scattering component I2 of the second reflected light pulse reaches the second imaging device 30b.
  • step S206 the processing device 50 causes the electronic shutter to stop exposure after a predetermined time has elapsed.
  • steps S105 and S106 signal charges are accumulated in the second floating diffusion layer 206 shown in FIG. The signal charges are called "second signal charges”.
  • step S207 the processing device 50 determines whether or not the number of times the above signal accumulation has been performed has reached a predetermined number. If the determination in step S207 is No, the processing device 50 repeats steps S201 to S206 until it determines Yes. If the determination in step S207 is Yes, the processing device 50 performs the operation of step S208.
  • step S208 the processing device 50 causes the second imaging device 30b to generate and output a first signal based on the first signal charge, and the processing device 50 causes the second imaging device 30b to output the second signal A second signal is generated and output based on the charge. Internal information of the subject 11 is reflected in the first signal and the second signal.
  • the processing device 50 performs a first operation of causing the first light source 20a to emit the first light pulse Ip1 and causing the second imaging device 30b to detect at least part of the fall period of the first reflected light pulse. .
  • the processing device 50 causes the second light source 20b to emit the second light pulse Ip2 , and performs a second operation of causing the second imaging device 30b to detect at least part of the falling period of the second reflected light pulse.
  • the processing device 50 repeats a series of operations including the first operation and the second operation a predetermined number of times. Alternatively, the processing device 50 may repeat the first action a predetermined number of times, and then repeat the second action a predetermined number of times. The first action and the second action may be interchanged.
  • the operation shown in FIG. 9C allows the internal scattering component I2 to be detected with high sensitivity.
  • the attenuation rate of light inside is very large.
  • the emitted light can be attenuated to about 1/1,000,000 of the incident light. Therefore, in order to detect the internal scattering component I2 , the amount of light may be insufficient with one pulse irradiation. In the case of irradiation in class 1 of laser safety standards, the amount of light is particularly weak.
  • the light source 20 emits light pulses a plurality of times
  • the second imaging device 30b also exposes a plurality of times by means of the electronic shutter in response to this, so that detection signals can be integrated and sensitivity can be improved.
  • the multiple times of light emission and exposure are not essential, and are performed as necessary.
  • the second imaging device 30b by causing the second imaging device 30b to detect at least part of the rising period of each of the first and second reflected light pulses, the surface of each of the first and second reflected light pulses Reflection component I1 can be detected, making it possible to obtain surface information such as blood flow on the face and scalp.
  • two pixels 201 adjacent to each other in the row direction shown in FIG. 7 may be treated as one pixel.
  • the first floating diffusion layer 204 and the second floating diffusion layer 206 included in one pixel 201 respectively receive at least part of the fall period components of the first and second reflected light pulses.
  • the charge generated can be accumulated.
  • the first floating diffusion layer 204 and the second floating diffusion layer 206 included in the other pixel 201 receive the charge generated by receiving at least part of the rising period component of the first and second reflected light pulses, respectively. can accumulate. With such a configuration, internal information and surface information of the living body 10 can be obtained.
  • Equations (1) and (2) below represent examples of simultaneous equations.
  • ⁇ HbO 2 and ⁇ Hb represent the amount of change from the initial values of the concentrations of HbO 2 and Hb in blood, respectively.
  • ⁇ 750 OXY and ⁇ 750 deOXY represent the molar extinction coefficients of HbO 2 and Hb at a wavelength of 750 nm, respectively.
  • ⁇ 850 OXY and ⁇ 850 deOXY represent the molar extinction coefficients of HbO 2 and Hb at 850 nm wavelength, respectively.
  • I 750 ini and I 750 now represent the detected intensity at a wavelength of 750 nm at a reference time (initial time) and a certain time, respectively.
  • I 850 ini and I 850 now represent the detected intensity at a wavelength of 850 nm at a reference time (initial time) and a certain time, respectively. These symbols represent, for example, the detection strength in the non-activated state and the activated state of the brain.
  • the processing of S102 to S106 shown in FIG. 2 may be the processing of S102' to S106' shown below. These processes will be described with reference to FIG. 3C for explaining the displacement amount Q1 and the displacement amount Q2 in the first image, and FIG. 3D for explaining the first rotation amount and the second rotation amount.
  • the processing device 50 extracts the face region 112 including the face of the living body 10 from the first image 112a by machine learning processing, and extracts the center O112 of the face region and the center O112a of the first image 112a. is calculated.
  • the amount of deviation includes the amount of deviation Q1, which is the amount of deviation in the horizontal direction, and the amount of deviation Q2 in the vertical direction (see FIG. 3C).
  • the processing device 50 includes a cascade classifier (not shown) trained on human faces.
  • the cascade classifier reads the first image data, and obtains information specifying the face region 112 including the face of the living body 10 in the first image 112a (for example, the two-dimensional coordinates of each of the four corners of the frame of the face region 112). Output.
  • the processing device 50 makes a first determination as to whether the amount of deviation Q1 is equal to or less than a first threshold and/or a second determination as to whether or not the amount of deviation Q2 is equal to or less than a second threshold.
  • the first threshold may be half the horizontal width Q3 of the face region 112, and the second threshold may be half the vertical width Q4 of the face region 112. FIG. If the first determination is Yes or the second determination is Yes, the processing device 50 performs the operation of step S106. If the first determination is No and the second determination is No, the processing device 50 performs the operation of step S104.
  • the processing device 50 determines a first amount of pan rotation in the electric device 40 and a second amount of tilt rotation in the electric device 40 .
  • Each of the first rotation amount and the second rotation amount is determined based on the three-dimensional coordinates (x1, y1, z1) of the first point corresponding to the center O112 of the face area 112 (see FIG. 3D).
  • the three-dimensional coordinates (x1, y1, z1) may be determined by providing a stereo camera system in the imaging device 30 and using a technique of distance measurement of the first point. .
  • the three-dimensional coordinates (x1, y1, z1) may be determined by equipping the first imaging device with a function of measuring the distance of the first point with a single eye.
  • the first three-dimensional coordinates are defined in a three-dimensional space including the first imaging device 30a and the living body 30.
  • the z-axis of the three-dimensional space is defined to overlap the optical axis of the first imaging device 30a, and the z-axis of the three-dimensional space is defined to perpendicularly intersect the first plane containing the first point. .
  • the origin of the three-dimensional space may be the focal point of the first imaging device 30a.
  • the first rotation amount may be determined using x1 and z1.
  • a second rotation amount may be determined using y1 and z1.
  • Step S105′ processing replacing step S105
  • the processor 50 causes the motorized device 40 to pan rotate the first amount of rotation, and the processor 50 causes the motorized device 40 to tilt rotate the second amount of rotation.
  • the orientation of the first imaging device 30a and the orientation of the second imaging device 30b change synchronously.
  • the angle in the x-axis direction formed by the optical axis of the first imaging device 30a and the optical axis of the second imaging device 30b, and the angle in the y-axis direction formed by the optical axis of the first imaging device 30a and the optical axis of the second imaging device 30b do not change due to the pan rotation of the electric device 40.
  • the imaging system according to the present disclosure is capable of acquiring biometric information of a subject part of a living body. Imaging systems in the present disclosure are useful, for example, for biosensing.

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Animal Behavior & Ethology (AREA)
  • Public Health (AREA)
  • Pathology (AREA)
  • Veterinary Medicine (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Biophysics (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Physiology (AREA)
  • Cardiology (AREA)
  • Hematology (AREA)
  • Neurology (AREA)
  • General Physics & Mathematics (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
  • Investigating Or Analysing Materials By Optical Means (AREA)
PCT/JP2022/035983 2021-11-05 2022-09-27 撮像システム、処理装置、および撮像システムにおいてコンピュータによって実行される方法 Ceased WO2023079862A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2023557890A JPWO2023079862A1 (https=) 2021-11-05 2022-09-27
CN202280067568.8A CN118076301A (zh) 2021-11-05 2022-09-27 摄像系统、处理装置及在摄像系统中由计算机执行的方法
US18/624,249 US20240260832A1 (en) 2021-11-05 2024-04-02 Imaging system, processing device, and method to be performed by computer in imaging system

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2021-181379 2021-11-05
JP2021181379 2021-11-05

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/624,249 Continuation US20240260832A1 (en) 2021-11-05 2024-04-02 Imaging system, processing device, and method to be performed by computer in imaging system

Publications (1)

Publication Number Publication Date
WO2023079862A1 true WO2023079862A1 (ja) 2023-05-11

Family

ID=86241382

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/035983 Ceased WO2023079862A1 (ja) 2021-11-05 2022-09-27 撮像システム、処理装置、および撮像システムにおいてコンピュータによって実行される方法

Country Status (4)

Country Link
US (1) US20240260832A1 (https=)
JP (1) JPWO2023079862A1 (https=)
CN (1) CN118076301A (https=)
WO (1) WO2023079862A1 (https=)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2005008567A1 (en) * 2003-07-18 2005-01-27 Yonsei University Apparatus and method for iris recognition from all direction of view
JP2006302276A (ja) * 1994-09-02 2006-11-02 Sarnoff Corp 自動化された非侵入的な虹彩認識システムおよびその方法
KR101070389B1 (ko) * 2010-12-30 2011-10-06 김용중 환자 상태 모니터링 시스템
JP2017144225A (ja) * 2016-02-17 2017-08-24 パナソニックＩｐマネジメント株式会社 生体情報検出装置
JP2017217119A (ja) * 2016-06-03 2017-12-14 株式会社ニデック 眼科装置、および眼科装置制御プログラム
JP2018534025A (ja) * 2015-10-06 2018-11-22 コーニンクレッカ フィリップス エヌ ヴェＫｏｎｉｎｋｌｉｊｋｅ Ｐｈｉｌｉｐｓ Ｎ．Ｖ． 生体のバイタルサイン関連情報を取得するデバイス、システム及び方法
WO2022085276A1 (ja) * 2020-10-20 2022-04-28 日本電気株式会社 情報処理システム、目状態測定システム、情報処理方法および非一時的なコンピュータ可読媒体

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140132729A1 (en) * 2012-11-15 2014-05-15 Cybernet Systems Corporation Method and apparatus for camera-based 3d flaw tracking system
WO2018174311A1 (ko) * 2017-03-22 2018-09-27 스노우 주식회사 얼굴 인식 카메라의 동적 컨텐츠를 제공하는 방법 및 시스템
JP7386440B2 (ja) * 2017-12-19 2023-11-27 パナソニックＩｐマネジメント株式会社 生体計測装置、生体計測装置の作動方法、及び判定装置

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006302276A (ja) * 1994-09-02 2006-11-02 Sarnoff Corp 自動化された非侵入的な虹彩認識システムおよびその方法
WO2005008567A1 (en) * 2003-07-18 2005-01-27 Yonsei University Apparatus and method for iris recognition from all direction of view
KR101070389B1 (ko) * 2010-12-30 2011-10-06 김용중 환자 상태 모니터링 시스템
JP2018534025A (ja) * 2015-10-06 2018-11-22 コーニンクレッカ フィリップス エヌ ヴェＫｏｎｉｎｋｌｉｊｋｅ Ｐｈｉｌｉｐｓ Ｎ．Ｖ． 生体のバイタルサイン関連情報を取得するデバイス、システム及び方法
JP2017144225A (ja) * 2016-02-17 2017-08-24 パナソニックＩｐマネジメント株式会社 生体情報検出装置
JP2017217119A (ja) * 2016-06-03 2017-12-14 株式会社ニデック 眼科装置、および眼科装置制御プログラム
WO2022085276A1 (ja) * 2020-10-20 2022-04-28 日本電気株式会社 情報処理システム、目状態測定システム、情報処理方法および非一時的なコンピュータ可読媒体

Also Published As

Publication number Publication date
US20240260832A1 (en) 2024-08-08
JPWO2023079862A1 (https=) 2023-05-11
CN118076301A (zh) 2024-05-24

Similar Documents

Publication Publication Date Title
JP6998529B2 (ja) 撮像装置
US12042260B2 (en) Biometric apparatus, biometric method, and non-transitory computer-readable storage medium
JP6431535B2 (ja) 歯牙表面撮像装置のためのビデオベースの自動取得
JP7811747B2 (ja) 生体計測装置、及び生体計測方法
JP2024028235A (ja) コンピュータにおける情報処理方法および情報処理システム
US20160139039A1 (en) Imaging system and imaging method
CN108209867A (zh) 摄像装置
CN116829057A (zh) 用于组织的多光谱3d成像和诊断的系统和装置及其方法
KR101742049B1 (ko) 적외선 마이봄샘 촬영장치 및 마이봄샘 촬영방법
CN107468209A (zh) 摄像装置
JP7617518B2 (ja) 計測装置および計測装置を制御する方法
JP7417867B2 (ja) 光計測装置
WO2023079862A1 (ja) 撮像システム、処理装置、および撮像システムにおいてコンピュータによって実行される方法
JP7142246B2 (ja) 生体計測装置、ヘッドマウントディスプレイ装置、および生体計測方法
WO2020137276A1 (ja) 撮像装置
KR20220162110A (ko) 3차원 피부 스캔 방법
WO2022138063A1 (ja) 生体計測装置、生体計測方法、及び、コンピュータプログラム
JP2005211329A (ja) 眼球運動測定装置
WO2023090188A1 (ja) 光検出システム、処理装置、光検出システムを制御する方法、およびプログラム
US11388372B2 (en) Biological state detecting apparatus and biological state detection method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22889686

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 202280067568.8

Country of ref document: CN

ENP Entry into the national phase

Ref document number: 2023557890

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22889686

Country of ref document: EP

Kind code of ref document: A1