JP7355331B2 - Scanning imaging device, control method thereof, image processing device, control method thereof, scanning imaging method, image processing method, program, and recording medium - Google Patents

Description

The present invention relates to a scanning imaging device, a control method thereof, an image processing device, a control method thereof, a scanning imaging method, an image processing method, a program, and a recording medium.

One of the imaging techniques is scanning imaging. Scanning imaging is a technique that collects data by sequentially irradiating multiple locations on a sample with a beam, and constructs an image of the sample from the collected data.

Optical coherence tomography (OCT) is known as a typical example of scanning imaging using light. OCT is a technology that can image a light scattering medium at a resolution of micrometer level or lower, and is applied to medical imaging, nondestructive testing, and the like. OCT is a technique based on low coherence interferometry and typically utilizes near-infrared light to ensure deep penetration of the light scattering medium into the sample.

For example, in ophthalmological image diagnosis, OCT devices are becoming more and more popular, and not only two-dimensional imaging but also three-dimensional imaging, structural analysis, and functional analysis have been put into practical use, and are widely used as powerful diagnostic tools. It has reached this point. Furthermore, in the field of ophthalmology, scanning imaging other than OCT, such as scanning laser ophthalmoscopy (SLO), is also used. Note that scanning imaging that uses light (electromagnetic waves) in a wavelength band other than near-infrared light or ultrasonic waves is also known.

There are various scanning modes used in OCT and SLO, but the so-called "Lissajous scan" for the purpose of motion artifact correction has attracted attention in recent years (for example, Patent Documents 1 to 4, (See Non-Patent Documents 1 and 2).

In a typical Lissajous scan, the measurement light is scanned at high speed to draw multiple loops (multiple cycles that intersect with each other) of a certain size, so the difference in data acquisition time from one cycle is effectively can be ignored. Furthermore, since alignment between cycles can be performed with reference to the intersecting regions of different cycles, it is possible to correct motion artifacts caused by sample movement. Focusing on these characteristics of the Lissajous scan, the ophthalmology field is trying to deal with motion artifacts caused by eye movements.

Japanese Patent Application Publication No. 2016-17915 JP2018-68578A JP 2018-140004 Publication JP 2018-140049 Publication

Yiwei Chen, Young-Joo Hong, Shuichi Makita, and Yoshiaki Yasuno, “Three-dimensional eye motion correction by Lissajous scan optical coherence tomography”, Biomedical Optics EXPRESS, Vol. 8, No. 3, 1 Mar 2017, PP. 1783- 1802 Yiwei Chen, Young-Joo Hong, Shuichi Makita, and Yoshiaki Yasuno, “Eye-motion-corrected optical coherence tomography angiography using Lissajous scanning”, Biomedical Optics EXPRESS, Vol. 9, No. 3, 1 Mar 2018, PP. 1111- 1129

One object of this invention is to improve the accuracy of motion artifact correction.

A scanning imaging device according to some aspects includes a scanning unit that collects data from a three-dimensional region of a sample, and an image that applies one or more imaging processes to the data to construct one or more three-dimensional image data. a projection processing unit that applies a projection in the depth direction to each of the one or more three-dimensional image data to construct a plurality of frontal image data; a first data shift amount calculation unit that calculates a data shift amount in a direction perpendicular to the data shift amount; and a correction unit that performs motion artifact correction of image data based on the data collected by the scanning unit based on the data shift amount. including.

In some aspects, the imaging processing unit applies a plurality of different imaging processes to the data collected by the scanning unit to construct a plurality of three-dimensional image data, and the projection processing unit applies the plurality of three-dimensional image data to the data collected by the scanning unit, and the projection processing unit The projection is applied to each of the plurality of three-dimensional image data to construct the plurality of frontal image data.

The imaging processing section constructs a single three-dimensional image data from the data collected by the scanning section, and the projection processing section applies a plurality of different projections to the single three-dimensional image data. and constructs the plurality of frontal image data.

A scanning imaging device according to some aspects includes a scanning unit that collects data by applying a scan according to a two-dimensional pattern including a series of cycles to a sample, and a scanning unit that applies a plurality of different imaging processes to the data to collect data. an imaging processing unit that constructs three-dimensional image data of the plurality of front images; a projection processing unit that applies depth direction projection to each of the plurality of three-dimensional image data to construct a plurality of front image data; a first data shift amount calculation section that calculates a data shift amount in a direction perpendicular to the depth direction based on image data; and image data based on the data collected by the scanning section based on the data shift amount. and a correction unit that performs motion artifact correction.

In some aspects, the first data shift amount calculation unit calculates a correlation coefficient based on the plurality of front image data, and calculates the data shift amount based on the correlation coefficient.

In some aspects, the first data shift amount calculation unit includes a dividing unit that divides each of the plurality of frontal image data into partial image data groups, and a division unit that divides each of the plurality of frontal image data into a partial image data group; and a correlation coefficient calculation unit that calculates the correlation coefficient based on a plurality of partial image data groups.

In some aspects, for the first front image data and the second front image data among the plurality of front image data, the number of partial image data included in the first partial image data group of the first front image data is , equal to the number of partial image data included in the second partial image data group of the second front image data and corresponding to one partial image data included in the first partial image data group, collected by the scanning unit. The partial data of the data is the same as the partial data of the data corresponding to any partial image data included in the second partial image data group.

In some aspects, the correlation coefficient calculation unit includes a first image data set including at least two of the plurality of partial image data of the plurality of front image data corresponding to first partial data of the data collected by the scanning unit. the first partial data and the second partial data based on the first group and the second group including at least two of the plurality of partial image data of the plurality of front image data corresponding to the second partial data; A corresponding correlation coefficient is calculated, and the first data shift amount calculation unit calculates a data shift amount corresponding to the first partial data and the second partial data based on the correlation coefficient.

In some aspects, the correction unit further includes a second data shift amount calculation unit that calculates the data shift amount in the depth direction based on any one or more of the plurality of three-dimensional image data, and the correction unit Motion artifact correction based on the data shift amount calculated by the second data shift amount calculation section is applied to the image data based on the data collected by the scanning section.

In some aspects, the scanning unit applies optical coherence tomography (OCT) scanning to the sample, and the plurality of three-dimensional image data constructed by the imaging processing unit include OCT intensity image data, OCT intensity image data, It includes any of angiography image data, composite image data based on OCT intensity image data and OCT angiography image data, and OCT polarization image data.

In some aspects, the scanning unit collects the data by applying a scan to the three-dimensional area that follows a two-dimensional pattern that includes a series of cycles.

In some aspects, the scanning unit includes a deflector capable of deflecting light in a first direction and a second direction that are different from each other, and repeats changes in the deflection direction along the first direction in a first period. The scanning is applied to the sample by repeating changes in the deflection direction along the second direction at a second period different from the first period.

In some embodiments, the deflector is controlled based on a preset scanning protocol based on a Lissajous function.

In some embodiments, the series of cycles intersect with each other.

In some embodiments, the two-dimensional pattern is a raster pattern that includes a series of parallel line scans.

A method according to some aspects is a method of controlling a scanning imaging device that includes a scanning unit that applies a scan to a sample to collect data, and a processor, the scanning unit including a series of cycles. controlling the processor to apply a scan according to a two-dimensional pattern to the sample to collect data; and controlling the processor to apply a plurality of different imaging processes to the data to construct a plurality of three-dimensional image data, applying the projection to each of the plurality of three-dimensional image data to construct a plurality of front image data, and calculating a data shift amount in a direction perpendicular to the depth direction based on the plurality of front image data, Based on the data shift amount, control is performed to perform motion artifact correction of image data based on the data collected by the scanning unit.

An image processing device according to some aspects includes: a storage unit that stores data collected by applying scanning according to a two-dimensional pattern including a series of cycles to a sample; and a storage unit that stores data collected by applying scanning according to a two-dimensional pattern including a series of cycles to the data; an imaging processing unit that constructs a plurality of three-dimensional image data by applying depth direction projection to each of the plurality of three-dimensional image data, and a projection processing unit that constructs a plurality of front image data by applying depth direction projection to each of the plurality of three-dimensional image data; a first data shift amount calculation unit that calculates a data shift amount in a direction perpendicular to the depth direction based on front image data of the image data, and an image based on the data collected by the scanning based on the data shift amount. and a correction unit that performs motion artifact correction on the data.

A method according to some aspects is a method for controlling an image processing device including a storage unit and a processor, the method comprising: applying a scan according to a two-dimensional pattern including a series of cycles to a sample to store data collected in the storage unit; and causing the processor to apply a plurality of different imaging processes to the data to construct a plurality of three-dimensional image data, and apply a depth direction projection to each of the plurality of three-dimensional image data. construct a plurality of frontal image data, calculate a data shift amount in a direction orthogonal to the depth direction based on the plurality of frontal image data, and calculate the amount of data shift in the direction perpendicular to the depth direction based on the data shift amount, controlling to perform motion artifact correction of image data based on the data;

A scanning imaging method according to some aspects includes applying a scan to a sample according to a two-dimensional pattern including a series of cycles to collect data, and applying a plurality of different imaging processes to the data to obtain a plurality of three-dimensional images. constructing image data, applying depth direction projection to each of the plurality of three-dimensional image data to construct a plurality of frontal image data, and orthogonal to the depth direction based on the plurality of frontal image data. A data shift amount in the direction is calculated, and based on the data shift amount, motion artifact correction of image data based on the data collected by the scanning is performed.

An image processing method according to some embodiments includes applying a scan according to a two-dimensional pattern to a sample including a series of cycles to prepare collected data, and applying a plurality of different imaging processes to the data to obtain a plurality of data. construct three-dimensional image data, apply depth direction projection to each of the plurality of three-dimensional image data to construct a plurality of front image data, and create a plurality of front image data in the depth direction based on the plurality of front image data. A data shift amount in an orthogonal direction is calculated, and based on the data shift amount, motion artifact correction of the image data based on the prepared data is performed.

Some embodiments are programs that cause a computer to execute any of the methods described above.

Some embodiments are computer-readable non-transitory recording media recorded with a program that causes a computer to execute any of the methods described above.

It is possible to at least partially combine any two or more aspects. Furthermore, any aspect of the present disclosure can be at least partially combined with any aspect. Further, it is possible to at least partially combine any two or more aspects of the present disclosure with at least partially.

According to exemplary aspects, it is possible to improve the accuracy of motion artifact correction.

FIG. 1 is a schematic diagram illustrating an example of the configuration of a scanning imaging device (ophthalmological device) according to an exemplary embodiment. FIG. 1 is a schematic diagram illustrating an example of the configuration of a scanning imaging device (ophthalmological device) according to an exemplary embodiment. FIG. 1 is a schematic diagram illustrating an example of the configuration of a scanning imaging device (ophthalmological device) according to an exemplary embodiment. FIG. 1 is a schematic diagram illustrating an example of the configuration of a scanning imaging device (ophthalmological device) according to an exemplary embodiment. FIG. 1 is a schematic diagram illustrating an example of the configuration of a scanning imaging device (ophthalmological device) according to an exemplary embodiment. FIG. 2 is a schematic diagram for explaining an example of a process executed by a scanning imaging device (ophthalmological device) according to an exemplary embodiment. FIG. 2 is a schematic diagram for explaining an example of a process executed by a scanning imaging device (ophthalmological device) according to an exemplary embodiment. FIG. 2 is a schematic diagram for explaining an example of a process executed by a scanning imaging device (ophthalmological device) according to an exemplary embodiment. 1 is a flowchart illustrating an example of the operation of a scanning imaging device (ophthalmological device) according to an exemplary embodiment. 1 is a flowchart illustrating an example of the operation of a scanning imaging device (ophthalmological device) according to an exemplary embodiment. 1 is a flowchart illustrating an example of the operation of a scanning imaging device (ophthalmological device) according to an exemplary embodiment. 1 is a flowchart illustrating an example of the operation of a scanning imaging device (ophthalmological device) according to an exemplary embodiment.

A scanning imaging device, a control method thereof, an image processing device, a control method thereof, a scanning imaging method, an image processing method, a program, and a recording medium according to some exemplary embodiments will be described with reference to the drawings. .

The techniques disclosed in the documents cited herein and any other known techniques may be incorporated into the exemplary embodiments. Furthermore, unless otherwise specified, there will be no distinction between "image data" and an "image" based on it, and no distinction will be made between a "part" of the eye to be examined and its "image."

In an exemplary embodiment, a scan is applied to the sample according to a two-dimensional pattern that includes a series of cycles. This two-dimensional pattern may typically be based on a preset scanning protocol based on the Lissajous function, for example, as described in any of Patent Documents 1 to 4 and Non-Patent Documents 1 and 2. It may be a scanning protocol utilized (or available) in the technology. In the description of the exemplary embodiments, the scan mode that can be employed will be referred to as a Lissajous scan. Hereinafter, as an example, processing of data collected by the Lissajous scan described in Non-Patent Document 1 will be described.

First, Fourier transformation or the like is applied to data collected by a Lissajous scan to construct three-dimensional image data defined by a predetermined three-dimensional Lissajous coordinate system. This three-dimensional Lissajous coordinate system is, for example, a three-dimensional orthogonal coordinate system (referred to as an αβγ coordinate system), in which the α axis represents the positions of a plurality of A lines constituting a cycle, and the β axis represents the cycle identification number (scan The γ-axis represents the depth position (position in the axial direction). Three-dimensional image data expressed in the αβγ coordinate system consists of a group of cross-sectional image data (each cross-section represents an αγ plane) corresponding to a cycle group constituting a Lissajous scan, arranged in the β direction according to the scan order.

Next, the three-dimensional image data (volume) defined in the αβγ coordinate system is divided into multiple partial image data (subvolumes) without relatively large movements, and an en face projection image of each subvolume is created. image) is constructed. This front projection image is a two-dimensional front image defined on the αβ plane, constructed by projection in the γ direction. Such a front projection image is called a strip.

Subsequently, each strip defined in the Lissajous coordinate system (αβ coordinate system) is transformed into an image defined in real space (xy coordinate system). In other words, a coordinate transformation from the αβ coordinate system to the xy coordinate system is applied to each strip. This process is called remapping. Additionally, an interpolation process is applied to each remapped strip (ie, defined in the xy coordinate system). This results in a plurality of strips defined in the xy coordinate system.

As a characteristic of the Lissajous scan described in Non-Patent Document 1, two strips overlap (intersect) at four locations. In the data processing described in Non-Patent Document 1, the largest strip is adopted as an initial reference strip, and strips are sequentially registered and merged with respect to this initial reference strip in order of size. In registration, a correlation operation (cross-correlation function, correlation coefficient) is used to determine the relative position between a reference strip and other strips. Here, since the strips have an arbitrary shape, a correlation calculation of the overlap area between the strips is performed using a mask for treating each strip as an image of a specific shape (for example, a square shape) (see Non-Patent Document 1). See Appendix A). As a result, a merged image (composite image) of a plurality of strips is obtained. This merged image is the target image, that is, the image with motion artifacts corrected.

In an exemplary embodiment, motion artifact correction is optimized by performing registration using multiple types of images obtained from data collected with a Lissajous scan.

Note that in some exemplary embodiments, scan modes other than Lissajous scan may be employed. For example, as a two-dimensional pattern including a series of cycles, a pattern including a series of mutually parallel line scans (that is, a series of B scans) can be employed. Such a scan pattern is called a raster pattern, and a scan using this is called a raster scan (see, for example, Japanese Patent Application Laid-Open Nos. 2013-81763 and 2015-62678). More generally, any scanning pattern for collecting data from a three-dimensional region of a sample can be employed.

A scanning imaging device according to an exemplary embodiment described below is an ophthalmological device capable of measuring the fundus of a living eye using Fourier domain OCT (eg, swept source OCT). The type of OCT that can be employed in the embodiments is not limited to swept source OCT, but may be, for example, spectral domain OCT or time domain OCT.

Scanning imaging devices according to some example embodiments may be capable of utilizing scanning modalities other than OCT. For example, some example embodiments may employ any optical scanning modality such as SLO.

Additionally, scanning modalities applicable to scanning imaging devices according to some example embodiments are not limited to optical scanning modalities. For example, some exemplary embodiments may employ scanning modalities that utilize electromagnetic waves other than light, scanning modalities that utilize ultrasound, and the like.

In addition to processing data collected with an OCT scan and/or processing data collected with an SLO, the ophthalmological device according to some example embodiments can process data acquired with other modalities. It's fine. Other modalities may be, for example, a fundus camera, a slit lamp microscope, and an ophthalmic surgical microscope. Ophthalmic devices according to some example embodiments may have the functionality of such modalities.

The target (sample) to which OCT is applied is not limited to the fundus of the eye, but may be any part of the eye such as the anterior segment or vitreous body, or may be a part or tissue of a living body other than the eye. It may be an object other than a living body.

Ophthalmic devices according to exemplary embodiments described below include some aspects of a scanning imaging device, some aspects of a method of controlling a scanning imaging device, some aspects of an image processing device, and some aspects of an image processing device. Some aspects of methods for controlling devices, some aspects of image processing methods, and some aspects of scanning imaging methods are provided.

<Configuration of scanning imaging device>
The ophthalmologic apparatus 1 shown in FIG. 1 includes a fundus camera unit 2, an OCT unit 100, and an arithmetic control unit 200. The fundus camera unit 2 is provided with a group of elements (optical elements, mechanisms, etc.) for photographing the eye E from the front. The OCT unit 100 is provided with a part of a group of elements (optical elements, mechanisms, etc.) for applying an OCT scan to the eye E to be examined. Another part of the element group for OCT scanning is provided in the fundus camera unit 2. Arithmetic control unit 200 includes one or more processors that perform various calculations and controls. In addition to these, the ophthalmologic apparatus 1 may include an element for supporting the subject's face and an element for switching the region to which an OCT scan is applied. Examples of the former element include a chin rest and a forehead rest. An example of the latter element is a lens unit used to switch the OCT scan application site from the fundus to the anterior segment of the eye.

The functionality of the elements disclosed herein is implemented using circuitry or processing circuitry. Circuitry or processing circuitry may include a general purpose processor, a special purpose processor, an integrated circuit, a central processing unit (CPU), a graphics processing unit (GPU), an ASIC ( Application Specific Integrated Circuit), programmable logic devices (for example, SPLD (Simple Programmable Logic Device), CPLD (Complex Programmable Logic Device), FPGA (Field Programmable Gate Array), conventional circuit configurations, and any combination thereof A processor is considered to be a processing circuitry or circuitry that includes transistors and/or other circuitry. In this disclosure, the term circuitry, unit, means, or similar terms may be used to perform the disclosed functionality. Hardware that executes or is programmed to perform the functions disclosed. The hardware may be the hardware disclosed herein or that is programmed to perform the functions described. It may be known hardware programmed and/or configured to is a combination of hardware and software that is used to configure the hardware and/or processor.

<Funus camera unit 2>
The fundus camera unit 2 is provided with an optical system for photographing the fundus Ef of the eye E to be examined. The acquired image of the fundus Ef (referred to as a fundus image, fundus photograph, etc.) is a frontal image such as an observation image or a photographed image. Observation images are obtained, for example, by video shooting using near-infrared light, and are used for alignment, focusing, tracking, and the like. The photographed image is, for example, a still image using flash light in the visible region or infrared region.

The fundus camera unit 2 includes an illumination optical system 10 and a photographing optical system 30. The illumination optical system 10 irradiates the eye E to be examined with illumination light. The photographing optical system 30 detects the return light of the illumination light from the eye E to be examined. The measurement light from the OCT unit 100 is guided to the eye E through the optical path within the fundus camera unit 2, and the return light is guided to the OCT unit 100 through the same optical path.

Light (observation illumination light) output from the observation light source 11 of the illumination optical system 10 is reflected by the concave mirror 12, passes through the condensing lens 13, and passes through the visible cut filter 14 to become near-infrared light. Further, the observation illumination light is once focused near the photographing light source 15, reflected by a mirror 16, and passes through a relay lens system 17, a relay lens 18, an aperture 19, and a relay lens system 20. The observation illumination light is reflected at the periphery of the apertured mirror 21 (the area around the aperture), passes through the dichroic mirror 46, and is refracted by the objective lens 22 to illuminate the eye E (fundus Ef). do. The return light of the observation illumination light from the subject's eye E is refracted by the objective lens 22, passes through the dichroic mirror 46, passes through the hole formed in the center area of the perforated mirror 21, and passes through the dichroic mirror 55. , passes through the photographic focusing lens 31 and is reflected by the mirror 32. Furthermore, this returned light passes through the half mirror 33A, is reflected by the dichroic mirror 33, and is imaged by the imaging lens 34 on the light receiving surface of the image sensor 35. The image sensor 35 detects the returned light at a predetermined frame rate. Note that the focus (focal position) of the photographing optical system 30 is adjusted to match the fundus Ef or the anterior segment of the eye.

The light output from the photographing light source 15 (photographing illumination light) passes through the same path as the observation illumination light and is irradiated onto the fundus Ef. The return light of the imaging illumination light from the eye E is guided to the dichroic mirror 33 through the same path as the return light of the observation illumination light, passes through the dichroic mirror 33, is reflected by the mirror 36, and is reflected by the imaging lens 37. An image is formed on the light receiving surface of the image sensor 38.

A liquid crystal display (LCD) 39 displays a fixation target (fixation target image). A portion of the light beam output from the LCD 39 is reflected by the half mirror 33A, then reflected by the mirror 32, passes through the photographic focusing lens 31 and the dichroic mirror 55, and then passes through the hole of the perforated mirror 21. The light beam that has passed through the hole of the perforated mirror 21 is transmitted through the dichroic mirror 46, refracted by the objective lens 22, and projected onto the fundus Ef. By changing the display position of the fixation target image on the LCD 39, the direction (fixation direction, fixation position) in which the line of sight of the eye E to be examined is guided is changed.

Instead of a display device such as an LCD, for example, a light emitting element array or a combination of a light emitting element and a mechanism for moving the light emitting element may be used.

The alignment optical system 50 generates an alignment index used for positioning (alignment) the optical system with respect to the eye E to be examined. The alignment light output from the light emitting diode (LED) 51 passes through the aperture 52 , the aperture 53 , and the relay lens 54 , is reflected by the dichroic mirror 55 , passes through the hole of the perforated mirror 21 , and passes through the dichroic mirror 46 . The light is transmitted through the objective lens 22 and projected onto the eye E to be examined. Return light of the alignment light from the eye E to be examined (corneal reflected light, etc.) is guided to the image sensor 35 through the same path as the return light of the observation illumination light. Manual alignment or auto alignment can be performed based on the received light image (alignment index image).

As in the prior art, the alignment index image of this example consists of two bright spot images whose positions change depending on the alignment state. When the relative position between the eye E and the optical system changes in the xy direction, the two bright spot images are integrally displaced in the xy direction. When the relative position between the eye E and the optical system changes in the z direction, the relative position (distance) between the two bright spot images changes. When the distance between the eye E and the optical system in the z direction matches the predetermined working distance, the two bright spot images overlap. When the position of the eye E and the position of the optical system match in the xy direction, two bright spot images are presented within or near a predetermined alignment target. When the distance between the eye E and the optical system in the z direction matches the working distance, and the position of the eye E and the optical system match in the xy direction, the two bright spot images overlap and alignment occurs. presented within the target.

In auto-alignment, the data processing unit 230 detects the positions of two bright spot images, and the main control unit 211 controls the movement mechanism 150, which will be described later, based on the positional relationship between the two bright spot images and the alignment target. . In manual alignment, the main control unit 211 displays two bright spot images on the display unit 241 together with the observation image of the eye E, and the user uses the operation unit 242 while referring to the two displayed bright spot images. to operate the moving mechanism 150.

The focus optical system 60 generates a split index used for focus adjustment for the eye E to be examined. In conjunction with the movement of the photographing focusing lens 31 along the optical path of the photographing optical system 30 (photographing optical path), the focusing optical system 60 is moved along the optical path of the illumination optical system 10 (illumination optical path). The reflection rod 67 is inserted into and removed from the illumination optical path. When performing focus adjustment, the reflective surface of the reflective rod 67 is arranged at an angle to the illumination optical path. The focus light output from the LED 61 passes through a relay lens 62 , is separated into two beams by a split indicator plate 63 , passes through a two-hole diaphragm 64 , is reflected by a mirror 65 , and is reflected by a condensing lens 66 into a reflecting rod 67 . Once the image is formed on the reflective surface of the image, it is reflected. Further, the focus light passes through the relay lens 20, is reflected by the perforated mirror 21, passes through the dichroic mirror 46, and is projected onto the eye E through the objective lens 22. Return light of the focus light from the eye E to be examined (fundus reflected light, etc.) is guided to the image sensor 35 through the same path as the return light of the alignment light. Manual focusing or autofocusing can be performed based on the received light image (split index image).

Diopter correction lenses 70 and 71 can be selectively inserted into the photographing optical path between the perforated mirror 21 and the dichroic mirror 55. The diopter correction lens 70 is a plus lens (convex lens) for correcting severe hyperopia. The diopter correction lens 71 is a minus lens (concave lens) for correcting severe myopia.

The dichroic mirror 46 combines the fundus photographing optical path and the OCT optical path (measuring arm). The dichroic mirror 46 reflects light in a wavelength band used for OCT and transmits light for fundus photography. The measurement arm is provided with a collimator lens unit 40, a retroreflector 41, a dispersion compensation member 42, an OCT focusing lens 43, an optical scanner 44, and a relay lens 45 in this order from the OCT unit 100 side.

The retroreflector 41 is movable in the direction of the arrow shown in FIG. 1, thereby changing the length of the measurement arm. Changing the measurement arm length is used, for example, to correct the optical path length according to the axial length or to adjust the interference state.

The dispersion compensation member 42 acts together with a dispersion compensation member 113 (described later) disposed on the reference arm to match the dispersion characteristics of the measurement light LS and the dispersion characteristics of the reference light LR.

The OCT focusing lens 43 is moved along the measurement arm to perform focus adjustment of the measurement arm. Movement of the imaging focusing lens 31, movement of the focusing optical system 60, and movement of the OCT focusing lens 43 can be controlled in a coordinated manner.

The optical scanner 44 is arranged at a position that is substantially optically conjugate with the pupil of the eye E to be examined. The optical scanner 44 deflects the measurement light LS guided by the measurement arm. The optical scanner 44 is a galvano scanner capable of two-dimensional scanning, including, for example, a galvano mirror for scanning in the x direction and a galvano mirror for scanning in the y direction.

<OCT unit 100>
As illustrated in FIG. 2, the OCT unit 100 is provided with an optical system for applying swept source OCT. This optical system includes an interference optical system. This interference optical system splits the light from the wavelength variable light source (wavelength swept light source) into measurement light and reference light, and overlaps the return light of the measurement light from the eye E with the reference light that has passed through the reference optical path. Together, they generate interference light, and this interference light is detected. The data (detection signal) obtained by the interference optical system is a signal representing the spectrum of interference light, and is sent to the arithmetic and control unit 200.

The light source unit 101 includes, for example, a near-infrared variable wavelength laser that changes the emission wavelength at high speed at least in the near-infrared wavelength band. Light L0 output from the light source unit 101 is guided to a polarization controller 103 by an optical fiber 102, and its polarization state is adjusted. Further, the light L0 is guided to a fiber coupler 105 by an optical fiber 104 and is split into a measurement light LS and a reference light LR. The optical path of the measurement light LS is called a measurement arm, and the optical path of the reference light LR is called a reference arm.

The reference light LR is guided to a collimator 111 by an optical fiber 110, converted into a parallel light beam, and guided to a retroreflector 114 via an optical path length correction member 112 and a dispersion compensation member 113. The optical path length correction member 112 acts to match the optical path length of the reference light LR and the optical path length of the measurement light LS. The dispersion compensation member 113 acts together with the dispersion compensation member 42 disposed on the measurement arm to match the dispersion characteristics between the reference light LR and the measurement light LS. The retroreflector 114 is movable along the optical path of the reference light LR incident thereon, thereby changing the length of the reference arm. Changing the reference arm length is used, for example, to correct the optical path length according to the axial length or to adjust the interference state.

The reference light LR that has passed through the retroreflector 114 passes through the dispersion compensating member 113 and the optical path length correcting member 112, is converted from a parallel beam into a convergent beam by the collimator 116, and enters the optical fiber 117. The reference light LR incident on the optical fiber 117 is guided to a polarization controller 118 to have its polarization state adjusted, guided to an attenuator 120 through an optical fiber 119 to have its light amount adjusted, and then sent to a fiber coupler 122 through an optical fiber 121. be guided.

On the other hand, the measurement light LS generated by the fiber coupler 105 is guided by the optical fiber 127 and converted into a parallel light beam by the collimator lens unit 40. The light passes through the relay lens 45, is reflected by the dichroic mirror 46, is refracted by the objective lens 22, and is projected onto the eye E to be examined. The measurement light LS is scattered and reflected at various depth positions of the eye E to be examined. The return light of the measurement light LS from the eye E to be examined travels the same path as the outgoing path in the opposite direction, is guided to the fiber coupler 105, and reaches the fiber coupler 122 via the optical fiber 128.

The fiber coupler 122 superimposes the measurement light LS incident on the optical fiber 128 and the reference light LR incident on the optical fiber 121 to generate interference light. The fiber coupler 122 generates a pair of interference lights LC by branching the generated interference lights at a predetermined branching ratio (for example, 1:1). A pair of interference lights LC are guided to a detector 125 through optical fibers 123 and 124, respectively.

Detector 125 includes, for example, a balanced photodiode. The balanced photodiode includes a pair of photodetectors that respectively detect a pair of interference lights LC, and outputs a difference between a pair of detection results obtained by these photodetectors. Detector 125 sends this output (detection signal) to data acquisition system (DAS) 130.

The data collection system 130 is supplied with a clock KC from the light source unit 101. The clock KC is generated in the light source unit 101 in synchronization with the output timing of each wavelength swept within a predetermined wavelength range by the variable wavelength light source. For example, the light source unit 101 splits the light L0 of each output wavelength to generate two branched lights, optically delays one of these branched lights, combines these branched lights, and generates the resulting composite light. A clock KC is generated based on the detection result. The data acquisition system 130 performs sampling of the detection signal input from the detector 125 based on the clock KC. Data acquisition system 130 sends the results of this sampling to arithmetic and control unit 200 .

In this example, both an element for changing the measurement arm length (for example, retroreflector 41) and an element for changing the reference arm length (for example, retroreflector 114 or reference mirror) are provided. However, only one element may be provided. Moreover, the elements for changing the difference (optical path length difference) between the measurement arm length and the reference arm length are not limited to these, and may be any elements (optical members, mechanisms, etc.).

<Control system/processing system>
Configuration examples of the control system and processing system of the ophthalmologic apparatus 1 are shown in FIGS. 3, 4A, and 4B. The control section 210, the image construction section 220, and the data processing section 230 are provided in the arithmetic control unit 200, for example. The ophthalmological apparatus 1 may include a communication device for performing data communication with an external device. The ophthalmological apparatus 1 may include a drive device (reader/writer) for reading data from a recording medium and writing data to the recording medium.

<Control unit 210>
The control unit 210 executes various controls. Control section 210 includes a main control section 211 and a storage section 212. Further, as shown in FIG. 4A, in this embodiment, the main control unit 211 includes a scan control unit 2111, and the storage unit 212 stores a scan protocol 2121.

<Main control unit 211>
The main control unit 211 includes a processor and controls each element of the ophthalmologic apparatus 1 (including the elements shown in FIGS. 1 to 4B). The main control unit 211 is realized by cooperation between hardware including a processor and control software. The scan control unit 2111 controls OCT scanning regarding a scan area of a predetermined shape and a predetermined size.

The photographic focus drive unit 31A moves the photographic focus lens 31 disposed in the photographic optical path and the focus optical system 60 disposed in the illumination optical path under the control of the main control unit 211. The retroreflector (RR) driving section 41A moves the retroreflector 41 provided on the measurement arm under the control of the main control section 211. The OCT focusing drive section 43A moves the OCT focusing lens 43 disposed on the measurement arm under the control of the main control section 211. A retroreflector (RR) drive unit 114A provided on the measurement arm moves the retroreflector 114 placed on the reference arm under the control of the main control unit 211. Each of the above-described drive units includes an actuator such as a pulse motor that operates under the control of the main control unit 211. The optical scanner 44 operates under the control of the main control section 211 (scanning control section 2111).

The moving mechanism 150 typically moves the fundus camera unit 2 three-dimensionally, for example, an x-stage that can move in the ±x direction (left-right direction), an x-movement mechanism that moves the x-stage, and a Includes a y stage that can move in the y direction (up and down direction), a y movement mechanism that moves the y stage, a z stage that can move in the ±z direction (depth direction), and a z movement mechanism that moves the z stage. . Each movement mechanism includes an actuator such as a pulse motor that operates under the control of the main control unit 211.

<Storage unit 212>
The storage unit 212 stores various data. The data stored in the storage unit 212 includes OCT images, fundus images, eye information to be examined, control information, and the like. The eye information to be examined includes patient information such as patient ID and name, left eye/right eye identification information, electronic medical record information, and the like. Control information is information regarding specific control. The control information in this embodiment includes a scanning protocol 2121.

The scanning protocol 2121 is an agreement regarding the contents of control for OCT scanning regarding a scanning area of a predetermined shape and a predetermined size, and includes a set of various control parameters (scanning control parameters). The scanning protocol 2121 includes a protocol for each scanning mode. The scanning protocol 2121 of this embodiment includes at least a Lissajous scan protocol, and may further include protocols such as B scan (line scan), cross scan, radial scan, and raster scan.

The scanning control parameters of this embodiment include at least parameters indicating the content of control for the optical scanner 44. This parameter includes, for example, a parameter indicating a scan pattern, a parameter indicating a scan speed, a parameter indicating a scan interval, and the like. The scan pattern indicates the shape of a scan path, and examples thereof include a Lissajous pattern, a line pattern, a cross pattern, a radial pattern, and a raster pattern. The scan speed is defined, for example, as the A-scan repetition rate. The scan interval is defined, for example, as the interval between adjacent A scans, that is, as the arrangement interval of scan points.

Note that, similar to the conventional techniques disclosed in Patent Documents 1 to 4 and Non-Patent Documents 1 and 2, the "Lissajous scan" of this embodiment is based on the point that two mutually orthogonal simple harmonic motions are obtained as an ordered pair. Not only Lissajous scan in a “narrow sense” where the path follows a pattern drawn by a trajectory (Lissajous pattern, Lissajous figure, Lissajous curve, Lissajous function, Bowditch curve), but also Lissajous in a “broad sense” that follows a predetermined two-dimensional pattern including a series of cycles. It may also be a scan.

In this embodiment, the optical scanner 44 includes a first galvano mirror that deflects the measurement light LS in the x direction and a second galvano mirror that deflects the measurement light LS in the y direction. In the Lissajous scan, the first galvanometer mirror is controlled so that the deflection direction along the x direction is repeated in the first cycle, and the second galvano mirror is controlled so that the deflection direction along the y direction is repeated in the second cycle. This is achieved by controlling the galvanometer mirror. Here, the first period and the second period are different from each other.

For example, the Lissajous scan of this embodiment not only scans a Lissajous pattern in a narrow sense obtained from a combination of two sine waves, but also scans a pattern in which a specific term (for example, an odd-order polynomial) is added to the Lissajous pattern, or scans a triangular wave pattern. It may also be a pattern scan based on .

"Cycle" generally means an object consisting of a plurality of sampling points of a certain length, which may be, for example, a closed or nearly closed curve (i.e., the start and end points of the cycle coincide or nearly coincide). ).

Typically, the scanning protocol 2121 is set based on a Lissajous function. As shown in equation (9) of Non-Patent Document 1, the Lissajous function is expressed, for example, by the following parametric equation system: x(t _i )=A・cos(2π・(f _A /n)・t _i ), y(t _i )=A・cos(2π・(f _A・(n-2)/n ² )・t _i ).

Here, x is the horizontal axis of the two-dimensional coordinate system in which the Lissajous curve is defined, y is the vertical axis, t _i is the acquisition time of the i-th A line in the Lissajous scan, A is the scan range (amplitude), f _A is the acquisition rate of the A-line (scan speed, repetition rate of A-scan), and n is the number of A-lines in each cycle in the x (horizontal axis) direction.

FIG. 5 shows an example of the distribution of scan lines applied to the fundus Ef in such a Lissajous scan.

Collected from any pair of cycles in a Lissajous scan (in a narrow or broad sense) because any pair of cycles included in the Lissajous scan intersects each other at one or more points (particularly two or more points, typically four points) Registration can be performed between the created data pairs. This makes it possible to use the image construction method and motion artifact correction method disclosed in Non-Patent Document 1 (or 2). Hereinafter, unless otherwise mentioned, the method described in Non-Patent Document 1 is applied.

The control information stored in the storage unit 212 is not limited to the above example. For example, the control information may include information for performing focus control (focus control parameters).

The focus control parameter is a parameter indicating the content of control for the OCT focusing drive section 43A. Examples of the focus control parameters include a parameter indicating the focal position of the measurement arm, a parameter indicating the moving speed of the focal position, and a parameter indicating the moving acceleration of the focal position. The parameter indicating the focal position is, for example, a parameter indicating the position of the OCT focusing lens 43. The parameter indicating the moving speed of the focal position is, for example, a parameter indicating the moving speed of the OCT focusing lens 43. The parameter indicating the moving acceleration of the focal position is, for example, a parameter indicating the moving acceleration of the OCT focusing lens 43. The moving speed may or may not be constant. The same applies to movement acceleration.

According to such focus control parameters, it becomes possible to perform focus adjustment according to the shape of the fundus Ef (typically a concave shape that is deep at the center and shallow at the periphery) and the aberration distribution. Focus control is executed in conjunction with scanning control (Lissajous scan repetition control), for example. This corrects motion artifacts and provides a high-quality image that is in focus throughout the scan range.

<Scan control unit 2111>
The scan control unit 2111 controls at least the optical scanner 44 based on the scan protocol 2121. The scan control unit 2111 may further control the light source unit 101 in conjunction with the control of the optical scanner 44 based on the scan protocol 2121. The scan control unit 2111 is realized by cooperation between hardware including a processor and scan control software including a scan protocol 2121.

<Image construction unit 220>
The image construction unit 220 includes a processor, and constructs various OCT image data based on the signal (sampling data) input from the data acquisition system 130. Typical OCT image data construction includes denoising, filtering, fast Fourier transform (FFT), etc., similar to traditional Fourier domain OCT (swept source OCT). When another type of OCT technique is employed, the image construction unit 220 constructs OCT image data by executing known processing according to the type. As a result, a plurality of A-scan image data are obtained. These A-scan image data are given coordinates according to the scanning protocol.

As described above, in this embodiment, the Lissajous scan is applied to the fundus Ef. The image construction unit 220 applies the image construction method and motion artifact correction method disclosed in Non-Patent Documents 1 and 2, respectively, to the data acquired through the Lissajous scan and the sampling by the data collection system 130. By applying this, three-dimensional image data is constructed. In some example aspects, image construction unit 220 may be configured to perform three-dimensional image data construction in conjunction with data processing unit 230.

The image construction unit 220 and/or the data processing unit 230 can form a display image by applying rendering to the three-dimensional image data. Examples of applicable rendering methods include volume rendering, surface rendering, maximum intensity projection (MIP), minimum intensity projection (MinIP), and multiplanar reconstruction (MPR).

The image construction unit 220 and/or the data processing unit 230 can construct an enface OCT image based on the three-dimensional image data. For example, the image construction unit 220 and/or the data processing unit 230 can construct projection data by projecting three-dimensional image data in the z direction (A-line direction, depth direction). Further, the image construction unit 220 and/or the data processing unit 230 can construct a shadowgram by projecting partial three-dimensional image data, which is a part of three-dimensional image data, in the z direction. This partial three-dimensional image data is set using, for example, segmentation. Segmentation is a process of identifying partial regions in an image. In this example, segmentation can be performed to identify an image region corresponding to a predetermined tissue of the fundus Ef.

The ophthalmological device 1 may be capable of performing OCT-Angiography. OCT angiography is an imaging technique that constructs images in which blood vessels are emphasized (for example, see Non-Patent Document 2, Japanese Patent Publication No. 2015-515894, etc.). Generally, the fundus tissue (structure) does not change over time, but the blood flow inside blood vessels changes over time. In OCT angiography, images are generated by emphasizing portions (blood flow signals) where such temporal changes exist. Note that OCT angiography is also called OCT motion contrast imaging. Furthermore, images obtained by OCT angiography are called angiography images, angiograms, motion contrast images, and the like.

When OCT angiography is performed, the ophthalmologic apparatus 1 repeatedly scans the same region of the fundus Ef a predetermined number of times. For example, the ophthalmologic apparatus 1 repeatedly performs the above-described scanning control (Lissajous scan repetition control) a predetermined number of times. Thereby, a plurality of three-dimensional data (three-dimensional data sets) corresponding to the Lissajous scan application area are collected by the data collection system 130. The image construction unit 220 and/or the data processing unit 230 can construct a motion contrast image from this three-dimensional data set. This motion contrast image is an angiographic image in which temporal changes in interference signals caused by blood flow in the fundus Ef are emphasized. This angiographic image is three-dimensional angiographic image data expressing the three-dimensional distribution of blood vessels in the fundus Ef.

The image construction unit 220 and/or the data processing unit 230 can construct arbitrary two-dimensional angiography image data and/or arbitrary pseudo three-dimensional angiography image data from this three-dimensional angiography image data. be. For example, the image constructing unit 220 can construct two-dimensional angiographic image data representing an arbitrary cross section of the fundus Ef by applying multi-sectional reconstruction to the three-dimensional angiographic image data. Furthermore, the image construction unit 220 can construct frontal angiography image data of the fundus Ef by applying projection imaging or shadowgram formation to the three-dimensional angiography image data.

The image construction unit 220 of this embodiment constructs a plurality of different types of images by applying a plurality of different imaging processes to the data collected by the data collection system 130. These multiple types of images are used for registration (ie, motion artifact correction). Note that conventional motion artifact correction technology uses a single type of image, so there is room for improvement in correction accuracy. This embodiment aims to improve correction accuracy by using a plurality of types of images.

An example of the configuration of the image construction unit 220 for executing such processing is shown in FIG. 4B. The image construction section 220 of this example includes an imaging processing section 221, a projection processing section 222, a lateral shift amount calculation section 223, an axial shift amount calculation section 224, and a correction section 225. Note that unless otherwise specified, this example follows the technology described in Non-Patent Document 1 or 2.

<Imaging processing unit 221>
The imaging processing unit 221 is configured to construct a plurality of three-dimensional image data by applying a plurality of different imaging processes to data collected by a Lissajous scan on a three-dimensional region of the eye E to be examined.

Typically, the imaging processing unit 221 is configured to apply at least fast Fourier transform (FFT) to construct three-dimensional image data (volume). Furthermore, the imaging processing unit 221 may be configured to perform predetermined processing (pre-processing and/or post-processing) before and/or after the fast Fourier transform.

The type of three-dimensional image data constructed by the imaging processing unit 221 may be arbitrary. Some examples are OCT intensity image data, OCT angiography image data, composite image data based on OCT intensity image data and OCT angiography image data, and OCT polarization image data.

The OCT intensity image data is three-dimensional image data defined in a Lissajous coordinate system (for example, the αβγ coordinate system described above), constructed by the method described in Non-Patent Document 1, for example. The OCT intensity image data is image data that visualizes the three-dimensional structure of the eye E to be examined.

OCT angiography image data is three-dimensional image data defined by a Lissajous coordinate system (for example, αβγ coordinate system), which is constructed by the method described in, for example, Non-Patent Document 2. OCT angiography image data is image data that visualizes the three-dimensional distribution of blood flow in the eye E (fundus Ef) to be examined.

Composite image data based on OCT intensity image data and OCT angiography image data is composed of at least a part of the OCT intensity image data, or image data constructed by processing the same, and at least a part of the OCT angiography image data. This is one piece of three-dimensional image data created by applying arbitrary compositing processing to the processed and constructed image data. This composite image data is image data that three-dimensionally visualizes the structure and blood flow of the eye E to be examined.

OCT polarization image data is three-dimensional image data obtained when a modality called polarization OCT (polarization sensitive OCT) is employed. The OCT polarization image data is image data that visualizes the three-dimensional distribution of melanin in the tissue of the eye E (fundus Ef). Polarized OCT is a modality that uses polarized light to detect polarization dependence (polarization information) due to birefringence of a sample. Polarized OCT is disclosed, for example, in JP-A No. 2004-28970, JP-A No. 2007-298461, and JP-A No. 2015-230297. As described in these documents, a scanning imaging device capable of performing polarized OCT has an OCT interference optical system similar to that of the ophthalmological device 1, but typically includes various types of systems for controlling polarization. (e.g., polarization controller, polarization modulator, electro-optic modulator), a polarization beam splitter that splits the interference light (interference signal) into two polarization components, and two optical detectors that detect each of these polarization components. (not shown).

The types of three-dimensional image data constructed by the imaging processing unit 221 are not limited to the examples described above. For example, in order to obtain a desired type of three-dimensional OCT image data, the Jones matrix OCT system described in Japanese Patent Application Publication No. 2014-228473, the OCT blood flow measuring device described in Japanese Patent Application Publication No. 2019-54993, It is also possible to use any of the other OCT modalities.

<Projection processing unit 222>
The projection processing unit 222 applies projection in the depth direction (A-line direction, axial direction, axial direction, γ direction, z direction) to each of the plurality of three-dimensional image data constructed by the imaging processing unit 221. is configured to do so. Thereby, front image data is constructed from each three-dimensional image data constructed by the imaging processing unit 221. The projection processing unit 222 can construct at least one type of front image data from one three-dimensional image data.

When OCT intensity image data is obtained by the imaging processing unit 221, the projection processing unit 222 performs, for example, a log-scaled linear projection, a log-scale projection, a maximum value projection ( Any type of projection processing can be applied to the OCT intensity image data, such as maximum intensity projection). Here, logarithmic scale linear projection is a projection method that adds pixel values (intensities) in the A-line direction (depth direction) on a linear scale and calculates the logarithm of the sum. It can be thought of as the addition of Logarithmic scale projection is a projection method in which the logarithm of a pixel value is calculated and then added in the A-line direction, and can essentially be considered as intensity multiplication. Maximum value projection is a projection method that selects the maximum value among the values of pixels on each A line. Note that the projection method of OCT intensity image data mentioned here is merely an example, and the method is not limited thereto.

When OCT angiography image data is constructed by the imaging processing unit 221, the projection processing unit 222 may, for example, project the entire OCT angiography image data in the A-line direction or project only a part of the OCT angiography image data. can be projected in the A-line direction. An example of the former is the whole depth en face OCT-A image shown in FIG. 5 of Non-Patent Document 2. Further, as an example of the latter, Fig. 9 of Yin, X.; Chao, J. R.; Wang, R. K., “User-guided segmentation for volumetric retinal optical coherence tomography images”, J. Biomed. Opt 19, 086020 (2014) shows There are images of the vascular network in three different layers (NFL, IPL/INL, OPL). These are projections of slabs extracted by segmentation, and correspond to the shadowgrams described above. Note that the projection method of OCT angiography image data mentioned here is merely an example, and the method is not limited thereto.

When composite image data based on OCT intensity image data and OCT angiography image data is constructed by the imaging processing unit 221, the projection processing unit 222 applies projection to this composite image data to create a retinal vessel enhanced image, for example. can be constructed. As an example of a retinal vessel-enhanced image, see Figure 13(c) of Makita, S., et al., “Clinical prototype of pigment and flow imaging optical coherence tomography for posterior eye investigation”, Biomedical Optics Express 9, 4372 (2018). There is a retinal angiography image shown. Also, emphasizes “hyperpenetration” described in Hong, Y.J., et al, “Simultaneous Investigation of Vascular and Retinal Pigment Epithelial Pathologies of Exudative Macular Diseases by Multifunctional Optical Coherence Tomography”, IOVS August 2014, Vol.55, 5016-5031. It is also possible to construct an image based on the above-mentioned composite image data. Note that the composite image data projection methods mentioned here are merely examples, and the present invention is not limited to these.

When OCT polarization image data is constructed by the imaging processing unit 221 (that is, when three-dimensional polarization information is acquired), the projection processing unit 222 applies projection to this OCT polarization image data, for example, Any 2D image or any 2D map, such as DOPU map, birefringence map, RPE-sensitive contrast map, RPE-elevation map, etc. It is possible to construct. As an example of a polarization uniformity map, see Miura M., et al., “Evaluation of intraretinal migration of retinal pigment epithelial cells in age-related macular degeneration using polarimetric imaging”, Sci Rep. 2017;7(1):3150. The images (minimum polarization uniformity projection (minimum DOPU projection)) are shown in FIGS. 1-4(f). As an example of birefringence map, see Sugiyama, S., et al., “Birefringence imaging of posterior eye by multi-functional Jones matrix optical coherence tomography”, Biomedical Optics Express Vol. 6, Issue 12, pp. 4951-4974 (2015 ) is the image shown in FIG. 7(b) (maximum birefringence projection). As an example of a retinal pigment epithelium-sensitive contrast map, see Azuma, S., et al., “Pixel-wise segmentation of severely pathologic retinal pigment epithelium and choroidal stroma using multi-contrast Jones matrix optical coherence tomography”, Biomedical Optics Express Vol. 9, Issue 7, pp. 2955-2973 (2018), there is an image obtained by adding the value shown in equation (1) in the A-line direction. In addition to three-dimensional polarization information, OCT intensity image data and OCT angiography image data are also used in constructing this map. An example of a retinal pigment epithelium elevation map is the image shown in FIG. 4 of the same document. In constructing this map as well, OCT intensity image data and OCT angiography image data are used in addition to three-dimensional polarization information.

When three-dimensional image data other than the above examples are constructed by the imaging processing unit 221, the projection processing unit 222 is configured to be able to apply projection processing determined according to the type of the three-dimensional image data.

<Lateral shift amount calculation unit 223>
The lateral shift amount calculation unit 223 is configured to calculate a data shift amount in a direction orthogonal to the depth direction (γ direction, z direction) based on the plurality of front image data constructed by the projection processing unit 222. ing. In this embodiment, this data shift amount is referred to as a lateral shift amount.

The direction perpendicular to the depth direction is called the lateral direction, and the direction perpendicular to the γ direction in the αβγ coordinate system is an arbitrary direction (αβ direction) within the αβ plane, and perpendicular to the z direction in the xyz coordinate system. The direction is an arbitrary direction (xy direction) within the xy plane.

For example, the lateral shift amount calculation unit 223 may be configured to calculate a correlation coefficient based on a plurality of front image data, and calculate the lateral shift amount based on this correlation coefficient. For this purpose, the exemplary lateral shift amount calculation section 223 includes a division section 2231, a correlation coefficient calculation section 2232, and a shift amount calculation section 2233.

<Divided part 2231>
The dividing unit 2231 is configured to divide each frontal image data constructed by the projection processing unit 222 into a plurality of partial image data. Each partial image data corresponds to the strip described above.

For example, the dividing unit 2231 is configured to divide the front image data into a plurality of strips (strip group) using the method described in Non-Patent Document 1. For example, the dividing unit 2231 is configured to analyze the frontal image data and divide the frontal image data into a plurality of strips in which there is no relatively large movement. In another example, the dividing unit 2231 performs a process of dividing the three-dimensional image data (volume) constructed by the imaging processing unit 221 into a plurality of sub-volumes in which relatively large movement does not occur, and a process that corresponds to each sub-volume. and determining strip groups corresponding to these sub-volumes by identifying portions of the frontal image data. Each strip thus obtained is defined in a Lissajous coordinate system (eg, an αβ coordinate system).

FIG. 6 shows an example of a strip defined in the Lissajous coordinate system (the αβ coordinate system described above). Reference numeral 300 indicates a front projection image of the entire three-dimensional image data (volume). The entire volume is defined by the αβγ coordinate system, and a front projection image (front image data) 300 obtained by projecting this in the γ direction (depth direction) is defined by the αβ coordinate system. Here, the α-axis represents the positions of a plurality of A lines constituting a cycle, and the β-axis perpendicular to this represents the cycle identification number (scan order). Typically, both the γ direction and the z direction coincide with the direction along the A line of the OCT scan (the traveling direction of the measurement light LS).

The example shown in FIG. 6 shows N strips 300(0) to 300(N-1). Here, N is an integer of 2 or more, typically from several tens to several hundreds. Each strip 300(n) is a partial image of the frontal projection image 300 of the entire volume.

According to the above processing, as shown in FIG. 7, a plurality of front image data 300-m (m=1, 2, . (an integer number), a plurality of strip groups 310-m are obtained. The strip group 310-m consists of a plurality of strips 310-m(1), 310-m(2), . . . , 310-m (N _m ).

Here, the number of strips N ₁ to N M included in each of the M strip groups 310-1 to 310- _M is generally arbitrary. However, in some exemplary embodiments, at least some of the strip numbers N ₁ -N _M may be equal. As a typical example, in this embodiment, all of the strip numbers N ₁ to N _M may be equal. In this case, predetermined M strips 310-1 (n ₁ ), 310-2 (n ₂ ), . . . , 310-M (n _M ) may be obtained from the same partial data of collected data (sampling data, three-dimensional image data).

In other words, any pair of the M pieces of front image data 300-1 to 300-M (first front image data 300-m ₁ , second front image data 300-m ₂ : m ₁ =1, 2, . . , M; m ₂ =1, 2, . . . , M; m ₁ ≠ m ₂ ) satisfies the following two conditions. 310-M can be formed. The first condition is that the number n _m1 of strips included in the first strip group 310- _m1 formed from the first front image data 300- _m1 is formed from the second front image data 300- _m2 . This is equal to the number n _m2 of strips included in the second strip group 310-m ₂ (n _m1 =n _m2 ). Another condition is that partial data of collected data (sampling data, three-dimensional image data) corresponding to any strip included in the first strip group 310- _m1 is included in the second strip group 310- _m2 . It must be the same as partial data of collected data (sampling data, three-dimensional image data) corresponding to any strip. In other words, the collected data (sampling data, three-dimensional image data) is divided into a plurality of partial data, and one strip of the M strip groups 310-1 to 310-M is obtained from each partial data. It is that you are.

When these two conditions are satisfied, the dividing unit 2231 performs processing for data division (strip formation) on only one of the M pieces of front image data 300-1 to 300-M. (In other words, it is sufficient to apply the analysis described in Non-Patent Document 1 for determining whether there is a relatively large movement). For example, similar to the method described in Non-Patent Document 1, the dividing unit 2231 applies the analysis to the OCT intensity image data (or the front image data obtained from this) to generate a strip group (reference strip group). ) to form. Furthermore, the dividing unit 2231 can form strip groups of other three-dimensional image data (for example, the aforementioned OCT angiography image data, composite image data, OCT polarization image data, etc.) based on this reference strip group. can. At this time, the dividing unit 2231 divides the other three-dimensional image data in the same manner as the reference strip group. Specifically, the number of strips formed from other three-dimensional image data is equal to the number of strips in the reference strip group. In addition, for each strip of other three-dimensional image data, the area of the eye E represented by that strip matches the area of the eye E represented by any strip of the reference strip group. By performing such processing, processing resources can be saved compared to applying data division (strip formation) to the M pieces of front image data 300-1 to 300-M individually. , processing time can be shortened.

<Correlation coefficient calculation unit 2232 and shift amount calculation unit 2233>
The correlation coefficient calculation unit 2232 calculates a correlation coefficient based on the plurality of strip groups (310-1 to 310-M) acquired from the plurality of frontal image data (300-1 to 300-M) by the division unit 2231. is configured to calculate. Here, each strip is defined in a Lissajous coordinate system (eg, αβ coordinate system). Unless otherwise mentioned, the correlation coefficient calculation unit 2232 can perform the correlation calculation according to the method described in Non-Patent Document 1.

For example, the correlation coefficient calculation unit 2232 first applies coordinate transformation to each strip defined in the Lissajous coordinate system. This coordinate transformation is a coordinate transformation from a Lissajous coordinate system (for example, αβ coordinate system) to a real space coordinate system (for example, xy coordinate system), and corresponds to remapping described in Non-Patent Document 1. .

The process for creating a group of strips defined in the real space coordinate system from the three-dimensional image data constructed by the imaging processing unit 221 is not limited to the process described above. For example, you can construct a front projection image of the entire 3D image data, apply a coordinate transformation to this front projection image (defined in the Lissajous coordinate system), and then apply a coordinate transformation to the front projection image (defined in the real space coordinate system). ) can be divided to build multiple strips.

Subsequently, the correlation coefficient calculation unit 2232 can interpolate the pixels of each strip formed by coordinate transformation. This pixel interpolation is data processing in which a value derived from the values of surrounding pixels is assigned to a pixel to which no value has been assigned in coordinate transformation.

Pixel interpolation may include, for example, any known pixel interpolation operation. For example, the correlation coefficient calculation unit 2232 first calculates, among the pixels of the strip after the coordinate transformation, those pixels to which no value is assigned from the pixels of the strip before the coordinate transformation (in other words, the pixels are not included in the value range of the coordinate transformation). pixel). The pixels identified by this become the targets of interpolation. Next, the correlation coefficient calculating unit 2232 calculates, among pixels in a predetermined neighborhood area of the interpolation target pixel, pixels to which values from the pixels of the strip before coordinate transformation are assigned (in other words, those included in the value range of coordinate transformation). pixel) and calculate the value of the interpolation target pixel from the values of these pixels.

Here, the predetermined neighborhood area is arbitrary, and the calculation for calculating the value of the interpolation target pixel from the value of the pixel inside the area is also arbitrary. For example, among the pixels adjacent to the interpolation target pixel (typically 8 pixels located above, below, right, left, upper right, lower right, upper left, and lower left of the interpolation target pixel), The average value of the included pixels can be calculated, and this average value can be used as the value of the interpolation target pixel.

In this way, a plurality of strips defined in a real space coordinate system (for example, an xy coordinate system) with pixel interpolation is obtained. The correlation coefficient calculation unit 2232 executes the correlation calculation described in Non-Patent Document 1 based on the plurality of obtained strips.

As a characteristic of the Lissajous scan, any two strips included in a strip group of one frontal image data have an overlapping area. Similar to the method described in Non-Patent Document 1, the correlation coefficient calculation unit 2232 performs a correlation calculation between strips using the overlap between strips.

In the method described in Non-Patent Document 1, strip groups are first ordered according to size (area, etc.), and a series of processes (correlation calculation, lateral shift amount calculation, registration, and merge processing) are performed according to this order. is being executed repeatedly. In this correlation calculation, the correlation coefficient ρ(r') shown in equation (18) of Non-Patent Document 1 is calculated (see the following equation).

In this way, in the conventional technology, a correlation calculation between strips is performed using a group of strips obtained from a single front image data, but in this embodiment, a correlation calculation between strips is performed using a group of strips obtained from a plurality of front image data. A correlation calculation between strips is performed using a group of strips. As mentioned above, in some exemplary embodiments, the division 2231 can form a plurality of strip groups to satisfy the above two conditions.

In this case, for any pair (first partial data, second partial data) of the plurality of partial data of the collected data (sampling data, three-dimensional image data) described above, the correlation coefficient calculation unit 2232 first calculates the first A plurality of strips corresponding to the partial data are selected from a plurality of strip groups formed from the plurality of front image data. Specifically, the correlation coefficient calculation unit 2232 selects one strip corresponding to the first partial data from a plurality of strip groups formed from a plurality of front image data. Further, the correlation coefficient calculation unit 2232 sets a first group including at least two of the selected plurality of strips. Similarly, the correlation coefficient calculation unit 2232 selects strips corresponding to the second partial data one by one from the plurality of strip groups, and selects a second group including at least two strips from among the plurality of selected strips. Set. Then, the correlation coefficient calculation unit 2232 calculates the correlation coefficient corresponding to the first partial data and the second partial data based on the set first group and second group.

Such a correlation coefficient is calculated based on a plurality of strip groups obtained from a plurality of frontal image data, and therefore takes into consideration information of a plurality of different types of images. The correlation coefficient is calculated using the following equation, for example.

Here, the sum subscript (index) i spans multiple different types of frontal image data, and f _i and g _i indicate two strips (images combined with the mask image) based on the same type of frontal image. . For example, if the above first and second groups are considered, f _i and g _i belong to the first and second groups, respectively. In addition, r = (x, y) indicates the position expressed in the real space coordinate system (xy coordinate system), and σ _fg ² is the covariance in the overlap region of f(r) and g(r+r'). , and σ _f ² and σ _g ² are the variances of f(r) and g(r+r′) in the overlap region, respectively. Here, the calculation for the pair f _i and g _i follows the conventional method, but differs from the conventional method in that the covariance and variance obtained for a plurality of different types of frontal image data are combined.

The correlation coefficients obtained from the first group and the second group in this manner reflect the amount of lateral shift between the first partial data and the second partial data described above. The shift amount calculation unit 2233 can calculate the lateral shift amount corresponding to the first partial data and the second partial data by detecting the peak of the correlation coefficient.

For example, the correlation coefficient calculation unit 2232 and the shift amount calculation unit 2233 apply the above processing to various pairs of a plurality of partial data of the aforementioned collected data (sampling data, three-dimensional image data). The lateral shift amount is obtained for all of the plurality of partial data (that is, for the entire collected data (sampling data, three-dimensional image data)).

Such processing corresponds to "rough lateral motion correction" in Non-Patent Document 1. Furthermore, the lateral shift amount calculation unit 223 uses "fine lateral motion correction" described in Non-Patent Document 1 to remove small motion artifacts in the lateral direction caused by eye movements such as slow drift and tremor. ” may be configured to execute the process. Regarding "fine lateral motion correction," it is also possible to improve accuracy by using multiple strip groups based on multiple frontal image data.

<Axial shift amount calculation unit 224>
The axial shift amount calculation unit 224 calculates the data shift amount (axial shift amount) in the depth direction (z direction, axial direction) based on any one or more of the plurality of three-dimensional image data constructed by the imaging processing unit 221. ) is configured to calculate.

For example, the axial shift amount calculation unit 224 is configured to execute either or both of the "rough axial motion correction" process and the "fine axial motion correction" process described in Non-Patent Document 1. The former applies processing similar to the above-described "rough lateral motion correction" to a plurality of sub-volumes of three-dimensional image data (volume) constructed by the imaging processing unit 221. The latter applies processing similar to the former between cycles on the premise that no motion artifact exists in each cycle. Note that the process for calculating the axial shift amount is not limited to these, and for example, any image data having the z direction as one dimension can be used.

Similar to the calculation of the lateral shift amount, it is possible to improve the accuracy in calculating the axial shift amount by using a plurality of three-dimensional image data. For example, any two of a plurality of different types of three-dimensional image data constructed by the imaging processing unit 221 (for example, OCT intensity image data, OCT angiography image data, composite image data, and OCT polarization image data) It is possible to perform either or both of the "rough axial motion correction" processing and the "fine axial motion correction" processing using the above).

<Correction unit 225>
In this embodiment, the lateral shift amount calculated by the lateral shift amount calculation section 223 and the axial shift amount calculated by the axial shift amount calculation section 224 are input to the correction section 225. Based on these shift amounts, the correction unit 225 performs motion artifact correction of the image data based on the data collected by the Lissajous scan.

Here, the image data to which the motion artifact correction is applied may be any image data based on data collected by Lissajous scan, for example, the plurality of three-dimensional image data constructed by the imaging processing unit 221. or any of the plurality of frontal image data constructed by the projection processing unit 222. Further, the image data to which motion artifact correction is applied is 3D image data constructed by the imaging processing unit 221 separately from the plurality of 3D image data, or separately from the plurality of frontal image data. The front image data constructed by the projection processing unit 222 may be used. Further, the image data to which motion artifact correction is applied is image data obtained by processing arbitrary three-dimensional image data constructed by the imaging processing section 221, or arbitrary image data constructed by the projection processing section 222. The image data may be obtained by processing front image data.

When applying motion artifact correction to the image data used in the processing up to this point, the correction unit 225, for example, creates multiple partial images of the image data according to the data division (strip formation) performed by the division unit 2231. Registration based on the lateral shift amount and the axial shift amount can be applied to the data. Thereby, the relative positions of the plurality of partial image data are adjusted, and motion artifacts in the image data are reduced or removed.

When applying motion artifact correction to image data that has not been used in the processing up to this point, the correction unit 225 divides the image data into a plurality of partial image data according to data division (strip formation) performed by the division unit 2231, for example. It is possible to apply registration based on the lateral shift amount and the axial shift amount to these partial image data.

The image construction unit 220 is realized by cooperation between hardware including a processor and image construction software.

<Data processing unit 230>
The data processing unit 230 includes a processor and applies various data processing to the image of the eye E to be examined. For example, the data processing unit 230 is realized by cooperation between hardware including a processor and data processing software.

The data processing unit 230 may be configured to perform registration between two images acquired for the eye E to be examined. For example, the data processing unit 230 performs registration between three-dimensional image data obtained by OCT (for example, three-dimensional image data obtained by Lissajous scan) and a frontal image obtained by the fundus camera unit 2. may be configured to do so. Further, the data processing unit 230 may be configured to perform registration between two OCT images obtained by OCT. Further, it may be configured to apply registration to the analysis results of OCT images or the analysis results of frontal images. These registrations can be performed using known techniques, and include, for example, feature point extraction and affine transformation.

<User interface 240>
User interface 240 includes a display section 241 and an operation section 242. The display section 241 includes the display device 3. The operation unit 242 includes various operation devices and input devices. The user interface 240 may include a device such as a touch panel that has a display function and an operation function integrated. It is also possible to construct embodiments that do not include at least a portion of user interface 240. For example, the display device may be an external device connected to the ophthalmological apparatus 1.

<motion>
The operation of the ophthalmologic apparatus 1 will be explained. An example of the operation of the ophthalmologic apparatus 1 is shown in FIG. The same preparatory processes as before, such as entering the patient ID, setting the scan mode (specifying Lissajous scan), presenting a fixation target, alignment, focus adjustment, and OCT optical path length adjustment, have already been performed. do.

(S1: Collect OCT interference signals with Lissajous scan)
Upon receiving a predetermined scan start trigger signal, the scan control unit 2111 starts applying an OCT scan (Lissajous scan) to the eye E (fundus Ef) to be examined.

The scan start trigger signal is generated, for example, in response to completion of a predetermined preparatory operation (alignment, focus adjustment, OCT optical path length adjustment, etc.), or in response to a scan start instruction operation performed using the operation unit 242. is generated in response to.

The scan control unit 2111 applies the Lissajous scan to the eye E by controlling the optical scanner 44, the OCT unit 100, etc. based on the scanning protocol 2121 (protocol compatible with the Lissajous scan). OCT interference signals as three-dimensional data are collected by this Lissajous scan. The collected three-dimensional data is sent to the image construction section 220.

(S2: Build multiple types of 3D image data)
The imaging processing unit 221 of the image construction unit 220 applies a plurality of different imaging processes to the three-dimensional data collected in step S1. Thereby, multiple types of three-dimensional image data are obtained. Different types of imaging processing result in three-dimensional image data containing different information.

In an exemplary embodiment, the imaging processor 221 may be configured to construct OCT intensity image data and OCT angiography image data. However, as described above, the imaging processing unit 221 can construct composite image data of OCT intensity image data and OCT angiography image data. Furthermore, when the ophthalmologic apparatus 1 has a polarization OCT function, the imaging processing unit 221 can construct OCT polarization image data.

The constructed three-dimensional image data of multiple types are sent to the projection processing section 222. Further, any of the plurality of types of three-dimensional image data constructed is sent to the axial shift amount calculation unit 224.

(S3: Build multiple types of frontal image data)
The projection processing unit 222 constructs multiple types of front image data by applying projection in the depth direction (axial direction) to the multiple types of three-dimensional image data constructed in step S2. Different types of frontal image data contain different information.

In a typical example of the present embodiment, the projection processing unit 222 constructs one or more types of frontal intensity image data from the OCT intensity image data, and constructs one or more types of frontal angiography image data from the OCT angiography image data. may be configured to construct. Note that when composite image data of OCT intensity image data and OCT angiography image data is constructed, one or more types of frontal composite image data can be constructed from this composite image data. Moreover, when OCT polarization image data is constructed, one or more types of frontal polarization image data can be constructed from this OCT polarization image data. The constructed plural types of frontal image data are sent to the lateral shift amount calculation unit 223.

(S4: Calculate the lateral shift amount)
The lateral shift amount calculation unit 223 calculates the data shift amount (lateral shift amount) in the lateral direction orthogonal to the axial direction based on the plurality of types of front image data constructed in step S3.

In a typical example of the present embodiment, the lateral shift amount calculation unit 223 is configured to calculate the lateral shift amount based on one or more types of frontal intensity image data and one or more types of frontal angiography image data. good. Note that when one or more types of frontal composite image data are constructed, the lateral shift amount can be calculated based on multiple types of frontal image data including the one or more types of frontal composite image data. Furthermore, when one or more types of front polarized image data are constructed, the lateral shift amount can be calculated based on multiple types of front image data including the one or more types of front polarized image data.

In this embodiment, the lateral shift amount calculation unit 223 divides each of the plurality of front image data into partial image data groups (strip groups) by the division unit 2231 to form a plurality of strip groups, and the correlation coefficient calculation unit 2232 calculates a correlation coefficient from a plurality of strip groups, and a shift amount calculation unit 2233 calculates a lateral shift amount from the correlation coefficient. Here, for example, each correlation coefficient reflects the amount of deviation between two strips. Information on the calculated lateral shift amount is sent to the correction unit 225.

(S5: Calculate the axial shift amount)
The axial shift amount calculation unit 224 calculates the data shift amount in the axial direction (axial shift amount) based on any one or more of the plurality of three-dimensional image data constructed in step S2. Here, in order to improve the accuracy of the process of calculating the axial shift amount, two or more types of three-dimensional image data may be used.

In a typical example of this embodiment, the axial shift amount calculation unit 224 may be configured to calculate the axial shift amount based on either or both of OCT intensity image data and OCT angiography image data. Note that when composite image data is constructed, the axial shift amount can be calculated based on one or more three-dimensional image data including this composite image data. Further, when OCT polarization image data is constructed, the axial shift amount can be calculated based on one or more three-dimensional image data including this OCT polarization image data. Information on the calculated axial shift amount is sent to the correction unit 225.

(S6: Perform motion artifact correction)
Regarding the image data constructed based on the three-dimensional data collected in step S1, the correction unit 225 performs motion artifact correction in the lateral direction based on the lateral shift amount calculated in step S4, and also performs motion artifact correction in the lateral direction based on the lateral shift amount calculated in step S5. Axial direction motion artifact correction is performed based on the calculated axial shift amount.

The image data to which these motion artifact corrections are applied may be, for example, any of the multiple types of three-dimensional image data constructed in step S2, and/or any of the multiple types of frontal image data constructed in step S3. It may be

(S7: Display and save corrected image data)
The main control unit 211 can cause the display unit 241 (and/or other display device) to display an image based on the image data corrected in step S6. Further, when a rendered image of the image data corrected in step S6 is created by the image construction section 220 or the data processing section 230, the main control section 211 displays this rendered image on the display section 241 (and/or other display section). device).

Further, the main control unit 211 can store the image data corrected in step S6 in the storage unit 212 (and/or other storage device) (end).

<Action, effect, etc.>
Certain features of exemplary embodiments will be described, and some of the operations and advantages provided by them will be described.

The ophthalmic device 1 according to the exemplary embodiments described above provides several aspects of a scanning imaging device. In addition, the ophthalmological apparatus 1 according to the exemplary embodiments described above may include some aspects of the method of controlling a scanning imaging device, some aspects of the image processing device, and some aspects of the method of controlling the image processing device. Some aspects of image processing methods, and some aspects of scanning imaging methods are provided.

Now, the scanning imaging device (ophthalmological device 1) according to the exemplary embodiment includes a scanning section, an imaging processing section (221), a projection processing section (222), and a first data shift amount calculation section (lateral A shift amount calculation section 223) and a correction section (225) are included.

The scanning unit is configured to collect data by applying scanning to the sample (eye E, fundus Ef). This scanning is performed according to a two-dimensional pattern that includes a series of cycles.

In some exemplary embodiments, the series of cycles included in the two-dimensional pattern of scans applied to the sample may intersect with each other.

In some example aspects, the scan applied to the sample may be performed based on a preset scan protocol based on a Lissajous function. An example of such a scan is the Lissajous scan in the exemplary embodiment described above.

In some exemplary embodiments, the scanning unit includes a deflector (light scanner 44) capable of deflecting light in a first direction (x direction) and a second direction (y direction) that are different from each other, and The sample is configured to apply scanning according to a two-dimensional pattern to the sample by repeating changes in the deflection direction along the direction in a first period and repeating changes in the deflection direction along the second direction in a second period different from the first period. It's good that it has been done. Here, the deflector is controlled based on a preset scanning protocol based on a Lissajous function.

In the above exemplary embodiment, the scanning section includes the OCT unit 100 and the elements in the fundus camera unit 2 that constitute the measurement arm (retroreflector 41, OCT focusing lens 43, optical scanner 44, objective lens 22, etc.) and applying an OCT scan to the eye E to be examined to collect OCT data.

The imaging processing section (221) is configured to construct a plurality of three-dimensional image data by applying a plurality of different imaging processes to the data collected by the scanning section.

In some exemplary embodiments, when the scanning unit applies an OCT scan to the sample, the plurality of three-dimensional image data constructed by the imaging processing unit (221) includes OCT intensity image data, OCT angiography image data. , composite image data based on OCT intensity image data and OCT angiography image data, and OCT polarization image data.

The projection processing unit (222) is configured to construct a plurality of front image data by applying depth direction projection to each of the plurality of three-dimensional image data constructed by the imaging processing unit (221). ing.

The first data shift amount calculation unit (lateral shift amount calculation unit 223) calculates the data shift amount (lateral shift amount) in the direction perpendicular to the depth direction based on the plurality of front image data constructed by the projection processing unit (222) amount).

The correction unit (225) executes motion artifact correction of the image data based on the data collected by the scanning unit, based on the data shift amount calculated by the first data shift amount calculation unit (lateral shift amount calculation unit 223). is configured to do so.

According to the scanning imaging device according to such an exemplary embodiment, the motion artifact correction is performed using a plurality of image data obtained by a plurality of types of imaging processing, so that a single It is possible to perform motion artifact correction with higher accuracy than conventional methods that use image data obtained by different types of imaging processing.

The present inventor focused on the fact that conventional motion artifact correction methods only refer to one type of information (for example, intensity information), and instead We found that correction accuracy can be improved by comprehensively referring to The exemplary embodiments and several exemplary aspects described above have been conceived to implement this technical idea.

In some exemplary aspects, the first data shift amount calculation unit (lateral shift amount calculation unit 223) calculates a correlation coefficient based on the plurality of front image data constructed by the projection processing unit (222). , the data shift amount (lateral shift amount) may be calculated based on this correlation coefficient. With such a configuration, the method disclosed in Non-Patent Document 1 etc. can be extended to a method using multiple types of image data.

Furthermore, in some exemplary embodiments, the first data shift amount calculation section (lateral shift amount calculation section 223) may include a division section (2231) and a correlation coefficient calculation section (2232). The dividing unit (2231) is configured to divide each of the plurality of front image data constructed by the projection processing unit (222) into partial image data groups (strip groups). The correlation coefficient calculation unit (2232) is configured to calculate a correlation coefficient based on a plurality of partial image data groups (a plurality of strip groups) acquired from a plurality of front image data by the division unit (2231). ing. Note that in the exemplary embodiment described above, the soft amount calculation unit 2233 calculates the data shift amount (lateral shift amount) based on the correlation coefficient calculated by the correlation coefficient calculation unit 2232. With such a configuration, the method disclosed in Non-Patent Document 1 etc. can be extended to a method using multiple types of image data.

Further, some example aspects may be configured as follows. Any two of the plurality of front image data (first front image data, second front image data) constructed by the projection processing unit (222) are considered.

First, the number of partial image data (strips) included in the first partial image data group (first strip group) of the first front image data is ) is equal to the number of partial image data (strips) included in That is, a plurality of strip groups including the same number of strips are formed from a plurality of front image data.

Furthermore, for any partial image data (strip) included in the first partial image data group (first strip group), the partial data (first strip) of the data collected by the scanning unit that corresponds to the partial image data (strip) is 1 partial data) is the same as the partial data (second partial data) of the data corresponding to any partial image data (strip) included in the second partial image data group (second strip group). In other words, assuming that the data collected by the scanning unit is divided into multiple partial data, the partial image data corresponding to each partial data (strip representing the area where the partial data was collected) is divided into multiple frontal images. One from each of the data.

According to such an exemplary aspect, it is possible to set a one-to-one correspondence relationship between a plurality of partial image data groups from a plurality of front image data, with the data collection area of the sample as a reference. Therefore, by selecting the partial image data corresponding to the first region of the sample one by one from the plurality of partial image data, it is possible to set the plurality of partial image data corresponding to the first region. Similarly, a plurality of partial image data corresponding to the second area can be set by selecting partial image data corresponding to the second area of the sample one by one from a plurality of partial image data. Then, the motion artifact between the data collected from the first region (first partial data) and the data collected from the second region (second partial data) is generated in the plurality of partial image data corresponding to the first region. and a plurality of partial image data corresponding to the second area.

On the other hand, in the conventional method described in Non-Patent Document 1, for example, the difference between the data collected from the first region (first partial data) and the data collected from the second region (second partial data) is The motion artifact of is corrected based on the single partial image data corresponding to the first region and the single partial image data corresponding to the second region. According to the method of this aspect, correlation calculation can be performed with higher precision than such conventional methods, and therefore it is possible to improve the precision of motion artifact correction.

In some exemplary aspects, the correlation coefficient calculation unit (2232) calculates at least two of the plurality of partial image data of the plurality of frontal image data corresponding to the first partial data of the data collected by the scanning unit. and a second group including at least two of the plurality of partial image data of the plurality of front image data corresponding to the second partial data. The configuration may be configured to calculate a correlation coefficient.

That is, the first group consists of two or more partial image data (strips) corresponding to the first partial data collected from the first region of the sample. Moreover, the second group consists of two or more partial image data (strips) corresponding to the second partial data collected from the second region of the sample. The correlation coefficient calculation unit (2232) can calculate the correlation coefficient between the first partial data and the second partial data based on the first group and the second group.

Furthermore, the first data shift amount calculation section (shift amount calculation section 2233) calculates the data shift amount (lateral shift amount). According to such an exemplary aspect, it is possible to calculate the correlation coefficient with higher accuracy than conventional methods, and therefore it is possible to improve the accuracy of motion artifact correction.

The scanning imaging device (ophthalmological device 1) according to some exemplary embodiments may further include a second data shift amount calculation section (axial shift amount calculation section 224). The second data shift amount calculation unit (axial shift amount calculation unit 224) calculates the amount of data in the depth direction (axial direction) based on any one or more of the plurality of three-dimensional image data constructed by the imaging processing unit (221). It is configured to calculate the data shift amount (axial shift amount) at .

Further, the correction unit (225) performs motion artifact correction based on the data shift amount (axial shift amount) calculated by the second data shift amount calculation unit (axial shift amount calculation unit 224) on the data collected by the scanning unit. can be configured to be applied to image data based on .

According to these exemplary aspects, in addition to lateral motion artifacts, axial motion artifacts can also be corrected.

Further, an axial shift amount is calculated based on two or more three-dimensional image data among the plurality of three-dimensional image data constructed by the imaging processing unit (221), and based on this axial shift amount, By performing motion artifact correction in the axial direction, it becomes possible to correct motion artifacts in the axial direction with higher accuracy than before.

Exemplary aspects of a method of controlling a scanning imaging device that can be provided by the ophthalmological apparatus 1 according to the exemplary embodiment described above will be described. In some exemplary embodiments, a scanning imaging device includes a scanning portion that applies a scan to a sample to collect data, and a processor, and a method for controlling the same includes the steps described below. It may be included.

First, a method for controlling a scanning imaging apparatus controls a scanning unit to collect data by applying scanning to a sample according to a two-dimensional pattern including a series of cycles. Furthermore, this control method causes the processor to apply a plurality of different imaging processes to the data collected by the scanning unit to construct a plurality of three-dimensional image data, and to apply a plurality of three-dimensional image data to each of the constructed three-dimensional image data. Apply depth direction projection to construct multiple frontal image data, calculate the amount of data shift in the direction orthogonal to the depth direction based on the constructed multiple frontal image data, and calculate the calculated data shift amount. control to perform motion artifact correction of the image data based on the data collected by the scanning unit.

According to such a method of controlling a scanning imaging device, it is possible to improve the accuracy of motion artifact correction, similarly to the scanning imaging device (for example, the ophthalmological device 1) according to the exemplary embodiment. It should be noted that any of the matters described with respect to the ophthalmological device 1 according to the exemplary embodiments above can be combined with the method of controlling a scanning imaging device according to the exemplary aspects.

Exemplary aspects of the image processing device that can be provided by the ophthalmologic apparatus 1 according to the above exemplary embodiment will be described. In some exemplary aspects, the image processing device includes a storage unit, an imaging processing unit, a projection processing unit, a first data shift amount calculation unit, and a correction unit.

The storage unit is configured to store data collected by applying a scan to the sample according to a two-dimensional pattern including a series of cycles. In the exemplary embodiment described above, this storage corresponds to storage 212.

The imaging processing section is configured to construct a plurality of three-dimensional image data by applying a plurality of different imaging processes to the data stored in the storage section. In the exemplary embodiment described above, this imaging processing unit corresponds to the imaging processing unit 221.

The projection processing section is configured to construct a plurality of front image data by applying depth direction projection to each of the plurality of three-dimensional image data constructed by the imaging processing section. In the exemplary embodiment described above, this projection processing section corresponds to the projection processing section 222.

The first data shift amount calculation section is configured to calculate a data shift amount in a direction orthogonal to the depth direction based on the plurality of front image data constructed by the projection processing section.

The correction section is configured to perform motion artifact correction of the image data based on the data collected by scanning, based on the data shift amount calculated by the first data shift amount calculation section.

According to such an image processing device, it is possible to improve the accuracy of motion artifact correction, similarly to the scanning imaging device (for example, the ophthalmological device 1) according to the exemplary embodiment. Note that any of the matters described with respect to the ophthalmological apparatus 1 according to the exemplary embodiments above can be combined into the image processing apparatus according to the exemplary embodiments.

An exemplary aspect of a method for controlling an image processing device that can be provided by the ophthalmologic apparatus 1 according to the exemplary embodiment described above will be described. In some exemplary embodiments, an image processing device includes a storage unit and a processor, and a method for controlling the same may include the steps described below.

First, a method for controlling an image processing apparatus applies scanning according to a two-dimensional pattern including a series of cycles to a sample, and stores collected data in a storage unit. Furthermore, this control method causes the processor to construct a plurality of three-dimensional image data by applying a plurality of different imaging processes to the data stored in the storage unit, and to apply a plurality of three-dimensional image data to each of the constructed three-dimensional image data. Apply depth direction projection to construct multiple frontal image data, calculate the amount of data shift in the direction orthogonal to the depth direction based on the constructed multiple frontal image data, and calculate the calculated data shift amount. control to perform motion artifact correction of the image data based on the data collected by the scanning unit.

According to such a method of controlling an image processing device, it is possible to improve the accuracy of motion artifact correction, similarly to the scanning imaging device (for example, the ophthalmological device 1) according to the exemplary embodiment. Note that any of the matters described regarding the ophthalmological apparatus 1 according to the above exemplary embodiment can be combined with the method for controlling an image processing apparatus according to the exemplary embodiment.

Exemplary aspects of the scanning imaging method that can be provided by the ophthalmologic apparatus 1 according to the exemplary embodiments described above will be described. In some example aspects, a scanning imaging method may include the steps described below.

First, scanning imaging methods collect data by applying a scan to a sample that follows a two-dimensional pattern that includes a series of cycles, and then apply multiple different imaging processes to the collected data to create multiple three-dimensional image data. Build. Furthermore, the scanning imaging method applies depth direction projection to each of the constructed plurality of three-dimensional image data to construct a plurality of frontal image data, and performs depth projection based on the constructed plurality of frontal image data. Calculate the amount of data shift in the direction perpendicular to the horizontal direction. In addition, the scanning imaging method performs motion artifact correction of the image data based on the data collected by the scanning, based on the calculated data shift amount.

According to such a scanning imaging method, it is possible to improve the accuracy of motion artifact correction, similar to the scanning imaging device (eg, ophthalmological device 1) according to the exemplary embodiment. It should be noted that any of the matters described with respect to the ophthalmic device 1 according to the exemplary embodiments above can be combined in the scanning imaging method according to the exemplary aspects.

Exemplary aspects of the image processing method that can be provided by the ophthalmological apparatus 1 according to the above exemplary embodiment will be described. In some example aspects, an image processing method may include the steps described below.

First, the image processing method prepares the collected data by applying a scan to the sample according to a two-dimensional pattern that includes a series of cycles. For example, sample data is accepted via a communication line or recording medium. Alternatively, data is collected by applying a scan to the sample.

Furthermore, the image processing method applies a plurality of different imaging processes to the prepared data to construct a plurality of three-dimensional image data, and performs a depth direction projection on each of the plurality of constructed three-dimensional image data. is applied to construct multiple frontal image data. Furthermore, the image processing method calculates a data shift amount in a direction perpendicular to the depth direction based on the plurality of constructed frontal image data, and based on the calculated data shift amount, an image based on the prepared data is calculated. Perform motion artifact correction on the data.

According to such an image processing method, it is possible to improve the accuracy of motion artifact correction similarly to the scanning imaging device (for example, ophthalmological device 1) according to the exemplary embodiment. Note that any of the matters described with respect to the ophthalmologic apparatus 1 according to the exemplary embodiments above can be combined with the image processing method according to the exemplary embodiments.

In some exemplary embodiments, a program causes a computer to execute any aspect of a method for controlling a scanning imaging device, a program causes a computer to execute any aspect of a method for controlling an image processing device, and image processing. It is possible to provide a program that causes a computer to perform any aspect of the method or a program that causes a computer to perform any aspect of the scanning imaging method.

Furthermore, it is possible to create a computer-readable non-temporary recording medium that records such a program. This non-transitory recording medium may be in any form, examples of which include magnetic disks, optical disks, magneto-optical disks, and semiconductor memory.

Although the above exemplary embodiments have been described in detail where Lissajous scans are used, other scan modes may also be employed, as mentioned above. For example, in some exemplary embodiments, the scanning unit may collect data by applying a scan (raster scan) to the sample that follows a raster pattern that includes a series of parallel line scans.

In this case, the imaging processing unit can construct a plurality of three-dimensional image data by applying a plurality of different imaging processes to data collected by raster scanning. The projection processing unit can construct a plurality of front image data by applying depth direction projection to each of the constructed three-dimensional image data. The first data shift amount calculation unit can calculate the data shift amount in the direction orthogonal to the depth direction based on the plurality of constructed front image data. The correction unit can perform motion artifact correction of (arbitrary) image data based on the data collected by raster scan, based on the calculated data shift amount.

According to these exemplary aspects, the motion artifact correction is configured to be performed using a plurality of image data obtained by applying a plurality of types of imaging processing to data collected by raster scanning. Therefore, it is possible to perform motion artifact correction with higher accuracy than conventional methods that use image data obtained by a single type of imaging processing.

Any of what has been described with respect to the ophthalmic device 1 according to the exemplary embodiments above can be combined into a scanning imaging device according to such exemplary aspects. Additionally, the scanning imaging device according to the exemplary aspects described herein includes some aspects of the scanning imaging device, as well as some aspects of the method of controlling the scanning imaging device, and some aspects of the image processing device. Some aspects of the method for controlling an image processing device, some aspects of the image processing method, and some aspects of the scanning imaging method are provided. Furthermore, it is possible to configure a program that causes a computer to perform a method according to any of these exemplary aspects, and it is also possible to create a computer-readable non-transitory storage medium recording this program. .

In the exemplary embodiments and aspects described above, a plurality of different imaging processes are applied to data collected with a Lissajous scan (or other modes of scanning) to construct a plurality of types of three-dimensional image data. Although motion artifact correction is performed by determining a data shift amount from a plurality of frontal image data constructed from dimensional image data, it is also possible to implement a mode different from this.

For example, some exemplary embodiments construct three-dimensional image data from data collected with a Lissajous scan (or other modes of scanning), and perform multiple different processes on this three-dimensional image data. (Projection) may be applied to construct a plurality of types of front image data, and a data shift amount may be determined from these front image data to perform motion artifact correction. Some specific examples of such aspects will be described below. Note that the following specific examples are merely illustrative and are not intended to be limiting.

Some example embodiments construct OCT intensity image data as three-dimensional image data based on collected data, and as frontal image data based on the OCT intensity image data, any two or more of the following frontal image data: may be configured to construct: frontal image data with logarithmic scale linear projection; frontal image data with logarithmic scale projection; frontal image data with maximum intensity projection; frontal image data with other projection techniques. Two or more pieces of frontal image data constructed from OCT intensity image data are used to calculate the data shift amount. The calculated data shift amount is used for motion artifact correction of arbitrary image data based on the collected data.

Some example aspects construct OCT angiography image data as three-dimensional image data based on the acquired data, and as en-face image data based on the OCT angiography image data, any of the following en-face image data: Image data obtained by projecting the entire OCT angiographic image data (full-depth frontal OCT angiographic image); projection of a portion of the OCT angiographic image data (slab projection) ) image data obtained by other projection methods; frontal image data obtained by other projection methods. Two or more pieces of frontal image data constructed from OCT angiography image data are used to calculate the data shift amount. The calculated data shift amount is used for motion artifact correction of arbitrary image data based on the collected data.

Some example embodiments construct OCT polarization image data as three-dimensional image data based on collected data, and as frontal image data based on the OCT polarization image data, any two or more of the following frontal image data: may be configured to construct: polarization uniformity (DOPU) maps; birefringence maps; retinal pigment epithelial-sensitive contrast maps; retinal pigment epithelium elevation maps; en-face image data from other projection techniques. Two or more pieces of front image data constructed from OCT polarization image data are used to calculate the data shift amount. The calculated data shift amount is used for motion artifact correction of arbitrary image data based on the collected data.

According to these specific examples, a scanning imaging device having the following aspects can be obtained, and various aspects corresponding thereto (scanning imaging device control method, image processing device, image processing device control method, scanning imaging device) can be obtained. An imaging method, an image processing method, a program, and a recording medium) are obtained.

That is, a scanning imaging device according to some example aspects includes a scanning unit that collects data by applying a scan to a sample according to a two-dimensional pattern that includes a series of cycles, and an imaging process that applies an imaging process to the data. an imaging processing unit that constructs three-dimensional image data by applying a plurality of different processes (projections) to the three-dimensional image data, a projection processing unit that constructs a plurality of frontal image data, and a plurality of frontal image data. a first data shift amount calculating section that calculates a data shift amount in a direction perpendicular to the depth direction based on the data shift amount; and a motion artifact of image data based on the data collected by the scanning section based on the data shift amount. and a correction unit that performs correction.

For such a scanning imaging device, it is possible to combine any of the features described with respect to the ophthalmological device 1 according to the exemplary embodiments above. For example, the following scanning imaging device can be obtained.

That is, a scanning imaging device according to some exemplary aspects includes a scanning unit that collects data by applying a scan to a sample according to a two-dimensional pattern that includes a series of cycles; and one or more imaging processes on the data. an imaging processing unit that applies one or more three-dimensional image data to construct one or more three-dimensional image data; and one or more processing (projection) to each of the one or more three-dimensional image data to construct a plurality of frontal image data. a projection processing unit; a first data shift amount calculation unit that calculates a data shift amount in a direction perpendicular to the depth direction based on the plurality of frontal image data; and a correction unit that performs motion artifact correction of the image data based on the data. For such a scanning imaging device, it is possible to combine any of the features described with respect to the ophthalmological device 1 according to the exemplary embodiments above.

Also, as previously discussed, the scan mode applicable to the exemplary embodiments is any, as long as data collection from a three-dimensional region of the sample is possible, and scanning imaging such as It is possible to configure the device.

That is, a scanning imaging device according to some example aspects includes a scanning unit that collects data from a three-dimensional region of a sample and applies one or more imaging processes to the data to generate one or more three-dimensional images. an imaging processing unit that constructs data; a projection processing unit that applies one or more processes (projections) to each of the one or more three-dimensional image data to construct a plurality of frontal image data; and a projection processing unit that constructs a plurality of frontal image data. a first data shift amount calculation unit that calculates a data shift amount in a direction perpendicular to the depth direction based on the data; and a motion of image data based on the data collected by the scanning unit based on the data shift amount. and a correction unit that performs artifact correction. For such a scanning imaging device, it is possible to combine any of the features described with respect to the ophthalmological device 1 according to the exemplary embodiments above.

Several other examples of the operation of the ophthalmological apparatus 1 according to the above exemplary embodiment will be described (see FIGS. 9 to 11). The same preparatory processes as before, such as entering the patient ID, setting the scan mode (specifying Lissajous scan), presenting a fixation target, alignment, focus adjustment, and OCT optical path length adjustment, have already been performed. do.

First, the operation example shown in FIG. 9 will be explained. This operation example is a modification of step S2 of the operation example shown in FIG.

(S11: Collect OCT interference signals with Lissajous scan)
In the same manner as step S1 of the operation example in FIG. 8, Lissajous scanning is applied to the eye E (fundus Ef) to be examined, and OCT interference signals as three-dimensional data are collected. The collected three-dimensional data is sent to the image construction section 220.

(S12: Build single 3D image data)
The imaging processing section 221 of the image construction section 220 applies one imaging process to the three-dimensional data collected in step S11 to construct a single three-dimensional image data. The constructed single three-dimensional image data is sent to the projection processing section 222.

Note that in step S2 in FIG. 8, which corresponds to this step, a plurality of different imaging processes are applied to the three-dimensional data to construct a plurality of types of three-dimensional image data.

(S13: Build multiple types of frontal image data)
The projection processing unit 222 constructs multiple types of front image data by applying projection in the depth direction (axial direction) to the single three-dimensional image data constructed in step S12. Here, the projection processing unit 222 applies a plurality of different projections to a single three-dimensional image data. As a result, a plurality of different types of frontal image data containing mutually different information are constructed. The constructed plural types of frontal image data are sent to the lateral shift amount calculation unit 223. Further, any of the plurality of types of three-dimensional image data is sent to the axial shift amount calculation unit 224.

In some examples, if the OCT intensity image data is constructed in step S12, then in step S13 the OCT intensity image data is subjected to one of, for example, a logarithmic scale linear projection, a logarithmic scale projection, and a maximum value projection. Any two or all may be applied.

In some examples, when OCT angiography image data is constructed in step S12, step S13 may include, for example, a projection of the entire OCT angiography image data and/or a portion of the OCT angiography image data. Projection of only the part is executed. The latter may be, for example, a projection of one or more slabs extracted from this OCT angiography image data by segmentation.

In some examples, when OCT polarization image data is constructed in step S12, in step S13, a polarization uniformity map, a birefringence map, a retinal pigment epithelium-sensitive contrast map, etc. , and any two, any three, or all of the retinal pigment epithelial elevation maps are constructed.

Note that in step S3 in FIG. 8, which corresponds to this step, one or more types of imaging processing can be applied. For example, step S3 in FIG. 8 is configured to construct a plurality of different types of front image data by applying the same type of projection to each of the multiple types of three-dimensional image data constructed in step S2. It's okay to be. In another example, step S3 in FIG. 8 is configured to classify the plurality of types of three-dimensional image data constructed in step S2 into a plurality of groups, and apply the same type of projection to each group. good. In yet another example, step S3 in FIG. 8 may be configured to apply a plurality of different types of projection to each of the plurality of types of three-dimensional image data constructed in step S2. In still another example, step S3 in FIG. 8 classifies the plurality of types of three-dimensional image data constructed in step S2 into a plurality of groups, and applies one or more types of projection to each group. may be configured.

(S14-S17)
Steps S14, S15, S16, and S17 are executed in the same manner as steps S4, S5, S6, and S7 in FIG. 8, respectively (end).

Next, an example of operation shown in FIG. 10 will be explained. This operation example is a modification (generalization) of step S1 of the operation example shown in FIG. That is, although step S1 in FIG. 8 employs a Lissajous scan, step S21 in this operational example is not limited to the Lissajous scan, and any three-dimensional scan may be employed.

(S21: Apply scanning to the three-dimensional region of the sample and collect OCT interference signals)
The scan control unit 2111 applies scanning to a three-dimensional region of the sample (tested eye E, fundus Ef) and collects an OCT interference signal as three-dimensional data. The collected three-dimensional data is sent to the image construction section 220.

Any scan pattern can be employed in this step, and any three-dimensional scan mode can be used, such as raster scan, Lissajous scan, radial scan, and multi-cross scan.

The three-dimensional region of the sample is defined by, for example, a fixation position and a scan pattern. For example, when a fixation position for macular imaging is applied, a three-dimensional region surrounding the center of the macula (fovea) is defined. When a fixation position for optic disc imaging is applied, a three-dimensional region surrounding the optic disc is defined. When a raster pattern is applied as a scan pattern, the boundaries (outer edges) of the B-scan groups that make up this raster scan define a three-dimensional area. When a Lissajous pattern is employed as a scan pattern, a rectangle (typically a square) circumscribing this Lissajous pattern defines a three-dimensional area. When a radial pattern is employed as the scan pattern, a figure (typically a circle or a polygon) circumscribing the radial pattern defines a three-dimensional area. The same applies when other scan patterns such as a multi-cross pattern are employed. The range thus defined is the range in the lateral direction (xy direction).

The range in the axial direction (z-direction) is defined, for example, by the imaging range of A-scan (the range in the z-direction to be imaged). The imaging range of the A-scan is determined by, for example, the measurement arm length and the reference arm length (e.g., in the ophthalmic apparatus 1 of the exemplary embodiment described above, the position of the retroreflector 41 of the measurement arm and/or the reference arm retroreflector 41). (position of reflector 114). Here, the imaging range of the A-scan may be determined by taking into account the focus position of the measurement arm (for example, the position of the OCT focusing lens 43 in the ophthalmologic apparatus 1 of the exemplary embodiment described above).

By the method exemplified above, the range in the lateral direction (xy direction) and the axial direction (z direction) can be determined, and as a result, the three-dimensional region of the sample can be defined. Note that the method for defining the three-dimensional region of the sample is not limited to this, and any other method may be adopted.

(S22-S27)
Steps S22, S23, S24, S25, S26, and S27 are executed in the same manner as steps S2, S3, S4, S5, S6, and S7 in FIG. 8, respectively (end).

Next, an example of operation shown in FIG. 11 will be explained. This operation example is obtained by generalizing step S1 of the operation example shown in FIG. 8 and modifying step S2.

(S31: Apply scanning to the three-dimensional region of the sample and collect OCT interference signals)
In the same manner as step S21 in FIG. 10, scanning is applied to the three-dimensional region of the sample (eye E, fundus Ef), and OCT interference signals as three-dimensional data are collected. The collected three-dimensional data is sent to the image construction section 220.

(S32: Build single 3D image data)
In the same manner as step S12 in FIG. 9, the imaging processing section 221 of the image construction section 220 constructs a single three-dimensional image data from the three-dimensional data collected in step S31. That is, the imaging processing unit 221 applies one imaging process to the three-dimensional data collected in step S31 to construct a single three-dimensional image data. The constructed single three-dimensional image data is sent to the projection processing section 222.

(S33: Build multiple types of frontal image data)
In the same manner as step S13 in FIG. 9, the projection processing unit 222 applies a plurality of different projections to the single three-dimensional image data constructed in step S32. As a result, a plurality of different types of frontal image data containing mutually different information are constructed. The constructed plural types of frontal image data are sent to the lateral shift amount calculation unit 223. Further, any of the plurality of types of three-dimensional image data is sent to the axial shift amount calculation unit 224.

(S34-S37)
Steps S34, S35, S36, and S37 are executed in the same manner as steps S4, S5, S6, and S7 in FIG. 8, respectively (end).

According to some operational examples as shown in FIGS. 9 to 11, one or more types of imaging processing are applied to data collected from a three-dimensional region of a sample to construct one or more three-dimensional image data. However, it is possible to construct multiple types of frontal image data by applying one or more types of projection to the one or more three-dimensional image data, and to perform motion artifact correction using these frontal image data. Therefore, similar to the ophthalmological apparatus 1 according to the exemplary embodiment described above, it is possible to perform motion artifact correction with higher accuracy than conventional methods using a single type of image data.

A scanning imaging device control method, an image processing device, an image processing device control method, a scanning imaging method, an image processing method, a program, and Similarly, depending on the recording medium, it is possible to perform motion artifact correction with higher accuracy than conventional methods that use a single type of image data.

This disclosure is merely illustrative of some embodiments and is not intended to be a limitation of the invention. Those who wish to carry out this invention can make arbitrary modifications (omissions, substitutions, additions, etc.) within the scope of the gist of this invention.

1 Ophthalmological apparatus 44 Optical scanner 100 OCT unit 211 Main control section 2111 Scanning control section 212 Storage section 2121 Scanning protocol 220 Image construction section 221 Imaging processing section 222 Projection processing section 223 Lateral shift amount calculation section 2231 Division section 2232 Correlation coefficient calculation Section 2233 Shift amount calculation section 224 Axial shift amount calculation section 225 Correction section

Claims (21)

a scanning unit that collects data from a three-dimensional area of the sample;
an imaging processing unit that applies one or more imaging processes to the data to construct one or more three-dimensional image data;
a projection processing unit that constructs a plurality of frontal image data by applying depth direction projection to each of the one or more three-dimensional image data;
a first data shift amount calculation unit that calculates a data shift amount in a direction orthogonal to the depth direction based on the plurality of front image data;
and a correction section that performs motion artifact correction on image data based on the data collected by the scanning section, based on the data shift amount. The imaging processing unit applies a plurality of different imaging processes to the data collected by the scanning unit to construct a plurality of three-dimensional image data,
The projection processing unit applies the projection to each of the plurality of three-dimensional image data to construct the plurality of front image data.
The scanning imaging device of claim 1. The imaging processing unit constructs a single three-dimensional image data from the data collected by the scanning unit,
The projection processing unit applies a plurality of different projections to the single three-dimensional image data to construct the plurality of front image data.
The scanning imaging device of claim 1. The first data shift amount calculation unit calculates a correlation coefficient based on the plurality of front image data, and calculates the data shift amount based on the correlation coefficient.
A scanning imaging device according to any one of claims 1 to 3. The first data shift amount calculation unit includes:
a dividing unit that divides each of the plurality of frontal image data into partial image data groups;
The scanning imaging device according to claim 4, further comprising: a correlation coefficient calculation section that calculates the correlation coefficient based on a plurality of partial image data groups acquired from the plurality of front image data by the division section. Regarding the first front image data and the second front image data among the plurality of front image data,
The number of partial image data included in the first partial image data group of the first front image data is equal to the number of partial image data included in the second partial image data group of the second front image data,
The partial data of the data collected by the scanning unit, which corresponds to one partial image data included in the first partial image data group, is included in any partial image data included in the second partial image data group. is the same as the corresponding partial data of said data;
A scanning imaging device according to claim 5. The correlation coefficient calculation section includes a first group including at least two of the plurality of partial image data of the plurality of front image data corresponding to the first partial data of the data collected by the scanning section; A correlation coefficient corresponding to the first partial data and the second partial data is calculated based on a second group including at least two of the plurality of partial image data of the plurality of front image data corresponding to the partial data. Calculate,
The first data shift amount calculation unit calculates a data shift amount corresponding to the first partial data and the second partial data based on the correlation coefficient.
7. A scanning imaging device according to claim 6. further comprising a second data shift amount calculation unit that calculates a data shift amount in the depth direction based on any one or more of the one or more three-dimensional image data,
The correction unit applies motion artifact correction based on the data shift amount calculated by the second data shift amount calculation unit to the image data based on the data collected by the scanning unit.
A scanning imaging device according to any one of claims 1 to 7. The scanning unit applies optical coherence tomography (OCT) scanning to the sample,
The one or more three-dimensional image data constructed by the imaging processing unit include OCT intensity image data, OCT angiography image data, composite image data based on OCT intensity image data and OCT angiography image data, and OCT containing any of the polarization image data,
A scanning imaging device according to any one of claims 1 to 8. The scanning unit collects the data by applying scanning to the three-dimensional area according to a two-dimensional pattern including a series of cycles.
A scanning imaging device according to any one of claims 1 to 9. The scanning unit includes:
including a deflector capable of deflecting light in a first direction and a second direction that are different from each other, and
The scanning is applied to the sample by repeating the change in the deflection direction along the first direction in a first period and the change in the deflection direction along the second direction in a second period different from the first period. do,
11. The scanning imaging device of claim 10. The deflector is controlled based on a preset scanning protocol based on a Lissajous function.
The scanning imaging device of claim 11. the series of cycles intersect with each other;
A scanning imaging device according to any one of claims 10 to 12. the two-dimensional pattern is a raster pattern including a series of line scans parallel to each other;
11. The scanning imaging device of claim 10. A method for controlling a scanning imaging device including a scanning unit that applies scanning to a sample to collect data, and a processor, the method comprising:
controlling the scanning unit to collect data from a three-dimensional region of the sample;
The processor,
Applying one or more imaging processes to the data to construct one or more three-dimensional image data,
constructing a plurality of frontal image data by applying depth direction projection to each of the one or more three-dimensional image data;
calculating a data shift amount in a direction perpendicular to the depth direction based on the plurality of front image data;
controlling to perform motion artifact correction of image data based on the data collected by the scanning unit based on the data shift amount;
A method for controlling a scanning imaging device. a storage unit that stores data collected by applying scanning to a three-dimensional region of the sample;
an imaging processing unit that applies one or more imaging processes to the data to construct one or more three-dimensional image data;
a projection processing unit that constructs a plurality of frontal image data by applying depth direction projection to each of the one or more three-dimensional image data;
a first data shift amount calculation unit that calculates a data shift amount in a direction perpendicular to the depth direction based on the plurality of front image data;
and a correction unit that performs motion artifact correction of image data based on the data collected by the scanning based on the data shift amount. A method for controlling an image processing device including a storage unit and a processor, the method comprising:
storing data collected by applying scanning to a three-dimensional region of the sample in the storage unit;
The processor,
Applying one or more imaging processes to the data to construct one or more three-dimensional image data,
constructing a plurality of frontal image data by applying depth direction projection to each of the one or more three-dimensional image data;
calculating a data shift amount in a direction perpendicular to the depth direction based on the plurality of front image data;
controlling to perform motion artifact correction of image data based on the data collected by the scanning based on the data shift amount;
A method for controlling an image processing device. collecting data by applying a scan to a three-dimensional region of the sample;
Applying one or more imaging processes to the data to construct one or more three-dimensional image data,
constructing a plurality of frontal image data by applying depth direction projection to each of the one or more three-dimensional image data;
calculating a data shift amount in a direction perpendicular to the depth direction based on the plurality of front image data;
performing motion artifact correction of image data based on the data collected by the scanning, based on the data shift amount;
Scanning imaging method. preparing the collected data by applying a scan to a three-dimensional region of the sample;
Applying one or more imaging processes to the data to construct one or more three-dimensional image data,
constructing a plurality of frontal image data by applying depth direction projection to each of the one or more three-dimensional image data;
calculating a data shift amount in a direction perpendicular to the depth direction based on the plurality of front image data;
performing motion artifact correction of image data based on the prepared data based on the data shift amount;
Image processing method. A program that causes a computer to execute the method according to any one of claims 15, 17, 18, and 19. A computer-readable non-transitory recording medium on which the program according to claim 20 is recorded.

Images