JP2021013506A - Optical coherence tomography (oct) data processing method, oct imaging method, oct data processing apparatus, control method thereof, oct apparatus, control method thereof, program, and recording medium - Google Patents

Abstract

To improve efficiency of optical coherence tomography (OCT) data processing.SOLUTION: An OCT apparatus includes an OCT scanner, an image data generation unit, a registration unit, and an image data synthesis unit. The OCT scanner applies OCT scan to a plurality of three-dimensional mutually different regions of a sample, and collects a plurality of three-dimensional data sets. The image data generation unit generates a plurality of pieces of image data from the plurality of three-dimensional data sets. The registration unit executes first registration between the plurality of pieces of image data by applying an image correlation operation to each of a plurality of overlapping regions in the plurality of pieces of image data. The image data synthesis unit synthesizes the plurality of pieces of image data on the basis of a result of the first registration.SELECTED DRAWING: Figure 2

Description

The present invention relates to an OCT data processing method, an OCT imaging method, an OCT data processing device, a control method thereof, an OCT device, a control method thereof, a program, and a recording medium.

OCT is a technique capable of imaging a light scattering medium with a resolution of a micrometer level or less, and is used for medical imaging, non-destructive inspection, and the like. OCT is a technique based on low coherence interferometry, which typically utilizes near-infrared light to ensure the depth of light scattering media into a sample.

Patent Document 1 states that in order to efficiently collect OCT data and to collect OCT data from a specific region of a sample accurately and in a short time, the optical depth is measured by backscattering or backreflection. A method of processing the OCT data set obtained as a function of the above, which analyzes the OCT data set to identify at least the first subset of landmark area data and positions the OCT data set based on this landmark area data. A method of processing at least a second subset of the OCT data set based on the correspondence between the OCT data set and the landmark area data is disclosed.

Further, in Patent Document 2, in order to monitor the progression of a disease, an OCT survey data set is acquired as a function of optical depth by measuring backward scattering or backward reflection, and this survey scan data set is analyzed. Of a survey scan dataset that represents at least a portion of the affected tissue region associated with the landmark region by identifying the landmark region and assigning a location in the sample or a location associated with a fixed location to an element of the survey scan dataset. Methods are disclosed for registering parts and monitoring changes in the area of affected tissue at different time points.

U.S. Pat. No. 7,884,945 U.S. Pat. No. 8,405,834

An object of the present invention is to improve the efficiency of OCT data processing.

Some exemplary embodiments are methods of processing data collected by optical coherence stromography (OCT) scans (OCT data processing methods), wherein a plurality of data collected from a plurality of different three-dimensional regions of a sample. The plurality of images are prepared by preparing the three-dimensional data set of the above, generating a plurality of image data from the plurality of three-dimensional data sets, and applying an image correlation calculation to each of the plurality of overlapping regions in the plurality of image data. A first registration is performed between the data, and the plurality of image data are combined based on the result of the first registration.

The OCT data processing method of some exemplary embodiments can be combined with any of the following embodiments: The image correlation calculation for the first and second image data including overlapping regions. Calculates at least one of the parallel movement amount and the rotational movement amount between the overlapping region in the first image data and the overlapping region in the second image data; the image correlation calculation is phase-limited. Correlation is included; a frontal image of the sample is prepared and a second registration between the plurality of image data is performed based on the frontal image prior to the first registration.

Some exemplary embodiments are imaging methods using optical coherence tomography (OCT), in which a plurality of 3D datasets are collected from a plurality of different 3D regions of a sample and said to the plurality of 3Ds. A plurality of image data are generated from the data set, and an image correlation calculation is applied to each of the plurality of overlapping regions in the plurality of image data to perform a first registration between the plurality of image data. The plurality of image data are combined based on the result of registration in 1.

The imaging method of some exemplary embodiments can be combined with any of the following embodiments: a imaging region is designated for the sample and multiple target regions are set to include the imaging region. , The plurality of three-dimensional data sets are collected by sequentially applying OCT scans to the plurality of target regions; two three-dimensional data sets are applied to two target regions of the plurality of target regions. Data sets are collected, two image data are generated from the two three-dimensional data sets, overlapping areas of the two image data are evaluated, and depending on the result of the evaluation, the said for at least the two image data. Apply the first registration or reapply the OCT scan to at least one of the two target areas; modify at least one of the two target areas based on the results of the evaluation and change the target Reapply the OCT scan to the region; prepare a frontal image of the sample, modify at least one of the two target regions based on the frontal image, and modify the OCT to the modified target region. The scan is reapplied; the partial image data corresponding to the shooting area is extracted from the composite image data obtained by synthesizing the plurality of image data; the plurality of images are based on the result of the first registration. The partial image data group corresponding to the shooting region is extracted from the image data, and the partial image data group is synthesized; the image correlation calculation is performed on the first image data and the second image data including the overlapping region. At least one of the parallel movement amount and the rotational movement amount between the overlapping region in the first image data and the overlapping region in the second image data is calculated; the image correlation calculation is a phase-limited correlation calculation. Including.

Some exemplary embodiments are devices that process data collected by optical coherence stromography (OCT) scans, with multiple 3D datasets collected from different 3D regions of a sample. The plurality of reception units are received, the image data generation unit that generates a plurality of image data from the plurality of three-dimensional data sets, and the plurality of overlapping regions in the plurality of image data by applying an image correlation calculation. It includes a registration unit that performs a first registration between image data and an image data synthesizing unit that synthesizes the plurality of image data based on the result of the first registration.

Some exemplary embodiments are methods of controlling an optical coherence stromography (OCT) data processor, including a processor, in which multiple 3D datasets are collected from multiple different 3D regions of a sample. The processor is controlled to accept, the processor is controlled to generate a plurality of image data from the plurality of three-dimensional data sets, and an image correlation calculation is applied to each of a plurality of overlapping regions in the plurality of image data. By doing so, the processor is controlled so as to perform the first registration between the plurality of image data, and the processor is combined so as to synthesize the plurality of image data based on the result of the first registration. Control.

Some exemplary embodiments include an OCT scanner that applies an optical coherence stromography (OCT) scan to multiple different three-dimensional regions of a sample to collect multiple three-dimensional data sets, and the plurality of three-dimensional data. A first registration between the plurality of image data is performed by applying an image correlation calculation to each of the image data generation unit that generates a plurality of image data from the set and the plurality of overlapping regions in the plurality of image data. It is an OCT apparatus including a registration unit for performing the registration and an image data synthesizing unit for synthesizing the plurality of image data based on the result of the first registration.

Some exemplary embodiments are methods of controlling an OCT apparatus that includes an OCT scanner and processor that applies an optical coherence tomography (OCT) scan to a sample, from multiple three-dimensional regions that differ from each other in the sample. The OCT scanner is controlled to collect the three-dimensional data sets of the above, the processor is controlled to generate a plurality of image data from the plurality of three-dimensional data sets, and a plurality of overlapping regions in the plurality of image data are generated. By applying an image correlation calculation to each of the above, the processor is controlled so as to perform a first registration between the plurality of image data, and the plurality of image data are based on the result of the first registration. The processor is controlled to synthesize the above.

Some exemplary embodiments are programs that cause a computer to perform the methods of any of the embodiments.

Some exemplary embodiments are computer-readable non-temporary recording media on which the program of any embodiment is recorded.

According to an exemplary embodiment, it is possible to improve the efficiency of OCT data processing.

It is the schematic which shows the structure of the ophthalmic apparatus (OCT apparatus) which concerns on an exemplary aspect. It is the schematic which shows the structure of the ophthalmic apparatus (OCT apparatus) which concerns on an exemplary aspect. It is a flowchart which shows the operation of the ophthalmic apparatus (OCT apparatus) which concerns on an exemplary aspect. It is a flowchart which shows the operation of the ophthalmic apparatus (OCT apparatus) which concerns on an exemplary aspect. It is a flowchart which shows the operation of the ophthalmic apparatus (OCT apparatus) which concerns on an exemplary aspect. It is the schematic for demonstrating the operation of the ophthalmic apparatus (OCT apparatus) which concerns on an exemplary aspect. It is the schematic for demonstrating the operation of the ophthalmic apparatus (OCT apparatus) which concerns on an exemplary aspect. It is the schematic for demonstrating the operation of the ophthalmic apparatus (OCT apparatus) which concerns on an exemplary aspect. It is the schematic for demonstrating the operation of the ophthalmic apparatus (OCT apparatus) which concerns on an exemplary aspect. It is the schematic for demonstrating the operation of the ophthalmic apparatus (OCT apparatus) which concerns on an exemplary aspect. It is the schematic for demonstrating the operation of the ophthalmic apparatus (OCT apparatus) which concerns on an exemplary aspect. It is the schematic which shows the structure of the ophthalmic apparatus (OCT apparatus) which concerns on an exemplary aspect.

Some exemplary embodiments of the embodiments will be described below. It is possible to refer to the matters disclosed in the documents cited in this specification and the matters relating to any known technology in an exemplary manner.

Some exemplary embodiments relate to techniques for applying OCT scans to a 3D region of a sample to process the collected 3D dataset. This technology can be applied to various processes including OCT scan, setting of OCT scan application area, registration between OCT images, analysis / measurement / segmentation of OCT images, and OCT data processing and OCT scan. Contributes to the efficiency of.

In some exemplary embodiments, the "processor" is, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), a programmable logic device (eg, a SPLD Digital Device). ), CPLD (Complex Programmable Logic Device), FPGA (Field Programmable Gate Array)) and the like. The processor provides some examples for achieving the desired function, for example, by reading a program stored in a storage circuit or a storage device and executing the program.

The type of OCT applicable to some exemplary embodiments is arbitrary, typically a swept source OCT or a spectral domain OCT, but may be of any other type. ..

The swept source OCT divides the light from the variable wavelength light source into measurement light and reference light, superimposes the return light of the measurement light from the sample on the reference light to generate interference light, and this interference light is used as an optical detector. This is a method of constructing an image by performing Fourier transformation or the like on the detection data collected in response to the sweeping of the wavelength and the scanning of the measurement light.

Spectral domain OCT divides the light from a low coherence light source (broadband light source) into measurement light and reference light, and superimposes the return light of the measurement light from the sample on the reference light to generate interference light. This is a method of constructing an image by detecting the spectral distribution of light source with a spectroscope and performing Fourier transform or the like on the detected spectral distribution.

That is, the swept source OCT is an OCT method for acquiring the spectral distribution of interference light by time division, and the spectral domain OCT is an OCT method for acquiring the spectral distribution of interference light by spatial division.

Types other than such Fourier domain OCT include time domain OCT in which scanning in the axial direction (Z direction) is performed mechanically and sequentially, and two-dimensional imaging of the XY plane orthogonal to the Z direction. There is an en-face OCT or a full field OCT.

The exemplary embodiments described below are used in the field of ophthalmology for eye imaging, analysis, measurement, evaluation and the like. Note that some exemplary embodiments may be used in other fields. Examples of other fields include clinical departments other than ophthalmology (dermatology, dentistry, surgery, etc.) and industrial fields (non-destructive inspection, etc.).

1 and 2 show the configuration of an OCT device (ophthalmic device) 100 according to one exemplary embodiment. The ophthalmic apparatus 100 provides an imaging method using OCT.

First, an outline of some features of the ophthalmic apparatus 100 will be described. The ophthalmic apparatus 100 is configured to collect a plurality of 3D data sets from a plurality of different 3D regions of a sample (eye). Typically, the plurality of 3D regions are set so that each 3D region partially overlaps with at least one other 3D region. The ophthalmic apparatus 100 collects a three-dimensional data set corresponding to each three-dimensional region by sequentially performing OCT scans targeting a plurality of three-dimensional regions. The order in which the OCT scans are applied, that is, the order in which they are assigned to the plurality of three-dimensional regions, is arbitrarily set. For example, an order can be assigned to a plurality of 3D regions (n is a positive integer) so that the nth 3D region and the n + 1th 3D region partially overlap. In another example, if one (or two or more) 3D regions partially overlaps all other 3D regions, the order can be arbitrarily assigned to these 3D regions. Considering eye movements, it is possible that the area to which the OCT scan targeting a certain three-dimensional area is actually applied does not match the three-dimensional area, but use fixation or tracking. Can scan an area that almost matches the three-dimensional area. Further, the number of 3D data sets collected from one 3D area is not limited to one. For example, by collecting two or more 3D data sets from one 3D area, averaging for high image quality and motion contrast data generation (OCT angiography, etc.) are executed in the subsequent processing. Is possible.

Further, the ophthalmic apparatus 100 is configured to generate a plurality of image data from a plurality of three-dimensional data sets collected by applying an OCT scan to a plurality of three-dimensional regions. As a result, a plurality of image data corresponding to the plurality of targeted three-dimensional regions can be obtained. The above-mentioned addition averaging and motion contrast data generation may be performed at this stage.

In addition, the ophthalmic apparatus 100 applies an image correlation calculation to each of the plurality of overlapping regions in the plurality of image data corresponding to the plurality of targeted three-dimensional regions, thereby performing a registration between the plurality of image data. It is configured to perform registration (first registration). In other words, the ophthalmic apparatus 100 is configured to arrange (align) a plurality of image data according to the image correlation of their overlapping regions. As described above, the typical plurality of 3D regions are set so that each 3D region partially overlaps with at least one other 3D region, so that each image data is at least one other. It partially overlaps with one other image data. By utilizing the image correlation of such overlapping regions, it is possible to perform registration of these image data, and finally, it is possible to perform integrated registration of a plurality of image data.

In addition, the ophthalmic apparatus 100 is configured to synthesize a plurality of image data based on the result of integrated registration of the plurality of image data. The image data constructed in this manner is called panoramic image data or montage image data, and is computer graphics image data that depicts a wider range than each three-dimensional area.

The ophthalmic apparatus 100 may be configured to specify an imaging area for a sample (eye). For example, the ophthalmic apparatus 100 can specify the imaging area to which the OCT scan is applied in response to a command from the user to specify the imaging area. In another example, the ophthalmic apparatus 100 can specify the imaging region based on the image data obtained from the sample by itself and / or the image data of the sample obtained from another apparatus. In yet another example, the ophthalmologic apparatus 100 can specify the imaging area by referring to the information of the imaging area previously designated for the sample. This past information is typically stored in a medical data system such as an electronic medical record system. Further, the ophthalmic apparatus 100 may be configured to set a plurality of target regions so as to include a designated imaging region. That is, the union of the plurality of target regions includes the imaging region specified for the sample. For example, the ophthalmic apparatus 100 can set a plurality of target areas in response to a command from the user to set the target area. In another example, the ophthalmic apparatus 100 can set a plurality of target regions based on the image data obtained from the sample by itself and / or the image data of the sample obtained from another apparatus. In yet another example, the ophthalmic apparatus 100 can set a plurality of target regions by referring to the information of the target regions previously set for the sample. This past information is typically stored in a medical data system such as an electronic medical record system. In addition, the ophthalmic apparatus 100 collects a plurality of 3D data sets corresponding to the plurality of 3D regions by sequentially applying OCT scans to the plurality of target regions set to include the imaging region. To do. As described above, due to the influence of eye movements and the like, a certain target region and the corresponding three-dimensional region to which the OCT scan is applied may not match. However, in an exemplary embodiment, a target region, a corresponding 3D region to which an OCT scan has been applied, and a 3D data set collected by this OCT scan can be associated with each other.

The ophthalmic apparatus 100 collects two 3D data sets by applying an OCT scan to two target regions (target region pairs) of a plurality of target regions set to include an imaging region, and two. Two image data are generated from the 3D dataset. Further, the ophthalmic apparatus 100 can evaluate the overlapping region of the two generated image data. As an example of this evaluation, evaluation of the size of the overlapping area, evaluation of the size of the image correlation value, the size of the relative movement amount (translation amount, rotation movement amount, etc.) from the predetermined positional relationship of both image data, etc. There is an evaluation of. The ophthalmic apparatus 100 can select the subsequent processing according to the result of this evaluation. For example, the ophthalmologic apparatus 100 applies registration (the first registration described above) to the two image data or reapplies the OCT scan to at least one of the target region pairs, depending on the result of this evaluation. Can be selectively executed. Typically, if the evaluation result is good, registration is selected, and if the evaluation result is poor, OCT scan reapplication is selected. The evaluation typically involves a comparison with a threshold, for example, if the dimensions of the overlapping region are greater than or equal to the threshold, if the image correlation value is greater than or equal to the threshold, or relative to the default positional relationship of both image data. Good evaluation results are obtained when the amount of movement is less than the threshold value. Such a series of processes (application of OCT scan to target area pair, generation of image data pair, evaluation of overlapping area, selective processing execution according to evaluation result) is applied to all of a plurality of target areas. Will be done. For example, a plurality of target region pairs can be set according to the order assigned to the plurality of target regions. Typically, the nth target region and the n + 1th target region can be paired (n is a positive integer). As a specific example, when the nth target region is expressed as "n", the target region pairs (n, n + 1) set from the five target regions are (1, 2), (2, 3), There are four (3,4) and (4,5). The mode of the target region pair is not limited to (n, n + 1). For example, a group of target region pairs can be set so that the set of target regions included in all the target region pairs covers a plurality of target regions set to include the imaging region. However, a group of target region pairs may be set so that the set of target regions included in all the target region pairs includes only a part of a plurality of target regions set to include the imaging region.

If the OCT scan is reapplied in response to the results of the overlap region evaluation above, the target region pair applied to this new OCT scan will be the target region pair applied to the previous (or earlier) OCT scan. It may be the same, or at least partially different.

As an example of the latter, the ophthalmic apparatus 100 may be configured to modify at least one of the target region pairs applied to the previous (or earlier) OCT scan based on the results of the overlap region assessment. Typically, the ophthalmic apparatus 100 can utilize the result of the previous overlap region evaluation so that a good result can be obtained in the next overlap region evaluation. For example, in the ophthalmic apparatus 100, the size of the overlapping region is equal to or greater than the threshold value, the image correlation value is equal to or greater than the threshold value, or the relative movement amount of both image data from the predetermined positional relationship is less than the threshold value. One or both of the target region pairs can be changed (adjusted, corrected) so as to be. The ophthalmic apparatus 100 may be configured to reapply an OCT scan to a target region pair that reflects the change. In addition, for example, when only one of the target region pairs is changed, the OCT scan may be reapplied only to the changed target region.

As another example of changing the target region pair, the ophthalmic apparatus according to another aspect may be configured to prepare a frontal image of the sample. For example, the ophthalmic apparatus of this example has a function of photographing a sample from the front, a function of photographing a sample from one or more positions other than the front to construct a front image, a function of receiving a front image of a sample from an external device, or It may have a function of reading the front image of the sample from the recording medium. In addition, the ophthalmic apparatus of this example may be configured to modify at least one of the target area pairs based on the acquired frontal image. Typically, the modification of the target area may include a series of processes: identification of a partial area of the front image corresponding to the target area; to a 3D data set collected by an OCT scan on the target area. Registration between the image data generated based on the image and the front image; calculation of the displacement amount of the image data with respect to the partial region; movement of the target region to cancel the displacement amount. The change of the target area is not limited to the movement of the target area, and includes a change of any aspect regarding the target area such as a change of the size of the target area, a change of the shape of the target area, and a change of the number of the target areas. You may be. The ophthalmic apparatus of this example may be configured to reapply OCT scans to the two target areas after such changes have been made. The ophthalmic apparatus of this example will be described after the explanation of the ophthalmic apparatus 100.

When a imaging region is specified for the sample (eye) and a plurality of target regions are set so as to include the imaging region, the ophthalmic apparatus 100 synthesizes a plurality of image data corresponding to the plurality of target regions. The partial image data corresponding to the shooting area may be extracted from the composite image data obtained in the above-mentioned. As a result, three-dimensional image data depicting the designated shooting area can be obtained. Typically, the extraction of the partial image data includes the alignment between the shooting area and the composite image data and the extraction of the partial image data of the composite image data corresponding to the shooting area specified by the alignment. Including. When the above-mentioned front image is acquired, the alignment may be performed using this front image.

Instead of being configured to synthesize the plurality of image data corresponding to the plurality of target regions and then to extract the partial image data corresponding to the imaging region, the ophthalmic apparatus 100 has the plurality of image data corresponding to the plurality of target regions. Partial image data may be extracted from each of the above, and a plurality of partial image data (partial image data group) obtained thereby may be combined. That is, when an imaging region is specified for the sample (eye) and a plurality of target regions are set to include this imaging region, for example, the ophthalmic apparatus 100 is based on the result of the first registration. A partial image data group corresponding to the photographing region may be extracted from a plurality of image data corresponding to the plurality of target regions, and the partial image data group may be combined. This also provides three-dimensional image data that depicts the designated shooting area.

As described above, among the plurality of image data generated based on the plurality of 3D data sets collected from the plurality of different 3D regions of the sample (eye), the two image data including the overlapping region are The ophthalmic apparatus 100 applies the first registration using the image correlation calculation. Arbitrary two image data having an overlapping region are referred to as "first image data and second image data". Further, in the first image data, the partial image data corresponding to the overlapping area is called "overlapping area in the first image data", and in the second image data, the partial image data corresponding to the overlapping area is "second". It is called "overlapping area in the image data of". The image correlation calculation performed by the ophthalmologic apparatus 100 is executed so as to calculate at least one of a translation amount and a rotational movement amount between the overlapping area in the first image data and the overlapping area in the second image data. You can. The translation amount may be a three-dimensional translation amount (typically, a translation amount in each direction of the three coordinate axes of the three-dimensional Cartesian coordinate system). Further, the rotational movement amount may be a three-dimensional rotational movement amount (typically, a rotational movement amount with each of the three coordinate axes of the three-dimensional Cartesian coordinate system as the rotation axis). An example of an operation that can be adopted for such an image correlation operation is phase-only correlation (POC). This phase-limited correlation may be, for example, a two-dimensional method described in Japanese Patent No. 6276943 (International Publication No. 2015/029675), or a three-dimensional method based on the same. In the case of the two-dimensional method, for example, the ophthalmic apparatus 100 is configured to use the two-dimensional method for each of the XY coordinate system, the YZ coordinate system, and the ZX coordinate system of the three-dimensional Cartesian coordinate system (XYZ coordinate system). May be done. In the case of the three-dimensional method, for example, the ophthalmic apparatus 100 may be configured to use an arithmetic expression that is a natural extension of the arithmetic expression of the two-dimensional method to three dimensions. As a specific example, a three-dimensional phase-limited correlation calculation formula can be obtained as a natural extension of the two-dimensional phase-limited correlation calculation formula shown in [Equation 1] to [Equation 7] of Japanese Patent No. 6276943.

The ophthalmic apparatus 100 having at least the features exemplified above will be described below. The exemplary ophthalmic apparatus 100 shown in FIG. 1 includes a light source 102 (eg, a broadband light source or a tunable light source) for generating a light beam. The beam splitter (BS) 104 divides the light beam from the light source 102 into a sample light beam (measurement light) and a reference light beam (reference light). In other words, the beam splitter 104 directs a part of the light beam from the light source 102 to the sample arm 106 and the other part to the reference arm 108.

The reference arm 108 includes a polarization controller 110 for adjusting the reference light beam (for example, for maximizing interference efficiency) and a collimator 112 for outputting the reference light beam as a parallel light beam. The reference light beam output from the collimator 112 is converted into a focused light beam by the lens 114 and projected onto the reflector 115. The reference light beam reflected by the reflector 115 returns to the beam splitter 104 through the reference arm 108. The lens 114 and the reflector 115 are integrally movable, thereby changing the distance from the collimator 112 (in other words, changing the length of the path of the reference light beam).

The sample arm 106 projects a sample light beam onto the eye 120 as a sample via the collimator 117, the two-dimensional scanner 116, and one or more objective lenses 118. The two-dimensional scanner 116 is, for example, a galvanometer mirror scanner or a MEMS scanner. The return light of the sample light beam projected onto the eye 120 returns to the beam splitter 104 through the sample arm 106. The two-dimensional scanner 116 enables OCT scanning of the three-dimensional region of the eye 120.

The beam splitter 104 superimposes the return light of the reference light beam and the return light of the sample light beam to generate an interference light beam. The interference light beam is guided by the detection unit 122 and detected. As a result, the echo time delay of light is measured from the interference spectrum.

The detector 122 sets a plurality of output sets based on the composition (that is, interferogram data) of the return light of the sample light beam supplied from the sample arm 106 and the return light of the reference light beam supplied from the reference arm 108. To generate. For example, each of the plurality of output sets generated by the detection unit 122 can correspond to the light intensity received at different wavelengths output from the light source 102. When the sample light beam is sequentially projected to a plurality of XY positions by the two-dimensional scanner 116, the detected light intensity is the reflection intensity distribution (backscattering) inside the eye 120 in the depth direction (Z direction) at each XY position. Contains information about (intensity distribution).

In this way, a three-dimensional data set is obtained. The three-dimensional data set includes a plurality of A scan data corresponding to a plurality of XY positions. Each A scan data represents a spectral intensity distribution at the corresponding XY position. The three-dimensional data set collected by the detection unit 122 is sent to the processing device 124.

The processing device 124 executes various data processing (information processing). For example, the processing device 124 applies an image correlation calculation to each of a process of generating a plurality of image data from a plurality of three-dimensional data sets of a sample and a plurality of overlapping regions in the generated plurality of image data. It is configured to execute a process of performing registration between a plurality of image data (the first registration described above) and a process of synthesizing the plurality of image data based on the result of the first registration. To. Further, the processing device 124 is configured to execute processing for realizing at least one of the above-described exemplary features. The processing device 124 includes a processor that operates according to the processing program. A specific example of the processing device 124 will be described later.

The control device 126 controls each part of the ophthalmic device 100. For example, control device 126 is configured to perform control to prepare a plurality of 3D data sets collected from a plurality of different 3D regions of a sample. Controls for preparing the 3D data set include, for example, control for applying an OCT scan to the eye 120 and / or acquiring the 3D data set collected from the eye 120 from another device or recording medium. Includes control for. Further, the control device 126 is configured to perform control for realizing at least one of the above-mentioned exemplary features. The control device 126 includes a processor that operates according to a control program. A specific example of the control device 126 will be described later.

Although not shown, the ophthalmic apparatus 100 may further include a display device, an operation device, a communication device, and the like.

The processing device 124 and the control device 126 will be further described with reference to FIG. The processing device 124 includes an image data generation unit 201, a registration unit 202, and an image data synthesis unit 203. Further, the processing device 124 includes a photographing area designation unit 204 and a target area setting unit 205. In addition, the processing apparatus 124 includes an overlapping region evaluation unit 206. Further, the control device 126 includes a scan control unit 210.

The OCT scanner 220 shown in FIG. 2 applies an OCT scan to a sample (eye 120). The OCT scanner 220 of this embodiment is, for example, the optical element group shown in FIG. 1, that is, a light source 102, a beam splitter 104, a sample arm 106 (collimator 117, a two-dimensional scanner 116, an objective lens 118, etc.), and a reference arm 108 ( A collimator 112, a lens 114, a reflecting mirror 115, etc.), and a detection unit 122 are included. The OCT scanner in some embodiments may have other configurations.

In this embodiment, the OCT scanner 220 corresponds to the plurality of three-dimensional regions by applying the OCT scan to each of the plurality of different three-dimensional regions of the eye 120 under the control of the scan control unit 210. Collect the 3D dataset of.

The control device 126 controls each part of the ophthalmic device 100. The control related to the OCT scan among the various controls is executed by the scan control unit 210. The scan control unit 210 of this embodiment is configured to execute control of the OCT scanner 220, for example, control of the light source 102, control of the two-dimensional scanner 116, movement control of the lens 114 and the reflector 115, and the like. .. The scan control unit 210 includes a processor that operates according to the scan control program.

The processing device 124 executes various data processing (calculation, analysis, measurement, image processing, etc.). The series of three processes described above, that is, the generation of a plurality of image data based on a plurality of three-dimensional data sets, the first registration, and the synthesis of the plurality of image data are performed by the image data generation unit 201 and the registration, respectively. It is executed by the relation unit 202 and the image data composition unit 203.

The image data generation unit 201 generates image data based on the data collected by the OCT scanner 220. For example, the image data generation unit 201 forms image data of a tomographic image of the eye 120 based on the output (sampling data, interference signal data) from the OCT scanner 220. This image data generation process includes filtering, a fast Fourier transform (FFT), and the like, as in the conventional (swept source or spectral domain) OCT. By such processing, the reflection intensity profile (reflection intensity profile along the Z direction) in the A line (scan path of the measured light beam in the eye 120) corresponding to each XY position is acquired, and this reflection intensity profile is imaged. The image data of this A line (A scan image data) is formed by the conversion.

Further, the image data generation unit 201 forms a plurality of A-scan image data according to the mode of OCT scan (deflection of the measurement light beam, movement of the A-scan position), and arranges these A-scan image data in two dimensions. Image data and 3D image data can be constructed.

When a plurality of tomographic image data are obtained by raster scanning or the like, the image data generation unit 201 embeds these tomographic image data in a single three-dimensional coordinate system to construct stack data, and voxelizes the stack data. Can be applied to construct voxel data (volume data).

The image data generation unit 201 can render stack data or volume data. The rendering method is arbitrary and may include, for example, volume rendering, multi-section reconstruction (MPR), surface rendering, and the like. Further, the image data generation unit 201 can construct a plane image (for example, a front image, an ENFACE image) from the stack data or the volume data. For example, the image data generation unit 201 can construct a projection image by integrating stack data or volume data along each A line.

As described above, in this aspect, the OCT scanner 220 collects a plurality of 3D data sets corresponding to a plurality of different 3D regions of the eye 120. The collected three-dimensional data sets are input to the image data generation unit 201. The image data generation unit 201 generates (three-dimensional) image data from each of the plurality of input three-dimensional data sets. As a result, a plurality of image data corresponding to a plurality of different three-dimensional regions of the eye 120 can be obtained. The image data generation unit 201 may be able to execute arbitrary processing related to image data generation in the above-described exemplary features.

The plurality of image data generated by the image data generation unit 201 are input to the registration unit 202. The registration unit 202 performs registration (first registration) between a plurality of image data by applying an image correlation calculation to each of a plurality of overlapping regions in the plurality of input image data.

One method that can be adopted for the image correlation calculation performed by the registration unit 202 is described in, for example, Japanese Patent No. 6276943. When this method is used, the registration unit 202 can obtain the displacement amount between the two overlapping regions in the two image data by applying the phase-limited correlation to the two image data. This displacement amount is, for example, relative to the predetermined positional relationship between the two 3D regions corresponding to the two overlapping regions (that is, the two 3D regions to which the OCT scan for acquiring the two image data is applied). It is the displacement amount of the positional relationship between the two overlapping regions actually obtained, and typically includes one or both of the parallel movement amount and the rotational movement amount. Registration is performed to change the relative position of these two image data so as to cancel the obtained displacement amount. That is, the registration between the two image data is executed so as to correct the error of the positional relationship between the two overlapping regions actually obtained.

In the actual processing, the registration unit 202 can first select two image data presumed to have overlapping regions from a plurality of image data. This selection refers, for example, to an array of different three-dimensional regions of the eye 120. Then, the registration unit 202 can apply the phase-limited correlation to the two overlapping regions in the two selected image data.

Typically, such registration is applied at least once to each of the plurality of image data corresponding to the plurality of different three-dimensional regions of the eye 120. That is, registration is applied to all of the plurality of image data corresponding to the plurality of three-dimensional regions different from each other in the eye 120. The registration unit 202 may be capable of performing any process relating to the first registration in the above-mentioned exemplary features.

For details of the phase-limited correlation method and the registration method using the method, refer to Japanese Patent No. 6276943. Further, the applicable registration method is not limited to the above examples, and any method within the scope of the invention described in Japanese Patent No. 6276943 or any modification thereof can be applied. Further, for registration, it is also possible to use an arbitrary image correlation method other than the phase-limited correlation method, or to use an arbitrary image comparison method other than the image correlation method.

The result of the registration executed by the registration unit 202 is input to the image data synthesis unit 203. Further, a plurality of image data generated by the image data generation unit 201 are input to the image data synthesis unit 203. The image data synthesizing unit 203 synthesizes a plurality of image data input from the image data generation unit 201 based on the registration result input from the registration unit 202.

More specifically, the image data synthesizing unit 203 first determines the positional relationship (relative position) of a plurality of image data based on the registration result. For example, the image data synthesizing unit 203 arranges the two image data having the overlapping region so that the overlapping regions overlap each other according to the registration result. By applying such an image arrangement process to all combinations of two image data having overlapping regions with each other, it is possible to determine all the positional relationships of the plurality of image data (that is, a plurality of images). All of the data can be represented in a single 3D coordinate system, or multiple image data can be embedded in the same 3D image space). It is not necessary to perform the image arrangement processing on all the combinations of the two image data having overlapping regions with each other, and it is sufficient to apply the image arrangement processing to each of the plurality of image data at least once.

Further, the image data synthesizing unit 203 constructs a single image data by applying a known image synthesizing to a plurality of image data to which the image arrangement processing is applied. This composite image data is also called montage image data, panoramic image data, or the like.

Instead of synthesizing all of the plurality of image data generated by the image data generation unit 201, any two or more of the plurality of image data may be combined. In this case, processing such as registration may be applied only to two or more image data to be combined.

The image data synthesizing unit 203 may be configured to extract a part of the composite image data obtained by synthesizing a plurality of image data. For example, the image data synthesizing unit 203 may be configured to extract partial image data corresponding to a shooting area designated by the shooting area designation unit 204 described later.

Further, the image data synthesizing unit 203 of each of the plurality of image data (or each of two or more image data among the plurality of image data) based on the result of the registration executed by the registration unit 202. It may be configured to extract a part and synthesize a plurality of (or two or more) extracted partial image data. For example, the image data synthesizing unit 203 changes each of the plurality of image data (or each of two or more image data of the plurality of image data) into a shooting area designated by the shooting area designation unit 204 described later. It may be configured to extract the corresponding partial image data and synthesize the extracted partial image data group.

The photographing area designation unit 204 is configured to designate the photographing area to the eye 120. The imaging area indicates the area of the eye 120 to which the OCT scan is applied. The region of the eye 120 to which the OCT scan is applied is a three-dimensional region, but the imaging region designated by the imaging region designation unit 204 is typically XY coordinates orthogonal to the axial direction (Z direction, Z coordinate axis). It is a two-dimensional region defined by the system. The range in the Z direction is defined as, for example, an imaging range (default) in the Z direction in the OCT system mounted on the ophthalmic apparatus 100. When the imaging range in the Z direction is variable, the photographing area designation unit 204 can specify a three-dimensional area including the range in the Z direction as the photographing range.

The ophthalmic apparatus 100 can be configured so that the user can specify the imaging area by using the operation device (and the display device) described above. In this case, the shooting area designation unit 204 can specify the shooting area in response to a command from the user to specify the shooting area. For example, the control device 126 displays the front image of the eye 120 on the display device, the user specifies a desired area in the front image using the operation device, and the photographing area designation unit 204 sets the area specified by the user. Record as coordinate information in the front image. The area defined by this coordinate information is treated as a shooting area.

In another example, the imaging area designation unit 204 designates an imaging area based on the image data acquired by the ophthalmic apparatus 100 photographing the eye 120 and / or the image data of the eye 120 obtained from another apparatus. be able to. In this case, the photographing area designation unit 204 can detect, for example, a feature portion in the image data and determine the photographing area with reference to the detected feature portion. Here, the characteristic points of the fundus include the papilla of the optic nerve, the yellow spot, the blood vessel, and the lesion, and the characteristic points of the anterior segment include the center of the pupil, the center of the pupil, the center of the iris, the center of the iris, the apex of the cornea, and the crystalline lens. There are anterior apex, posterior lens apex, ciliary body, zonule of Zinn, blood vessels, lesions, etc. Further, the photographing area is, for example, an area in which a feature portion is arranged, and is an area having a predetermined shape and / or a predetermined dimension. For example, the imaging region may be a square region of predetermined dimensions defined on the XY plane, including the optic disc and the macula. Further, for example, the midpoint of the line segment connecting the center of the papilla and the fovea centralis is located at the center of the square region.

In still another example, the photographing area designation unit 204 can specify the current photographing area by referring to the information of the photographing area previously designated for the eye 120. For example, the control device 126 accesses the medical data system via the communication device described above, searches the past imaging area information of the eye 120 from the medical data system, and transfers the searched imaging area information to the imaging area designation unit 204. input. The imaging area designation unit 204 uses the imaging area information as the imaging area applied to the current imaging of the eye 120.

Information indicating the shooting area designated by the shooting area designation unit 204 is input to the target area setting unit 205. The target area setting unit 205 sets a plurality of target areas so as to include the shooting area designated by the shooting area designation unit 204. The target area setting unit 205 can assign an order to a plurality of target areas. This ordering (ordering) is performed, for example, as described above.

The ophthalmic apparatus 100 can be configured so that the user can specify a plurality of target areas by using the operation device (and the display device) described above. In this case, the target area setting unit 205 can set a plurality of target areas in response to a command from the user to specify the target area. For example, the control device 126 displays the front image of the eye 120 on the display device, overlays the shooting area on the front image, the user specifies a plurality of areas using the operation device, and the target area setting unit 205 sets the user. Each of the plurality of areas specified by is recorded as coordinate information in the front image. A plurality of regions defined by such coordinate information are treated as a plurality of target regions.

In another example, the target region setting unit 205 sets a plurality of target regions based on the image data acquired by the ophthalmic apparatus 100 by photographing the eye 120 and / or the image data of the eye 120 obtained from another apparatus. Can be set. In this case, the target area setting unit 205 may detect, for example, a feature portion in the image data, and set a plurality of target regions so that the region representing the detected feature portion is included in the single target region. it can. As a result, the image of the featured portion can be drawn well. When the featured portion is large, a plurality of target regions can be set so that the region representing the featured portion is divided into two or more target regions and included. Similarly, when the image of the feature portion is referred to in the registration executed by the registration unit 202, a plurality of target regions are set so that the region representing the feature portion is divided into two or more target regions and included. be able to. The featured portion may be any or other of the above-mentioned examples. Each target region may be, for example, a region of a predetermined shape and / or a predetermined size.

In still another example, the target area setting unit 205 can set the plurality of target areas this time by referring to the information of the plurality of target areas set in the past for the eye 120. For example, the control device 126 accesses the medical data system via the communication device described above, searches the past target area information of the eye 120 from the medical data system, and sends the searched target area information to the target area setting unit 205. input. The target area setting unit 205 uses the target area information as a plurality of target areas applied to the current imaging of the eye 120.

Information indicating a plurality of target areas set by the target area setting unit 205 is input to the scan control unit 210. The scan control unit 210 controls the OCT scanner 220 so as to sequentially apply the OCT scan to the plurality of input target regions. Under this control, the OCT scanner 220 collects a plurality of three-dimensional data sets corresponding to a plurality of target regions. In addition, it should be noted

The scan control unit 210 controls the OCT scanner 220 so as to sequentially apply the OCT scan to a plurality of target areas according to the order assigned by the target area setting unit 205, for example.

In some examples, the scan control unit 210 may be configured to continuously perform sequential OCT scans on a plurality of target areas. In other words, the scan control unit 210 may be configured to perform sequential OCT scans on a plurality of target areas as a predetermined series of operations.

Instead of performing the sequential OCT scan in combination with other actions in this way, in some other examples, the scan control unit 210 arbitrarily performs the sequential OCT scan and other actions on a plurality of target areas. Can be executed in combination with. For example, the ophthalmic apparatus 100 may be configured to advance the OCT scans for a plurality of target regions one by one while confirming whether or not the OCT scans for each target region are appropriately performed. Some examples will be described below.

OCT scanning for any target region (referred to as a reference target region in this example) among a plurality of target regions, and generation of image data (referred to as reference image data) based on the three-dimensional data set collected thereby. Consider when it is complete. Next, the scan control unit 210 controls the OCT scanner 220 so as to apply the OCT scan to the target region (referred to as a new target region) to which the next order of the reference target region is assigned. Further, the image data generation unit 201 generates image data (referred to as new image data) from the three-dimensional data set collected from the new target region. The two target regions consisting of the reference target region and the new target region correspond to the target region pair (n, n + 1) described above. Further, the two image data composed of the reference image data and the new image data correspond to the above-mentioned image data pair.

In this aspect, the reference image data and the new image data are input to the overlapping area evaluation unit 206. The overlapping area evaluation unit 206 evaluates the overlapping area between the reference image data and the new image data. This is to evaluate whether or not the overlapping region of the reference image data and the new image data is appropriate for synthesizing these image data.

For example, the overlapping area evaluation unit 206 evaluates based on the dimensions of the overlapping area between the reference image data and the new image data, and the evaluation based on the image correlation value between the overlapping area in the reference image data and the overlapping area in the new image data. And, at least one of the evaluations based on the relative movement amount with respect to the predetermined positional relationship between the reference image data and the new image data may be performed. In some embodiments, the image correlation value is determined by a phase-limited correlation calculation and its evaluation is performed based on at least one magnitude of translation and rotational movement. Alternatively, evaluation may be performed based on a correlation value indicating the degree of similarity between the overlapping region in the reference image data and the overlapping region in the new image data. Further, the default positional relationship between the reference image data and the new image data corresponds to, for example, the positional relationship between the reference target area and the new target area.

The evaluation result obtained by the overlapping area evaluation unit 206 is input to the control device 126. The control device 126 can select the control content according to the input evaluation result. The control device 126 of this embodiment applies registration to the reference target region and the new target region (the first registration described above), or OCT to at least one of the reference target region and the new target region, depending on the evaluation result. You can selectively reapply the scan. The control device 126 of this embodiment may be configured to select registration if a good evaluation result is obtained, and to select OCT scan reapplication if a good evaluation result is not obtained. ..

The evaluation performed by the overlapping area evaluation unit 206 may include threshold processing (comparison with a default threshold). For example, the overlapping area evaluation unit 206 calculates the dimension of the overlapping region between the reference image data and the new image data, compares this dimension with the threshold value, and determines that the dimension is "good" when the dimension is equal to or larger than the threshold value. It may be configured. Examples of overlapping area dimensions are volume (eg, dimensions defined in the XYZ coordinate system), area (eg, XY coordinate system, YZ coordinate system, ZX coordinate system, and any two-dimensional coordinate system in any orientation. Dimension defined by), length (dimension defined by any of the X, Y, Z, and any orientation coordinates), surface area, circumference, diagonal area, diagonal length and so on. That is, the overlapping area evaluation unit 206 of this example is configured to determine whether or not the overlapping area of the reference image data and the new image data has sufficient dimensions for image data synthesis.

The overlapping area evaluation unit 206 according to another example applies an image correlation calculation to the combination of the overlapping area in the reference image data and the overlapping area in the new image data, and the image correlation obtained by this image correlation calculation. The value may be compared with the threshold value, and if the image correlation value is equal to or greater than the threshold value, it may be determined as “good”. That is, the overlapping area evaluation unit 206 of this example determines whether or not the overlapping area in the reference image data and the overlapping area in the new image data have a sufficient correlation for image data synthesis. It is composed.

The overlapping area evaluation unit 206 according to another example obtains the positional relationship (relative position) between the reference image data (overlapping area included therein) and the new image data (overlapping area included therein), and obtains the positional relationship (relative position) with the reference target area. The error (relative movement amount) of the relative position with respect to the positional relationship with the new target region is obtained, the relative movement amount is compared with the threshold value, and if the relative movement amount is less than the threshold value, it is judged as "good". It may be configured. That is, the overlapping area evaluation unit 206 of this example is the overlapping area of the actually obtained reference image data and the new image data with respect to the relative position between the reference target area and the new target area set for the OCT scan. It is configured to determine if the error is small enough for image data composition.

The ophthalmologic apparatus 100 is such that the sequential OCT scan for a plurality of target regions, the sequential generation of image data, the sequential evaluation of overlapping regions, and the sequential application of processing selection according to the evaluation result are performed in combination. It is composed of. An example of the operation realized by this will be described below.

The operation of the ophthalmic apparatus 100 will be described. A flow of an example of the operation of the ophthalmic apparatus 100 is shown in FIGS. 3A, 3B and 3C. The operation shown in FIG. 3B and the operation shown in FIG. 3C are selectively applied. That is, the operation shown in one of FIGS. 3B and 3C can be combined with the operation shown in FIG. Preparatory operations such as alignment, focusing, optical path length adjustment, and polarization adjustment may be performed at arbitrary timings using known means.

First, the photographing area designation unit 204 designates a photographing area for the eye 120 (S1).

In the example shown in FIG. 4A, the imaging region 310 is designated to include the optic disc and macula of the fundus 300.

Next, the target area setting unit 205 sets a plurality of target areas so as to include the imaging area specified in step S1, and sets the order in which the OCT scan is applied to these target areas (S2). .. In this example, N target regions are set. Here, N is an integer of 2 or more. Further, a plurality of target areas are referred to as "first target area", "second target area", ..., "Nth target area" according to the set scan order.

In the example shown in FIG. 4B, seven target regions 321 to 327 are set so as to include the imaging region 310 shown in FIG. 4A. The target regions 321 to 327 are arranged so that two adjacent target regions partially overlap each other. For example, two adjacent target regions 322 and 323 are arranged so as to partially overlap each other, and two adjacent target regions 323 and 324 are arranged so as to partially overlap each other. Further, a scan order corresponding to the order of reference numerals 321 to 327 is assigned to the target areas 321 to 327. According to this scan order, the target area 321 is referred to as the "first target area", the target area 322 is referred to as the "second target area", ..., The target area 327 is referred to as the "seventh target area". Call.

After the target area setting is completed, the OCT scan is started in response to a predetermined trigger. First, the scan control unit 210 applies the OCT scan to the first target region among the plurality of target regions set in step S2 (S3). In the example shown in FIG. 4B, an OCT scan is first applied to the first target region 321.

The image data generation unit 201 generates image data based on the three-dimensional data set collected from the first target region by the OCT scan in step S3 (S4). This image data is called "first image data".

The control device 126 stores the first image data generated in step S4 in a storage device (not shown) (S5). The storage device is typically provided inside the ophthalmic device 100 (eg, inside the control device 126), but may be provided outside the ophthalmic device 100.

Next, the scan control unit 210 applies the OCT scan to the target area to which the following order is assigned (S6). After step S5, an OCT scan is applied to the second target area. As shown in FIG. 3A, this step S6 is repeated until appropriate image data is obtained from all of the plurality of target regions set in step S2. For the following explanation, let n = 2, ..., N. Here, N is the number of target regions set in step S2.

The image data generation unit 201 generates image data based on the three-dimensional data set collected from the nth target region by the OCT scan in step S6 (S7). This image data is called "nth image data".

The control device 126 reads the n-1st image data and the nth image data from the storage device and sends them to the overlapping area evaluation unit 206. The overlapping area evaluation unit 206 evaluates the overlapping area between the n-1st image data and the nth image data (S8).

If the evaluation result in step S8 is good (S9: Yes), the process proceeds to step S10. On the other hand, if the evaluation result in step S8 is not good (S9: No), the process proceeds to step S13. That is, when the nth image data (overlapping area) sufficient for image data synthesis is obtained, the process proceeds to step S10, and when it is not obtained, the process proceeds to step S13.

When the evaluation result in step S8 is good (S9: Yes), the control device 126 stores the nth image data generated in step S7 in the storage device (S10).

Here, the control device 126 determines whether n = N (S11). That is, the control device 126 determines whether good image data has been obtained from all of the plurality of target regions set in step S2. When it is determined that n = N (S11: Yes), the process proceeds to step S14 of FIG. 3B (or step S21 of FIG. 3C). On the other hand, when it is determined that n = N (S11: No), that is, when n <N, the process proceeds to step S12.

When it is determined in step S11 that n = N (S11: No), the control device 126 controls to shift to the process relating to the target region (n + 1st target region) in the next order (S12). .. This control includes, for example, control for changing the OCT scan application area (for example, control of the two-dimensional scanner 116, control of switching the fixation position).

As described above, when the evaluation result in step S8 is not good (S9: No), the process proceeds to step S13. The target region setting unit 205 corrects the nth target region (S13). For example, the target region setting unit 205 can correct the nth target region based on the evaluation result in step S8.

When the OCT scan is sequentially applied to a plurality of target regions, the OCT scan is applied later among the n-1st target region and the nth target region as in this example. It is considered preferable to change the nth target region obtained, but the n-1th target region may be changed, or the n-1th target region and the nth target region may be changed. Both may be changed.

The process returns to step S6 under the control of step S12. Steps S6 to S13 are repeated until it is determined in step S11 that n = N (S11: Yes). As a result, suitable image data for image data synthesis is acquired for all of the plurality of target regions set in step S2. When it is determined in step S11 that n = N (S11: Yes), the process of this example proceeds to step S14 of FIG. 3B or step S21 of FIG. 3C.

See FIG. 3B. When shifting from step S11 to step S14, first, the control device 126 reads out the first to Nth image data corresponding to the first to Nth target regions from the storage device (S14). The control device 126 sends the read N image data (image data group) to the registration unit 202.

The registration unit 202 applies registration to the image data group input by the control device 126 (S15). Here, each image data included in the image data group has an overlapping region suitable for image data synthesis in relation to adjacent image data. In other words, each image data included in the image data group has an overlapping region suitable for registration in relation to the adjacent image data.

FIG. 4C represents image data 334 corresponding to the target region 324 shown in FIG. 4B. Even if the OCT scan is applied to the target area 324, the area to which the OCT scan is actually applied may deviate from the target area 324 due to the influence of eye movements and the like.

FIG. 4D shows a state in which seven image data 331-337 corresponding to the seven target regions 321 to 327 shown in FIG. 4B are arranged according to the arrangement of the seven target regions 321 to 327. When seven image data 331-337 are arranged in this way, that is, when seven image data 331-337 are arranged without considering the influence of eye movements, for example, an image 340 of a certain tissue is displayed. It is drawn intermittently.

By applying the registration in step S15, the influence of eye movement and the like can be canceled out. By applying registration to the seven image data 331-337 shown in FIG. 4D, the relative positions of the seven image data 331-337 can be adjusted, and the tissue image 340 is smooth like an actual tissue. (See Fig. 4E).

The result of the registration executed in step S15 is input to the image data synthesis unit 203. The image data synthesizing unit 203 synthesizes an image data group arranged according to the registration result (S16).

For example, the image data synthesizing unit 203 synthesizes seven image data 331-337 arranged as shown in FIG. 4E. As a result, composite image data corresponding to the region of the eye 120 including the imaging region designated in step S1 can be obtained.

The image data synthesizing unit 203 extracts the partial image data corresponding to the shooting area designated in step S1 from the composite image data generated in step S16 (S17).

Based on the partial image data obtained in step S17, the control device 126 causes the display device described above to display the image of the shooting area designated in step S1 (S18).

The displayed image is obtained by rendering partial image data which is three-dimensional image data. It is also possible to render the composite image data obtained in step S16, or to render any one or more of the first to Nth image data. The user can specify a desired viewpoint and a desired cross section for rendering. Alternatively, the viewpoint and / or cross section for rendering may be preset.

The control device 126 illustrates, for example, any one or more of the partial image data obtained in step S17, the composite image data obtained in step S16, and the first to Nth image data. It can be stored in a storage device that does not (S19).

The storage device may be, for example, any of an element of the ophthalmic device 100, a peripheral device of the ophthalmic device 100, a device that can be connected to the ophthalmic device 100 via a communication line, and a portable recording medium. This completes the operations shown in FIGS. 3A and 3B (end).

See FIG. 3C. When shifting from step S11 to step S21, first, the control device 126 obtains the first to Nth image data corresponding to the first to Nth target regions, as in step S14 of FIG. 3B. Read from the storage device (S21).

Next, the registration unit 202 applies registration to the image data group input by the control device 126 (S22).

The result of the registration executed in step S22 is input to the image data synthesis unit 203. The image data synthesizing unit 203 extracts partial image data corresponding to the shooting area from each image data included in the image data group (S23). As a result, a partial image data group corresponding to the image data group can be obtained.

Next, the image data synthesizing unit 203 synthesizes the partial image data group extracted from the image data group in step S23 according to the registration result (S24). As a result, the composite image data corresponding to the shooting area specified in step S1 can be obtained.

The control device 126 renders the composite image data obtained in step S24 to display the image of the shooting area specified in step S1 on the display device described above (S25).

The control device 126, for example, obtains one or more of the composite image data obtained in step S24, the partial image data group obtained in step S23, and the first to Nth image data. It can be stored in a storage device (S26) (not shown). This completes the operations shown in FIGS. 3A and 3C (end).

The target regions 321 to 327 shown in FIG. 4B are arranged in a row, but the arrangement mode of the plurality of target regions is not limited to such. For example, as shown in FIG. 5, a target region 421 can be set in the center of the imaging region 310, and six target regions 422 to 427 can be set around the target region 421. Each of the surrounding target areas 422-427 partially overlaps the central target area 421. In addition, any two target regions adjacent to each other among the surrounding target regions 422 to 427 do not overlap. In this case, registration is applied to each of the surrounding target regions 422-427 and the combination of the central target region 421.

An ophthalmic apparatus according to another aspect will be described. The ophthalmic apparatus 500 shown in FIG. 6 includes a control apparatus 510, an OCT scanner 520, an imaging apparatus 530, and a processing apparatus 540. The control device 510 may be configured in the same manner as the control device 126 of the ophthalmic device 100. The OCT scanner 520 may be configured in the same manner as the OCT scanner 220 of the ophthalmic apparatus 100. The processing device 540 may be configured in the same manner as the processing device 124 of the ophthalmic device 100. That is, the ophthalmology apparatus 500 may be an ophthalmology apparatus 100 to which an imaging apparatus 530 is added. The sample of this aspect is the eye. Hereinafter, the description will be given with reference to the elements of the ophthalmic apparatus 100 (see FIG. 2 and the like).

The photographing device 530 photographs the eye and acquires a front image. The photographing device 530 may include a camera for photographing the eye (fundus, anterior eye portion, etc.) from the front. Alternatively, the photographing device 530 may include a camera that photographs the eye from one or more positions other than the front, and a processor that converts an image acquired by the camera into a front image. Instead of the photographing device 530, a communication device that receives the front image of the eye from an external device or a reader that reads the front image of the eye from the recording medium may be included. Typically, the imaging apparatus 530 may be a known ophthalmic modality apparatus such as a fundus camera, a slit lamp microscope, or a surgical microscope.

The ophthalmic apparatus 500 (target region setting unit 205 in the processing apparatus 540) can correct the target region based on the front image acquired by the imaging apparatus 530 (see step S13 in FIG. 3A for comparison). ..

Therefore, for example, the processing device 540 of this embodiment identifies a partial region of the front image corresponding to the target region, and the image data generated based on the three-dimensional data set collected by the OCT scan for the target region. It is possible to perform registration with the front image and calculate the displacement amount of the image data with respect to the partial region.

The control device 510 controls to change the target region so as to cancel the displacement amount calculated by the processing device 540. The change of the target region may be any of, for example, movement of the target region, change of the size of the target region, change of the shape of the target region, and change of the number of the target regions. The control device 510 (scan control unit 210) reapplies the OCT scan to the target region with such a change.

In the field of ophthalmology (particularly the diagnosis of the fundus), infrared illumination is widely used for moving the fundus without inducing miosis. The use of the image obtained by infrared observation (infrared front image) is not limited to the correction of the target region. For example, it is possible to use an infrared front image for specifying a shooting area, setting a target area, setting a scan order, evaluating an overlapping area, registration (first registration), synthesizing image data, and the like. is there. The ophthalmic apparatus 500 may be configured to display or analyze an infrared frontal image for any of these processes.

Prior to the first registration, a "rough" registration (second registration) may be performed with reference to the infrared front image. The second registration is performed by comparing each of the plurality of image data corresponding to the plurality of target regions with the infrared front image to specify a partial region of the infrared front image corresponding to the image data. It is said.

Here, the image data corresponding to the target region is three-dimensional image data (typically image data defined in the XYZ coordinate system), and the infrared front image is two-dimensional image data (typically in the XY coordinate system). Image data defined in). The second registration may include, for example, a process of converting the three-dimensional image data corresponding to the target region into the two-dimensional image data defined in the same two-dimensional coordinate system as the infrared front image. The two-dimensional image data obtained by converting the three-dimensional image data may be, for example, the projection image described above.

When the projection is applied, the registration result between the infrared front image and the projection image can be used to perform the registration between the infrared front image and the three-dimensional image data.

According to such a second registration, rough image alignment in the XY direction can be performed. In the first registration performed after the second registration, registration in the Z direction can be performed in addition to precise registration in the XY direction. It is possible to perform registration in the rotation direction in the same manner as registration in the coordinate axis direction.

Some effects of the ophthalmic apparatus (OCT apparatus) 100 and 500 of the exemplary embodiment will be described.

The ophthalmic apparatus 100 (500) includes an OCT scanner 220 (520), an image data generation unit 201, a registration unit 202, and an image data synthesis unit 203. The OCT scanner 220 collects a plurality of 3D data sets from a plurality of different 3D regions of the eye 120. The image data generation unit 201 generates a plurality of image data from a plurality of three-dimensional data sets collected by the OCT scanner 220. The registration unit 202 performs the first registration between the plurality of image data by applying the image correlation calculation to each of the plurality of overlapping regions in the plurality of image data generated by the image data generation unit 201. .. The image data synthesizing unit 203 synthesizes a plurality of image data corresponding to a plurality of three-dimensional regions based on the result of the first registration executed by the registration unit 202.

According to such an ophthalmic apparatus 100, image data synthesis can be performed after registration is performed by image correlation calculation between three-dimensional image data, and therefore, Patent Document 1 (US Pat. No. 7,884,945) and Patent. It is possible to obtain composite image data corresponding to a wide area of a sample without performing processing related to landmarks as in the invention described in Document 2 (US Pat. No. 8,405,834). Therefore, it is possible to improve the efficiency of resources required for processing and shorten the processing time, and it is possible to further improve the efficiency of OCT data processing. Thereby, for example, real-time processing can be preferably performed.

The ophthalmic apparatus 100 (500) may further include an imaging area designation unit 204 and a target area setting unit 205. The shooting area designation unit 204 designates a shooting area for the sample. The target area setting unit 205 sets a plurality of target areas so as to include the shooting area designated by the shooting area designation unit 204. The OCT scanner 220 (520) sequentially applies an OCT scan to a plurality of target regions set by the target region setting unit 205, thereby corresponding to a plurality of target regions (that is, a plurality of three-dimensional regions). A 3D dataset can be collected.

According to such a configuration, it is possible to suitably acquire the composite image data in a desired photographing region.

The ophthalmic apparatus 100 (500) may further include an overlapping region evaluation unit 206. In this case, the OCT scanner 220 (520) can apply the OCT scan to two target regions out of the plurality of target regions set by the target region setting unit 205 to collect two three-dimensional data sets. .. The image data generation unit 201 can generate two image data from the two three-dimensional data sets collected from the two target regions. The overlapping area evaluation unit 206 evaluates the overlapping area of the two image data corresponding to the two target areas generated by the image data generation unit 201. The ophthalmologic apparatus 100 (500) applies the first registration to at least the two image data or to at least one of the two target regions, depending on the result of the evaluation performed by the overlapping region evaluation unit 206. Reapply the OCT scan.

With such a configuration, the first registration or reapplication of the OCT scan can be selectively performed while evaluating the overlapping region. Typically, the first registration is selected when the overlapping area between the image data is suitable, and the reapplying of the OCT scan is selected when the overlapping area between the image data is not suitable. As a result, if a suitable overlapping region cannot be obtained, the image data can be acquired again, and the first registration and image data synthesis can be performed using the suitable overlapping region.

In the ophthalmic apparatus 100, the target region setting unit 205 can change at least one of the two target regions based on the result of the evaluation performed by the overlapping region evaluation unit 206. The OCT scanner 220 can reapply the OCT scan to the target area changed by the target area setting unit 205.

According to such a configuration, the target area for the reapplication of the OCT scan can be adjusted, so that it is highly possible that a suitable overlapping area can be obtained by the reapplying of the OCT scan.

The ophthalmic apparatus 500 may further include means for preparing a frontal image of the sample (eg, imaging apparatus 530, communication device, reader, etc.). In this case, the target area setting unit 205 can change at least one of the two target areas based on the front image. The OCT scanner 220 can reapply the OCT scan to the target area changed by the target area setting unit 205.

According to such a configuration, the target area for the reapplication of the OCT scan can be adjusted, so that it is highly possible that a suitable overlapping area can be obtained by the reapplying of the OCT scan.

If the ophthalmic apparatus 500 includes means for preparing a frontal image of the sample (eg, imaging apparatus 530, communication device, reader, etc.), the registration unit 202 is based on the frontal image prior to the first registration. A second registration can be performed between the plurality of image data.

According to such a configuration, after performing the second registration using the front image and roughly aligning the relative alignment of the plurality of image data, the precise first registration considering the overlapping region is performed. It can be performed. This makes it possible to improve the quality of registration.

In the ophthalmic apparatus 100 (500), the image data synthesizing unit 203 converts the composite image data obtained by synthesizing a plurality of image data corresponding to a plurality of target areas into a photographing area designated by the photographing area designation unit 204. The corresponding partial image data can be extracted.

According to such a configuration, image data (partial image data) representing a desired photographing region can be reliably acquired.

In the ophthalmic apparatus 100 (500), the image data synthesizing unit 203 designates an imaging area from a plurality of image data corresponding to a plurality of target areas based on the result of the first registration executed by the registration unit 202. A partial image data group corresponding to the photographing area designated by the unit 204 can be extracted. Further, the image data synthesizing unit 203 can generate composite image data by synthesizing a group of partial image data extracted from a plurality of image data.

According to such a configuration, image data (partial image data) representing a desired photographing region can be reliably acquired.

In the ophthalmic apparatus 100 (500), the image correlation calculation executed by the registration unit 202 for the first image data and the second image data including the overlapping region is the overlapping region and the second image data in the first image data. At least one of the amount of translation and the amount of rotational movement between the overlapping region in the image data can be calculated. This image correlation calculation may include a phase-limited correlation calculation.

With such a configuration, the position between the first image data and the second image data by image correlation (eg, phase-limited correlation) without going through a resource-intensive process such as landmark detection. It is possible to find relationships efficiently.

As described above, the sample of this embodiment is a living eye, but the same function and configuration can be applied to an OCT device for a sample other than the living eye. That is, it is possible to combine any matter (function, hardware configuration, software configuration, etc.) relating to the ophthalmic apparatus 100 (500) with the OCT apparatus of any aspect. At this time, the action and effect according to the combined items are exhibited.

Some aspects relate to a method of controlling an OCT apparatus including an OCT scanner and a processor that applies an OCT scan to a sample. This control method may include at least the following steps: controlling the OCT scanner to collect multiple 3D datasets from different 3D regions of the sample; from multiple 3D datasets. A step of controlling the processor to generate a plurality of image data; a first registration between the plurality of image data is performed by applying an image correlation calculation to each of the plurality of overlapping regions in the plurality of image data. The step of controlling the processor so as to synthesize a plurality of image data based on the result of the first registration.

According to such a control method of the OCT apparatus, it is possible to acquire composite image data corresponding to a wide area of a sample without performing processing related to landmarks. Therefore, it is possible to improve the efficiency of resources required for processing and shorten the processing time, and it is possible to further improve the efficiency of OCT data processing. Thereby, for example, real-time processing can be preferably performed.

It is possible to combine any matter (function, hardware configuration, software configuration, etc.) relating to the ophthalmic apparatus 100 (500) with the control method of this embodiment. At this time, the action and effect according to the combined items are exhibited.

Some aspects relate to a program that causes a computer to perform such a method of controlling an OCT apparatus. It is possible to combine any of the items described with respect to the ophthalmic apparatus 100 (500) with this program. Also, some aspects relate to computer-readable non-temporary recording media on which such programs are recorded. It is possible to combine this recording medium with any of the items described with respect to the ophthalmic apparatus 100 (500).

The ophthalmic apparatus 100 (500) functions as an apparatus (OCT data processing apparatus) for processing the data collected by the OCT scan. The ophthalmic apparatus 100 (500) includes an OCT scanner 220 (520) as a means for applying the OCT scan, but the OCT data processing apparatus does not have to include this. The OCT data processing device includes, for example, a reception unit including the above-mentioned communication device and / or reader, and receives a plurality of three-dimensional data sets collected from a plurality of different three-dimensional regions of a sample from the outside. The image data generation unit 201 generates a plurality of image data from a plurality of three-dimensional data sets received from the outside by the reception unit. The registration unit 202 performs the first registration between the plurality of image data by applying the image correlation calculation to each of the plurality of overlapping regions in the plurality of image data generated by the image data generation unit 201. .. The image data synthesizing unit 203 synthesizes a plurality of image data based on the result of the first registration executed by the registration unit 202.

According to such a control method of the OCT data processing device, it is possible to acquire composite image data corresponding to a wide area of a sample without performing processing related to landmarks. Therefore, it is possible to improve the efficiency of resources required for processing and shorten the processing time, and it is possible to further improve the efficiency of OCT data processing. Thereby, for example, real-time processing can be preferably performed.

It is possible to combine any matter (function, hardware configuration, software configuration, etc.) relating to the ophthalmic apparatus 100 (500) with the control method of this embodiment. At this time, the action and effect according to the combined items are exhibited.

Some aspects relate to a program that causes a computer to execute such a control method of an OCT data processor. It is possible to combine any of the items described with respect to the ophthalmic apparatus 100 (500) with this program. Also, some aspects relate to computer-readable non-temporary recording media on which such programs are recorded. It is possible to combine this recording medium with any of the items described with respect to the ophthalmic apparatus 100 (500).

Some aspects of the OCT apparatus (eg, ophthalmic apparatus 100 or 500), some aspects of the OCT apparatus control method, some aspects of the OCT data processing apparatus, or some aspects of the OCT data processing apparatus. The control method provides an imaging method using OCT.

The OCT imaging method of some embodiments may include at least the following steps: the step of collecting multiple 3D data sets from different 3D regions of a sample; multiple 3D data sets. Step of generating image data; step of performing first registration between a plurality of image data by applying an image correlation calculation to each of a plurality of overlapping regions in a plurality of image data; result of the first registration Steps to combine multiple image data based on.

It is possible to combine any matter (function, hardware configuration, software configuration, etc.) relating to the ophthalmic apparatus 100 (500) with the OCT imaging method of this embodiment. At this time, the action and effect according to the combined items are exhibited.

Some aspects relate to a program that causes a computer to perform such an OCT imaging method. It is possible to combine any of the items described with respect to the ophthalmic apparatus 100 (500) with this program. Also, some aspects relate to computer-readable non-temporary recording media on which such programs are recorded. It is possible to combine this recording medium with any of the items described with respect to the ophthalmic apparatus 100 (500).

Some aspects of the OCT apparatus (eg, ophthalmic apparatus 100 or 500), some aspects of the OCT apparatus control method, some aspects of the OCT data processing apparatus, or some aspects of the OCT data processing apparatus. The control method provides a method of processing OCT data.

The OCT data processing method of some embodiments may include at least the following steps: preparing multiple 3D datasets collected from multiple different 3D regions of the sample; multiple 3D data. A step of generating a plurality of image data from a set; a step of performing a first registration between a plurality of image data by applying an image correlation calculation to each of a plurality of overlapping regions in the plurality of image data; a first step. The step of synthesizing multiple image data based on the registration result.

It is possible to combine any matter (function, hardware configuration, software configuration, etc.) related to the ophthalmic apparatus 100 (500) with the OCT data processing method of this embodiment. At this time, the action and effect according to the combined items are exhibited.

Some aspects relate to a program that causes a computer to perform such an OCT data processing method. It is possible to combine any of the items described with respect to the ophthalmic apparatus 100 (500) with this program. Also, some aspects relate to computer-readable non-temporary recording media on which such programs are recorded. It is possible to combine this recording medium with any of the items described with respect to the ophthalmic apparatus 100 (500).

The non-temporary recording medium may be in any form, and examples thereof include a magnetic disk, an optical disk, a magneto-optical disk, and a semiconductor memory.

Some of the embodiments described above are merely examples of embodiments of the present invention. Therefore, it is possible to make arbitrary modifications (omission, replacement, addition, etc.) within the scope of the gist of the present invention.

100 Ophthalmic equipment (OCT equipment)
124 Processing device 126 Control device 201 Image data generation unit 202 Registration unit 203 Image data synthesis unit 204 Imaging area designation unit 205 Target area setting unit 206 Overlapping area evaluation unit 210 Scan control unit 220 OCT scanner 500 Ophthalmology device 510 Control device 520 OCT Scanner 530 Imaging device 540 Processing device

Claims (19)

A method of processing data collected by optical coherence tomography (OCT) scans.
Prepare multiple 3D datasets collected from multiple 3D regions that differ from each other in the sample.
A plurality of image data are generated from the plurality of three-dimensional data sets, and a plurality of image data are generated.
By applying the image correlation calculation to each of the plurality of overlapping regions in the plurality of image data, the first registration between the plurality of image data is performed.
The plurality of image data are synthesized based on the result of the first registration.
OCT data processing method. For the first image data and the second image data including the overlapping region, the image correlation calculation is a translation between the overlapping region in the first image data and the overlapping region in the second image data. Calculate at least one of the amount and the amount of translation,
The OCT data processing method according to claim 1. The image correlation calculation is the OCT data processing method of claim 2, which includes a phase-limited correlation calculation. Prepare a front image of the sample
Prior to the first registration, a second registration between the plurality of image data is performed based on the front image.
The OCT data processing method according to any one of claims 1 to 3. An imaging method using optical coherence tomography (OCT).
Collect multiple 3D datasets from different 3D regions of the sample
A plurality of image data are generated from the plurality of three-dimensional data sets, and a plurality of image data are generated.
By applying the image correlation calculation to each of the plurality of overlapping regions in the plurality of image data, the first registration between the plurality of image data is performed.
The plurality of image data are synthesized based on the result of the first registration.
OCT imaging method. Specify the shooting area for the sample,
A plurality of target areas are set so as to include the imaging area.
Collecting the plurality of 3D data sets by sequentially applying OCT scans to the plurality of target regions.
The OCT imaging method according to claim 5. An OCT scan was applied to two of the plurality of target regions to collect two 3D data sets.
Two image data are generated from the two three-dimensional data sets,
The overlapping area of the two image data is evaluated, and
Depending on the result of the evaluation, the first registration is applied to at least the two image data, or the OCT scan is reapplied to at least one of the two target regions.
The OCT imaging method according to claim 6. Based on the result of the evaluation, at least one of the two target regions was changed.
Reapply the OCT scan to the changed target area.
The OCT imaging method according to claim 7. Prepare a front image of the sample
At least one of the two target areas is modified based on the front image.
Reapply the OCT scan to the changed target area.
The OCT imaging method according to claim 7. Partial image data corresponding to the shooting region is extracted from the composite image data obtained by synthesizing the plurality of image data.
The OCT imaging method according to any one of claims 6 to 9. OCT imaging according to any one of claims 6 to 9, in which a partial image data group corresponding to the photographing region is extracted from the plurality of image data based on the result of the first registration, and the partial image data group is synthesized. Method. For the first image data and the second image data including the overlapping region, the image correlation calculation is a translation between the overlapping region in the first image data and the overlapping region in the second image data. Calculate at least one of the amount and the amount of translation,
The OCT imaging method according to any one of claims 5 to 11. The OCT imaging method according to claim 12, wherein the image correlation calculation includes a phase-limited correlation calculation. A device that processes data collected by optical coherence tomography (OCT) scans.
A reception unit that accepts multiple 3D datasets collected from multiple 3D regions that differ from each other in the sample.
An image data generation unit that generates a plurality of image data from the plurality of three-dimensional data sets,
A registration unit that performs a first registration between the plurality of image data by applying an image correlation calculation to each of the plurality of overlapping regions in the plurality of image data.
An OCT data processing apparatus including an image data synthesizing unit that synthesizes a plurality of image data based on the result of the first registration. A method of controlling an optical coherence tomography (OCT) data processor, including a processor.
The processor is controlled to accept multiple 3D datasets collected from different 3D regions of the sample.
The processor is controlled to generate a plurality of image data from the plurality of 3D data sets.
The processor is controlled so as to perform the first registration between the plurality of image data by applying the image correlation calculation to each of the plurality of overlapping regions in the plurality of image data.
The processor is controlled to synthesize the plurality of image data based on the result of the first registration.
Control method of OCT data processing device. An OCT scanner that collects multiple 3D data sets by applying optical coherence tomography (OCT) scans to multiple 3D regions of the sample.
An image data generation unit that generates a plurality of image data from the plurality of three-dimensional data sets,
A registration unit that performs a first registration between the plurality of image data by applying an image correlation calculation to each of the plurality of overlapping regions in the plurality of image data.
An OCT apparatus including an image data synthesizing unit that synthesizes the plurality of image data based on the result of the first registration. A method of controlling an OCT device that includes an OCT scanner and processor that applies an optical coherence tomography (OCT) scan to a sample.
The OCT scanner is controlled to collect multiple 3D datasets from multiple 3D regions that differ from each other in the sample.
The processor is controlled to generate a plurality of image data from the plurality of 3D data sets.
The processor is controlled so as to perform the first registration between the plurality of image data by applying the image correlation calculation to each of the plurality of overlapping regions in the plurality of image data.
The processor is controlled to synthesize the plurality of image data based on the result of the first registration.
Control method of OCT device. A program that causes a computer to execute any of the methods 1 to 13, 15 and 17. A computer-readable non-temporary recording medium on which the program of claim 18 is recorded.

Images