JP2020116410A - Image processing apparatus and actuation method for image processing apparatus - Google Patents

Abstract

To provide an image processing apparatus that enables acquisition of an excellent two-dimensional image regardless of an acquisition condition on information in a depth direction of a retina and the like.SOLUTION: An image processing apparatus for processing an image acquired by acquisition means for acquiring pixel value rows arranged in a depth direction of an object to be measured on the basis of interference light obtained by making return light of scanned measurement light from the object to be measured interfere with reference light corresponding to the measurement light comprises: generation means for generating a two-dimensional image on the basis of a pixel value selected from each of the plurality of pixel value rows according to a predetermined selection criterion; setting means for setting a selection range of a pixel value selected in the depth direction for each of the plurality of pixel value rows; and criterion change means for changing the predetermined selection criterion depending on the set selection range.SELECTED DRAWING: Figure 3

Description

The present invention relates to an image processing apparatus and an image processing method for obtaining a two-dimensional image from information on a tomographic image obtained by optical interference.

An imaging apparatus (hereinafter referred to as an OCT apparatus) using optical coherence tomography (OCT) utilizing multi-wavelength light wave interference is known. The OCT device is used to obtain, for example, built-in information in an endoscope and retina information in an ophthalmologic device, and is expanding its application field to the human body. In particular, the OCT device applied to the eye is becoming an indispensable device in an outpatient specialized in retina as an ophthalmic device.

The OCT apparatus irradiates a sample with measurement light that is low-coherent light and measures backscattered light from the sample using an interference system, thereby enabling image acquisition. When applied to the eye, the tomographic image of the eye to be inspected can be captured with high resolution by scanning the eye to be inspected with the measurement light.

Further, there is known a technique of generating a two-dimensional image in which the fundus is viewed from the front in a pseudo manner (hereinafter, referred to as “two-dimensional image”) from a plurality of tomographic images. As a method of generating a two-dimensional image, a pixel value is selected from the pixel value sequence in the depth direction obtained by one A scan based on the order of the magnitude of the pixel value. Then, by obtaining the selected pixel values for all A scans, a two-dimensional image (Projection image) similar to the planar image of the retina with only the tomographic image is generated (Patent Document 1).

Patent Document 1 also discloses a method of generating a two-dimensional image of a predetermined layer when the predetermined layer of the retina is selected. In this method, a pixel value is selected from a pixel value sequence in the layer in the order of the magnitude of the pixel value, and a two-dimensional image is generated.

JP, 2014-45869, A

By the way, the ratio of the object to be measured occupying in the pixel value sequence greatly differs between all the pixel value sequences in the depth direction of the retina and the pixel value sequences in the range including a predetermined layer in the depth direction. Therefore, the pixel value is selected based on the same criterion when selecting one pixel value from all the pixel value columns in the depth direction and when selecting one pixel value from the pixel value column within the predetermined range. Then, a good two-dimensional image may not be obtained.

Further, for example, when trying to obtain a two-dimensional image of a layer having a lesion, the pixel value corresponding to the tissue affected by the lesion may greatly differ from the peripheral portion. Also in this case, there is a possibility that the pixel value corresponding to the lesion may be deleted when the uniform standard is used.

In view of the above problems, it is an object of the present invention to provide an image processing apparatus and an image processing method capable of obtaining a good two-dimensional image regardless of the acquisition condition of information in the depth direction of the retina or the like.

In order to achieve the above object, the image processing apparatus according to the present invention,
Acquisition means for acquiring a pixel value sequence arranged in the depth direction of the measured object based on the interference light obtained by causing the return light of the scanned measurement light from the measured object and the reference light corresponding to the measured light to interfere with each other. When,
Generating means for generating a two-dimensional image based on pixel values selected from each of the plurality of pixel value columns according to a predetermined selection criterion;
Setting means for setting a selection range of the selected pixel value in the depth direction in each of the plurality of pixel value columns;
A reference changing unit that changes the predetermined selection criterion according to the set selection range.

According to the present invention, it is possible to obtain a good two-dimensional image regardless of the acquisition condition of information in the depth direction of the retina or the like.

It is a block diagram of an imaging system having an image processing apparatus according to the first embodiment of the present invention. It is a side view of the ophthalmologic apparatus which comprises the imaging system illustrated in FIG. It is a block diagram which shows the structure of the image processing apparatus in the 1st Embodiment of this invention. FIG. 2 is a configuration diagram of an optical system of a photographing device used in the photographing system shown in FIG. 1. 6 is a flowchart showing a flow of two-dimensional image generation in the first mode of the first embodiment. It is explanatory drawing which shows the pixel value sequence of an A scan image. It is explanatory drawing which shows an example of rearrangement of a pixel value. 6 is a flowchart showing a flow of two-dimensional image generation in the second mode of the first embodiment. It is explanatory drawing which shows the example of a setting of a depth range. It is a figure which shows the example of a display of the tomographic image and two-dimensional image in the imaging system shown in FIG. It is a figure which shows the example of a display of the Projection image and EnFace image in the imaging system shown in FIG.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the following embodiments do not limit the present invention related to the claims, and all combinations of the features described in the present embodiments are essential to the solving means of the present invention. Not necessarily.

[First Embodiment]
FIG. 1 is a diagram showing a configuration example of an image capturing system 1000 having an image processing apparatus 100 according to a first embodiment of the present invention and an image capturing apparatus 1 connected thereto. The image processing apparatus 100 according to this embodiment includes a central processing unit (CPU) 10, a main memory 11, a magnetic disk 12, and a display memory 13. The image capturing system 1000 also includes a monitor 928, a mouse 929-1, and a keyboard 929-2. Note that the monitor 928 may have a touch panel.

The CPU 10 mainly controls the operation of each component of the image processing apparatus 100. The main memory 11 stores a control program executed by the CPU 10 and provides a work area when the CPU 10 executes the program. The magnetic disk 12 stores an operating system (OS), device drivers of peripheral devices, various application software including programs for performing reconstruction processing described later, and the like. The display memory 13 temporarily stores display data for the monitor 928. The monitor 928 is, for example, a CRT monitor, a liquid crystal monitor, or the like, and displays an image based on the data from the display memory 13. A mouse 929-1 and a keyboard 929-2 are used by the user to perform pointing input and input of characters and the like, respectively. The above components are connected to each other via a common bus 17 so that they can communicate with each other.

The image processing apparatus 100 is connected to the image capturing apparatus 1 via a local area network (LAN) and can acquire image data from the image capturing apparatus 1. Note that the form of the present invention is not limited to this, and the connection with these devices may be performed, for example, via another interface such as USB or IEEE1394. Further, it may be configured to read necessary data from an external device 3 such as a data server that manages these data via a LAN or the like. A storage device such as an FDD, a CD-RW drive, an MO drive, or a ZIP drive may be connected to the image processing apparatus 100, and necessary data may be read from these drives.

FIG. 2 is a side view of the photographing apparatus 1 constructed in the form of an ophthalmologic apparatus with the above configuration. The imaging apparatus 1 shown in the figure has an external fixation target 324, a chin rest 323, a base portion 951, a stage portion 950, and an optical head 900. The optical head 900 is a measurement optical system for capturing an anterior eye image of the eye to be inspected, and a surface image and a tomographic image of the fundus. The stage unit 950 constitutes a moving unit that allows the optical head 900 to move in the xyz directions in the drawing using a motor (not shown). The base section 951 incorporates a spectroscope described later.

A personal computer (hereinafter referred to as “personal computer”) 925 doubles as a control unit of the stage unit and has the image processing apparatus 100. The chin rest 323 fixes the chin of the subject and the forehead, thereby urging the fixation of the subject's eye (the subject's eye). The external fixation target 324 is used to fix the eye of the subject. Further, the image processing apparatus 100 can be incorporated in the optical head 900 or the stage section 950. In this case, the photographing device 1 and the image processing device 100 are integrally configured as a photographing device.

FIG. 3 is a block diagram showing the functional configuration of the image processing apparatus 100. The image processing apparatus 100 has a reconstruction unit 1100, a generation unit 1200, and a layer recognition unit 1300. The reconstructing unit 1100 functions as a reconstructing unit, and obtains a tomographic image of a predetermined range of the measured object based on the interference light obtained by causing the return light of the measuring light from the measured object and the reference light to interfere with each other. More specifically, the output value from the sensor is subjected to wave number conversion and fast Fourier transform (FFT) processing, and reconstructed as a tomographic image (A scan image) in the depth direction at one point on the fundus of the eye to be inspected. The generation unit 1200 functions as a generation unit that selects a predetermined pixel for each pixel value sequence from each pixel value sequence in the depth direction of the tomographic image obtained by the reconstruction unit 1100 and generates a two-dimensional image. To do. The layer recognition unit 1300 extracts the layer structure of the measured object (retina) in the tomographic image obtained by the reconstruction unit 1100 and specifies the shape of each layer boundary.

FIG. 4 is a diagram illustrating the configurations of the measurement optical system and the spectroscope of the image capturing apparatus 1. The optical elements arranged inside the optical head 900 and the base 951 will be described below.

In the present embodiment, the following description will be given by exemplifying the subject's eye 107 as the measured object. In the optical head 900, an objective lens 135-1 is installed so as to face the subject's eye 107. A first dichroic mirror 132-1 and a second dichroic mirror 132-2 are arranged on the optical axis of the objective lens 135-1. The optical path to the eye 107 to be inspected is branched by the first dichroic mirror 132-1 to the optical path 353 for anterior eye observation according to the wavelength band. Further, the optical path after branching is further branched by the second dichroic mirror 132-2 into an optical path 352 for a fixation lamp and an optical path 351 of an OCT optical system for fundus observation for each wavelength band.

A focusing lens 135-3, a lens 135-4, a perforated mirror 303, and a third dichroic mirror 132-3 are arranged in this order from the eye 107 side on the fixation lamp optical path 352. The CCD 172 is arranged in the reflection direction of the third dichroic mirror 132-3, and the fixation lamp 191 is arranged in the transmission direction. Here, the focusing lens 135-3 is driven in the direction along the optical path 352 by a motor (not shown) for adjusting the focusing of the fixation target 191 and the CCD 172 for fundus observation.

The perforated mirror 303 is disposed between the lens 135-4 and the third dichroic mirror 132-3, and branches the optical path from the optical path 352 to the optical path 354.

The optical path 354 forms an illumination optical system that illuminates the fundus of the subject's eye 107. An LED light source 316, which is a fundus observation illumination light source used for alignment of the eye 107 to be inspected, and a strobe tube 314 used to image the fundus of the eye 107 to be inspected are installed. The illumination optical system has a condenser lens 315, a strobe tube 314, a condenser lens 313, a mirror 317, a ring slit 312, a lens 311 and a lens 309, which are arranged in order from the innermost LED light source 316.

Illumination light from the LED light source 316 and the strobe tube 314 becomes a ring-shaped luminous flux by the ring slit 312, is reflected by the perforated mirror 303, and illuminates the fundus 127 of the subject's eye 107. The LED light source 316 is a light source having a center wavelength near 780 nm.

As described above, the third dichroic mirror 132-3 is arranged after the perforated mirror 303 on the optical path 352. The branching of the optical path to the CCD 172 for eye fundus observation and the fixation lamp 191 by the third dichroic mirror 132-3 is performed for each wavelength band like other dichroic mirrors.

The CCD 172 is sensitive to the central wavelength of the light emitted from the LED light source 316 which is the illumination light for fundus observation, specifically, near 780 nm, and is connected to the CCD control unit 102. On the other hand, the fixation target 191 generates visible light and promotes the fixation of the subject, and is connected to the fixation target control unit 103.

The CCD control unit 102 and the fixation target control unit 103 are both connected to the calculation unit 104, and data is input to and output from the personal computer 925 through the calculation unit 104.

A lens 135-2 and an infrared CCD 171 for observing the anterior eye are arranged in order from the branching portion on the optical path 353 branched to the first dichroic mirror 132-1. The CCD 171 has sensitivity in the wavelength of illumination light for anterior eye observation (not shown), specifically, in the vicinity of 970 nm. Further, an image split prism (not shown) is arranged in the optical path 353, and the distance in the z direction of the optical head 900 portion with respect to the eye 107 to be inspected (direction of approaching and separating from the eye 107 to be inspected) is measured in the anterior eye observation image. Can be detected as a split image of.

The optical path 351 constitutes an OCT optical system as described above, and a configuration for actually capturing a tomographic image of the retina of the subject's eye 107 is arranged. More specifically, a configuration for obtaining an interference signal for forming a tomographic image is arranged. An XY scanner 134, a focusing lens 135-5, and a lens 135-6 are arranged on the optical path 351 in order from the second dichroic mirror 132-2, and the measurement light is supplied to the optical path 351 from the end of the fiber 131-2. To be done.

The XY scanner 134 is used to scan the measurement light on the fundus. Although the XY scanner 134 is shown as a single mirror, it scans in the XY biaxial directions. The focusing lens 135-5 is driven along an optical path 351 by a motor (not shown) in order to focus the light from the light source 101 emitted from the end of the optical fiber 131-2, which will be described later, on the fundus 127. .. Further, by this focusing adjustment, the light from the fundus 127 is simultaneously imaged and incident on the tip of the optical fiber 131-2 in a spot shape.

Next, the configurations of the optical path from the light source 101, the reference optical system, and the spectroscope will be described.

The light source 101 is connected to the optical coupler 131 via the optical fiber 131-1. The optical coupler 131 is connected to the optical fibers 131-3 and 131-4 in addition to the optical fibers 131-1 and 131-2. These optical fibers are single mode optical fibers connected to and integrated with the optical coupler 131. The light emitted from the light source 101 is split into a measurement light and a reference light by the optical coupler 131, the measurement light is passed through the optical fiber 131-2 to the optical path 351, and the reference light is passed through the optical fiber 131-3 and the reference optical path described later. Be led to. In the reference optical path, a mirror 132-4, a dispersion compensating glass 115, and a lens 135-7 are arranged in this order from the depth of the optical path. The dispersion compensating glass 115 is inserted in the reference optical path in order to match the dispersions of the measurement light and the reference light.

The reference light having passed through the reference optical path is incident on the end portion of the optical fiber 131-3, is similarly combined with the measuring light returned to the optical coupler 131, and is guided to the spectroscope 180 via the optical fiber 131-4. These configurations in this embodiment form a Michelson interference system.

The measurement light is applied to the fundus of the eye 107 to be inspected, which is the observation target, through the optical path of the OCT optical system described above, and reaches the optical coupler 131 through the same optical path due to reflection and scattering by the retina. On the other hand, the reference light reaches the mirror 132-4 via the optical fiber 131-3, the lens 135-7, and the dispersion compensation glass 115 and is reflected. Then, the same optical path is returned to reach the optical coupler 131.

The measurement light and the reference light are combined by the optical coupler 131 to become interference light. Here, interference occurs when the optical path length of the measurement light and the optical path length of the reference light become substantially the same. The mirror 132-4 is held by a motor and a driving mechanism (not shown) so as to be adjustable in the optical axis direction, and the optical path length of the reference light can be matched with the optical path length of the measurement light that changes depending on the eye 107 to be inspected. The interference light is guided to the spectroscope 180 via the optical fiber 131-4.

Further, the polarization adjusting unit 139-1 on the measurement light side is provided in the optical fiber 131-2. The polarization adjusting unit 139-2 on the reference light side is also provided in the optical fiber 131-3. These polarization adjusting units have several portions in which the optical fiber is looped around, and twist the fiber by rotating the looped portion around the longitudinal direction of the fiber. By this operation, the polarization states of the measurement light and the reference light can be adjusted and matched. In this device, the polarization states of the measurement light and the reference light are adjusted and fixed in advance.

Next, the spectroscope 180 will be described. The spectroscope 180 includes a lens 135-8, a lens 135-9, a diffraction grating 181, and a line sensor 182. The interference light emitted from the optical fiber 131-4 becomes substantially parallel light via the lens 135-8, is then dispersed by the diffraction grating 181, and is imaged on the line sensor 182 by the lens 135-3. The output of the line sensor 182 is input to the personal computer 925.

Next, the periphery of the light source 101 will be described. As the light source 101, an SLD (Super Luminescent Diode), which is a typical low coherent light source, is used. The center wavelength of the light emitted from the light source 101 is 855 nm, and the wavelength band width is about 100 nm. Here, the bandwidth is an important parameter because it affects the resolution of the obtained tomographic image in the optical axis direction. Although SLD is selected here as the type of light source, ASE (Amplified Spontaneous Emission) or the like may be used as long as it can emit low-coherent light. The near-infrared light is suitable as the center wavelength in view of measuring the eye. Further, since the central wavelength affects the lateral resolution of the obtained tomographic image, it is desirable that the central wavelength be as short as possible. The center wavelength was set to 855 nm for both reasons.

Although the Michelson interferometer is used as the interferometer in this embodiment, a Mach-Zehnder interferometer may be used. It is desirable to use a Mach-Zehnder interferometer when the light amount difference is large according to the light amount difference between the measurement light and the reference light, and to use a Michelson interferometer when the light amount difference is relatively small.

Next, an image pickup method of the eye to be inspected using the image pickup apparatus will be described.

First, the examiner seats the patient in front of the image capturing apparatus according to this embodiment and starts capturing a surface image of the fundus of the eye to be examined. The light emitted from the light source 316 becomes a ring-shaped luminous flux by the ring slit 312 and is reflected by the perforated mirror 303 toward the eye 107 to be inspected. The reflected light illuminates the fundus 127 of the subject's eye 107. The light flux reflected from the fundus 127 by the illumination passes through the perforated mirror 303 and is imaged on the CCD 172. The reflected light of the fundus 127 formed on the CCD 172 is imaged as a surface image of the fundus by the CCD control unit 102 and transmitted to the image processing apparatus 100.

Next, the image capturing apparatus 1 controls the XY scanner 134 to capture a tomographic image of a desired site on the fundus of the subject's eye 107, and more specifically, obtains brightness information in the depth direction at each position of the desired site. To do.

The measurement light directed to the subject's eye passes through the optical fiber 131-2, is emitted from the fiber end, and enters the XY scanner 134. The measurement light deflected by the XY scanner 134 illuminates the fundus 127 of the eye to be inspected via the optical system 135-1. Then, the reflected light reflected by the eye to be examined follows the reverse path and is returned to the optical coupler 131.

On the other hand, the reference light traveling toward the reference mirror passes through the optical fiber 131-3, is emitted from the fiber end, and reaches the reference mirror 132-4 through the collimating optical system 135-7 and the dispersion compensating optical system 115. The reference light reflected by the reference mirror 132-4 follows the reverse path and is returned to the optical coupler 131.

The measurement light and the reference light that have returned to the optical coupler 131 interfere with each other, become interference light, enter the optical fiber 131-4, and are substantially collimated by the optical system 135-8 and enter the diffraction grating 181. The interference light input to the diffraction grating 181 is imaged on the line sensor 182 by the imaging lens 135-9, and an interference signal at one point on the fundus of the eye to be inspected can be obtained.

The output value as an image signal having the interference information acquired by the plurality of elements of the line sensor 182 is output to the image processing apparatus 100. In FIG. 4, the mode in which the surface image of the fundus is acquired at once by the light emission of the strobe tube 314 has been described, but the surface image of the fundus may be obtained by the SLO type in which the light emitted from the SLD light source is scanned.

In the present embodiment, the configuration of the OCT optical system described above is based on the interference light obtained by causing the return light of the scanned measurement light from the measurement object and the reference light corresponding to the measurement light to interfere with each other. This corresponds to an acquisition unit that acquires a pixel value string arranged in the depth direction.

Next, the flow of the image processing method of the image processing apparatus 100 will be described.

After acquiring the tomographic information at one point on the fundus of the eye to be inspected, the imaging apparatus 1 moves the irradiation position of the measurement light in the X direction by driving the XY scanner 134 as a scanning unit, and interferes with another point on the fundus of the eye to be inspected. Generate light. The interference light at the other point is input to the reconstruction unit 1100 via the line sensor 182. The reconstruction unit 1100 forms a tomographic image (A scan image) in the depth direction at another point on the fundus of the eye to be examined based on the input data. The position of the XY scanner 134 that captured the interference signal of the A scan and the coordinates of the A scan image on the fundus image are stored in association with each other.

The reconstruction unit 1100 reconstructs one horizontal tomographic image (B-scan image) of the fundus of the eye to be inspected by continuously moving the measurement light irradiation position in the X direction by the XY scanner 134.

After moving the measurement light irradiation position by a certain amount in the Y direction by the XY scanner 134, the scanning in the X direction described above is performed again, so that a horizontal tomographic image (B scan image) of the fundus at another Y direction position on the fundus of the eye to be examined. ) Is reconfigured by the reconstructing unit 1100. By repeating the movement of the measurement light irradiation position in the Y direction by the XY scanner 134, it is possible to form a plurality of tomographic images covering a predetermined range of the fundus 127. In the image capturing apparatus 1, 128 tomographic images are generated by repeatedly forming B scan images while performing a certain amount of minute driving in the Y direction 128 times. The reconstruction unit 1100 also reconstructs (forms) a three-dimensional tomographic image from 128 tomographic images.

Next, the generation unit 1200 generates a two-dimensional image of the retina from the tomographic image generated by the reconstruction unit 1100.

In this embodiment, the generation unit 1200 has a first mode and a second mode as operation modes. The first mode produces a two-dimensional image from pixels corresponding to all depth ranges for which luminance information for the A-scan image was obtained. The second mode generates a two-dimensional image from a predetermined depth range within the depth range in which the brightness information is obtained. The two-dimensional image (hereinafter, referred to as a projection image) generated in the first mode is also called an intensity image and is similar to the surface image of the fundus. On the other hand, the two-dimensional image generated in the second mode (hereinafter, referred to as EnFace image) is a planar image generated based on information of an arbitrary depth in the retina, and is a retinal layer structure due to eye disease. Used to visualize changes. The user can select one of the first mode and the second mode by using at least one of the mouse 929-1, the keyboard 929-2, and the touch panel. For example, the user can select a mode by clicking one of the GUI showing the first mode and the GUI showing the second mode displayed on the monitor with the mouse 929-1. The monitor 928 may display the first mode and the second mode in a selectable manner by pulling down.

Here, a process of generating a projection image in the first mode will be described with reference to FIG. As described above, the tomographic image generated by the reconstruction unit 1100 is input to the generation unit 1200 (S2000).

The A scan image is a tomographic image in the depth direction at one point on the fundus of the eye to be inspected, and is composed of a plurality of brightness information in the depth direction as shown in FIG.

The two-dimensional tomographic image in FIG. 6 is a set of A scan images. This two-dimensional tomographic image may be a B-scan image, or may show a cross section of a three-dimensionally reconstructed tomographic image. As can be seen from FIG. 6, the tomographic image includes the retinal region and other portions. The retinal region is a region sandwiched by ILM and RPE, for example.

For example, the image capturing apparatus 1 used in the present embodiment uses the line sensor 182 having 2048 pixels, and the A-scan image Ai after FFT has a pixel value column composed of 1176 pixel values. Here, P0 represents the pixel value value as the brightness information of the shallowest part in the depth direction by the color density, and P1175 represents the pixel value as the brightness information of the deepest part in the depth direction. ing. When obtaining a two-dimensional image from these A-scan images, the photographing apparatus 1 selectively extracts one piece of brightness information from the plurality of pieces of brightness information. The pixel value corresponding to the extracted luminance information is a representative intensity signal of one point on the fundus of the eye to be inspected from which the A-scan image is obtained. That is, one pixel value is selected from the 1176 pixel values obtained by the A scan.

Here, the generation unit 1200 may be configured to process the reconstructed tomographic image acquired from the external device 3 to generate a projection image. In this case, it is preferable that the brightness information and the like be directly input from a data acquisition unit (not shown) without going through the reconstruction unit 1100.

The generation unit 1200 rearranges the brightness information of the tomographic images corresponding to each A scan in descending order of brightness as shown in FIG. 7. That is, the pixel values are ranked based on the magnitude relationship of the pixel values for each of the 1176 pixel value columns, and the pixel values are rearranged (sort processing) (S2010). In the figure, R0 is a pixel having the brightest luminance information as a pixel value, and R1175 is a pixel having the darkest luminance information as a pixel value. Since the brightness indicates the intensity of interference, the pixel value also corresponds to the intensity of interference. The generation unit 1200 further selects a pixel Rx of a predetermined rank in the rearranged pixel value sequence. Here, the pixel of a predetermined rank is a pixel located at the x-th position from the beginning after rearranging in the descending order of luminance information.

When generating a Projection image in the first mode, a pixel value of a predetermined rank is selected from all 1176 pixel values included in the A scan image. When a tomographic image of the retina is taken, the ratio of the object to be measured (retina) in the A scan image is about 20%. That is, about 20% of all 1176 pixels forming the A scan image are pixels in the bright retina, and the remaining about 80% are dark pixels other than the object to be measured. Therefore, when acquiring the average luminance of the retina, x is preferably a pixel positioned higher than half the total number of pixels. Specifically, when using an A-scan image composed of a pixel value sequence having a total number of pixels of 1176, the 118th pixel from the beginning corresponding to the top 10% position is selected as the pixel Rx of a predetermined rank (S2020).

The generation unit 1200 selects pixels Rx corresponding to the top 10% positions for all A-scan images, and generates a projection image using those pixel values (S2030).

This projection image is an image similar to the surface image of the fundus obtained by the CCD 172 or the fundus image obtained by another fundus camera or SLO, and the surface of the fundus can be visualized in a pseudo manner. In addition, since only effective information is selectively acquired from a plurality of luminance information, it is possible to obtain a suitable surface image without being affected by a dark component region in which the intensity of noise components and interference included in the A-scan image is low. Is possible.

Here, an example has been described in which the reconstruction unit 1100 reconstructs all data and then the generation unit 1200 generates a projection image. However, the configuration may be such that the tomographic images reconstructed for each A scan are sequentially transmitted to the generation unit 1200, or the tomographic images reconstructed for each B scan are sequentially transmitted to the generation unit 1200.

Next, the EnFace image generation process in the second mode will be described with reference to FIG.

The layer recognition unit 1300 extracts the layer structure (layer boundary) of the retina from the two-dimensional tomographic image and specifies the shape of each layer boundary. The shape of the layer boundary specified here includes ILM (retinal-vitreous boundary), NFL/GCL boundary, GCL/IPL boundary, IPL/INL boundary, IS/OS line, RPE, BM and the like. Generally, in the EnFace image, the shape of the RPE or the estimated RPE (BM) is often used, but an arbitrary shape obtained by image analysis such as the cornea shape may be used. The shape of the specified layer boundary is input to the generation unit 1200 together with the tomographic image (S3000). Here, the generation unit 1200 may be configured to generate the EnFace image using the shape of the layer boundary and the tomographic image acquired from the external device 3 by the acquisition unit (not shown). In this case, the generation unit 1200 receives an input directly from the acquisition unit 2000 without going through the reconstruction unit 1100 and the layer recognition unit 1300.

The generation unit 1200 sets a part of the pixel value sequence as the depth range Zi for generating the EnFace image in each A-scan image forming the tomographic image based on the input layer boundary shape. When obtaining an EnFace image of all layers of the retina, a depth range Z from ILM to RPE is set as shown in FIG. Further, for example, when observing edema and the like near the RPE, two lines having an estimated RPE shape are arranged at equal distances above and below the RPE as shown in FIG. 9B to determine the depth range Zi. You may set it.

Further, the two lines that set the depth range Zi can be configured to be movable up and down with a mouse 929-1 or the like. In this case, by setting one of the lines having the estimated RPE shape on the RPE and arranging the other one under the choroid, it is possible to set the depth range Zi for obtaining the EnFace image of the choroid. In the present embodiment, when setting the selection range of the pixel value in the depth direction, the depth range is set by using the display form displayed in FIG. However, the display form when setting the depth range is not limited to this, and various display forms such as clicking and selecting each layer boundary with the cursor can be used. The display in this display form is executed by the module functioning as a display control unit in the personal computer 925. For example, when the two layer boundaries in the tomographic image displayed in FIG. 9A are selected by the user using a click or the like, the area sandwiched between the selected two layer boundaries is set as the depth range Z. It The layer boundary selected by the user using a click or the like may be a display indicating a layer boundary superimposed and displayed on the tomographic image, or may be a layer boundary included in the tomographic image itself.

Further, the depth range Z can be set without using the shape of the layer boundary. For example, in order to obtain an EnFace image of the vitreous, the depth range Z can be set from the upper end of the tomographic image to the ILM boundary. It is also possible to set two straight lines at arbitrary depth positions of the tomographic image and set the depth range Z between the two straight lines.

Then, the generation unit 1200 sets the depth range Zi for all A scan images Ai (S3010). The generation unit 1200 generates an EnFace image based on the set depth range. That is, the module functioning as the setting unit in the generation unit 1200 sets, as the selection range, each part of each pixel value sequence in the depth direction as the depth range Zi.

As described above, each A-scan image Ai has a pixel value sequence composed of 1176 pixel values. The generation unit 1200 acquires only pixel values corresponding to the depth range Zi based on the depth range Zi set for each A-scan image Ai, and generates a pixel value sequence in the depth range Zi. Hereinafter, for example, it is assumed that the pixel value sequence in the depth range includes 192 pixel values.

The generation unit 1200 ranks the pixel values in the pixel value sequence of the depth range set for each A scan based on the magnitude relationship of the pixel values, and rearranges the pixel values (S3020). Then, the generation unit 1200 selects the pixel Rx having a predetermined rank. Here, the pixel of a predetermined rank is a pixel located at the x-th position from the beginning after rearranging in the descending order of luminance information.

When the EnFace image is generated in the second mode, the pixel value of the predetermined rank is selected from all 192 pixel values included in the set depth range. When setting the depth range in the A-scan image, the ratio of the region of interest (for example, the retina that is the measurement target) to the depth range is almost 100%. This is because when creating an EnFace image, the depth range is set so that only the region of interest is included. Therefore, when acquiring the average luminance of the region of interest, x is inappropriate at the same upper 10% position as the projection image, and is preferably about half the total number of pixels. Specifically, in the case of using the pixel value sequence having the total number of pixels of 192, the 81st pixel from the top corresponding to the position of the upper 50% is selected as the pixel Rx of the predetermined rank (S3030).

The generation unit 1200 selects the pixel Rx corresponding to the position of the upper 50% from the pixel value sequence in the depth range Zi for all the A-scan images, and generates the EnFace image using those pixel values (S3040). ).

In the above description, when generating the EnFace image, the pixels corresponding to the top 50% positions in the pixel value sequence within the depth range are selected, but this is not limited to the pixels at the top 50% positions. The predetermined order can be set according to the brightness distribution (histogram) of the pixel value sequence, or the predetermined order can be changed according to the ratio of the region of interest in the depth range. For example, when setting the predetermined rank according to the luminance distribution (histogram) of the pixel value sequence, the upper percentage of the pixels to be selected is changed according to the ratio of the retina included in the depth range. You can also With such a configuration, since an appropriate pixel is selected according to the proportion of the retina included in the image, a good Enface image can be generated regardless of the difference in the retinal thickness of each patient. For example, the layer recognition unit 1300 extracts ILM and RPE from the tomographic image, and recognizes the area sandwiched between ILM and RPE as the area of the retina. That is, the layer recognition unit 1300 acquires the coordinates of the retina area in the tomographic image. On the other hand, the generation unit 1200 acquires the coordinates of the depth range Zi designated by the user. Then, the generation unit 1200 can calculate the proportion of the retina in the depth range Zi by acquiring the coordinates of the retina region in the tomographic image from the layer recognition unit 1300 and comparing with the coordinates of the depth range Zi. That is, the generation unit 1200 can calculate the ratio occupied by the attention area included in the pixel value sequence. Based on this ratio, the generation unit 1200 can automatically change what percentage of higher-order pixels the pixel is selected. The generation unit 1200 selects the pixel at a higher position as the proportion of the retina included in the depth range Zi is smaller. In other words, the generation unit 1200 selects a pixel at a lower position as the proportion of the retina included in the depth range Zi increases. That is, the generation unit 1200, which is an example of the reference changing unit, changes the order of the pixel values used for generating the two-dimensional image in the second mode according to the ratio of the attention area included in the pixel value sequence.
Note that the generation unit 1200 may calculate the proportion of the retina occupied for each A-scan image and change what percentage of higher-order pixels are selected for each A-scan image. It is also possible to change what percentage of higher-order pixels are to be selected based on the average value of the proportion occupied by. Also, the generation unit 1200 may change the uppermost percentage of pixels to be selected according to the proportion of the retina included in the A-scan image even in the first mode.

The change of the predetermined selection criterion according to the set selection range described as the predetermined order in the present embodiment is performed according to the eye characteristics of each patient exemplified by the proportion of the retina as described herein. Is preferably carried out. Further, it may be arbitrarily selected from pre-stored individual patient data and performed based on conditions. In this case, the processing is executed by the module functioning as the reference changing unit in the image processing apparatus 100. Further, this reference change may be executed by determining whether or not the value of the proportion occupied by the retina exceeds a threshold value, for example, or may be executed in response to an input or instruction from the examiner.

This EnFace image is different from the surface image of the fundus obtained by the CCD 172, and various types of images can be obtained by setting the depth range. Since the two-dimensional image is generated only with the information within the specific depth range, the structure of the measured object within the depth range and the change in the structure can be remarkably visualized.

Next, in the imaging apparatus 1 according to the present embodiment, the generated fundus surface image, tomographic image, projection image, and EnFace image are displayed on the monitor 928. As shown in FIG. 10, the surface image S of the fundus oculi, the tomographic image Ti, and the projection image P are displayed side by side on the monitor 928. Further, the acquisition position Li of the tomographic image Ti is displayed on the projection image P and the surface image S in an overlapping manner. The projection image P can also be switched to the EnFace image E and displayed by a switching unit (not shown). Alternatively, as shown in FIG. 11, the tomographic image Ti, the projection image P, and the EnFace image E can be displayed side by side on the monitor 928.

Note that the image processing apparatus 100 according to this embodiment generates 128 tomographic images. However, on the monitor 928, a tomographic image Ti (i=0 to 128) as one selected cross section or a cross-sectional image Ti of a tomographic image reconstructed in three dimensions (in this case, an arbitrary number i Is shaken) is displayed. The examiner can operate the input units 929-1 and 929-2 so that the tomographic images to be displayed can be switched and displayed. Here, when the tomographic image Ti is switched, the display position of the acquisition position Li of the tomographic image Ti displayed on the projection image P, the EnFace image E, and the surface image S is also updated. Thereby, the examiner can easily know which position on the fundus 127 of the eye to be inspected the displayed tomographic image Ti is.

When the acquisition position Li is placed on the EnFace image E, it is also possible to observe the structural change of the measured object in a specific depth range and the tomographic image Ti while comparing them.

In the present embodiment, the projection image and the EnFace image of the retina are generated based on the tomographic image of the fundus of the eye to be inspected. It may be generated.

In the present embodiment, the generation unit 1200 that generates the projection image and the EnFace image described above generates a two-dimensional image based on the pixel value selected from each of the plurality of pixel value columns according to a predetermined selection criterion. In this case, the predetermined selection criterion is exemplified by the criterion of the above-mentioned percentage.

Further, this predetermined selection criterion is changed according to the number of pixels used as a pixel value sequence in the A scan image used when selecting pixel values. In this case, in each of the plurality of pixel value columns, how many or what condition of the pixel value is included in the depth direction functions as a setting unit that sets the selection range of the selected pixel value. It is executed by a module in the image processing apparatus 100. Further, the change of the predetermined selection criterion according to the set selection range is executed by the module functioning as the criterion changing unit in the image processing apparatus 100.

As described above, according to the present invention, it is possible to generate a two-dimensional image based on all information in the depth direction and to generate a two-dimensional image based on information within a predetermined range in the depth direction. It becomes possible to obtain a good two-dimensional image.

[Other Embodiments]
The present invention is not limited to the above-described embodiment, and various modifications and changes can be made without departing from the spirit of the present invention. For example, in the above embodiment, the case where the object to be measured is the eye, particularly the fundus of the eye is described, but the present invention can also be applied to the object to be measured such as skin and organs other than the eye. In this case, the present invention has an aspect as a medical device such as an endoscope other than the ophthalmologic apparatus. Therefore, it is desirable that the present invention be understood as an image processing apparatus for an inspection apparatus exemplified by an ophthalmologic apparatus, and that the eye to be inspected be understood as one mode of the object to be inspected.

Further, the present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus execute the program. It can also be realized by a process of reading and executing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

The present invention relates to an image processing apparatus for obtaining a two-dimensional image from information on a tomographic image obtained by optical interference and an operating method of the image processing apparatus .

In view of the above problems, it is another object to provide a method of operating an image processing apparatus and an image processing apparatus which makes it possible to obtain a good good two-dimensional images.

In order to achieve the above object, an image processing device according to an aspect of the present invention is
Display control means for superimposing a line relating to a layer boundary on a tomographic image of the eye to be inspected and displaying the display means,
In the displayed tomographic image, the position of the line with respect to the layer boundary, changing means for changing in accordance with an instruction from the examiner,
Generating means for generating a two-dimensional image that is a front image of the eye to be inspected corresponding to the depth range of the eye to be inspected selected using information about the position of the line related to the layer boundary,
Equipped with .

According to one aspect of the present invention, it is possible to obtain a good good two-dimensional images.

Claims (12)

Acquisition means for acquiring a pixel value sequence arranged in the depth direction of the measured object based on the interference light obtained by causing the return light of the scanned measurement light from the measured object and the reference light corresponding to the measured light to interfere with each other. When,
Generating means for generating a two-dimensional image based on pixel values selected from each of the plurality of pixel value columns according to a predetermined selection criterion;
Setting means for setting a selection range of the selected pixel value in the depth direction in each of the plurality of pixel value columns;
An image processing apparatus comprising: a reference changing unit that changes the predetermined selection criterion according to the set selection range. The generation unit is the two-dimensional image using the pixel value selected in the order of magnitude of the pixel value as the predetermined selection criterion, the selection range is all of the pixel value sequence in the depth direction. A first mode that produces
Pixels selected as a part of each pixel value sequence in the depth direction as the selection range are set in the setting means, and selected as the predetermined selection criterion based on the order of the pixel value size in the selection range. The image processing apparatus according to claim 1, further comprising: a second mode for generating a two-dimensional image using the value. The generation means ranks the pixel values based on the magnitude relationship of the pixel values for each pixel value column,
Generating the two-dimensional image based on pixel values of a predetermined rank in the ranked pixel value sequence,
The image processing apparatus according to claim 2, wherein the predetermined order is different between the first mode and the second mode. The image processing apparatus according to claim 3, wherein the reference changing unit changes the predetermined order according to a ratio of the attention area included in the pixel value sequence in the second mode. .. The image processing apparatus according to claim 3, wherein the predetermined rank in the first mode is higher than the predetermined rank in the second mode. The image processing apparatus according to claim 1, further comprising a display unit that displays the two-dimensional image generated by the generation unit. When the setting means sets the selection range of the pixel value in the depth direction, the display means has a display control means for displaying a display form for causing the setting means to set the selection range. The image processing apparatus according to claim 6. The display unit can switch and display a two-dimensional image in the first mode and a two-dimensional image in the second mode generated by the generating unit. The image processing device described. 8. The display unit according to claim 6, wherein the display unit can display the two-dimensional image in the first mode and the two-dimensional image in the second mode generated by the generating unit side by side. Image processing device. The image processing apparatus according to claim 1, wherein the measured object is an eye. An acquisition step of acquiring a pixel value sequence arranged in the depth direction of the measured object based on the interference light obtained by causing the return light of the scanned measurement light from the measured object and the reference light corresponding to the measured light to interfere with each other. When,
A generating step of generating a two-dimensional image based on pixel values selected according to a predetermined selection criterion from each of the plurality of pixel value columns;
In each of the plurality of pixel value columns, a setting step of setting a selection range of the selected pixel value in the depth direction,
And a reference changing step of changing the predetermined selection criterion according to the set selection range. A program that causes a computer to execute each step of the image processing method according to claim 11.