JP6526145B2 - Image processing system, processing method and program - Google Patents

Description

  The present invention relates to an image processing system, a processing method, and a program.

  There is known a tomographic imaging apparatus using optical coherence tomography (OCT) using interference by low coherent light. By imaging the fundus by such a tomographic imaging apparatus, it is possible to three-dimensionally observe the state inside the retinal layer.

  Imaging by a tomographic imaging apparatus has been attracting attention in recent years because it is a useful technique for more accurately diagnosing diseases. As a form of such OCT, for example, TD-OCT (Time domain OCT) in which a broadband light source and a Michelson interferometer are combined is known. In TD-OCT, interference light with the backscattered light of the signal arm can be measured by scanning the delay of the reference arm, and information on depth resolution can be obtained.

  However, in TD-OCT, it is difficult to obtain an image at high speed. Therefore, SD-OCT (Spectral domain OCT) which acquires an interferogram with a spectrometer using a broadband light source as a method of acquiring an image faster is known. Moreover, SS-OCT (Swept Source OCT) which measures spectrum interference with a single channel photodetector is also known by using a high-speed wavelength swept light source as a light source.

  Here, if it is possible to measure the shape change of the retina in the tomographic images captured by these OCTs, it is possible to quantitatively diagnose the degree of progression of a disease such as glaucoma and the degree of recovery after treatment. In relation to such a technique, Patent Document 1 detects the boundary of each layer of the retina from a tomographic image using a computer in order to quantitatively measure the morphological change of the retina, and based on the detection result, a layer is formed. A technique for measuring the thickness of

JP, 2008-073099, A

  In the technique of Patent Document 1 described above, a tomographic image in a range designated on a two-dimensional image is acquired, a layer boundary is detected, and a layer thickness is determined. However, three-dimensional shape analysis of the retinal layer is not performed, and the result of shape analysis is not effectively displayed.

  The present invention has been made in view of the above problems, and is a technology that can display the shape feature of an eye to be diagnosed together with an index that is a criterion for determining whether the eye to be examined is a disease. Intended to be provided.

In order to solve the above problems, an image processing system according to an aspect of the present invention processes the tomographic image acquired by the tomographic imaging apparatus for acquiring a tomographic image of the fundus, and forms the retinal layer of the fundus Acquisition means for acquiring shape feature quantities ;
The shape feature quantity acquired by the acquisition means is referred to as the shape feature quantity of the retinal layer with respect to the plurality of eyes with reference to a database in which the shape feature quantity of the retinal layer with respect to the plurality of eyes classified by eye axis length is stored. to characterized in that it comprises a control means for displaying in a comparable state.

  According to the present invention, it is possible to display the shape feature quantity of the subject eye to be diagnosed, as well as the index serving as the determination criterion of whether the subject eye is a disease.

FIG. 1 is a diagram showing an example of the configuration of an image processing system 10 according to an embodiment of the present invention. FIG. 6 is a view showing an example of a tomographic image photographing screen 60. 5 is a flowchart showing an example of the flow of processing of the image processing apparatus 30 shown in FIG. 1; The figure for demonstrating the outline | summary of a three-dimensional shape analysis process. The figure for demonstrating the outline | summary of a three-dimensional shape analysis process. FIG. 8 is a view showing an example of a tomographic image observation screen 80. FIG. 7 is a view showing an example of each part of a tomographic image observation screen 80. FIG. 7 is a view showing an example of each part of a tomographic image observation screen 80. FIG. 7 is a view showing an example of each part of a tomographic image observation screen 80. FIG. 8 is a view showing an example of a tomographic image observation screen 80. FIG. 7 is a view showing an example of each part of a tomographic image observation screen 80. FIG. 8 is a view showing an example of a tomographic image observation screen 80. FIG. 6 is a view showing an example of a tomographic image observation screen 400. FIG. 6 is a view showing an example of each part of a tomographic image observation screen 400. FIG. 6 is a view showing an example of a tomographic image observation screen 400. FIG. 6 is a view showing an example of each part of a tomographic image observation screen 400. FIG. 7 is a diagram showing an example of the configuration of an image processing system 10 according to a third embodiment. 12 is a flowchart showing an example of the flow of processing of the image processing apparatus 30 according to the third embodiment. The figure which shows the outline | summary of the acquisition method of the feature-value which shows the shape feature of a retinal layer. The figure which shows the outline | summary of the acquisition method of the feature-value which shows the shape feature of a retinal layer. FIG. 8 is a view showing an example of a tomographic image observation screen 80. FIG. 14 is a diagram showing an example of the configuration of an image processing system 10 according to a fourth embodiment. FIG. 6 is a view showing an example of a tomographic image observation screen 900.

  Hereinafter, embodiments according to the present invention will be described in detail with reference to the attached drawings.

(Embodiment 1)
FIG. 1 is a diagram showing an example of the configuration of an image processing system 10 according to an embodiment of the present invention.

  The image processing system 10 includes an image processing device 30, a tomographic imaging device 20, a fundus imaging device 51, an external storage device 52, a display device 53, and an input device 54.

  The tomographic imaging apparatus 20 is realized by, for example, SD-OCT, SS-OCT, etc., and uses optical coherence tomography (OCT) using interference by low coherent light to display a tomographic image showing a three-dimensional shape of the fundus. Take a picture. The tomographic imaging apparatus 20 includes a galvano mirror 21, a drive control unit 22, a parameter setting unit 23, an internal fixation lamp 24, and a coherence gate stage 25.

  The galvano mirror 21 has a function of two-dimensionally scanning the measurement light (irradiation light) on the fundus, and defines the imaging range of the fundus by the tomographic imaging apparatus 20. The galvano mirror 21 is composed of, for example, two mirrors of a mirror for X scanning and a mirror for Y scanning, and scans measurement light on a plane orthogonal to the optical axis with respect to the fundus of the eye to be examined. .

  The drive control unit 22 controls the drive (scanning) range and speed of the galvano mirror 21. Thereby, the imaging range and the number of scanning lines (scanning speed in the plane direction) in the plane direction (direction orthogonal to the optical axis direction of the measurement light) in the fundus are defined.

  The parameter setting unit 23 sets various parameters used for drive control of the galvano mirror 21 by the drive control unit 22. Based on this parameter, imaging conditions for tomographic images by the tomographic imaging apparatus 20 are determined. For example, the scanning position of the scanning line, the number of scanning lines, the number of shots, etc. are determined. In addition, the position of the internal fixation lamp, the scan range (scan range), the scan pattern (scan pattern), the setting of the coherence gate position, and the like are also performed. The setting of the parameters is performed based on an instruction from the image processing apparatus 30.

  The internal fixation lamp 24 places a bright spot in the field of view so that eye movement does not occur during imaging of a tomographic image, thereby suppressing movement of the viewpoint. The internal fixation lamp 24 is configured to include a display unit 24a and a lens 24b. The display unit 24 a is realized, for example, by arranging a plurality of light emitting diodes (LDs) in a matrix. The lighting position of the light emitting diode is changed by the control of the drive control unit 22 in accordance with the part to be imaged. The light from the display unit 24a is guided to the subject's eye through the lens 24b. The light emitted from the display unit 24a is, for example, 520 nm, and a desired pattern is displayed (lighted) by the control of the drive control unit 22.

  The coherence gate stage 25 is provided to cope with the difference in axial length of the eye to be examined. More specifically, the optical path length of the reference light (for interfering with the measurement light) is controlled, and the imaging position along the depth direction (optical axis direction) of the fundus is adjusted. Thereby, the optical path lengths of the reference light and the measurement light can be made to coincide with each other for the subject's eyes having different eye axial lengths. The coherence gate stage 25 is controlled by the drive control unit 22.

  Here, the coherence gate indicates a position at which the optical distance between the measurement light and the reference light in the tomographic imaging apparatus 20 is equal. By controlling the position of the coherence gate, it is possible to switch between photographing on the retinal layer side or photographing by the EDI (Enhanced Depth Imaging) method in which the retinal layer is located deeper than the retinal layer. When imaging is performed by the EDI method, the position of the coherence gate is set to the deep side of the retinal layer, so when the retinal layer is imaged beyond the upper side of the tomographic image, the retinal layer is in the tomographic image It will not be reflected back.

  The fundus oculi imaging device 51 is realized by, for example, a fundus camera, an SLO (Scanning Laser Othlasmoscope) or the like, and captures a (two-dimensional) fundus oculi image of the fundus oculi.

  The external storage device 52 is realized by, for example, an HDD (Hard Disk Drive) or the like, and stores various data. The external storage device 52 associates and holds, for example, captured image data, imaging parameters, image analysis parameters, and parameters set by the operator with information on the eye to be examined (patient's name, age, gender, etc.) Do.

  The input device 54 is realized by, for example, a mouse, a keyboard, a touch operation screen, etc., and inputs various instructions from the operator. For example, the operator performs various instructions, settings, and the like on the image processing device 30, the tomographic imaging device 20, and the fundus imaging device 51 via the input device 54. The display device 53 is realized by, for example, a liquid crystal display or the like, and displays (presents) various information to the operator.

  The image processing apparatus 30 is realized by, for example, a personal computer or the like, and processes various images. That is, the image processing apparatus 30 incorporates a computer. The computer is equipped with main control means such as a central processing unit (CPU) and storage means such as a read only memory (ROM) and a random access memory (RAM).

  Here, the image processing apparatus 30 includes an image acquisition unit 31, a storage unit 32, an image processing unit 33, an instruction unit 34, and a display control unit 35 as its functional configuration. The configuration other than the storage unit 32 is realized, for example, by the CPU reading and executing a program stored in the ROM or the like.

  The image processing unit 33 includes a detection unit 41, an analysis unit 42, a determination unit 43, and a registration unit 47.

  The image acquisition unit 31 acquires the tomographic image captured by the tomographic imaging device 20 and the fundus image captured by the fundus imaging device 51 and stores the acquired image in the storage unit 32. The storage unit 32 is realized by, for example, a ROM, a RAM, and the like.

  The detection unit 41 detects a layer of retina (retinal layer) from the tomographic image stored in the storage unit 32.

  The analysis unit 42 analyzes the retinal layer to be analyzed. The analysis unit 42 is provided with a determination unit 43, a retinal layer analysis unit 44, an analysis result generation unit 45, and a shape data generation unit 46.

  The determination unit 43 determines whether or not to perform three-dimensional shape analysis processing of the retinal layer according to the imaging mode (an imaging mode for myopia analysis and an imaging mode for non-myopia analysis). Note that three-dimensional shape analysis refers to creating three-dimensional shape data and performing shape analysis of the retinal layer using the shape data.

  The retinal layer analysis unit 44 performs analysis processing on the retinal layer to be analyzed based on the determination result of the determination unit 43. In the present embodiment, the case of analyzing the macula of the myopic eye is described as the three-dimensional shape analysis process. The analysis result creation unit 45 creates various data for presenting the analysis result (information indicating the state of the retinal layer). The shape data creation unit 46 aligns a plurality of tomographic images obtained by imaging and creates three-dimensional shape data. That is, three-dimensional shape data is created based on layer information of the retinal layer.

  The alignment unit 47 aligns the analysis result with the fundus image, aligns the fundus images, and the like. The instruction unit 34 instructs information such as imaging parameters according to the imaging mode set by the tomographic imaging apparatus 20.

  The above is the description of an example of the configuration of the image processing system 10. The functional configuration provided in each device described above does not have to be necessarily realized as illustrated, and all or a part of the functional configuration may be realized in any device in the system. For example, although the external storage device 52, the display device 53, and the input device 54 are provided outside the image processing device 30 in FIG. 1, these configurations may be provided inside the image processing device 30. . Further, for example, the image processing apparatus and the tomographic imaging apparatus 20 may be integrally configured.

  Next, an example of the tomographic image capturing screen 60 displayed on the display device 53 shown in FIG. 1 will be described using FIG. 2. This screen is displayed at the time of imaging a tomographic image.

  The tomographic image capturing screen 60 is configured to include a display region 61 for tomographic images, a display region 62 for fundus images, a combo box 63 for setting an imaging mode, and a capture button 64 for instructing imaging. Reference numeral 65 in the display area 62 of the fundus oculi image is a mark indicating the imaging area, and is displayed superimposed on the fundus oculi image. Also, M indicates a macular region, D indicates an optic disc, and V indicates a blood vessel.

  In the combo box 63, for example, setting of a photographing mode for myopic eye analysis (for the macula) and a photographing mode for non-myopic eye analysis (for the macula) is performed. In this case, the imaging mode for myopic eye analysis is set.

  Here, a tomographic image of the fundus is displayed in the display region 61 of the tomographic image. L1 indicates the inner limiting membrane (ILM), L2 indicates the boundary between the nerve fiber layer (NFL) and the ganglion cell layer (GCL), and L3 indicates the photoreceptor inner nodal segment junction (ISOS) . L4 indicates the retinal pigment epithelial layer (RPE), and L5 indicates Bruch's membrane (BM). The detection unit 41 described above detects any one of the boundaries L1 to L5.

  Next, an example of the process flow of the image processing apparatus 30 shown in FIG. 1 will be described using FIGS. 3A and 3B. First, with reference to FIG. 3A, the overall flow of processing at the time of imaging a tomographic image will be described.

[S101]
First, the image processing device 30 acquires a subject identification number from the outside as information for identifying the subject's eye. Then, based on the subject identification number, information on the subject's eye held by the external storage device 52 is acquired and stored in the storage unit 32.

[S102]
The image processing unit 30 causes the image acquisition unit 31 to acquire a fundus oculi image from the fundus oculi imaging device 51 as a prescan image for imaging position confirmation at the time of imaging, and acquires a tomographic image from the tomographic imaging device 20.

[S103]
The image processing device 30 sets the photographing mode. The setting of the imaging mode is performed based on the selection of the operator from the combo box 63 for imaging mode setting described with reference to FIG. Here, the case of performing imaging in the imaging mode for myopic eye analysis will be described.

[S104]
The image processing device 30 instructs the tomographic imaging device 20 to capture imaging parameters corresponding to the imaging mode set from the combo box 63 in the instruction unit 34. Thereby, the tomographic imaging apparatus 20 sets imaging parameters in the parameter setting unit 23 according to the instruction. Specifically, setting of at least one of the position of the internal fixation lamp, the scan range (scan range), the scan pattern (scan pattern), and the coherence gate position is instructed.

  Here, parameter setting in the imaging mode for myopic eye analysis will be described. In the parameter setting of the position of the internal fixation lamp, the position of the internal fixation lamp 24 is set so that the macular center can be photographed. The image processing device 30 controls the light emitting diode of the display unit 24 a in the drive control unit 22 in accordance with the imaging parameter. In the case of an apparatus configured to obtain a sufficiently wide imaging range, control may be performed so that the centers of the macular region and the optic papilla become the centers of imaging. Such control is performed in order to image the area where the macular region appears in order to analyze the shape of the retinal layer in the myopic eye.

  Further, in the parameter setting of the scanning range, for example, a range of 9 to 15 mm is set as the limit value of the imaging range of the apparatus. This numerical value is merely an example, and may be appropriately changed in accordance with the specifications of the apparatus. The imaging range is preferably a wider area in order to be able to detect the portion where the shape change is occurring without omission.

  In the parameter setting of the scan pattern, for example, raster scan or radial scan is set so that the three-dimensional shape of the retinal layer can be photographed.

  In parameter setting of the position of the coherence gate, the gate position is set so that imaging can be performed by the EDI method. In highly myopic eyes, the degree of curvature of the retinal layer becomes tight, and the retinal layer is photographed beyond the upper side of the tomographic image. In that case, since the retinal layer beyond the top of the tomographic image is folded back and reflected in the tomographic image, setting of such parameters is necessary to prevent it. In addition, when using SS-OCT with high depth as the tomographic imaging apparatus 20, good tomographic images can be obtained even if the position of the retinal layer is moved away from the gate position, so imaging is not necessarily performed by the EDI method. good.

[S105]
The image processing device 30 instructs the tomographic imaging device 20 to capture an eye to be examined in the instruction unit 34. This instruction is issued, for example, when the operator presses the capture button 64 of the tomographic image photographing screen 60 via the input device 54. When this instruction is issued, the tomographic imaging apparatus 20 controls the drive control unit 22 based on the imaging parameters set by the parameter setting unit 23. Thereby, the galvano mirror 21 operates and imaging of a tomographic image is performed.

  As described above, the galvano mirror 21 is configured by the X scanner for the horizontal direction and the Y scanner for the vertical direction. Therefore, tomographic images can be taken along the horizontal direction (X) and the vertical direction (Y) in the device coordinate system by respectively changing the orientations of these scanners. By simultaneously changing the orientations of these scanners, scanning can be performed in a direction in which the horizontal direction and the vertical direction are combined, so that imaging can be performed along any direction on the fundus plane. At this time, the image processing device 30 causes the display control unit 35 to display the captured tomographic image on the display device 53. Thereby, the operator can confirm the imaging result.

[S106]
The image processing device 30 causes the image processing unit 33 to detect and analyze the retinal layer from the tomographic image stored in the storage unit 32. That is, detection / analysis processing of the retinal layer is performed on the tomographic image captured in the processing of S105.

[S107]
The image processing device 30 determines whether to end imaging of a tomographic image. This determination is made based on an instruction from the operator via the input device 54. That is, it is determined whether the imaging of the tomographic image is ended based on whether or not the operator has instructed the end.

  When the end of imaging is instructed, the image processing apparatus 30 ends this process. On the other hand, when the photographing is continued without ending the processing, the image processing device 30 performs the processing of S102 and thereafter again.

  When the operator manually corrects the detection result of the retinal layer and the position of the fundus image and the map by the processing of S206 and S209 described later, the image processing apparatus 30 receives the correction when the imaging is finished. The imaging parameters changed accordingly are stored in the external storage device 52. At this time, a confirmation dialog as to whether to save the changed parameter may be displayed, and the operator may be inquired as to whether or not to change the imaging parameter.

  Next, the detection / analysis process of the retinal layer shown in S106 of FIG. 3A will be described using FIG. 3B.

[S201]
When detection / analysis processing of the retinal layer starts, the image processing device 30 detects the retinal layer from the tomographic image in the detection unit 41. This process will be specifically described using the tomographic image (display area 61) shown in FIG. In the case of the macular region, the detection unit 41 first applies a median filter and a Sobel filter to the tomographic image to create an image (hereinafter referred to as a median image and a Sobel image). Subsequently, the detection unit 41 creates a profile for each A scan from the created median image and the Sobel image. For median images it is a profile of luminance values and for Sobel images it is a profile of gradients. Then, the detection unit 41 detects a peak in the profile created from the Sobel image. Finally, the boundary of each region of the retinal layer is detected with reference to the profile of the median image corresponding to the front and rear of the detected peak and the peak between the peaks. That is, detection of L1 (ILM), L2 (boundary between NFL and GCL), L3 (ISOS), L4 (RPE), L5 (BM), etc. is performed. In the present embodiment, the layer to be analyzed is described as RPE.

[S202]
The image processing device 30 determines in the determination unit 43 whether or not to analyze the three-dimensional shape of the retinal layer. More specifically, when imaging is performed in the imaging mode for myopic eye analysis, it is determined that three-dimensional shape analysis of the retinal layer is to be performed. In addition, when imaging is performed without using the myopic eye analysis mode (imaging mode for non-myopic eye analysis), it is determined that the three-dimensional shape analysis is not performed. In this case, analysis is performed based on the detection result of the detection unit 41 (analysis that does not use three-dimensional shape data). Even in the imaging mode for myopic eye analysis, it is determined based on the tomographic image whether or not the macular region is shown in the tomographic image, and if it is not (for example, only the optic papilla is shown) It may be determined that the (three-dimensional) shape analysis of the retinal layer is not performed.

  Here, in the present embodiment, the image processing system 10 in which the tomographic imaging apparatus 20 and the image processing apparatus 30 are integrated is described as performing a series of processes from imaging to analysis. It is not limited. That is, the image processing device 30 only needs to perform shape analysis of the retinal layer when a tomographic image in which the macular region is captured is input using a scanning pattern for acquiring a three-dimensional shape, and this is not necessarily required. It does not have to be such an integrated system. Therefore, the shape analysis of the retinal layer can be performed on tomographic images captured by other than the tomographic imaging device 20 based on the information at the time of imaging. However, when shape analysis is not required, the shape analysis processing may not be performed.

[S203]
The image processing device 30 creates three-dimensional shape data in the shape data creation unit 46. Three-dimensional shape data is created to perform shape analysis based on the detection result of the retinal layer in the process of S201.

  Here, for example, if the scanning pattern at the time of imaging is raster scan, alignment between a plurality of adjacent tomographic images is performed. In alignment of tomographic images, for example, an evaluation function representing the degree of similarity between two tomographic images is defined in advance, and the tomographic image is deformed so that the value of this evaluation function becomes the best.

  As an evaluation function, for example, a method of evaluating by pixel value (for example, a method of evaluating using a correlation coefficient) can be mentioned. Further, examples of image deformation processing include processing that performs translation and rotation using affine transformation. The shape data creation unit 46 creates three-dimensional shape data of the layer to be subjected to the shape analysis after the alignment processing of the plurality of tomographic images is completed. The three-dimensional shape data may be created, for example, by preparing voxel data of 512 × 512 × 500 and assigning a label to a position corresponding to the coordinate value of layer data in which the retinal layer is detected.

  Also, if the scanning pattern at the time of imaging is a radial scan, in this case as well, three-dimensional shape data is created after the shape data creating unit 46 performs alignment between tomographic images. However, in this case, alignment in the depth direction (Z direction in the tomographic image (display region 61) in FIG. 2) is performed using only region information in the vicinity of the center of the adjacent tomographic image. This is because, in the case of a radial scan, even if it is an adjacent tomographic image, information is sparse at both ends compared with the vicinity of the center of the tomographic image, and the change in shape is large. It is for good. As a method of alignment, the method described above may be used. The shape data generation unit 46 generates three-dimensional shape data of a layer to be subjected to shape analysis after the alignment processing is completed.

  In the case of the radial scan, for example, voxel data of 512 × 512 × 500 is prepared, and first, layer data to be a target of shape analysis of each tomographic image is expanded by rotating it into a uniform circle. Thereafter, in the expanded layer data, interpolation processing is performed between shape data adjacent in the circumferential direction. By interpolation processing, shape data of a portion not taken is created. As a method of interpolation processing, processing such as linear interpolation and non-linear interpolation may be applied. Three-dimensional shape data may be created by assigning labels to locations corresponding to coordinate values obtained by interpolating between layer data in which a retinal layer is detected.

  The numerical values of the voxel data shown here are merely examples, and may be appropriately changed according to the number of A-scans at the time of imaging and the memory status of the apparatus to be executed. In the case of large voxel data, since the resolution is high, the shape data can be accurately represented, but the execution speed is slow and a large amount of memory is also consumed. On the other hand, in the case of small voxel data, the resolution is lower, but the execution speed is faster and the memory consumption is reduced.

[S204]
The image processing apparatus 30 causes the retinal layer analysis unit 44 to analyze the three-dimensional shape of the retinal layer. Here, a method of measuring the area and volume of the retinal layer will be described as an example of a method of shape analysis.

  This process will be described with reference to FIG. In FIG. 4, three-dimensional shape data (RPE), a measurement surface (MS), an area (Area), and a volume (Volume) are schematically shown.

  First, area measurement will be described. In the three-dimensional shape data of the RPE created in the process of S203, a flat (flat) measurement surface (MS) is prepared at the place of the layer data located deepest in the Z direction (optical axis direction). Then, the measurement surface (MS) is moved in the shallow direction (the direction of the origin of the Z axis) at regular intervals from that. When the measurement surface (MS) is moved in the direction shallow from the deep part of the layer, the measurement surface (MS) crosses the boundary of the RPE.

  The area (Area) is obtained by measuring an internal planar area surrounded by the boundary between the measurement surface (MS) and the RPE. More specifically, the area (Area) is obtained by measuring the area of the intersection region of the boundary between the measurement surface (MS) and the RPE. As described above, the area (Area) is a cross-sectional area of three-dimensional retinal layer shape data. When the cross-sectional area of the position of the reference region is measured, the area decreases if the curvature of the retina is tight, and the area increases if the curvature of the retina is gentle.

  In addition, Volume (Volume) measures the entire internal area surrounded by the boundary between the measurement surface (MS) and the RPE using the measurement surface (MS) used when measuring the area (Area) Obtained by When measuring the lower volume along the Z direction from the position of the reference part, the volume increases if the curvature of the retina is tight, and the volume decreases if the curvature of the retina is gentle. Here, the reference position may be, for example, the position of the Bruch's membrane opening with reference to the part. Alternatively, it may be based on a constant height of 100 μm, 500 μm, etc. from the position of the deepest part of RPE. In addition, when measuring an area or volume, when counting the number of voxels contained inside, it calculates by multiplying the physical size per voxel.

[S205]
After the shape analysis, the image processing apparatus 30 causes the analysis result generation unit 45 to generate an analysis result (for example, a map, a graph, numerical information) based on the three-dimensional shape analysis result.

  Here, the case of creating a contour map as a result of three-dimensional shape analysis will be described. The contour map is used when displaying the result of measuring the area and volume.

  FIG. 5A shows an example of the contour map 71. Here, reference numeral 71 denotes the entire contour map, reference numeral 72 denotes contours drawn at a constant interval, and reference numeral 73 denotes the deepest portion in the Z direction of the three-dimensional retinal layer shape data. ing.

  When creating the contour map 71, a lookup table for the contour map is prepared, and the color is classified according to the volume with reference to the table. Here, a lookup table for contour map may be prepared, for example, according to the volume. Thus, on the map, changes in shape and volume can be grasped at one time.

More specifically, the contour lines 72 are drawn at constant intervals with the reference numeral 73 in FIG. 5A at the bottom, and the color of the map is set according to the volume. Therefore, when the height (depth) of the measurement surface is changed from the position located deepest in the Z direction, it is possible to grasp how the volume increases. Specifically, if 1 mm ³ is set to blue and 2 mm ³ is set to yellow, the height (depth) of the measurement surface is 100 μm from the deepest part in the Z direction and blue The operator can grasp the relationship between the shape and the volume according to whether it is blue or 300 μm. Therefore, the operator can grasp the value of the overall volume when measuring the shape of the retinal layer to be measured by confirming the color of the outermost contour of the map.

  Also, the operator can grasp the value of the volume corresponding to the height (depth) by confirming the color in the vicinity of the internal contour line. The look-up table used when setting the color of the contour map may not be limited to the volume, but may be provided according to the area, and the most from the measurement surface (MS) to the Z direction of the retinal layer data. It may be provided according to the height (depth) to the location 73 located in the deep part. Although not shown, numerical values may be displayed together on each contour line so that the height (depth) of the contour line can be known. As an interval of the distance which expresses a contour line, it is 100 micrometers along a Z direction, for example. The contour map may be color or gray scale, but color has better visibility.

  In the contour map, the size of the outer shape changes in accordance with the height (depth) from the deepest portion in the Z direction to the measurement surface (MS). FIG. 5B shows an example of the contour map 74 when the height (depth) of the measurement surface (MS) is changed.

  Next, the case where the curvature of the retinal layer is measured as three-dimensional shape analysis will be described using a tomographic image (display area 61) shown in FIG. Here, the case where the horizontal axis is x-coordinate and the vertical axis is z-coordinate, and the curvature of the boundary of the layer (RPE) to be analyzed is calculated. The curvature κ can be obtained by calculating “Equation 1” at each point of the boundary. The sign of curvature κ indicates whether it is convex upward or downward, and the magnitude of the numerical value indicates how curved the shape is. Therefore, in the case where the upward convex is + and the downward convex is-, in each tomographic image, when the sign of the curvature is-area, + area, or-area, it has a W shape.

[Equation 1]

  Here, although the case of calculating the curvature at the boundary of the tomographic image has been described, the curvature calculation is not limited to this, and three-dimensional curvature may be calculated from three-dimensional shape data. In this case, after the shape analysis, the image processing device 30 generates a curvature map based on the analysis result in the analysis result generation unit 45.

  An example of a curvature map is shown in FIG.5 (c). In this case, the color of the portion where the curvature is tight is dark and the color of the portion where the curvature is smooth is lightly represented. That is, the color depth is changed according to the curvature. The color to be set in the curvature map may be changed to be positive or negative with reference to the curvature value 0. Thus, the operator can grasp whether the retinal shape is smooth, or convex upward or downward by looking at the map.

[S206]
The image processing apparatus 30 controls the display control unit 35 to generate tomographic images, detection results of the layer (RPE) detected by the detection unit 41, and various shape analysis results (maps, graphs, etc.) generated by the analysis result generation unit 45. , Numerical information) is displayed on the display device 53.

  FIG. 6 is a view showing an example of a tomographic image observation screen 80 displayed on the display device 53 shown in FIG. This screen is displayed after analysis of the tomographic image is completed (ie, displayed by the process of S206).

  The tomographic image observation screen 80 is configured to include a tomographic image display unit 91 including a display region 81 of tomographic images, and a fundus image display unit 94 including a display region 82 of a fundus image. The tomographic image observation screen 80 is also provided with a first analysis result display unit 96 including a first analysis result 84 and a second analysis result display unit 98 including second analysis results 85 and 86.

  Here, first, details of the tomographic image display unit 91 including the display region 81 of the tomographic image will be described. In the display area 81 of the tomographic image, segmentation results (L1 to L5) in which each layer of the retinal layer is detected and a measurement surface (MS) are superimposed and displayed on the captured tomographic image. In the display area 81 of the tomographic image, the segmentation result of the retinal layer to be analyzed (in the present embodiment, RPE (L4)) is emphasized and displayed.

  In the tomographic image display area 81, a hatched area 81a surrounded by the measurement plane (MS) and the retinal layer to be analyzed (RPE (L4)) is an area and volume measurement target area. At this time, in the shaded area 81a, a color according to the volume measurement result is displayed with a predetermined transparency α. The color to be set may be the same as the look-up table for contour map. The transparency α is, for example, 0.5.

  The combo box 92 is provided for the operator to select whether to display in OCT ratio or 1: 1 ratio when displaying a tomographic image. Here, the OCT ratio is a ratio represented by the resolution in the horizontal direction (X direction) obtained from the number of A scans at the time of imaging and the resolution in the vertical direction (Y direction). The 1: 1 ratio is the ratio of the physical size per pixel in the horizontal direction and the physical size per pixel in the vertical direction, which can be obtained by measuring how many mm the range is captured by how many A scans. It is

  The combo box 93 is provided to switch between two-dimensional display (2D) and three-dimensional display (3D) when displaying a tomographic image. Here, in the case of 2D display, one slice of the tomographic image is displayed, and in the case of 3D display, the three-dimensional shape of the retinal layer created from boundary line data of the retinal layer is displayed.

  Specifically, when 3D display is selected in the combo box 93, tomographic images shown in FIG. 7A to FIG. 7C are displayed in the display region 81 of the tomographic image.

  FIG. 7A shows an aspect in which RPE is displayed in 3D at an OCT ratio. In this case, the measurement plane (MS) is simultaneously displayed in 3D.

  Here, below the display area 81 of the tomographic image, check boxes 101 to 104 corresponding to the layers of the retina are displayed. Specifically, check boxes corresponding to ILM, PRE, BM, and MS are displayed, and the operator can instruct switching of display / non-display of each layer by using these check boxes.

  When the measurement surface (MS) is expressed by a surface, the transparency α is a value larger than 0 and smaller than 1. When the retinal layer shape and the measurement surface are overlapped in a state where the transparency is 1, the lid is in a state of being covered, and the three-dimensional shape of the retinal layer can not be seen from the upper side. Alternatively, the measurement surface (MS) may be expressed not as a surface but as a grid. In the case of a lattice, the transparency α of the measurement surface (MS) may be 1. The color of the measurement surface (MS) may be selected according to the measurement value (area or volume) of the position where the measurement surface (MS) is located, by referring to the lookup table of the contour map.

  Here, the operator can move the position of the measurement surface (MS) via the input device 54. Therefore, when the position of the measurement surface (MS) is changed, the image processing apparatus 30 changes the contour map shape as described in FIGS. 5A and 5B in conjunction with the change. Let

  The text box 105 is an item for specifying a numerical value. The operator inputs a numerical value indicating the height (depth) of the measurement surface (MS) from the deepest position in the Z direction into the text box 105 via the input device 54. For example, when a numerical value such as 100 μm or 300 μm is input, the measurement surface (MS) moves to that location, and the contour map also changes accordingly. Thus, the operator can simultaneously grasp the position in the three-dimensional shape and the contour map at that time, and can also grasp the volume value and the area value at that time.

As another example, the value of volume may be input to the text box 105. In this case, when a numerical value such as 1 mm ³ or 2 mm ³ is input, the operator can simultaneously grasp the position of the measurement surface (MS) in the three-dimensional shape corresponding to the volume and the contour map at that time. . Furthermore, the operator can also know the height (depth) of the measurement surface (MS) from the position located deepest in the Z direction at that time.

  Then, FIG.7 (b) has shown the display mode at the time of making a measurement surface (MS) non-display. The other display items are the same as in FIG. 7 (a). In the case of FIG. 7B, the check box 102 is selected, and only the RPE is displayed. Thus, only the three-dimensional shape of the fundus can be displayed.

  Here, in FIG. 7 (b), 3D display of RPE is performed at an OCT ratio, while FIG. 7 (c) is an embodiment in which 3D display of RPE is performed at a 1: 1 ratio. It shows. That is, in the combo box 92, a 1: 1 ratio is selected.

  In the present embodiment, since RPE is described as an analysis target, when 2D / 3D display is switched by the combo box 93, the RPE shape is displayed in 2D / 3D, but the present invention is not limited to this. For example, when the analysis target is a Bruch's membrane (BM), the Bruch's membrane (BM) shape is displayed in 2D / 3D.

  Here, in the display area 81 of the tomographic image, a display may be performed to schematically show which position the measurement surface is measuring. The display mode in this case will be described using FIGS. 8 (a) to 8 (d).

  FIG. 8A shows an aspect in which an object 110 indicating which position the measurement plane is measuring in the tomographic image being displayed is displayed superimposed on the tomographic image. The symbol MS 'represents a measurement surface (a schematic measurement surface) on the object 110. The three-dimensional shape of the retinal layer is rotated up, down, left, and right in accordance with an instruction from the operator via the input device 54. Therefore, an indicator of the positional relationship is presented by the object 110 and the schematic measurement surface MS 'so that the operator can grasp the positional relationship between the measurement surface (MS) and the retinal layer.

  When the operator changes the positional relationship between the object 110 and the schematic measurement surface MS ′ via the input device 54, the display of the three-dimensional shape of the retinal layer is also changed in conjunction with that. In this case, since the region of the retinal layer to be analyzed is also changed, the first analysis result 84 and the second analysis results 85 and 86 are also changed in conjunction.

  Here, some display modes of the object 110 will be described as an example. For example, as shown in FIGS. 8B and 8C, tomographic images corresponding to each cross-sectional position may be displayed. In this case, in the object 110, tomographic images of the vertical position and the horizontal position crossing the center position in the three-dimensional case are displayed. Alternatively, as shown in FIG. 8 (d), English abbreviations such as S and I indicating superior and inferior may be displayed.

  Next, the details of the fundus oculi image display unit 94 including the display area 82 of the fundus oculi image shown in FIG. 6 will be described. In the display area 82 of the fundus oculi image, the imaging position and the scan pattern mark 83 thereof are superimposed on the fundus oculi image. Further, the fundus oculi image display unit 94 is provided with a combo box 95 for switching the display format of the fundus oculi image. In this case, an SLO image is displayed as a fundus image.

  Here, the case where the display format of the fundus oculi image is switched from the combo box 95 will be described with reference to FIGS. 9 (a) and 9 (b). Here, when displaying “SLO image + map” simultaneously and “Displaying fundus image (second fundus image) + SLO image (first fundus image) + map” simultaneously as the display format of the fundus image And will be described as an example. The SLO image (first fundus image) may be a two-dimensional fundus image taken simultaneously with the tomographic image, and is, for example, an integrated image generated by integrating the tomographic image in the depth direction. It may be. Further, the fundus picture (second fundus image) may be a two-dimensional fundus image taken at a timing different from that of the tomographic image, and examples thereof include a contrast test image and the like.

  FIG. 9A shows the display mode when “SLO image + map” is selected from the combo box 95. Reference numeral 201 denotes an SLO image, and reference numeral 200 denotes a map. Alignment between the SLO image 201 and the map 200 is performed by the alignment unit 47 described with reference to FIG. The alignment is performed by setting the position and size of the map based on the position of the fixation lamp at the time of photographing and the scanning range.

  When the result of superposing the map 200 on the SLO image 201 is displayed as a fundus image, the transparency α of the SLO image 201 is displayed as 1, and the transparency α of the map 200 is smaller than 1 and displayed as 0.5, for example. Do. The parameter of the transparency α is a parameter which is set when the operator selects target data for the first time. These parameters can be appropriately changed by the operator via the input device 54. The parameters changed by the operator are stored, for example, in the external storage device 52. When the same target data is opened next time, the display is performed according to the parameters set by the previous operator.

  Subsequently, FIG. 9 (b) shows a display mode when “fundus photograph + SLO image + map” is selected from the combo box 95. Reference numeral 202 denotes a fundus picture (second fundus image).

  In this case, in order to superimpose the map 200 on the fundus picture 202, the SLO image 201 is used. The reason is that the fundus picture 202 is taken at a timing different from that of the tomographic image, and the photographing position and the photographing range can not be known only by the map 200. Therefore, by using the SLO image 201, alignment between the fundus picture 202 and the map 200 can be performed. The alignment between the fundus photograph 202 and the SLO image 201 is performed by the alignment unit 47 described with reference to FIG.

  The alignment method may be performed using, for example, a blood vessel feature. As a blood vessel detection method, since the blood vessel has a thin linear structure, first, the blood vessel is extracted using a filter emphasizing the linear structure. As a filter for emphasizing a linear structure, a filter that calculates the difference between the average value of the image density values in the structural element and the average value in the local region surrounding the structural element when the line segment is the structural element You should use it. Of course, the present invention is not limited to this, and a differential filter such as a Sobel filter may be used. Alternatively, the eigenvalues of the Hessian matrix may be calculated for each pixel of the density value image, and a line segment-like region may be extracted from a combination of two eigenvalues obtained as a result. Furthermore, top hat operation may be used in which line segments are simply structural elements. The alignment unit 47 aligns the fundus picture 202 and the SLO image 201 using the blood vessel position information detected by these methods.

  Since the SLO image 201 and the map 200 can be aligned by the method described above, as a result, the alignment between the fundus picture 202 and the map 200 can also be performed. When the result of superposing the map 200 on the fundus picture 202 is displayed in the display area 82 of the fundus image display unit 94, the transparency α of the fundus picture 202 is displayed as 1. Further, the transparency α of the SLO image 201 and the transparency α of the map 200 are set smaller than 1, and displayed as, for example, 0.5. Of course, the values of the transparency α of the SLO image 201 and the map 200 do not have to be the same. For example, the value of the transparency α of the SLO image 201 may be set to 0.

  When the eye to be examined is a diseased eye, alignment processing by the alignment unit 47 may fail. In the determination of alignment failure, when calculating the inter-image similarity, the maximum similarity does not exceed the threshold. Alternatively, even if the threshold value is exceeded, it can be determined that a failure has occurred when the alignment processing is completed at an anatomically unnatural location. In that case, the SLO image 201 and the map 200 may be displayed at the position of the initial parameter set in advance (for example, the center of the image) and the initial transmittance. Then, a message indicating that the alignment processing has failed is displayed to prompt the position correction processing by the operator via the input device 54.

  Here, when the operator performs position correction and parameter change of transparency α, when the operator performs movement, enlargement / reduction, and rotation operations on the SLO image 201, the map 200 on the SLO image 201 is also simultaneously , Move, zoom in / out, rotate. That is, the SLO image 201 and the map 200 operate as one and the same image. However, regarding the transparency α, the SLO image 201 and the map 200 are set independently. The parameters changed by the operator via the input device 54 are stored in the external storage device 52, and display is performed according to the set parameters from the next time onwards.

  As described above, in the display area 82 of the fundus oculi image in the fundus oculi image display unit 94, in addition to the two-dimensional fundus oculi image, display in which a map or the like is superimposed on the fundus oculi image is performed. Although FIGS. 9A and 9B illustrate the case where the contour map is associated with the corresponding position on the fundus image to be superimposed and displayed, the present invention is not limited thereto. That is, a curvature map or a layer thickness map may be displayed.

  Further, when changing the layer to be analyzed, the operator selects, for example, the segmentation results (L1 to L5) from the tomographic image of the display area 61 shown in FIG. 2 via the input device 54. Just do it. When the layer to be analyzed is switched, the image processing apparatus 30 causes the display control unit 35 to return the layer of the segmentation result that has been highlighted so far to the normal display, and highlights the new layer to be analyzed. Thereby, analysis results of any layer can be displayed.

  Next, details of the first analysis result display unit 96 including the first analysis result 84 shown in FIG. 6 will be described.

  In the first analysis result 84, the shape analysis map generated by the analysis result generation unit 45 is displayed. Further, in the combo box 97, the type of map of the first analysis result 84 can be selected. In this case, the shape analysis map shown as the first analysis result 84 is a contour map.

  The type of shape analysis map shown as the first analysis result 84 and the type of shape analysis map displayed superimposed on the fundus image described in FIG. 9A and FIG. It can be linked and changed by specifying the type from. Furthermore, the display of the second analysis result display unit 98 described below is also changed in conjunction with the change.

  Here, an example of the tomographic image observation screen 80 when the result of curvature is displayed as the analysis result in FIG. 10 will be described.

  When the display of the analysis result is switched from the analysis result of the area and volume to the curvature analysis result, the tomographic image display unit 91, the fundus oculi image display unit 94, the first analysis result display unit 96, and the second analysis result display unit 98. The result displayed on the screen is the content shown in FIG.

  Specifically, in the display area 81 of the tomographic image, segmentation results (L1 to L5) in which each layer of the retinal layer is detected on the tomographic image are superimposed and displayed on the captured tomographic image. In addition, a curvature map is displayed as the first analysis result 84, and a curvature graph is displayed as the second analysis result 88. In the display area 82 of the fundus oculi image, an image in which the SLO image (first fundus oculi image) 201, the fundus oculi photograph (second fundus oculi image) 202, and the curvature map 203 are superimposed and displayed is displayed.

  Next, the details of the second analysis result display unit 98 including the second analysis results 85 and 86 shown in FIG. 6 will be described.

  In the second analysis result 85, the shape analysis graph created by the analysis result creation unit 45 is displayed. In this case, the graph which measured area and volume is shown, a horizontal axis shows height (depth) and a vertical axis shows volume. The solid line indicated by reference numeral 87 indicates a volume.

  In the second analysis result 86, the shape analysis results are displayed as a table. The table displays the area and volume when the height of a certain reference value (for example, 100 μm, 500 μm, etc.) is displayed, and the height when referring to a certain part (the position of the Bruch's membrane opening) The corresponding area and volume are displayed.

  Here, the case where the area and volume results are displayed in one graph (second analysis result 85) will be described with reference to FIG. In this case, the horizontal axis indicates height (depth), the vertical axis on the left side of the graph indicates volume, and the vertical axis on the right side of the graph indicates area. The broken line 89 is a graph showing the area, and the solid line 87 is a graph showing the volume.

[S207]
The image processing apparatus 30 analyzes the shape of the retinal layer analysis unit 44 based on the detection result of the retinal layer. In this analysis process, analysis using detection results of the retinal layer is performed without creating three-dimensional shape data and the like. For example, analysis of layer thickness is performed.

[S208]
The image processing device 30 causes the analysis result creation unit 45 to create an analysis result (for example, a map, a graph, numerical information) based on the analysis result.

[S209]
The image processing apparatus 30 includes a tomographic image, detection results of layers (RPE, ILM, etc.) detected by the detection unit 41, and analysis results (maps, graphs, numerical information) generated by the analysis result generation unit 45. It is displayed on the display device 53.

  FIG. 12 is a view showing an example of the tomographic image observation screen 80 displayed on the display device 53 by the process of S209.

  In this case, a layer thickness map is shown as a first analysis result 302, and a layer thickness graph is shown as a second analysis result 301. That is, analysis results using the detection result of the retinal layer are displayed as the first analysis result 302 and the second analysis result 301.

  Also in this case, the parameter changed by the operator via the input device 54 is stored in the external storage device 52, and the display is performed according to the set parameter from the next time on, as in the process of S206.

  As described above, according to the first embodiment, a plurality of imaging modes including an imaging mode for myopic eye analysis can be provided, and analysis processing can be switched and executed according to the imaging mode at the time of imaging of a tomographic image.

  More specifically, at least a photographing mode for myopic eye analysis and a photographing mode for non-myopic eye analysis are provided as the plurality of photographing modes. In the imaging mode for myopic analysis, imaging of a tomographic image suitable for performing three-dimensional shape analysis (analysis of the macula in the myopic eye) is performed.

  Then, for the tomographic image captured in the imaging mode for myopic eye analysis, three-dimensional shape data is created, and three-dimensional shape analysis is performed based on the shape data. For tomographic images captured in other imaging modes, analysis processing based on the detection result of the retinal layer is executed without creating three-dimensional shape data.

Second Embodiment
Next, the second embodiment will be described. In the second embodiment, a case in which 2D display and 3D display of tomographic images are simultaneously displayed will be described. More specifically, in the second embodiment, a case in which tomographic images, three-dimensional shape data, and analysis results are displayed side by side will be described. In the second embodiment, a tomographic image captured by radial scan will be described as an example of a tomographic image.

  An example of the tomographic image observation screen 400 according to the second embodiment will be described using FIG. 13.

  The tomographic image observation screen 400 includes a first tomographic image display unit 410 including a display region 411 of a two-dimensional tomographic image, and a second tomographic image display unit 430 including a display region 431 of a three-dimensional tomographic image. Provided. Furthermore, on the tomographic image observation screen 400, a first analysis result display unit 440 including the first analysis result 442 and a second analysis result display unit 420 including the second analysis results 421 and 422 are also provided.

  Here, since the first analysis result display unit 440 and the second analysis result display unit 420 are the same as FIGS. 6 and 9A described in the first embodiment described above, the description thereof will be omitted. . In addition, the second tomographic image display unit 430 is also similar to FIGS. 7A to 7C described in the first embodiment, and thus the description thereof will be omitted. Here, the first tomographic image display unit 410 will be mainly described.

  The first tomographic image display unit 410 displays a slider bar 412 for changing the viewpoint position (slice position) of the tomographic image and the slice number of the tomographic image in addition to the display region 411 of the two-dimensional tomographic image. An area 413 is provided.

  Here, the relationship between the viewpoint direction 503 in the three-dimensional shape and the tomographic image (display area 411) shown in FIG. 13 will be described using FIG. Reference numeral 500 shows an outline of the fundus when the three-dimensional shape of the tomographic image is viewed from above in the direction of the deep portion (Z direction), and a radial straight line indicated by reference 501 indicates the imaging slice position of the tomographic image. A broken line indicated by reference numeral 502 indicates a slice position corresponding to the tomographic image (display area 411) being displayed in the imaging slice position 501, and is perpendicular to the viewpoint direction 503. That is, the tomographic image at the position indicated by the broken line 502 is the tomographic image displayed in the display area 411 of the tomographic image.

  Next, a case where the operator operates the tomographic image observation screen 400 described in FIG. 13 via the input device 54 will be described.

  It is assumed that the operator moves the position of the slider bar 412 from the center to the end via the input device 54. Then, in conjunction with the operation, the image processing device 30 changes the viewpoint position of the three-dimensional tomographic image (three-dimensional shape data) being displayed in the display area 431. Also, the slice position of the two-dimensional tomographic image being displayed in the display area 411 is changed to a viewpoint position corresponding to the tomographic image specified by the slider bar 412.

  When the slider bar 412 is operated, the bar does not change discretely, but its position is continuously changed. Therefore, the position of the three-dimensional shape data of the tomographic image being displayed also changes continuously. Specifically, the image processing device 30 displays the three-dimensional shape data of the tomographic image being displayed so as to rotate around the center in the vertical direction.

  FIG. 15 shows an example of screen display of a two-dimensional tomographic image in the display area 411 and a three-dimensional tomographic image (three-dimensional shape data) in the display area 431 when the position of the slider bar 412 is moved to the end. It shows. In this case, the relationship between the viewpoint direction 507 in the three-dimensional shape and the two-dimensional tomographic image (display area 411) is as shown in FIG.

  When the position of the slider bar 412 is manipulated, the image processing device 30 changes the slice position of the two-dimensional tomographic image being displayed in the display area 411, and the three-dimensional tomographic image (3 Change the viewpoint position of dimensional shape data). It is possible to change not only the viewpoint position of the three-dimensional shape data but also the reverse with the operation of the slider bar 412 for changing the slice position of the two-dimensional tomographic image. That is, when the operator changes the viewpoint position of the three-dimensional shape data via the input device 54, the slice position of the two-dimensional tomographic image and the position of the slider bar 412 are changed correspondingly. You may

  As described above, according to the second embodiment, a two-dimensional tomographic image and a three-dimensional tomographic image can be simultaneously displayed, and their display modes can be interlocked and changed according to the operation from the operator. it can.

(Embodiment 3)
Next, the third embodiment will be described. In the third embodiment, a case will be described where both the analysis result of the shape of the retinal layer and the statistical database held in the database are displayed comparably.

  FIG. 17 is a diagram illustrating an example of the configuration of the image processing system 10 according to the third embodiment. Here, points different from the configuration of FIG. 1 described in the first embodiment will be mainly described.

  Here, in addition to the configuration of the first embodiment, the image processing unit 33 of the image processing apparatus 30 according to the third embodiment is additionally provided with a feature amount acquisition unit 601 and a comparison operation unit 602.

  The feature amount acquisition unit 601 calculates a feature amount (shape feature amount) indicating a shape feature of the retinal layer based on three-dimensional shape analysis.

  The comparison operation unit 602 compares the shape feature amount (first feature amount) with the shape feature amount (second feature amount) held in the statistical database 55 stored in the external storage device 52, and compares the comparison. Output the result. The statistical database 55 is created from multi-eye data, and is created by integrating data by human type and age. That is, the statistical database 55 holds statistical feature quantities obtained from a plurality of subject eyes. In the field of ophthalmology, it may be classified according to eye-specific parameters such as right and left eye and eye axis length.

  Next, an example of the process flow of the image processing apparatus 30 according to the third embodiment will be described with reference to FIG. In addition, since the whole process at the time of imaging | photography of a tomographic image becomes the same as the content demonstrated in Fig.3 (a) of Embodiment 1, the description is abbreviate | omitted here and the process of FIG.3 (b) ( Differences from the detection / analysis process of the retinal layer) will be mainly described.

[S301 to S304]
When the detection / analysis process of the retinal layer is started, the image processing apparatus 30 performs the processes of S201 to S204 of FIG. 3B described in the first embodiment. Thus, the retinal layer is detected from the tomographic image, and three-dimensional shape analysis of the detected retinal layer is performed.

[S305]
The image processing apparatus 30 causes the feature amount acquisition unit 601 to calculate the shape feature amount based on the three-dimensional shape analysis performed in the process of S304.

  Here, as the shape feature of the retinal layer, for example, the feature amount of the area and volume of the retinal layer, the feature amount indicating the circular (concave and convex) degree of the retinal layer, the feature amount of the curvature of the shape of the retinal layer, etc. It can be mentioned.

  First, how to calculate the feature amount of the area and volume of the retinal layer will be described. The feature quantities of the area and volume of the retinal layer are determined, for example, by calculating the rate of increase and decrease (typically, the rate of increase) of the area and volume of the retinal layer. The area and volume of the retinal layer may be determined by the same method as in Embodiment 1 described above.

  When calculating the change rate of the area and volume, the feature acquisition unit 601 calculates 100 μm, 200 μm, 300 μm, etc. from the position of the deepest part of the RPE based on the analysis result of the three-dimensional shape analysis by the retina layer analysis unit 44. The area and volume when at a constant height (depth) are determined respectively. Then, based on the result, the increase / decrease rate of the area and volume is determined as the shape feature of the retinal layer.

  In addition, a feature amount indicating the degree of (roundness) of the retina layer is calculated, for example, from the shape of the cross section of the retina layer to be subjected to the area measurement (finally). Here, with reference to FIGS. 19 (a) and 19 (b), a method of calculating the feature amount indicating the circular (concave and convex) degree of the retinal layer will be described using a specific example.

FIG. 19 (a) shows the maximum horizontal chord length (CHORD H), the maximum vertical chord length (CHORD V), and the maximum absolute length (MAXIMUM LENGTH). The FIG. 19 (b), the area (AREA) and has peripheral length (PERIMETER) and are shown, it is possible to calculate the circularity _{C L} from these values. The equation of circularity is shown in “Equation 2”.

[Formula 2]

  In addition, A shows area (AREA) and L shows perimeter length (PERIMETER). When the reciprocal of the circularity is calculated, it becomes a feature amount indicating the degree of unevenness. Of course, the method of obtaining the feature amount indicating the degree of the circular asperity is not limited to this, and for example, a moment feature around the center of gravity may be used.

  Subsequently, how to obtain the characteristic amount of curvature of the shape of the retinal layer will be described. When calculating the feature amount of curvature, the feature amount acquisition unit 601 calculates the curvature at each portion of the retinal layer, and calculates the average, variance, and standard deviation at all portions.

  In addition, when the curvature value is determined not by an absolute value but by a signed curvature value, it is possible to know whether the shape is uneven in the positive or negative. Therefore, in the case of obtaining a signed curvature value, the number of positive curvature values and the number of negative curvature values are counted as curvature values. In the case of the count of positive and negative numbers, the curvature value is 0 in the case of a perfectly straight line, but there is some error in automatic or manual detection of the retinal layer. Therefore, the value near the curvature value 0 may be excluded from the count, and the curvature value may be large in positive and negative directions, that is, only the portion where the feature is large may be counted.

[S306]
The image processing device 30 causes the comparison operation unit 602 to compare the feature amount obtained in the process of S305 with the feature amount of the statistical database 55 stored in the external storage device 52. The statistical database 55 holds information in which the 95% range of normal data is normal, the 4% range is borderline, and the remaining 1% range is abnormal.

  The comparison operation unit 602 compares the range of the analysis result and the feature value obtained in the processing of S304 and S305 with the feature value held in the statistical database 55. For example, comparison may be performed only on the area or volume of the analysis result, or the value of the area or volume may be compared with the value of the feature amount as a position of two axes. Furthermore, in the case where the feature amount is curvature, two axes may be created by the average value of the curvature values and the variance value thereof, and the range in which the axis is included may be compared.

  This point will be further described. In the graph shown in FIG. 20A, the horizontal axis indicates the average value of the curvature, and the vertical axis indicates the dispersion value of the curvature. A rhombus indicated by reference numeral 701 indicates the curvature value of the retinal layer without myopic symptoms in the myopic eye, and a triangle indicated by reference numeral 702 indicates the curvature of the retinal layer when myopic eyes show the symptoms of mild posterior vesiculation. It shows the value. In addition, a square indicated by reference numeral 703 indicates a curvature value of the retinal layer in the case where there is a symptom of posterior vesicular edema in the myopic eye.

  In addition, posterior pelvis is a state in which the shape is deformed as the eyeball protrudes rearward. The fact that there are a plurality of plot points is the number of cases, and in this case, several tens of cases are plotted. In addition, although the description of the shape feature obtained by the process of S305 is omitted here, the shape feature can be discriminated from the feature stored in the statistical database 55 (for example, a display format Shapes are changed) and plotted on the graph. That is, the shape feature amount obtained by the process of S305 is displayed comparably to the feature amount stored in the statistical database 55.

  Here, the area 711 shows a curvature value area without symptoms of posterior vesiculation, the area 712 shows a curvature value area with symptoms of mild posterior chyproma, and the area 713 shows symptoms of posterior chyproma The curvature value area is shown. Therefore, for example, when the curvature value of the subject eye to be examined is measured, if the curvature value of the subject eye is located in the area 711, it is determined that there is no posterior edema and if it is located in the area 713 it can.

  As described above, in the present embodiment, the curvature value (the display form indicating the category of the subject's eye to be shown stepwise to show the stage of the disease) shows the curvature value The feature quantities are plotted and displayed on the graph. In addition, the display format (diamonds, triangles, squares) of the object indicating the curvature value is changed and displayed depending on which region it belongs to. In addition, although the case where it classify | categorizes into three area | regions was demonstrated here in order to simplify description, it does not restrict to this. For example, each region may be divided into a plurality of pieces and classified into about 10 in total, so that the size of the degree of the posterior fibroid may be determined.

[S307]
The image processing apparatus 30 causes the analysis result generation unit 45 to generate an analysis result based on the result of the process of S305. The analysis result is created, for example, in a form in which it can be grasped where the analysis value of the subject eye to be examined is located in the statistical database 55.

  As an example of the analysis result, as described in FIG. 20A, for example, when the statistical database 55 is expressed in two dimensions, a graph plotting where the curvature value of the subject eye is located can be mentioned. . Also, as shown in FIG. 20 (b), a color bar (bar graph) representing the features of the statistical database 55 in one dimension is created, and an analysis result indicating where the color bar (bar graph) is located is created. It is good.

  Furthermore, as in the curvature value, when one value corresponds to each location, comparison is made with any curvature value stored in the statistical database 55 at that location, and each location is located in the statistical database 55 It is also possible to create a two-dimensional map that can be understood. In the two-dimensional map, a value obtained by averaging a certain range may be compared with the statistical database 55 instead of the comparison at each place.

[S308]
The image processing apparatus 30 causes the display control unit 35 to display the tomographic image, the layer detection result detected by the detection unit 41, and the analysis result (map, graph, numerical information) generated by the analysis result generation unit 45. It is displayed on the display device 53. In addition, what is necessary is just to display the map, the graph, etc. which were produced by the process of S307 as an analysis result. Specifically, a tomographic image observation screen 80 shown in FIG. 21 is displayed.

  In this case, as a first analysis result 84, a curvature map of the retinal layer is displayed as a comparison result with the feature amount (curvature) of the statistical database 55. The curvature map shows the result of comparing a value obtained by averaging a certain range with the statistical database 55. The darker the color, the more the curvature value deviates from the normal value of the statistical database 55, and the lighter the lightness, the curvature value falls within the normal value range of the statistical database 55.

  In the second analysis result display unit 98, as a second analysis result, a graph 730 indicating where the mean value and the variance value of the curvature are located in the statistical database 55 is shown. The round shaped point 722 in the graph 730 is a plot of the analysis value of the subject eye in question in this graph.

  In the screen example shown in FIG. 21, only one map or graph is displayed, but the present invention is not limited to this. For example, the area / volume map and the curvature map may be displayed side by side in the first analysis result display section 96, and at the same time, the area / volume graph and the curvature graph may be displayed in the second analysis result display section It may be displayed side by side within 98.

  As described above, according to the third embodiment, the feature quantity (shape feature quantity: first feature quantity) indicating the shape feature of the retinal layer and the shape feature corresponding to a plurality of subject eyes held in advance in the statistical database The amount (second feature amount) can be displayed (presented) to the operator. Thus, it is possible to present the shape feature of the subject's eye to be diagnosed, as well as an index serving as a criterion for determining whether the subject's eye is a disease.

(Embodiment 4)
Next, the fourth embodiment will be described. In the fourth embodiment, the feature amount stored in the statistical database is searched using the shape feature (shape feature amount) of the retinal layer and the image feature (image feature amount) of the tomographic image or the fundus image. Then, a case where a tomographic image, an eye fundus image, and diagnostic information of an eye to be examined having a feature amount close to the shape feature of the current eye to be examined is displayed.

  FIG. 22 is a diagram illustrating an example of the configuration of the image processing system 10 according to the fourth embodiment. Here, points different from the configuration of FIG. 17 in which the third embodiment is described will be mainly described. In addition to the configuration of the third embodiment, a search unit 36 is newly provided in the image processing apparatus 30 according to the fourth embodiment.

  Further, in the external storage device 52 according to the fourth embodiment, the feature amount of the retina layer, the tomographic image, the fundus image, and the diagnostic information are stored as the statistical database 55 in association with each other. The search unit 36 searches the external storage device 52 (that is, the statistical database 55) using the shape feature of the retinal layer and the image feature in the tomographic image or the fundus image.

  Next, an example of the flow of processing of the image processing apparatus 30 according to the fourth embodiment will be described. The processing of the image processing apparatus 30 according to the fourth embodiment is performed according to the same flow as that of the third embodiment described above, so here, the different processing will be mainly described using FIG. 18 described above. The different processes include the processes of S305, S306, and S308.

[S306]
The image processing apparatus 30 causes the feature amount acquisition unit 601 to extract an image feature amount from a tomographic image or a fundus image, in addition to the shape feature amount described in the third embodiment. Examples of the image feature amount include edge features, color features, histogram features, and scale-in variant feature transform (SIFT) feature amounts.

[S307]
The image processing apparatus 30 causes the search unit 36 to express the shape feature amount and the image feature amount obtained in the process of S305 as a multidimensional vector, and stores the feature amount and the statistic database 55 in the external storage device 52. Compare the data with the Thereby, images with similar feature quantities are searched. A nearest neighbor search may be used as a search method. In order to speed up the process, approximate nearest neighbor search may be used.

[S308]
The image processing device 30 causes the display control unit 35 to display a screen including the photographed tomographic image and the similar tomographic image obtained by the search of the search unit 36 on the display device 53.

  Specifically, a tomographic image observation screen shown in FIG. 23 is displayed. The tomographic image observation screen 900 is provided with a first analysis result display unit 930 for displaying the current image analysis result and a second analysis result display unit 940 for displaying the past image analysis result.

  In the first analysis result display unit 930, a tomographic image 901, an SLO image 902, a fundus photograph 903, a (three-dimensional shape) analysis result map 904, and an analysis graph 905 are displayed. A circle-shaped point 906 in the analysis graph 905 indicates the result of plotting the analysis value of the subject eye to be examined in the graph. In addition, a star-shaped point 907 indicates a result of plotting the analysis value of the subject eye in the past displayed in the second analysis result display unit 940 in a graph. In this case, in the analysis graph 905, the horizontal axis indicates the shape feature, and the vertical axis indicates the image feature. Of course, the present invention is not limited to this, and may be displayed as a graph indicating the curvature value described in the third embodiment as an analysis result.

  The second analysis result display unit 940 includes tomographic images 911 and 921, SLO images 912 and 922, fundus photographs 913 and 923, (three-dimensional shape) analysis result maps 914 and 924, diagnostic information 915 and 925 of the subject's eye Is displayed. The diagnosis information is information indicating the progress of diagnosis, and includes, for example, age, diagnosis name, treatment history, visual acuity, reflex value, kerat value, axial length and the like. In the case of FIG. 23, although the image and diagnostic information of the same to-be-tested eye are displayed along a longitudinal row, of course, the image and diagnostic information of the same to-be-tested eye are arranged side by side and displayed horizontally. Also good.

  Here, when there are a plurality of images similar to the current eye to be examined, the operator can switch and display those results by operating the slider bar 950 via the input device 54. Note that switching of the display of similar past images may be performed using something other than the slider bar 950. For example, when the operator clicks the star-shaped point 907 of the analysis graph 905 via the input device (for example, a mouse) 54, an image having the feature amount is displayed side by side with the current image. You may do so.

  As described above, according to the fourth embodiment, the tomographic image of the eye to be examined that is captured based on the feature amount (shape feature amount) indicating the shape feature of the retinal layer and the image feature amount of the tomographic image or the fundus image Search from the statistical database for tomographic images similar to and information about them. Then, the similar tomographic image and information related thereto are displayed side by side with the captured tomographic image. Thus, it is possible to present the shape feature of the subject's eye to be diagnosed, as well as an index serving as a criterion for determining whether the subject's eye is a disease.

  The above is an example of a representative embodiment of the present invention, but the present invention is not limited to the embodiment described above and shown in the drawings, and can be appropriately modified and implemented within the scope not changing the gist of the present invention. .

  For example, in some screens described in the first to fourth embodiments described above, check boxes, combo boxes, and the like are arranged, but radio buttons, list boxes, buttons, and the like may be appropriately changed according to the application. good. For example, the configuration described as the combo box in the above description may be implemented as a list box.

(Other embodiments)
The present invention is also realized by performing the following processing. That is, software (program) for realizing the functions of the above-described embodiment is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU or the like) of the system or apparatus reads the program. It is a process to execute.

  10: Image processing system 20: Tomographic imaging device 30: Image processing device 51: Fundus imaging device

Claims (10)

An acquisition unit configured to process the tomographic image acquired by the tomographic imaging apparatus for acquiring a tomographic image of the fundus to acquire a shape feature of the shape of the retinal layer of the fundus;
The shape feature quantity acquired by the acquisition means is referred to as the shape feature quantity of the retinal layer with respect to the plurality of eyes with reference to a database in which the shape feature quantity of the retinal layer with respect to the plurality of eyes classified by eye axis length is stored. Control means for displaying in a comparable state ;
An image processing system comprising: It further comprises selection means for selecting a retinal layer to be analyzed,
The image processing system according to claim 1, wherein the acquisition unit acquires the shape feature of the retinal layer selected by the selection unit. The control means, the tomographic image and the image processing system according to claim 2, wherein the benzalkonium be identifiably displayed retinal layer selected by said selection means. The shape feature value includes the curvature of the retinal layer or the layer thickness of the retinal layer,
Before SL control means, the image processing system according to claim 3, characterized in that displaying a curvature map or thickness map of the layer thickness of the curvature. Has multiple imaging modes, including imaging modes for myopic analysis,
The acquisition unit, an image processing system according to claim 1 or 2, characterized in that performing by switching analysis processing according to the imaging mode at the time of photographing of the tomographic image. The image processing system according to claim 1 or 2, wherein the shape feature before Symbol retinal layers, the curvature value of the signed indicating which protrudes downward or is convex upward. And a generation unit configured to generate a curvature map based on the acquired curvature value.
It said control means, the image processing system according to claim 6, wherein the benzalkonium to further display the curvature map. Fundus image acquisition means for acquiring a fundus image of the fundus;
An alignment unit that aligns the fundus image and the curvature map;
Before SL control means, the image processing system according to claim 7, characterized in that displaying the curvature map superimposed on the fundus image based on a result of the alignment. A processing method of the image processing system,
Acquisition means processing the tomographic image acquired by the tomographic imaging apparatus for acquiring a tomographic image of the fundus to acquire a shape feature of the shape of the retinal layer of the fundus;
The analysis means refers to a database in which shape feature amounts of retinal layers related to a plurality of eyes classified by eye axis length are stored, and shapes obtained in the step of acquiring the retinal shape layer on the plurality of eyes Displaying in a state comparable to the shape feature of
And a processing method of an image processing system. A program for causing a computer to execute processing by the image processing system according to any one of claims 1 to 8 .