JP2023149647A - Ophthalmologic image processing device and ophthalmologic image processing program - Google Patents

Abstract

To provide an ophthalmologic image processing device and an ophthalmologic image processing program capable of appropriately processing data on a tomographic image of a tissue of an eye to be examined.SOLUTION: In an image acquisition step, a control part acquires an ophthalmologic image 62 captured by an ophthalmologic image capturing device. In an extraction step, the control part extracts data on an image in a target range for storing data from each of a plurality of partial images 70 arranged in a predetermined direction in the ophthalmologic image 62. In a position information acquisition step, the control part acquires position information indicating each position of a plurality of extraction images 80 extracted from each of the plurality of partial images 70 in the extraction step. In a storage step, the control part causes storage means to store the data on the plurality of extraction images 80 and the position information on each of the plurality of extraction images 80.SELECTED DRAWING: Figure 6

Description

The present disclosure relates to an ophthalmologic image processing apparatus that processes data of an ophthalmologic image that is a tomographic image of a tissue of a subject's eye, and an ophthalmologic image processing program that is executed in the ophthalmologic image processing apparatus.

2. Description of the Related Art Techniques for taking tomographic images of tissues of an eye to be examined are known. For example, the OCT apparatus disclosed in Patent Document 1 captures a two-dimensional tomographic image of a cross section passing through the scanning line by scanning measurement light on the scanning line. Furthermore, a three-dimensional tomographic image is obtained by arranging a plurality of two-dimensional tomographic images taken for a plurality of scanning lines.

JP 2019-150532 Publication

Generally, data of captured tomographic images is stored as is in a storage device. As a result, problems such as an increase in the storage capacity of the storage device and a decrease in processing speed when processing stored data tend to occur. In particular, in recent years, the resolution, angle of view, depth, and speed of tomographic imaging have become higher, and the frequency with which tomographic images are taken and used for diagnosis, etc. is also increasing. . Therefore, reducing the amount of tomographic image data stored in a storage device has become an important issue.

A typical object of the present disclosure is to provide an ophthalmological image processing device and an ophthalmic image processing program that can appropriately process data of a tomographic image of a tissue of an eye to be examined.

An ophthalmologic image processing apparatus provided by a typical embodiment of the present disclosure is an ophthalmologic image processing apparatus that processes data of an ophthalmologic image that is a tomographic image of a tissue of a subject's eye, and the control unit of the ophthalmologic image processing apparatus includes: , an image acquisition step of acquiring an ophthalmological image photographed by an ophthalmological image capturing device; and data of images within a target range in which data is saved from each of a plurality of partial images arranged in a predetermined direction within the ophthalmological image. an extraction step of extracting a plurality of extracted images; a position information acquisition step of obtaining position information indicating the position of each of the plurality of extracted images extracted from each of the plurality of partial images in the extraction step; A storing step of storing the data and position information of each of the plurality of extracted images in a storage means is executed.

An ophthalmologic image processing program provided by a typical embodiment of the present disclosure is an ophthalmologic image processing program executed by an ophthalmologic image processing apparatus that processes data of an ophthalmologic image that is a tomographic image of a tissue of an eye to be examined, an image acquisition step of acquiring an ophthalmologic image photographed by the ophthalmologic image capturing device by executing an ophthalmologic image processing program by a control unit of the ophthalmologic image processing device; an extraction step of extracting data of an image within a target range for data storage from each of the plurality of partial images; and a position of each of the plurality of extracted images extracted from each of the plurality of partial images in the extraction step. a position information acquisition step of acquiring position information indicating the plurality of extracted images, and a storage step of storing the data of the plurality of extracted images and the position information of each of the plurality of extracted images in a storage means, in the ophthalmological image processing apparatus. Let it run.

According to the ophthalmological image processing device and the ophthalmological image processing program according to the present disclosure, data of a tomographic image of a tissue of an eye to be examined is appropriately processed.

1 is a block diagram showing a schematic configuration of an ophthalmologic image processing system 100. FIG. FIG. 2 is an explanatory diagram for explaining a method in which the ophthalmologic image capturing apparatus 1 captures an ophthalmologic image of a tissue 50 of an eye to be examined. 6 is a diagram showing an example of an ophthalmologic image 61 captured by the ophthalmologic image capturing device 1. FIG. It is a flowchart showing an example of ophthalmological image processing performed by the ophthalmological image processing device 40. 6 is a diagram showing a photographed ophthalmologic image 61 and an ophthalmologic image 62 in which an image area is extracted from the ophthalmologic image 61. FIG. This is an image extracted from a partial image 70 within the ophthalmological image 62. 80 is an explanatory diagram for explaining an example of a method for extracting 80. FIG. 7 is a diagram illustrating a reference image 80A in which a plurality of extracted images 80 extracted from each of a plurality of partial images 70 are arranged in the X direction without consideration of their respective positions. FIG. 9 is a diagram showing an example of reconstructed image data 90 and background complementary image data 91. FIG.

<Summary>
The ophthalmological image processing device exemplified in the present disclosure processes data of an ophthalmological image that is a tomographic image of a tissue of a subject's eye. The control unit of the ophthalmological image processing device executes an image acquisition step, an extraction step, a position information acquisition step, and a storage step. In the image acquisition step, the control unit acquires an ophthalmologic image captured by the ophthalmologic image capturing device. In the extraction step, the control unit extracts data of an image within a target range for storing data from each of a plurality of partial images arranged in a predetermined direction within the ophthalmological image. In the position information acquisition step, the control unit acquires position information indicating the position of each of the plurality of extracted images extracted from each of the plurality of partial images in the extraction step. In the storage step, the control unit stores data of the plurality of extracted images and position information of each of the plurality of extracted images in the storage means.

According to the technology according to the present disclosure, an extraction image that is an image within a target range for storing data is extracted from each of a plurality of partial images that constitute an ophthalmological image. Furthermore, position information of each extracted image is stored together with data of a plurality of extracted images. That is, data of an appropriate range of images from the ophthalmological image is extracted and saved. Further, if each extracted image is arranged based on position information, the image of the extracted portion can be appropriately reconstructed. Therefore, compared to the case where the entire data of the ophthalmological image is saved, the storage capacity of the storage device is appropriately suppressed.

Various devices can be used as an ophthalmologic imaging device that captures (generates) tomographic images. For example, an OCT device that takes a tomographic image of the tissue of the eye to be examined using the principle of optical coherence tomography can be used. In this case, the tomographic image may be, for example, a motion contrast image (for example, an OCT angiography image) obtained by acquiring a plurality of OCT signals at different times from the same position in the retinal layer of the fundus. Furthermore, an MRI (magnetic resonance imaging) device, a CT (computed tomography) device, or the like may be used as the ophthalmologic imaging device. The tomographic image acquired in the image acquisition step may be a two-dimensional tomographic image or a three-dimensional tomographic image. Further, the tomographic image in the present disclosure is a tomographic image of the fundus of the eye to be examined. However, the tomographic image may be a tomographic image of a tissue other than the fundus of the eye to be examined (for example, the anterior segment of the eye).

Note that various devices can function as an ophthalmological image processing apparatus. For example, the ophthalmologic imaging device itself that captures tomographic images may function as the ophthalmologic image processing device in the present disclosure. In this case, the ophthalmologic imaging apparatus can appropriately store the captured tomographic image in the storage means while capturing the tomographic image of the eye to be examined. Further, a device (for example, a personal computer (PC) or a mobile terminal, etc.) that can exchange data with the ophthalmologic imaging device may function as the ophthalmologic image processing device. A plurality of control units (for example, a control unit of an ophthalmologic imaging device, a control unit of a PC, etc.) may cooperate to perform processing.

The control unit may further execute a reference region specifying step of specifying at least one of the plurality of regions within the tissue shown in the ophthalmological image as a reference region. In the extraction step, the control unit may extract, as an extraction image, data of an image within a target range based on a reference site from each of a plurality of partial images arranged in a predetermined direction within the ophthalmological image. . In this case, image data in an appropriate range based on the identified reference site is extracted and saved. Therefore, image data in an unnecessary range is appropriately excluded, and then image data in a necessary range is saved.

A method for specifying a reference site within a tissue that appears in an ophthalmological image can be selected as appropriate. For example, the control unit may identify the reference site within the ophthalmological image by known image processing (for example, edge detection, etc.). Also, a mathematical model trained by a machine learning algorithm to identify reference sites within an input ophthalmological image may be used. In this case, the control unit may specify the reference site by inputting the ophthalmological image into the mathematical model and acquiring the reference site identification result output by the mathematical model.

The ophthalmological image may be a tomographic image of a layer structure in which a plurality of layers are stacked (for example, a layer structure of the fundus of the eye) among the tissues of the eye to be examined. In the reference region identification step, the control unit identifies at least one of a plurality of layers included in the layered structure and a layer boundary (hereinafter sometimes referred to as "layer/boundary") as a reference region. Good too. The layers and boundaries included in the layered structure tend to have a shape that follows the overall shape of the layered structure. Therefore, even if the layer structure is not flat (for example, if the layer structure is curved or deformed due to a disease, etc.), the extracted image can be extracted using the layer/boundary as the reference site. By extracting the image, it becomes easier to save an image in an appropriate range according to the shape of the layer structure.

In the reference site specifying step, the control unit may specify at least the most superficial boundary or layer of the layer structure shown in the ophthalmological image as the reference site. The most superficial side of the layered structure is the boundary between the area where the layered structure exists and the area where the layered structure does not exist. Therefore, by using the layer/boundary on the most superficial side of the layer structure as the reference region, it becomes easier to extract an extraction image including the layer structure more appropriately. Note that when the tissue shown in the ophthalmological image is the retinal tissue of the fundus, the most superficial layer/boundary is the retinal surface layer or the boundary of the retinal surface layer (for example, the surface side boundary).

In the extraction step, the control unit may include a predetermined range from the most superficial reference region in the layered structure toward the deeper layer of the layered structure as a target range for extracting the extraction image from each of the plurality of partial images. good. In this case, the widest possible range of the layer structure is likely to be included in the extracted image. Furthermore, the problem that an image of the tissue on the surface layer side of the layered structure is not extracted is less likely to occur. Therefore, an image of the layer structure of the eye to be examined is more appropriately extracted and saved.

Note that the size of the image extraction range from the most superficial reference part in the layered structure toward the deeper layer (in other words, the above-mentioned "size of the predetermined range toward the deeper layer") can be set as appropriate. can. For example, the control unit sets the size of the image extraction range from the reference region toward the deep side (hereinafter also referred to as "offset amount toward the deep side") according to instructions input by the user. You may. In this case, the user can efficiently and appropriately save image data of a desired range of the layer structure of the eye to be examined. Further, the control unit may adjust the amount of offset toward the deeper layer depending on the photographing conditions of the ophthalmological image (for example, at least one of photographing magnification, photographing angle of view, photographing depth, etc.). In this case, image data in an appropriate range according to the shooting conditions is extracted. Further, the amount of offset toward the deeper layer side may be a predetermined fixed value.

In the extraction step, the control unit extracts a plurality of partial images in a predetermined range from the most superficial reference region in the layered structure toward the side opposite to the deep side of the layered structure (hereinafter referred to as the "front side" for convenience). The extracted images may be included in the target range to be extracted from each of the images. In this case, for example, even if the identification accuracy of the reference region closest to the superficial layer is reduced, an image of the tissue on the superficial side of the layered structure can be easily extracted appropriately. Further, even if some object is reflected on the surface side of the most superficial layer/boundary due to various influences (for example, the influence of a disease, etc.), the object reflected on the surface side is also likely to be extracted. Therefore, it becomes easier to extract images more appropriately.

In addition, similar to the amount of offset toward the deep side, the size of the image extraction range from the most superficial reference region toward the near side (hereinafter sometimes referred to as "the amount of offset toward the near side") is also Can be set as appropriate. For example, the control unit may set the amount of offset toward the front side according to an instruction input by the user. The control unit may adjust the amount of offset toward the front side according to the photographing conditions of the ophthalmological image (for example, at least one of photographing magnification, photographing angle of view, photographing depth, etc.). Further, the amount of offset toward the front side may be a predetermined fixed value.

Note that, of the layer structure shown in the ophthalmological image, a layer/boundary located deeper than the most superficial layer/boundary may be used as the reference site. For example, if the tissue shown in an ophthalmological image is the retinal tissue of the fundus, the layers and boundaries located deeper than the most superficial retinal surface layer (e.g. ganglion cell layer, inner plexiform layer, inner nuclear layer, outer plexiform layer) The reference site may be at least one of the following: outer limiting membrane, photoreceptor inner segment/outer segment junction, retinal pigment epithelium, Bruch's membrane, choroid, etc., and their boundaries). For example, by setting a layer/structure that is more clearly visible in an ophthalmological image as the reference region, the accuracy of identifying the reference region may be improved, and the image may be extracted and saved more appropriately. .

Further, among the layer structures shown in the ophthalmological image, the layer/boundary specified as the reference region may be changed according to various conditions. For example, there are layers and boundaries whose visibility in ophthalmological images changes depending on the position of the tissue. Therefore, the layer/boundary specified as the reference site may be changed depending on the position of the tissue shown in the ophthalmological image.

Further, the control unit may specify a plurality of layers/boundaries shown in the ophthalmological image as the reference region. In this case, in the extraction step, the control unit selects one reference region (e.g., the layer/boundary on the most superficial side) and another reference region (e.g., on the deeper side of the layer structure) out of the plurality of reference regions identified. The extracted image may be extracted from each of the plurality of partial images, with the range surrounded by the layer/boundary) as the target range. As a result, data of the image within a specific range of the layered structure shown in the ophthalmological image is appropriately extracted and saved.

Each of the plurality of partial images may be a long image extending in a direction intersecting the layers in the layered structure shown in the ophthalmological image. The plurality of partial images may be consecutively arranged within the ophthalmological image in a direction intersecting the direction in which each partial image extends. In this case, within each partial image, the layer/boundary serving as the reference region can be easily identified. As a result, the extracted image is extracted and saved more appropriately.

For example, when the ophthalmologic image is an OCT image, the plurality of partial images may be A-scan images extending in a direction along the optical axis of the OCT measurement light. The OCT measurement light is irradiated onto the tissue of the eye to be examined (for example, fundus tissue) from the front side. Therefore, the A-scan image extending in the direction along the optical axis of the OCT measurement light becomes an image (pixel row) extending in the direction intersecting the layers in the tissue. Therefore, by using a plurality of A-scan images constituting an ophthalmologic image as partial images, it becomes easier to extract an extraction image that includes an appropriate region of layer structure. Note that each A-scan image may be used as a partial image as it is, or a plurality of A-scan images may be used as a constituent unit of each partial image.

However, it is also possible to change partial images. For example, when the ophthalmologic image is an OCT image, a plurality of pixel columns (so-called C-scan image) that perpendicularly intersect with the A-scan image may be used as the partial image. Alternatively, the direction in which at least one of the layers/boundaries (for example, the layer/boundary serving as the reference region) extends may be detected, and a plurality of partial images intersecting the detected direction may be appropriately set. Further, when the layered structure is curved, a set of the plurality of partial images may be arranged in a fan shape or the like so that each of the plurality of partial images intersects the curved shape of the layered structure.

Moreover, it is also possible to use a part other than the layer/boundary as the reference part among the plurality of parts shown in the ophthalmological image. For example, a diseased site appearing in an ophthalmologic image may be specified as a reference site, and an extraction image may be extracted using the specified reference site as a reference. In this case, an extracted image is appropriately extracted and saved according to the disease site. Further, a blood vessel site or the like that appears in an ophthalmologic image may be specified as the reference site.

As a result of executing the reference part identification step, if there is a part where the reference part identification result is missing, the reference part identification result at the missing part is supplemented based on the reference part identification result in other parts. Further completion steps may also be performed. In this case, even if a part of the reference region is not identified due to various influences (for example, image quality or disease), multiple partial images can be generated based on the reference region whose identification results have been complemented in the interpolation step. An extracted image is extracted from. Therefore, images may be extracted and saved more appropriately.

The specific method of the interpolation step can also be selected as appropriate. For example, when there is a missing part for which the identification result is missing, the control unit may detect the reference part based on the identification result of the reference part in a part adjacent to the missing part (for example, a partial image adjacent to a partial image including the missing part). , the missing location identification results may be supplemented. For example, the control unit complements the position between the specific position of the partial image adjacent to one side of the partial image including the missing part and the specific position of the partial image adjacent to the other side as the position of the reference part. You may.

Note that instead of the method of extracting the extracted image from the ophthalmological image using the reference site as a reference, it is also possible to extract the extracted image using another method. For example, the control unit performs image processing on the acquired ophthalmological image, detects an image area that includes the image in the ophthalmological image, and converts data of the image within the target range including the detected image area. It may be extracted as an extracted image. The control unit also controls a region of the ophthalmological image that is likely to contain an image (for example, a predetermined region of the ophthalmological image that is predicted to have a high possibility of showing the image). Image data may be extracted as an extracted image.

An image (reference image) in which multiple extracted images extracted in the extraction step are arranged in the arrangement direction of the multiple partial images without considering position information is an image other than the layered structure from the ophthalmological image acquired in the image acquisition step. An image may be obtained in which at least a portion of the area is erased and the curvature of the layer structure is flattened. In this case, in the reference image, by erasing at least part of the area other than the layer structure from the ophthalmological image, the amount of image data is appropriately reduced, and layer curvature information reproduced by position information is removed. The layer structure becomes flat. Therefore, necessary data is appropriately extracted and stored from the ophthalmological image data.

The extracted image may be extracted from the ophthalmological image and stored while the thickness of the layer structure in the ophthalmological image is maintained. Note that the extracted image data may be stored uncompressed or after being reversibly compressed. In this case, if each extracted image is arranged based on the position information, the layer structure in the original ophthalmological image is appropriately reconstructed with the thickness maintained.

The control unit performs image reconstruction for reconstructing the area extracted in the extraction step of the ophthalmological image by arranging data of the plurality of extracted images stored in the storage means based on the respective position information. Further construction steps may be performed. In this case, the extracted image stored with an amount of data smaller than the amount of data of the original ophthalmological image is appropriately reconstructed.

Note that the control unit can also store the difference with data of other extracted images, instead of storing the data of the plurality of extracted images as they are. For example, the control unit may reduce the data amount of some of the extracted images by storing the difference between the some of the extracted images and data of nearby (for example, adjacent) extracted images. When reconstructing an image, the control unit may combine data of a part of extracted images with data of neighboring extracted images based on the difference.

<Embodiment>
One typical embodiment according to the present disclosure will be described below. In this embodiment, a case will be exemplified in which a tomographic image of the fundus tissue of the eye E to be examined is processed, which has been photographed by an OCT device that is one type of ophthalmological image photographing device. However, the ophthalmological image to be processed may be an image of a tissue other than fundus tissue. For example, the ophthalmological image to be processed may be an image of a tissue other than the fundus of the eye E to be examined (for example, the anterior segment of the eye). Furthermore, images of biological tissue (for example, skin, digestive organs, brain, etc.) other than the subject's eye E (that is, images other than ophthalmological images) may be processed. Further, as described above, the photographing device for photographing images is not limited to the OCT device.

With reference to FIG. 1, a schematic configuration of an ophthalmologic image processing system 100 according to the present embodiment will be described. The ophthalmologic image processing system 100 of this embodiment includes an ophthalmologic image capturing device 1 and an ophthalmologic image processing device 40. The ophthalmologic image capturing apparatus 1 captures a tomographic image of a living body (in this embodiment, the fundus of an eye to be examined). Specifically, the ophthalmological imaging device (OCT device) 1 of this embodiment of the present invention scans measurement light on the tissue of a living body and continuously receives light from the tissue in a first direction. (scan direction) and a second direction (depth direction, which is the direction of the optical axis of the measurement light) intersecting the first direction. In addition, the ophthalmologic image capturing apparatus 1 scans measurement light onto each of a plurality of scan lines within a two-dimensional measurement region of a living body to capture a plurality of two-dimensional tomographic images, thereby obtaining a three-dimensional tomographic image of a living body. It is also possible to acquire images. The ophthalmologic image processing device 40 processes data of ophthalmologic images acquired (photographed) by the ophthalmologic image capturing device 1 .

The configuration of the ophthalmologic image capturing apparatus 1 of this embodiment will be explained. The ophthalmologic image capturing apparatus (OCT apparatus) 1 includes an OCT section 10 and a control unit 30. The OCT section 10 includes an OCT light source 11, a coupler (light splitter) 12, a measurement optical system 13, a reference optical system 20, and a light receiving element 22.

The OCT light source 11 emits light (OCT light) for acquiring ophthalmological image data. The coupler 12 splits the OCT light emitted from the OCT light source 11 into measurement light and reference light. Moreover, the coupler 12 of this embodiment combines the measurement light reflected by the tissue (in this embodiment, the fundus of the eye E to be examined) and the reference light generated by the reference optical system 20 and causes them to interfere with each other. That is, the coupler 12 of this embodiment serves both as a branching optical element that branches the OCT light into a measurement light and a reference light, and as a combining optical element that combines the reflected light of the measurement light and the reference light. Note that it is also possible to change the configuration of at least one of the branching optical element and the multiplexing optical element. For example, elements other than couplers (eg, circulators, beam splitters, etc.) may be used.

The measurement optical system 13 guides the measurement light split by the coupler 12 to the subject, and returns the measurement light reflected by the tissue to the coupler 12. The measurement optical system 13 includes a scanning section (scanner) 14, an irradiation optical system 16, and a focus adjustment section 17. The scanning unit 14 is driven by the driving unit 15 and can scan the spot-shaped measurement light in a two-dimensional direction intersecting the optical axis of the measurement light. In this embodiment, two galvanometer mirrors capable of deflecting measurement light in mutually different directions are used as the scanning unit 14. However, another device that deflects light (for example, at least one of a polygon mirror, a resonant scanner, an acousto-optic device, etc.) may be used as the scanning unit 14. The irradiation optical system 16 is provided on the downstream side of the optical path (that is, on the subject side) than the scanning unit 14, and irradiates the tissue with measurement light. The focus adjustment unit 17 adjusts the focus of the measurement light by moving an optical member (for example, a lens) included in the irradiation optical system 16 in a direction along the optical axis of the measurement light.

Reference optical system 20 generates a reference light and returns it to coupler 12 . The reference optical system 20 of this embodiment generates reference light by reflecting the reference light split by the coupler 12 with a reflective optical system (for example, a reference mirror). However, the configuration of the reference optical system 20 can also be changed. For example, the reference optical system 20 may transmit the light incident from the coupler 12 without reflecting it, and return the light to the coupler 12. The reference optical system 20 includes an optical path length difference adjustment section 21 that changes the optical path length difference between the measurement light and the reference light. In this embodiment, the optical path length difference is changed by moving the reference mirror in the optical axis direction. Note that a configuration for changing the optical path length difference may be provided in the optical path of the measurement optical system 13.

The light receiving element 22 detects an interference signal by receiving interference light between the measurement light and the reference light generated by the coupler 12. In this embodiment, the principle of Fourier domain OCT is adopted. In Fourier domain OCT, the spectral intensity of interference light (spectral interference signal) is detected by the light receiving element 22, and a complex OCT signal is obtained by Fourier transform on the spectral intensity data. As an example of Fourier domain OCT, spectral-domain-OCT (SD-OCT), swept-source-OCT (SS-OCT), etc. can be adopted. Furthermore, it is also possible to employ, for example, Time-domain-OCT (TD-OCT).

The control unit 30 manages various controls of the ophthalmologic image capturing apparatus 1 . The control unit 30 includes a CPU 31, a RAM 32, a ROM 33, and a nonvolatile memory (NVM) 34. The CPU 31 is a controller that performs various controls. The RAM 32 temporarily stores various information. The ROM 33 stores programs executed by the CPU 31 and various initial values. The NVM 34 is a non-transitory storage medium that can retain stored contents even if the power supply is cut off.

A monitor 37 and an operation section 38 are connected to the control unit 30. The monitor 37 is an example of a display unit that displays various images. The operation unit 38 is operated by the user in order for the user to input various operation instructions to the ophthalmologic image capturing apparatus 1 . Various devices such as a mouse, a keyboard, a touch panel, a foot switch, etc. can be used as the operation unit 38, for example. Note that various operation instructions may be input to the ophthalmologic image capturing apparatus 1 by inputting sound to the microphone.

The schematic configuration of the ophthalmologic image processing device 40 will be described. In this embodiment, a personal computer (hereinafter referred to as "PC") is used as the ophthalmological image processing apparatus 40. However, devices other than PCs may be used as the ophthalmological image processing apparatus. For example, the ophthalmologic image capturing device 1 itself may function as an ophthalmologic image processing device that executes ophthalmologic image processing to be described later. The ophthalmological image processing device 40 includes a CPU 41, a RAM 42, a ROM 43, and an NVM 44. The CPU 41 is a controller that performs various controls. The RAM 42, ROM 43, and NVM 44 can store various information as described above. An ophthalmologic image processing program for executing ophthalmologic image processing (see FIG. 4), which will be described later, may be stored in the NVM 44. Further, a monitor 47 and an operation unit 48 are connected to the ophthalmological image processing device 40. The monitor 47 is an example of a display unit that displays various images. The operation unit 48 is operated by the user in order for the user to input various operation instructions to the ophthalmological image processing apparatus 40. As with the operation section 38 of the ophthalmologic image capturing apparatus 1, various devices such as a mouse, a keyboard, and a touch panel can be used for the operation section 48. Further, various operation instructions may be input to the ophthalmological image processing device 40 by inputting sound to the microphone 46 .

The ophthalmologic image processing device 40 can acquire various data from the ophthalmologic image capture device 1 (for example, data of an ophthalmologic image captured by the ophthalmologic image capture device 1). The various data may be acquired by at least one of wired communication, wireless communication, and a removable storage device (eg, USB memory), for example.

With reference to FIGS. 2 and 3, a method of photographing an ophthalmologic image to be processed by the ophthalmologic image processing apparatus 40 of this embodiment and an example of the structure of the ophthalmologic image will be described. In this embodiment, a case will be exemplified in which two-dimensional tomographic image data is processed and stored in a storage device (for example, NVM 44, etc.). However, the techniques exemplified in this disclosure can also be applied to the case where three-dimensional tomographic image data is processed and stored in a storage device.

As shown in FIG. 2, the ophthalmological image capturing apparatus 1 of this embodiment scans a spot-shaped light (measuring light) on a tissue 50 of a living body (in the example shown in FIG. 2, fundus tissue). Specifically, the ophthalmological image capturing apparatus 1 of the present embodiment scans light on a scan line 52 extending in a predetermined direction on the tissue 50, thereby scanning the Z direction along the optical axis of the light and the Z direction intersecting the Z direction. An ophthalmologic image (two-dimensional tomographic image) 61 (see FIG. 3) that extends in the X direction (which intersects perpendicularly in this embodiment) is photographed.

As shown in FIG. 3, in this embodiment, the direction in which the spot-shaped measurement light scans the tissue (referred to as the "B-scan direction") is the X direction. Further, the depth direction of the tissue 50 along the optical axis of the measurement light (that is, the direction perpendicularly intersecting the X direction; referred to as the "A scan direction") is defined as the Z direction. The ophthalmological image 61 is a tomographic image of a layered structure of a plurality of layers of a living body's tissue. Specifically, the ophthalmological image 61 of this embodiment shows the layer structure of the fundus tissue of the eye to be examined. As described above, the ophthalmological image capturing apparatus 1 scans the measurement light that is irradiated from the front side onto the tissue of the fundus in the X direction, so that the measurement light can be scanned in the X direction (B scan direction) and in the Z direction (A scan direction). A two-dimensional ophthalmologic image 61 is captured. Therefore, all of the layers in the layered structure shown in the ophthalmological image 61 intersect in the Z direction (A-scan direction).

In the ophthalmological image 61 of this embodiment, a long A-scan image (that is, a pixel row extending in the Z direction along the optical axis of measurement light) extending in the Z direction (A-scan direction) is It is composed of multiple arrays arranged in a direction (crossing the direction). The plurality of A-scan images extend in a direction intersecting the layers of the layered structure shown in the ophthalmological image 61. Although details will be described later, in this embodiment, each of the plurality of A-scan images is treated as a partial image, which is a unit from which images to be saved are extracted.

Ophthalmological image processing in this embodiment will be described with reference to FIGS. 4 to 8. In this embodiment, the ophthalmological image processing device 40, which is a PC, acquires the data of the ophthalmological image 61 (hereinafter sometimes simply referred to as "the ophthalmological image 61") from the ophthalmological image capturing device 1, and the acquired ophthalmological image 61. The data is processed and stored in the storage device. However, as mentioned above, other devices may function as the ophthalmic image processing apparatus. For example, the ophthalmologic image capturing apparatus 1 itself may perform ophthalmologic image processing. Further, a plurality of control units (for example, the CPU 31 of the ophthalmologic image capturing device 1 and the CPU 41 of the ophthalmologic image processing device 40) may cooperate to execute the ophthalmologic image processing. The CPU 41 of the ophthalmology image processing device 40 executes the ophthalmology image processing shown in FIG. 4 according to the ophthalmology image processing program stored in the NVM 44.

First, the CPU 41 acquires an ophthalmologic image 61 of the tissue of the subject's eye (in this embodiment, the retinal tissue of the fundus) (S1). As illustrated in FIGS. 3 and 5, the ophthalmological image 61 of this embodiment shows the layer structure of the retinal tissue of the fundus of the eye.

The CPU 41 extracts in advance a part of the region (image region) in which the tissue image is captured from the ophthalmological image 61 acquired in S1 (S2). As shown in FIG. 5, the ophthalmological image capturing apparatus 1 according to the present embodiment can capture images at a high depth even deeper than the end of the deepest layer of the retinal tissue (the lower end of the image in the ophthalmological image 61 of FIG. 5). Tissues can be photographed. However, when using the ophthalmological image 61 for diagnosis or the like, it is often sufficient to be able to observe the retinal tissue. Therefore, in the present embodiment, the CPU 41 identifies and detects an image area in which an image of the tissue is shown (the area shown within the dotted line frame in the ophthalmological image 61 in FIG. 5) in the ophthalmological image 61 acquired in S1. An ophthalmologic image 62 including an image region is extracted in advance from the ophthalmologic image 61 acquired in S1. As a result, the amount of processing of various processes to be executed thereafter (for example, the process of specifying the reference region in S3) and the amount of image data to be processed are reduced.

The method of specifying the image area within the ophthalmologic image 61 can be selected as appropriate. For example, the CPU 41 may specify the image area by performing known image processing on the ophthalmological image 61. Additionally, a mathematical model previously trained by a machine learning algorithm to identify image regions within the ophthalmological image 61 may be used. In this case, the CPU 41 may input the ophthalmological image 61 into the mathematical model to obtain the identification result of the image area output by the mathematical model. Note that it is also possible to omit the process of S2.

Next, the CPU 41 identifies at least one site as a reference site among the plurality of sites within the tissue captured in the ophthalmological image 61 acquired in S1 (S3). Although details will be described later, the reference region is used to extract images within the storage target range from the ophthalmological image 61 acquired in S1 (in this embodiment, the ophthalmological image 62 from which the image area has been extracted from the ophthalmological image 61). becomes the standard.

As shown in FIG. 6, the ophthalmological image 62 shows a layer structure in which a plurality of layers are laminated among the tissues of the eye to be examined. In S3 of the present embodiment, at least one of a plurality of layers included in the layer structure shown in the ophthalmologic image 62 and a layer boundary is specified as the reference site RS. Each layer/boundary included in the layered structure tends to have a shape that follows the overall shape of the layered structure. Therefore, even if the layer structure is not flat (for example, if the layer structure is curved), the layer/boundary is used as the reference region RS to extract the extraction image in S6 and S7, which will be described later. Images in an appropriate range depending on the shape of the structure can be easily saved.

Specifically, in S3 of this embodiment, the boundary or layer (the retinal surface layer in this embodiment) on the most superficial side (the upper end in FIG. 6) of the layer structure shown in the ophthalmological image 62 is identified as the reference site RS. Ru. The most superficial side of the layered structure is the boundary between the area where the layered structure exists and the area where the layered structure does not exist. Therefore, by setting the layer/boundary on the most superficial side of the layer structure as the reference region RS, it becomes easier to extract an image including the layer structure more appropriately.

Note that in S3 of the present embodiment, the CPU 41 specifies the reference site RS in the ophthalmological image 62 by performing known image processing (for example, edge detection, etc.) on the ophthalmological image 62. However, it is also possible to change the method of specifying the reference site RS. For example, the CPU 41 may input the ophthalmological image 62 to a mathematical model trained by a machine learning algorithm, thereby obtaining the identification result of the reference site RS output by the mathematical model.

Next, when there is a part in the ophthalmological image 62 where the identification result of the reference part RS is missing, the CPU 41 performs the identification based on the identification result of the reference part RS of another part (the part where the identification result was normally acquired). , complement the identification results of the reference region RS at the missing location (S4). As a result, even if a part of the reference region RS is not specified in S3, images within the storage range are appropriately extracted using the reference region RS as a reference in S6 and S7, which will be described later. As an example, in S4 of the present embodiment, the identification result of the missing part is based on the identification result of the reference region RS in a part adjacent to the missing part (for example, an A-scan image adjacent to an A-scan image that includes the missing part). complement. Specifically, the CPU 41 determines the specific position of the A-scan image adjacent to one side of the A-scan image (which may be a plurality of consecutive A-scan images) including the missing portion, and the A-scan image adjacent to the other side. The position between the specific positions is complemented as the position of the reference part RS in the missing part. However, it is also possible to change the interpolation method.

Next, the CPU 41 selects the extraction target range of the Nth partial image 70 (the initial value of N is "1") from among the plurality of partial images 70 arranged in a predetermined direction within the ophthalmological image 62, based on the reference region RS. (S6). As shown in FIG. 6, the plurality of partial images 70 are consecutively arranged in a predetermined direction within the ophthalmological image 62. In the present embodiment, each of the plurality of partial images 70 is an elongated image extending in a direction (in the present embodiment, the Z direction) intersecting the layers in the layered structure shown in the ophthalmological image 62. They are continuously arranged in a direction (X direction in this embodiment) that intersects with the extending direction. Therefore, the layer/boundary (in this embodiment, the layer/boundary closest to the surface layer) that is the reference region can be easily identified within each partial image 70. As described above, in this embodiment, the plurality of partial images 70 are A-scan images extending in the direction along the optical axis of the OCT measurement light.

An example of a method for specifying the extraction target range of each partial image 70 based on the reference region RS will be described in detail. In the present embodiment, the CPU 41 moves from the reference region RS on the most superficial side in the layered structure of the ophthalmological image 62 toward the deeper side of the layered structure (in the example shown in FIG. 6, the lower side of the drawing which is the +Z direction). Include the range in the extraction target range. In other words, the CPU 41 determines the extraction target range in the partial image 70 by adding the offset amount (+offset1) toward the deeper side (+Z direction) to the Z coordinate of the reference region RS in the partial image 70 extending in the Z direction. It is specified as the deep end DE. As a result, the range from the reference site RS on the most superficial side to the deepest layer of the layered structure is subsequently extracted, so that it is easier to extract as wide a range as possible of the layered structure.

Further, the CPU 41 moves from the reference site RS on the most superficial side in the layer structure of the ophthalmological image 62 to the near side, which is the side opposite to the deep layer side of the layer structure (in the example shown in FIG. 6, the upper side of the drawing, which is the -Z direction). A predetermined range is included in the extraction target range. In other words, the CPU 41 determines, in the partial image 70 extending in the Z direction, a position obtained by adding an offset amount (-offset2) toward the front side (+Z direction) to the Z coordinate of the reference region RS, as the extraction target in the partial image 70. It is specified as the near end SE of the range. As a result, for example, even if the identification accuracy of the reference site RS on the most superficial side is reduced, it becomes easier to appropriately extract an image of the tissue on the superficial side of the layered structure. Further, even if some object is reflected on the surface side rather than the most superficial layer/boundary due to various influences (for example, the influence of a disease, etc.), the object reflected on the surface side is also likely to be extracted.

In the present embodiment, the CPU 41 sets the offset amount toward the deep side (+offset1) and the offset amount toward the near side (-offset2) according to instructions input by the user. Therefore, the user can efficiently and appropriately save image data of a desired range of the layer structure of the eye to be examined. Further, the CPU 41 may adjust at least one of the two offset amounts depending on the photographing conditions of the ophthalmological image 61 (for example, at least one of photographing magnification, photographing angle of view, photographing depth, etc.). In this case, image data in an appropriate range according to the shooting conditions is extracted. Furthermore, at least one of the two offset amounts may be a predetermined fixed value.

Next, the CPU 41 extracts the data of the image (extracted image 80) within the extraction target range from the Nth partial image 70 (S7). As shown in FIG. 6, the extracted image 80 extracted from each partial image 70 tends to include an appropriate range based on the reference region RS, and tends to exclude unnecessary ranges.

FIG. 7 shows a reference image 80A in which a plurality of extracted images 80 extracted from each of a plurality of partial images 70 are arranged in the X direction (that is, the arrangement direction of the plurality of partial images 70) without considering the respective positions. shows. As shown in FIG. 7, the reference image 80A, which is a set of a plurality of extracted images 80, includes an appropriate range of tissue (layered structure in this embodiment) based on the specified reference site RS. Furthermore, it can be seen that the amount of image data is smaller than the ophthalmological images 61 and 62. That is, in the reference image 80A, at least a portion of the area other than the layer structure is erased from the ophthalmological images 61 and 62, and the curvature of the layer structure is flattened. By erasing at least part of the area other than the layer structure, the amount of image data is appropriately reduced, and the layer structure is made flat by removing the layer curvature information reproduced by position information. . In the reference image 80A shown in FIG. 7, the background portion (white portion in FIG. 8) in the reconstructed image data 90 shown in FIG. 8 (details will be described later) is deleted from the ophthalmological image 62 shown in FIG. , the curvature is flattened.

Furthermore, the extracted image 80 of this embodiment is extracted from the ophthalmological image 62 with the thickness of the layer structure maintained. Therefore, although details will be described later, by reconstructing each extracted image 80 based on position information, the layer structure in the original ophthalmological image 62 is appropriately reconstructed while maintaining the thickness.

The CPU 41 acquires position information indicating the position of the Nth extracted image 80 (that is, the extracted image 80 extracted from the Nth partial image 70) (S8). Various types of information can be used as location information. For example, the coordinate information of at least one of the position of the reference region RS, the position of the deep side end DE of the extraction target range, and the position of the near side end SE in the Nth partial image 70 is It may also be acquired as location information.

The CPU 41 stores the data of the extracted image 80 extracted from the N-th partial image 70 and the position information indicating the position of the N-th extracted image 80 in the storage means (for example, NVM 44, etc.) (S9). Note that the CPU 41 stores the data of the extracted image 80 in the storage means without compressing the data, or after performing reversible compression that can be reproduced after decoding the image including the layered structure. Next, the CPU 41 determines whether processing has been completed for all partial images 70 within the ophthalmologic image 62 (S10). If the processing for some of the partial images 70 has not yet been completed (S10: NO), "1" is added to the counter N indicating the order of the partial images 70 (S11), and the process returns to S6. -10 processes are repeated. When the processing for all partial images 70 is completed, the processing for saving the extracted image 80 and position information ends.

Referring to FIG. 8, an image reconstruction process for reconstructing an image based on the saved extracted image 80 and position information will be described. The CPU 41 of the ophthalmological image processing device 40 executes image reconstruction processing according to the ophthalmological image processing program stored in the NVM 44.

When data for which an image is to be reconstructed is specified by the user, the CPU 41 selects the data specified by the user (data of the plurality of extracted images 80 and each position information of the extracted image 80). The CPU 41 generates reconstructed image data 90 in which the image of the extracted area is reconstructed by arranging the data of the plurality of extracted images 80 based on the respective position information. As shown in FIG. 8, in the reconstructed image data 90, the regions extracted from the ophthalmological images 61 and 62 are appropriately reconstructed.

Note that the reconstructed image data 90 lacks information on areas that were not extracted as the extracted image 80. The CPU 41 can also generate background supplemented image data 91 by supplementing the reconstructed image data 90 with pixel information of a background region where information is missing. As shown in FIG. 8, the background supplemented image data 91 is an image that is even closer to the actually photographed ophthalmological images 61 and 62 by supplementing black pixel information as the background.

The techniques disclosed in the above embodiments are merely examples. Therefore, it is also possible to modify the techniques exemplified in the above embodiments. First, it is also possible to omit some of the plurality of processes exemplified in the above embodiment. For example, the process (S2) of extracting the image area in advance may be omitted. Furthermore, the process (SS4) of supplementing the missing portions of the identification results of the reference region RS can also be omitted.

In the embodiment described above, of the layer structure shown in the ophthalmological image 62, the layer/boundary on the most superficial side is set as the reference site RS. However, it is also possible to change the reference site. For example, if the tissue shown in the ophthalmological image 62 is the retinal tissue of the fundus, layers/boundaries located deeper than the most superficial retinal surface layer (for example, ganglion cell layer, inner plexiform layer, inner nuclear layer, outer nuclear layer, At least one of the following (reticular layer, external limiting membrane, photoreceptor inner segment/outer segment junction, retinal pigment epithelium, Bruch's membrane, choroid, etc., and their boundaries) may be used as the reference site. For example, among the layer structures that appear in the ophthalmological image 62, by setting the layer/structure that is more clearly visible as the reference region, the accuracy of identifying the reference region may be improved, and the image may be extracted and saved more appropriately. be.

Further, among the layer structures shown in the ophthalmological image 62, the layers and boundaries specified as reference parts may be changed according to various conditions. For example, there are layers and boundaries whose visibility in ophthalmological images changes depending on the position of the tissue. Therefore, the layer/boundary specified as the reference site may be changed depending on the position of the tissue shown in the ophthalmological image.

Further, the CPU 41 may specify a plurality of layers/boundaries shown in the ophthalmological image 62 as reference parts. In this case, the CPU 41 selects one reference part (for example, the layer/boundary on the most superficial side) and another reference part (for example, the layer/boundary located on the deeper side of the layer structure) among the plurality of reference parts identified. ) may be set as the extraction target range, and an extraction image may be extracted from each of the plurality of partial images. As a result, data of an image within a specific range of the layered structure shown in the ophthalmological image 62 is appropriately extracted and saved.

Moreover, it is also possible to use a part other than the layer/boundary as the reference part among the plurality of parts shown in the ophthalmological image. For example, a diseased site appearing in an ophthalmologic image may be specified as a reference site, and an extraction image may be extracted using the specified reference site as a reference. In this case, an extracted image is appropriately extracted and saved according to the disease site. Further, a blood vessel site or the like that appears in an ophthalmologic image may be specified as the reference site.

Note that the process of acquiring an ophthalmologic image in S1 in FIG. 4 is an example of an "image acquisition step." The process of specifying the reference part RS in S3 of FIG. 4 is an example of a "reference part specifying step." The process of extracting image data within the extraction target range from the partial image 70 in S6 and S7 of FIG. 4 is an example of an "extraction step." The process of acquiring the position information of the extracted image 80 in S8 of FIG. 4 is an example of a "position information acquisition step." The process of saving the data and position information of the extracted image 80 in S9 of FIG. 4 is an example of a "save step." The process of complementing the identification result of the reference region RS in S4 of FIG. 4 is an example of a "completion step." The process of reconstructing the image shown in FIG. 8 is an example of an "image reconstruction step."

1 Ophthalmology image capturing device 40 Ophthalmology image processing device 41 CPU
44 NVM
61, 62 Ophthalmological image 70 Partial image 80 Extracted image 90 Reconstructed image data 91 Background complemented image data RS Reference site

Claims (12)

An ophthalmological image processing device that processes data of an ophthalmological image that is a tomographic image of a tissue of an eye to be examined,
The control unit of the ophthalmological image processing device includes:
an image acquisition step of acquiring an ophthalmological image taken by an ophthalmological image capturing device;
an extraction step of extracting data of an image within a target range for storing data from each of a plurality of partial images arranged in a predetermined direction within the ophthalmological image;
a position information acquisition step of acquiring position information indicating the position of each of the plurality of extracted images extracted from each of the plurality of partial images in the extraction step;
a storing step of storing data of the plurality of extracted images and position information of each of the plurality of extracted images in a storage means;
An ophthalmological image processing device characterized by performing the following. The ophthalmological image processing device according to claim 1,
The control unit includes:
further performing a reference site specifying step of specifying at least one site among the plurality of sites within the tissue shown in the ophthalmological image as a reference site;
In the extraction step, data of an image within a target range with the reference site as a reference is extracted as the extraction image from each of the plurality of partial images arranged in a predetermined direction within the ophthalmological image. Features of the ophthalmological image processing device. The ophthalmological image processing device according to claim 2,
The ophthalmological image is a tomographic image obtained by photographing a layer structure in which a plurality of layers are stacked among tissues of the eye to be examined;
The ophthalmologic image processing apparatus is characterized in that, in the reference site specifying step, at least one of a plurality of layers included in the layer structure and a boundary between layers is specified as the reference site. The ophthalmological image processing device according to claim 3,
The ophthalmologic image processing apparatus is characterized in that, in the reference region specifying step, at least a boundary or layer on the most superficial side of the layer structure shown in the ophthalmologic image is identified as the reference region. The ophthalmological image processing device according to claim 4,
In the extraction step, a predetermined range from the reference region on the most superficial side in the layered structure toward the deeper side of the layered structure is included in the target range from which the extracted image is extracted from each of the plurality of partial images. An ophthalmological image processing device characterized by: The ophthalmological image processing device according to claim 4 or 5,
In the extraction step, the extraction image is extracted from each of the plurality of partial images in a predetermined range from the reference region on the most superficial side of the layered structure toward the side opposite to the deeper side of the layered structure. An ophthalmological image processing device characterized by being included in a target range. The ophthalmological image processing device according to any one of claims 3 to 6,
Each of the plurality of partial images is a long image extending in a direction intersecting the layers in the layered structure shown in the ophthalmological image, and each of the plurality of partial images extends in the direction in which each of the partial images extends within the ophthalmological image. An ophthalmological image processing apparatus characterized in that the apparatus is arranged continuously in a direction intersecting with the ophthalmological image processing apparatus. The ophthalmological image processing device according to any one of claims 2 to 7,
As a result of the execution of the reference part identifying step, if there is a part where the identification result of the reference part is missing, the reference part at the missing part is determined based on the identification result of the reference part in other parts. An ophthalmological image processing device characterized by further executing a complementing step of complementing a specific result. The fundus image processing device according to any one of claims 1 to 8,
The ophthalmological image is a tomographic image obtained by photographing a layer structure in which a plurality of layers are stacked among tissues of the eye to be examined;
An image in which the plurality of extraction images extracted in the extraction step are arranged in the arrangement direction of the plurality of partial images without considering the position information is an image in which at least one of the regions other than the layer structure is extracted from the ophthalmological image. A fundus image processing device characterized in that an image is obtained in which the curvature of the layer structure is flattened and the curvature of the layer structure is flattened. The fundus image processing device according to any one of claims 1 to 9,
The ophthalmological image is a tomographic image obtained by photographing a layer structure in which a plurality of layers are stacked among tissues of the eye to be examined;
The fundus image processing device is characterized in that the extracted image is extracted from the ophthalmological image and stored in a state where the thickness of the layer structure in the ophthalmological image is maintained. The ophthalmological image processing device according to any one of claims 1 to 10,
image reconstruction of reconstructing the area extracted in the extraction step of the ophthalmological image by arranging data of the plurality of extracted images stored in the storage means based on the respective position information; An ophthalmological image processing device further comprising the steps of: An ophthalmologic image processing program executed by an ophthalmologic image processing device that processes data of an ophthalmologic image that is a tomographic image of a tissue of an eye to be examined,
The ophthalmology image processing program is executed by the control unit of the ophthalmology image processing device,
an image acquisition step of acquiring an ophthalmological image taken by an ophthalmological image capturing device;
an extraction step of extracting data of an image within a target range for storing data from each of a plurality of partial images arranged in a predetermined direction within the ophthalmological image;
a position information acquisition step of acquiring position information indicating the position of each of the plurality of extracted images extracted from each of the plurality of partial images in the extraction step;
a storing step of storing data of the plurality of extracted images and position information of each of the plurality of extracted images in a storage means;
An ophthalmologic image processing program that causes the ophthalmologic image processing apparatus to execute the following.

Images