WO2022209574A1 - 医療画像処理装置、医療画像処理プログラム、および医療画像処理方法 - Google Patents

医療画像処理装置、医療画像処理プログラム、および医療画像処理方法 Download PDF

Info

Publication number
WO2022209574A1
WO2022209574A1 PCT/JP2022/009329 JP2022009329W WO2022209574A1 WO 2022209574 A1 WO2022209574 A1 WO 2022209574A1 JP 2022009329 W JP2022009329 W JP 2022009329W WO 2022209574 A1 WO2022209574 A1 WO 2022209574A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
region
image processing
dimensional
medical image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/009329
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
涼介 柴
佳紀 熊谷
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nidek Co Ltd
Original Assignee
Nidek Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nidek Co Ltd filed Critical Nidek Co Ltd
Priority to JP2023510719A priority Critical patent/JP7439990B2/ja
Publication of WO2022209574A1 publication Critical patent/WO2022209574A1/ja
Priority to US18/477,067 priority patent/US20240020839A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/761—Proximity, similarity or dissimilarity measures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—Three-dimensional [3D] image rendering
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/24—Aligning, centring, orientation detection or correction of the image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10081—Computed x-ray tomography [CT]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10088—Magnetic resonance imaging [MRI]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10101—Optical tomography; Optical coherence tomography [OCT]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20076—Probabilistic image processing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30024—Cell structures in vitro; Tissue sections in vitro
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/03—Recognition of patterns in medical or anatomical images

Definitions

  • the present disclosure relates to a medical image processing apparatus that processes image data of living tissue, a medical image processing program executed in the medical image processing apparatus, and a medical image processing method.
  • tissue structures for example, at least one of layers, boundaries between multiple layers, and specific sites in tissues, etc.
  • tissue structures for example, at least one of layers, boundaries between multiple layers, and specific sites in tissues, etc.
  • Non-Patent Document 1 a GPU, which has higher computational power than a CPU, is used to perform segmentation of a fundus layer using a neural network, thereby shortening the processing time.
  • high performance controller a controller such as a GPU with high computing power
  • high performance controllers are expensive. Therefore, it would be very useful if the amount of computation could be reduced while maintaining high detection accuracy when detecting tissue structures from a three-dimensional image.
  • a typical object of the present disclosure is a medical image processing apparatus, a medical image processing program, and a medical image that can reduce the amount of calculation while maintaining high detection accuracy when detecting tissue structures from three-dimensional images. It is to provide a processing method.
  • a medical image processing apparatus provided by a typical embodiment of the present disclosure is a medical image processing apparatus that processes data of a three-dimensional image of a biological tissue, wherein a control unit of the medical image processing apparatus comprises a three-dimensional image of a tissue.
  • a detection result of the specific structure in the first region is obtained by inputting the first region extracted in the extraction step into a mathematical model that outputs a detection result of the specific structure of the captured tissue. 1 structure detection step;
  • a medical image processing program provided by a typical embodiment of the present disclosure is a medical image processing program executed by a medical image processing apparatus that processes data of a three-dimensional image of living tissue, wherein the medical image processing program is An image acquisition step of acquiring a three-dimensional image of tissue, and an extraction of extracting a first region, which is a partial region, from the acquired three-dimensional image by being executed by the control unit of the medical image processing apparatus and inputting the first region extracted in the extracting step into a mathematical model that has been trained by a machine learning algorithm and outputs detection results of specific structures of the tissue in the input image. and a first structure detection step of obtaining a detection result of the specific structure in the first region.
  • a medical image processing method provided by a typical embodiment of the present disclosure is a medical image processing method executed in a medical image processing system for processing three-dimensional image data of living tissue, wherein the medical image processing system , a first image processing device and a second image processing device connected to each other via a network, wherein the first image processing device acquires a three-dimensional image of tissue; an extracting step of extracting a first region, which is a partial region, from the three-dimensional image; and said second image processing device is trained by a machine learning algorithm and outputs a detection result of a particular structure of said tissue in an input image to said first mathematical model.
  • a first structure detection step of obtaining a detection result of the specific structure in the first region by inputting the region.
  • the distortion of the image of the living tissue generated by scanning with light is appropriately corrected.
  • the medical image processing apparatus exemplified in the present disclosure processes data of three-dimensional images of living tissue.
  • a control unit of the medical image processing apparatus executes an image acquisition step, an extraction step, and a first structure detection step.
  • the controller acquires a three-dimensional image of the tissue.
  • the control unit extracts a first area, which is a partial area, from the three-dimensional image acquired in the image acquiring step.
  • the control unit inputs the first region to the mathematical model to acquire the detection result of the specific structure in the first region.
  • the mathematical model is trained by a machine learning algorithm and outputs detection results of specific structures of tissue in an input image (in this embodiment, an image of a first region extracted from a three-dimensional image).
  • a first region which is a partial region, is extracted from the entire three-dimensional image, and specific structure detection processing is performed on the extracted first region using a mathematical model.
  • the computational complexity of the processing using the machine learning algorithm is appropriately reduced compared to the case where the processing by the mathematical model is executed for the entire three-dimensional image.
  • the structure detection processing executed by the mathematical model for the first region may also be referred to as "first structure detection processing”.
  • the structure of the tissue to be detected from the image can be selected as appropriate.
  • the structures to be detected include the layers of the fundus tissue of the eye to be examined, the boundaries of the layers of the fundus tissue, the optic papilla existing in the fundus, the layers of the anterior ocular tissue, and the anterior ocular tissue. It may be at least one of the boundary of the layer of the tissue and the diseased part of the eye to be examined.
  • various devices can be used as an imaging device that captures (generates) a three-dimensional image.
  • an OCT apparatus that captures a tomographic image of tissue using the principle of optical coherence tomography can be used.
  • Imaging methods using an OCT apparatus include, for example, a method of acquiring a three-dimensional tomographic image by two-dimensionally scanning a spot of light (measurement light), or a method of acquiring a three-dimensional tomographic image by scanning light extending in a one-dimensional direction. (so-called line field OCT) or the like can be adopted.
  • an MRI (Magnetic Resonance Imaging) device, a CT (Computer Tomography) device, or the like may be used.
  • the control unit may further execute a second structure detection step.
  • the controller outputs a specific structure in the second region, which was not extracted as the first region in the extraction step, out of the entire region of the three-dimensional image, to the first region by the mathematical model. detection based on the detection result of the specific structure obtained.
  • the structure in the second region is also detected in addition to the structure in the first region, so the specific structure in the three-dimensional image is detected with higher accuracy. Further, detection results output by the mathematical model for the first region are used to detect specific structures in the second region. Therefore, the amount of calculation for the structure detection process in the second area is less than the amount of calculation for the structure detection process in the first area. Therefore, the structure detection process is executed for each of the first area and the second area while suppressing an increase in the amount of computation for the process.
  • the structure detection processing in the second region which is executed based on the detection result of the first structure detection processing, may be referred to as "second structure detection processing".
  • a specific method for executing the second structure detection step can be selected as appropriate.
  • the control unit compares the structure detection result and pixel information (e.g., luminance value, etc.) for each pixel forming the first region with the pixel information for each pixel forming the second region.
  • a structure detection result in the region may be calculated.
  • the positional relationship between each pixel forming the second region and each pixel forming the first region to be referred to (for example, the first region closest to the target second region, etc.) may be considered.
  • the detection result and pixel information regarding the pixels within the n-th closest distance from the pixel of interest in the second region may be compared with the pixel information of the pixel of interest.
  • the control unit may calculate the structure detection result for the pixel of interest in the second region by interpolation processing based on the structure detection result for the pixels in the first region located around the pixel of interest.
  • control unit may extract the first region from each of the plurality of two-dimensional images forming the three-dimensional image.
  • the amount of calculation for processing is appropriately reduced compared to the case where structure detection processing is performed on the entire region of each two-dimensional image using a mathematical model.
  • the control unit classifies each of the plurality of pixel strings forming the two-dimensional image into one of the plurality of groups based on the degree of similarity, and selects the pixel string representing each of the plurality of groups as the first group. It may be extracted as a region.
  • the control unit may input the pixel string extracted as the first region in the extraction step into the mathematical model. In this case, even if a large number of pixel rows are classified into one group, the structure detection processing using the mathematical model is performed on one or a small number of pixel rows representing the group. Therefore, the amount of computation for processing using the mathematical model is appropriately reduced.
  • the direction in which the pixel row extends may be defined as appropriate.
  • a row of pixels in a direction along the optical axis of OCT light among a plurality of two-dimensional images forming the three-dimensional image may be called an A-scan image.
  • each of the plurality of A-scan images forming the two-dimensional image may be classified into one of the plurality of groups.
  • each of the plurality of pixel columns that intersect perpendicularly with the A-scan image may be classified into one of the plurality of groups.
  • the control unit detects a specific structure in a pixel row (that is, a second region) that has not been detected as the first region from each group, and detects the structure detection result by the mathematical model for the first region of the same group. may be detected based on As described above, a plurality of pixel columns classified into the same group have a high degree of similarity. Therefore, by executing the first structure detection process and the second structure detection process for each group, the accuracy of the second structure detection process is further improved.
  • a specific method for extracting a pixel row representing each of the plurality of groups as the first region can be selected as appropriate.
  • the control unit may extract, as the first region, a pixel row obtained by performing averaging processing on a plurality of pixel rows classified into each group.
  • the control unit may extract the first region according to a predetermined rule or randomly from the plurality of pixel columns belonging to each group.
  • the number of first regions extracted from each group may be one or plural (however, the number is smaller than the number of pixel columns belonging to the group).
  • the method of detecting structures in units of a plurality of pixel strings forming a two-dimensional image is not limited to the method of classifying pixel strings into a plurality of groups.
  • the control unit may extract pixel rows as the first region at regular intervals from a plurality of pixel rows forming the two-dimensional image.
  • the control unit extracts the first region from the plurality of pixel rows arranged in the first direction, and extracts the first region from the plurality of pixel rows arranged in the second direction perpendicular to the first direction. Processing may be performed jointly.
  • a three-dimensional image may be configured by arranging a plurality of two-dimensional images in order in a direction intersecting the image area of each two-dimensional image.
  • the control unit may extract, as the first area, a rectangular image area showing the tissue from each of the plurality of two-dimensional images. In this case, regions without tissue are excluded from the regions targeted for detection of specific tissue by the mathematical model. Therefore, the amount of computation for processing using the mathematical model is appropriately reduced.
  • the control unit may detect the image area of the reference image by inputting a part of the reference image from the plurality of two-dimensional images into the mathematical model.
  • the control unit may extract the image area of the two-dimensional image other than the reference image from among the plurality of two-dimensional images as the first area based on the detection result of the image area of the reference image.
  • the image regions of the reference image are detected with high accuracy by means of the mathematical model.
  • the image area of the two-dimensional image other than the reference image is detected with a small amount of calculation based on the detection result of the image area of the reference image. Therefore, the image area is detected more appropriately.
  • the method of extracting the image area of the other two-dimensional image based on the detection result of the image area of the reference image can be appropriately selected.
  • the control unit compares the image area detection result and pixel information for each pixel that constitutes the reference image with the pixel information for each pixel that constitutes another two-dimensional image, thereby determining the other two-dimensional image.
  • Image regions may be extracted.
  • the positional relationship between each pixel forming the reference image and each pixel forming another two-dimensional image may be considered.
  • control unit may extract the image area based on pixel information of each pixel forming the two-dimensional image.
  • control unit may detect, as the image area, an area in which the brightness of the pixels is equal to or higher than a threshold value in the image area of the two-dimensional image.
  • the control unit may further execute a two-dimensional image alignment step of performing image alignment between a plurality of pixel columns forming each two-dimensional image.
  • the rectangular first regions registered and extracted in the two-dimensional image registration and extraction steps may be input to the mathematical model.
  • the image can be properly fit within the rectangular first region, and the size of the rectangular first region tends to be reduced. Therefore, the structure is appropriately detected with a small amount of calculation.
  • either the two-dimensional image alignment step or the extraction step may be executed first. That is, a rectangular image area may be extracted as the first area after image alignment is performed between a plurality of pixel columns. Further, the shape of the first area may be adjusted to be rectangular by aligning the images between the plurality of pixel rows after the image area is detected as the first area.
  • the control unit may further execute a two-dimensional image alignment step of performing image alignment between a plurality of two-dimensional images.
  • the processing is made more efficient in various respects. For example, when detecting a specific structure in one two-dimensional image (second region) based on the result of the first structure detection processing for another two-dimensional image (first region), the control unit detects a plurality of If the images are aligned between the two-dimensional images, the structure in the second region can be detected more appropriately by comparing the pixels whose coordinates are similar between the two-dimensional images. .
  • either the two-dimensional image registration step or the extraction step may be executed first.
  • either the intra-two-dimensional image registration step or the inter-two-dimensional image registration step may be executed first.
  • control unit may extract a part of the plurality of two-dimensional images included in the three-dimensional image as the first region.
  • the amount of computation for processing is appropriately reduced compared to the case where structure detection processing is performed using a mathematical model for all two-dimensional images that make up a three-dimensional image.
  • the control unit may execute the extraction step and the first structure detection step by using a part of the reference image as the first region from among the plurality of two-dimensional images included in the three-dimensional image. After that, the control unit may execute the extraction step and the first structure detection step using, among the plurality of two-dimensional images, a two-dimensional image having a degree of similarity with the reference image that is less than a threshold value as the first region. The control unit may repeatedly execute the above processing.
  • a 2D image as a first region at regular intervals from a plurality of 2D images that form a 3D image.
  • the first regions are only extracted at regular intervals even in portions where the structure changes drastically.
  • the detection accuracy of the structure may be degraded.
  • the first region is extracted using the degree of similarity with the reference image on which the first structure detection process has been performed, the first regions are densely extracted in portions where the structure changes significantly. Therefore, the structure detection accuracy is improved.
  • the method of extracting the first region in units of two-dimensional images is not limited to the method of extracting using the degree of similarity with the reference image.
  • the control unit may extract a two-dimensional image as the first region at regular intervals from a plurality of two-dimensional images forming the three-dimensional image.
  • the control unit may set the attention position within the image area of the three-dimensional image.
  • the control unit may set extraction patterns for a plurality of two-dimensional images with reference to the set attention position.
  • the control unit may extract a plurality of two-dimensional images that match the set extraction pattern from the three-dimensional image as the first regions. In this case, a plurality of two-dimensional images are extracted as the first region according to the extraction pattern with reference to the position of interest. Therefore, a specific structure is detected from the three-dimensional image in an appropriate manner corresponding to the site of interest.
  • the control unit may set the attention position within the image area of the 3D image according to the instruction input by the user. In this case, the first area is appropriately extracted based on the position where the user pays attention.
  • the control unit may detect a specific site (for example, a site having a specific structure, a site having a disease, etc.) in the three-dimensional image, and set the detected specific site as the position of interest. . In this case, the control unit may detect the specific part by known image processing or the like. Mathematical models may also be used to detect specific sites.
  • extraction patterns for multiple two-dimensional images can be selected as appropriate. For example, even if the extraction pattern is set so that when the three-dimensional image is viewed from the direction along the image capturing optical axis, the lines crossed by the extracted two-dimensional image are radial with the position of interest as the center. good. Also, the extraction pattern may be set such that the closer to the position of interest, the denser the two-dimensional image extracted as the first region.
  • control unit changes the extraction method of the first region according to the imaging conditions (for example, at least one of the imaging region, imaging method, imaging angle of view, etc.) when the three-dimensional image is captured. good too.
  • control unit may change the method of extracting the first region according to the processing capability of the control unit of the medical image processing apparatus.
  • a medical image processing method exemplified in the present disclosure is executed in a medical image processing system that processes three-dimensional image data of living tissue.
  • a medical image processing system includes a first image processing device and a second image processing device that are connected to each other via a network.
  • a medical image processing method includes an image acquisition step, an extraction step, a transmission step, and a first structure detection step.
  • the image acquisition step the first image processing device acquires a three-dimensional image of the tissue.
  • the extracting step the first image processing device extracts a first area, which is a partial area, from the three-dimensional image.
  • the transmitting step the first image processing device transmits the first region extracted in the extracting step to the second image processing device.
  • the second image processing device inputs the first region into a mathematical model that has been trained by a machine learning algorithm and that outputs detection results of specific structures of tissue appearing in the input image. By doing so, the detection result of the specific structure in the first region is obtained.
  • the first image processing device and the second image processing device are connected via a network. If so, the various processes described above are properly executed.
  • the first image processing device may be at least one of a PC, a mobile terminal, a medical imaging device, and the like.
  • the first image processing apparatus may be placed in a facility for diagnosing or examining a subject.
  • the second image processing apparatus may be a server (for example, a cloud server or the like).
  • the second image processing device may execute an output step of outputting the structure detection result obtained by the first structure detection step to the first image processing device.
  • the first image processing device converts a specific structure in a second region, which was not extracted as the first region in the extraction step, out of the entire region of the three-dimensional image, to a specific structure output by the mathematical model for the first region.
  • a second structure detection step of performing detection based on the structure detection result may be further performed. In this case, each of the first structure detection process and the second structure detection process is appropriately performed within the medical image processing system.
  • FIG. 1 is a block diagram showing schematic configurations of a mathematical model construction device 1, a medical image processing device 21, and medical image capturing devices 11A and 11B;
  • FIG. 3 is a diagram showing an example of a training image 30 that is a two-dimensional tomographic image of the fundus. 3 is a diagram showing an example of output data 31 indicating a specific structure of tissue appearing in the training image 30 shown in FIG. 2;
  • FIG. 11 is an explanatory diagram for explaining a method for the medical image capturing apparatus 11B to capture a three-dimensional image of the tissue 50 of the living body;
  • FIG. 6 is an explanatory diagram for explaining a state in which a three-dimensional image is composed of a plurality of two-dimensional images 61; 4 is a flowchart of first detection processing executed by the medical image processing apparatus 21; 6 is an explanatory diagram for explaining a state in which a plurality of A-scan images in a two-dimensional image 61 are classified into a plurality of groups; FIG. 4 is a flowchart of second detection processing executed by the medical image processing apparatus 21.
  • FIG. FIG. 6 is an explanatory diagram for explaining an example of a method of extracting an image area of a two-dimensional image 61B based on an image area of a reference image 61A; 4 is a flowchart of third detection processing executed by the medical image processing apparatus 21; FIG.
  • FIG. 4 is a diagram comparing two-dimensional images before and after alignment is performed; 4 is a flowchart of fourth detection processing executed by the medical image processing apparatus 21. FIG. It is a reference diagram for explaining the fourth detection process. 10 is a flowchart of fifth detection processing executed by the medical image processing apparatus 21; FIG. 7 is a diagram showing an example of a state in which a target position 73 and an extraction pattern 75 are set in a three-dimensional image; FIG. 11 is a block diagram showing a schematic configuration of a medical image processing system 100 according to a modified example;
  • a mathematical model construction device 1 a medical image processing device 21, and medical image capturing devices 11A and 11B are used.
  • the mathematical model building device 1 builds a mathematical model by training the mathematical model with a machine learning algorithm. Based on the input image, the constructed mathematical model outputs the detection result of the specific structure of the tissue in the image.
  • the medical image processor 21 uses a mathematical model to detect specific structures in the imaged tissue.
  • the medical image capturing apparatuses 11A and 11B capture images of living tissue (in this embodiment, fundus tissue of an eye to be examined).
  • a personal computer (hereinafter referred to as "PC") is used for the mathematical model construction device 1 of this embodiment.
  • the mathematical model construction device 1 utilizes an image (hereinafter referred to as “input data”) acquired from the medical imaging device 11A and output data indicating a specific tissue structure in the input data. Building a mathematical model by training a mathematical model.
  • a device that can function as the mathematical model construction device 1 is not limited to a PC.
  • the medical imaging device 11A may function as the mathematical model construction device 1.
  • the control units of a plurality of devices for example, the CPU of the PC and the CPU 13A of the medical imaging apparatus 11A may work together to construct the mathematical model.
  • a PC is used for the medical image processing apparatus 21 of this embodiment.
  • devices that can function as the medical image processing apparatus 21 are not limited to PCs either.
  • the medical image capturing device 11B, a server, or the like may function as the medical image processing device 21 .
  • the medical image capturing device (OCT device in this embodiment) 11B functions as the medical image processing device 21 .
  • the medical image capturing device 11B captures a three-dimensional image of a biological tissue, and the tissue of the captured three-dimensional image. Specific structures can be detected well.
  • a portable terminal such as a tablet terminal or a smartphone may function as the medical image processing apparatus 21 .
  • the controllers of a plurality of devices (for example, the CPU of the PC and the CPU 13B of the medical imaging apparatus 11B) may work together to perform various processes.
  • the mathematical model construction device 1 will be explained.
  • the mathematical model construction device 1 is installed, for example, in the medical image processing device 21 or a manufacturer that provides users with a medical image processing program.
  • the mathematical model construction device 1 includes a control unit 2 that performs various control processes, and a communication I/F 5 .
  • the control unit 2 includes a CPU 3, which is a controller for control, and a storage device 4 capable of storing programs, data, and the like.
  • the storage device 4 stores a mathematical model building program for executing a later-described mathematical model building process.
  • the communication I/F 5 connects the mathematical model construction device 1 with other devices (for example, the medical image capturing device 11A, the medical image processing device 21, etc.).
  • the mathematical model construction device 1 is connected to the operation unit 7 and the display device 8.
  • the operation unit 7 is operated by the user to input various instructions to the mathematical model construction device 1 .
  • a keyboard, a mouse, a touch panel, and the like can be used as the operation unit 7 .
  • a microphone or the like for inputting various instructions may be used together with the operation unit 7 or instead of the operation unit 7 .
  • the display device 8 displays various images.
  • Various devices capable of displaying images for example, at least one of a monitor, a display, a projector, etc.
  • the “image” in the present disclosure includes both still images and moving images.
  • the mathematical model construction device 1 can acquire image data (hereinafter sometimes simply referred to as "image") from the medical imaging device 11A.
  • image may acquire image data from the medical imaging device 11A, for example, by at least one of wired communication, wireless communication, a removable storage medium (eg, USB memory), and the like.
  • the medical image processing device 21 will be explained.
  • the medical image processing apparatus 21 is installed, for example, in a facility (for example, a hospital, a health checkup facility, or the like) for diagnosing or examining a subject.
  • the medical image processing apparatus 21 includes a control unit 22 that performs various control processes, and a communication I/F 25 .
  • the control unit 22 includes a CPU 23 which is a controller for control, and a storage device 24 capable of storing programs, data, and the like.
  • the storage device 24 stores a medical image processing program for executing medical image processing (first detection processing to fifth detection processing) to be described later.
  • the medical image processing program includes a program for realizing the mathematical model constructed by the mathematical model construction device 1 .
  • the communication I/F 25 connects the medical image processing apparatus 21 with other devices (for example, the medical image capturing apparatus 11B, the mathematical model construction apparatus 1, etc.).
  • the medical image processing device 21 is connected to an operation unit 27 and a display device 28.
  • Various devices can be used for the operation unit 27 and the display device 28, similarly to the operation unit 7 and the display device 8 described above.
  • the medical image capturing device 11 includes a control unit 12 (12A, 12B) that performs various control processes, and a medical image capturing section 16 (16A, 16B).
  • the control unit 12 includes a CPU 13 (13A, 13B), which is a controller for control, and a storage device 14 (14A, 14B) capable of storing programs, data, and the like.
  • the medical image capturing unit 16 has various configurations necessary for capturing an image of a living tissue (in this embodiment, an ophthalmologic image of an eye to be examined).
  • the medical image capturing unit 16 of the present embodiment includes an OCT light source, a branching optical element that branches the OCT light emitted from the OCT light source into measurement light and reference light, a scanning unit for scanning the measurement light, and a scanning unit for scanning the measurement light. It includes an optical system for irradiating an eye to be examined, a light receiving element for receiving the combined light of the light reflected by the tissue and the reference light, and the like.
  • the medical image capturing apparatus 11 can capture two-dimensional tomographic images and three-dimensional tomographic images of living tissue (in this embodiment, the fundus of the subject's eye).
  • the CPU 13 captures a two-dimensional tomographic image of a cross section that intersects the scan lines by scanning OCT light (measurement light) along the scan lines.
  • the two-dimensional tomographic image may be an averaging image generated by averaging a plurality of tomographic images of the same region.
  • the CPU 13 can capture a three-dimensional tomographic image of a tissue by two-dimensionally scanning the OCT light.
  • the CPU 13 acquires a plurality of two-dimensional tomographic images by causing measurement light to scan each of a plurality of scan lines at different positions in a two-dimensional region when the tissue is viewed from the front.
  • the CPU 13 acquires a three-dimensional tomographic image by combining a plurality of captured two-dimensional tomographic images (details of which will be described later).
  • the mathematical model building process executed by the mathematical model building device 1 will be described with reference to FIGS. 2 and 3.
  • the mathematical model building process is executed by CPU 3 according to a mathematical model building program stored in storage device 4 .
  • the mathematical model is trained with a plurality of training data to build a mathematical model that outputs detection results of specific structures in the tissue imaged.
  • Training data includes input data and output data.
  • the CPU 3 acquires training image data, which is an image captured by the medical imaging device 11A, as input data.
  • training image data is acquired by the mathematical model construction device 1 after being generated by the medical imaging device 11A.
  • the CPU 3 acquires a signal (for example, an OCT signal) that is used as a basis for generating a training image from the medical imaging apparatus 11A, and generates a training image based on the acquired signal. may be obtained.
  • the structure of the tissue to be detected from the image is at least one of the layers of the fundus tissue of the subject's eye and the boundary between the layers of the fundus tissue (hereinafter simply referred to as "layer/boundary").
  • layer/boundary the boundary between the layers of the fundus tissue
  • an image of the fundus tissue of the subject's eye is acquired as the training image.
  • the mode of the training image changes according to the mode of the image input to the mathematical model.
  • FIG. 2 shows an example of a training image 30, which is a two-dimensional tomographic image of the fundus. A plurality of layers/borders in the fundus appear in the training image 30 illustrated in FIG.
  • the image input to the mathematical model for detecting the structure is a one-dimensional image (for example, an A-scan image extending in the direction along the optical axis of the OCT measurement light)
  • One-dimensional images are also used as training images in the mathematical model building process.
  • FIG. 3 shows an example of output data 31 indicating the positions of specific boundaries when a two-dimensional tomographic image of the fundus is used as the training image 30 .
  • the output data 31 illustrated in FIG. 3 includes data of labels 32A to 32F indicating the positions of six boundaries of the fundus tissue shown in the training image 30 (see FIG. 2).
  • the data of the labels 32A to 32F in the output data 31 are generated by the operator operating the operation unit 7 while looking at the boundaries in the training image 30.
  • FIG. it is also possible to change the method of generating label data. Note that if the training image is a one-dimensional image, the output data will be data indicating the position of the specific structure on the one-dimensional image.
  • CPU 3 then executes training of the mathematical model using the training data by means of a machine learning algorithm.
  • machine learning algorithms for example, neural networks, random forests, boosting, support vector machines (SVM), etc. are generally known.
  • a neural network is a method that imitates the behavior of the neural network of living organisms.
  • Neural networks include, for example, feedforward neural networks, RBF networks (radial basis functions), spiking neural networks, convolutional neural networks, recurrent neural networks (recurrent neural networks, feedback neural networks, etc.), probability neural networks (Boltzmann machine, Baysian network, etc.), etc.
  • Random forest is a method of learning based on randomly sampled training data to generate a large number of decision trees.
  • branches of a plurality of decision trees learned in advance as discriminators are traced, and the average (or majority vote) of the results obtained from each decision tree is taken.
  • Boosting is a method of generating a strong classifier by combining multiple weak classifiers.
  • a strong classifier is constructed by sequentially learning simple and weak classifiers.
  • SVM is a method of constructing a two-class pattern classifier using linear input elements.
  • the SVM learns the parameters of the linear input element, for example, based on the criterion of finding the margin-maximizing hyperplane that maximizes the distance to each data point from the training data (hyperplane separation theorem).
  • a mathematical model refers to a data structure for predicting the relationship between input data and output data.
  • a mathematical model is built by being trained using training data.
  • the training data are pairs of input data and output data.
  • training updates the correlation data (eg, weights) for each input and output.
  • a multilayer neural network is used as the machine learning algorithm.
  • a neural network includes an input layer for inputting data, an output layer for generating data to be predicted, and one or more hidden layers between the input layer and the output layer.
  • a plurality of nodes are arranged in each layer.
  • CNN convolutional neural network
  • other machine learning algorithms may be used.
  • Generative Adversarial Networks (GAN) which utilize two competing neural networks, may be employed as a machine learning algorithm.
  • Programs and data for realizing the constructed mathematical model are installed in the medical image processing apparatus 21 .
  • FIG. 4 An example of a three-dimensional image from which tissue structure is to be detected will be described with reference to FIGS. 4 and 5.
  • FIG. 4 the medical imaging apparatus 11B of the present embodiment scans light (measurement light) within a two-dimensional region 51 in a biological tissue 50 (in the example shown in FIG. 4, fundus tissue). .
  • the medical imaging apparatus 11B of the present embodiment scans light on scan lines 52 extending in a predetermined direction within the region 51, thereby scanning the Z direction along the optical axis of the light and the Z direction perpendicular to the Z direction.
  • a two-dimensional image 61 (see FIG. 5) extending in the X direction is acquired (photographed).
  • FIG. 5 A two-dimensional image 61 (see FIG. 5) extending in the X direction is acquired (photographed).
  • the Z direction is the direction perpendicular to the two-dimensional region 51 (depth direction)
  • the X direction is the direction in which the scan lines 52 extend.
  • the medical imaging apparatus 11B repeats acquisition of the two-dimensional image 61 by moving the position of the scan line 52 in the Y direction within the area 51 .
  • the Y direction is a direction that intersects both the Z direction and the X direction (perpendicularly intersects in this embodiment).
  • a plurality of two-dimensional images 61 passing through each of the plurality of scan lines 52 and extending in the depth direction of the tissue are acquired.
  • a plurality of two-dimensional images 61 are arranged in the Y direction (that is, the direction intersecting the image area of each two-dimensional image), so that the three-dimensional image in the area 51 is generated.
  • the first to fifth detection processes executed by the medical image processing apparatus 21 of this embodiment will be described below.
  • the medical image processing device 21 which is a PC, acquires a three-dimensional image from the medical image capturing device 11B and detects a specific structure of tissue appearing in the acquired three-dimensional image.
  • other devices may function as medical image processors.
  • the medical imaging apparatus (OCT apparatus in this embodiment) 11B itself may perform the first to fifth detection processes described below.
  • a plurality of control units may cooperate to execute the first detection process to the fifth detection process.
  • the CPU 23 of the medical image processing apparatus 21 executes the first detection process to the fifth detection process according to the medical image processing program stored in the storage device 24.
  • the first detection process will be described with reference to FIGS. 6 and 7.
  • FIG. In the first detection process a process of detecting a structure from each of the two-dimensional images 61 is performed with pixel rows forming the two-dimensional images 61 as processing units.
  • the CPU 23 acquires a three-dimensional image from which a specific structure is to be detected (S1). For example, the user operates the operation unit 27 (see FIG. 1) to select a three-dimensional image as a structure detection target from a plurality of three-dimensional images. The CPU 23 acquires the data of the three-dimensional image selected by the user.
  • the CPU 23 selects the T-th (T is a natural number with an initial value of 1) two-dimensional image 61 from among the plurality of two-dimensional images 61 forming the three-dimensional image (S2).
  • T is a natural number with an initial value of 1
  • each of the plurality of two-dimensional images 61 forming the three-dimensional image is numbered in the order in which they are arranged in the Y direction.
  • the two-dimensional images 61 positioned on the outermost side in the Y direction are sequentially selected.
  • the CPU 23 classifies the multiple A-scan images in the two-dimensional image 61 selected in S2 into one of multiple groups (S3).
  • a two-dimensional image 61 captured by the OCT apparatus is composed of a plurality of A-scan images indicated by arrows in FIG.
  • An A-scan image is a pixel row extending in a direction along the optical axis of OCT measurement light.
  • the CPU 23 classifies a plurality of A-scan images having a high degree of mutual similarity into the same group regardless of the positional relationship between the A-scan images.
  • A-scan images of regions with no layer separation and a thin retinal nerve fiber layer are classified into group G1.
  • Group G2 includes A-scan images of regions where there is no layer detachment and where the retinal nerve fiber layer is thick. A-scan images of regions where IS/OS lines are peeled off are classified into group G3. Group G4 includes A-scan images of regions where both the IS/OS line and the retinal pigment epithelium layer are detached.
  • the CPU 23 extracts a representative A-scan image, which is a pixel row representing each of the plurality of groups, as a first region (S4).
  • a first region is a region in a three-dimensional image in which a specific structure is detected using a mathematical model trained by a machine learning algorithm.
  • a method for extracting a representative A-scan image from a plurality of A-scan images belonging to a group can be selected as appropriate.
  • the CPU 23 extracts, as a representative A-scan image, a pixel row obtained by performing averaging processing on a plurality of A-scan images classified into each group. As a result, a representative A-scan image representing the group is appropriately extracted.
  • the CPU 23 executes the first structure detection process on the representative A-scan image (first region) extracted from each group (S5).
  • the first structure detection process is a process of detecting a specific structure using a mathematical model. That is, when executing the first structure detection process, the CPU 23 inputs the first region extracted from the three-dimensional image (representative A-scan image in the example shown in FIG. 6) to the mathematical model trained by the machine learning algorithm. do.
  • the mathematical model outputs detection results of specific structures in the first region (layers/boundaries in the fundus in this embodiment).
  • CPU23 acquires the detection result output by the mathematical model. According to the first structure detection process, a specific structure in an image can be detected with high accuracy, although the amount of calculation is larger than that of conventionally used techniques such as image processing.
  • the CPU 23 sets the A-scan image that is not extracted as the first area (representative A-scan image in this embodiment) from each group as the second area, and executes the second structure detection process on the second area for each group.
  • the second structure detection process is a process of detecting a specific structure in a second area that was not detected as the first area out of the entire area of the three-dimensional image, based on the detection result of the first structure detection process. .
  • the amount of computation for the second structure detection process is less than the amount of computation for the first structure detection process.
  • the second structure detection process is executed based on the result of the first structure detection process performed with high accuracy. Therefore, the structure of the second region is properly detected.
  • the CPU 23 generates the structure detection result and pixel information for each pixel that constitutes the first region (representative A-scan image), and the pixel information for each pixel that constitutes the second region.
  • a detection result of the structure in the second region is calculated by comparing with the information.
  • the CPU 23 determines the positional relationship (for example, closeness of distance in the Z direction, etc.) between each pixel forming the second region and each pixel forming the first region to be referred to (representative A scan belonging to the same group). may be considered.
  • the CPU 23 may perform the second structure detection process on the second region by interpolation process based on the result of the first structure detection process.
  • the first structure detection process and the second structure detection process are executed for each group, thereby further improving the accuracy of the second structure detection process.
  • the CPU 23 determines whether or not the structure detection processing for all two-dimensional images has been completed (S8). If not completed (S8: NO), "1" is added to the counter T indicating the order of the two-dimensional images (S9), and the process returns to S2. When the structure detection processing for all two-dimensional images is completed (S8: YES), the first detection processing ends.
  • a plurality of pixel rows (A-scan images) forming a two-dimensional image are classified into a plurality of groups, and a first region is extracted for each group.
  • the CPU 23 may classify small regions (patches) each composed of a plurality of pixel rows (such as an A-scan image) into a plurality of groups, and extract the first region for each group. Further, the CPU 23 may extract the first regions at regular intervals from a plurality of pixel rows forming the two-dimensional image.
  • FIG. 8 An image region, which is a region in which an image of tissue is shown, is extracted from each two-dimensional image.
  • a first structure detection process using a mathematical model is performed on the extracted image region. Therefore, areas where tissue is not imaged are excluded from the areas targeted for detection of specific tissue by the mathematical model. It should be noted that the steps of the second to fifth detection processes that are the same as the steps described in the first detection process will be simplified.
  • the CPU 23 when starting the second detection process, acquires a three-dimensional image from which a specific structure is to be detected (S1).
  • the CPU 23 selects the T-th two-dimensional image 61 from among the plurality of two-dimensional images 61 forming the three-dimensional image (S2).
  • the CPU 23 determines whether or not the T-th two-dimensional image 61 is to be the reference image 61A (see FIG. 9) (S11).
  • the reference image 61A is an image that serves as a reference for the extraction position of the image area in the other two-dimensional image 61B.
  • a method for selecting the reference image 61A from the plurality of two-dimensional images 61 can be selected as appropriate.
  • the CPU 23 executes the first structure detection process on the reference image 61A (that is, the T-th two-dimensional image 61) (S12). In other words, the CPU 23 inputs the reference image 61A into the mathematical model to acquire the detection result of the specific structure of the tissue shown in the reference image 61A.
  • the CPU 23 detects an image region in which the tissue image is shown in the reference image 61A based on the structure detection result obtained in S12 (S13).
  • the first structure detection process using the mathematical model tends to detect a specific structure with high accuracy. Therefore, the image area detected based on the detection result acquired in S12 is also likely to be detected with high accuracy.
  • the area surrounded by two white solid lines is the image area based on the structure detection result (in this embodiment, the fundus layer/boundary detection result) by the first structure detection process. is detected as
  • the CPU 23 determines the T-th two-dimensional image 61B based on the already detected image area of the reference image 61A. (see FIG. 9) is extracted as the first area (S15). As a result, the T-th two-dimensional image 61B is detected with a smaller amount of calculation than when a mathematical model is used. In the example shown in FIG. 9, the area surrounded by two white broken lines is detected as the image area in the two-dimensional image 61B positioned near the reference image 61A.
  • the CPU 23 compares the image area detection result and pixel information for each pixel forming the reference image 61A with the pixel information for each pixel forming another two-dimensional image 61B. , the image area of the two-dimensional image 61B is detected. In addition, the CPU 23 also considers the positional relationship between each pixel forming the reference image 61A and each pixel forming the two-dimensional image 61B (in this embodiment, the positional relationship between the XZ coordinates). Detect regions.
  • the CPU 23 aligns the images (in this embodiment, in the Z direction) between the plurality of pixel columns (in this embodiment, the plurality of A-scan images described above) that constitute the T-th two-dimensional image 61B. image alignment) is performed (S16). For example, the CPU 23 aligns the image area of the two-dimensional image 61B with respect to the reference image 61 so that the image area of the two-dimensional image 61B and the image area of the reference image have the same shape (curved shape). In this state, the CPU 23 cuts out the curved shape so as to flatten it, or shifts the A-scan image in the Z direction, thereby making the image area 65 extracted from the two-dimensional image 61B rectangular (or substantially rectangular). good).
  • the image is appropriately contained within the rectangular image area 65 (first area), and the size of the rectangular image area 65 tends to be reduced.
  • the CPU 23 executes the first structure detection process on the rectangular image area 65 (S17). That is, the CPU 23 inputs the rectangular image area 65 into the mathematical model to acquire the detection result of the specific structure in the image area 65 .
  • the CPU 23 determines whether or not the structure detection processing for all two-dimensional images 61 has been completed (S18). If not completed (S18: NO), "1" is added to the counter T indicating the order of the two-dimensional image 61 (S19), and the process returns to S2. When the structure detection processing for all two-dimensional images 61 is completed (S18: YES), the second detection processing ends. It should be noted that in the second detection process, the reciprocal of the movement amount for alignment executed for each A-scan image in S16 is added to the structure detection result obtained in S17, thereby obtaining the final structure. A detection result is obtained.
  • the image area of the other two-dimensional image 61B is extracted based on the image area of the reference image 61A.
  • the CPU 23 may detect the image area by performing known image processing on the two-dimensional image 61 .
  • the third detection process will be described with reference to FIGS. 10 and 11.
  • FIG. 10 The third detection process, alignment of images within the two-dimensional images 61 and alignment of images between the two-dimensional images 61 are performed for a plurality of two-dimensional images 61 forming a three-dimensional image. An image region is then extracted and specific structures in the extracted image region are detected.
  • the CPU 23 acquires a three-dimensional image from which a specific structure is to be detected (S1).
  • the CPU 23 performs image alignment (in this embodiment, image alignment in the Z direction) between the plurality of two-dimensional images 61 forming the three-dimensional image (S21). Further, for each of the plurality of two-dimensional images 61 forming the three-dimensional image, the CPU 23 performs the Image registration (in this embodiment, image registration in the Z direction) is performed (S22).
  • the CPU 23 constructs a plurality of two-dimensional images extending in the YZ direction, and aligns the images among the constructed two-dimensional images to obtain a two-dimensional image extending in the XZ direction. Alignment between adjacent pixels in 61 is performed. As a result, the effects of noise and the like are suppressed as compared with the case where alignment is performed between a plurality of A-scan images. Note that the order of the processing of S21 and the processing of S22 may be reversed.
  • FIG. 11 compares two-dimensional images before and after image alignment (image alignment between two-dimensional images 61 and image alignment within the two-dimensional image 61).
  • the left side of FIG. 11 is the two-dimensional image before image alignment is performed, and the right side of FIG. 11 is the two-dimensional image after image alignment is performed.
  • intra-image and inter-image alignment is performed so that the positions of the images in each image are closer to each other.
  • the CPU 23 sets at least one of the plurality of two-dimensional images 61 forming the three-dimensional image as a reference image, and extracts a rectangular image area from the two-dimensional image 61 used as the reference image as a first area (S23 ).
  • a method for selecting a reference image from a plurality of two-dimensional images 61 can be appropriately selected as in S11 described above.
  • the CPU 23 selects a reference image from a plurality of two-dimensional images 61 at regular intervals. Of the plurality of two-dimensional images 61, the two-dimensional images 61 that have not been selected as the reference image serve as the second region where the structure detection processing based on the mathematical model is not performed.
  • the CPU 23 executes the first structure detection process for the first region extracted in S23 (S24). That is, the CPU 23 inputs the first region extracted in S23 to the mathematical model, thereby acquiring the detection result of the specific structure in the first region.
  • the CPU 23 executes the second structure detection process for the two-dimensional image 61 (second region) that has not been selected as the reference image (S25). That is, the CPU 23 detects a specific structure in the second area based on the result of the first structure detection processing for the first area, which is the reference image.
  • image alignment is performed between the plurality of two-dimensional images 61 in S21. Therefore, in S25, the structure in the second area is appropriately detected by comparing pixels having similar coordinates (XZ coordinates in this embodiment) between the first area and the second area.
  • the reciprocal of the sign (plus, minus) of the movement amount for alignment executed for each A-scan image in S21 and S22 is the structure obtained in S24 and S25. is added to the detection result of , the final structure detection result is obtained.
  • the CPU 23 extracts a rectangular (or substantially rectangular) image area from the three-dimensional image after aligning the images of the entire three-dimensional image in S21 and S22, and performs the first step on the extracted image area.
  • a structure detection process may be performed.
  • the CPU 23 may take an average of all A-scan images from the three-dimensional images that have undergone overall alignment in S21 and S22, and specify the range of the image from the average A-scan images.
  • the CPU 23 extracts a rectangular image area from each two-dimensional image 61 based on the specified image range, and inputs the extracted image area into the mathematical model to perform the first structure detection process. good.
  • the first structure detection process is omitted.
  • the first structure detection process is performed only for areas where there is a high possibility that an image exists, so the amount of processing is appropriately reduced.
  • the fourth detection process will be described with reference to FIGS. 12 and 13.
  • FIG. 1 a part of the plurality of two-dimensional images 61 forming the three-dimensional image is extracted as the first region targeted for the first structure detection process. Specifically, based on the degree of similarity between the multiple two-dimensional images 61, a portion of the multiple two-dimensional images 61 is extracted as the first region.
  • the CPU 23 acquires a three-dimensional image from which a specific structure is to be detected (S1).
  • the CPU 23 selects the T-th two-dimensional image 61 from among the plurality of two-dimensional images 61 forming the three-dimensional image (S2).
  • the CPU 23 determines whether the degree of similarity between the reference image at that time and the T-th two-dimensional image 61 is less than a threshold (S31).
  • the reference image in the fourth detection process is an image that serves as a reference for determining whether the other two-dimensional image 61 should be the first area or the second area.
  • the CPU 23 executes the first structure detection process on the T-th image, which is the reference image (S33).
  • the CPU 23 changes the T-th two-dimensional image 61 is the second region, and the second structure detection process is performed on the T-th two-dimensional image 61 (S34). In other words, the CPU 23 detects a specific structure in the T-th two-dimensional image 61 based on the result of the first structure detection processing for the reference image with high similarity.
  • the CPU 23 replaces the T-th two-dimensional image 61 with a new reference image. , and the T-th two-dimensional image 61 is extracted as the first area (S32). The CPU 23 executes the first structure detection process on the T-th image, which is the reference image (S33).
  • the CPU 23 determines whether or not the structure detection processing for all two-dimensional images 61 has been completed (S36). If not completed (S36: NO), "1" is added to the counter T indicating the order of the two-dimensional images (S37), and the process returns to S2. When the structure detection processing for all two-dimensional images is completed (S36: YES), the fourth detection processing ends.
  • the fifth detection process will be described with reference to FIGS. 14 and 15.
  • FIG. 14 a position of interest is set within the three-dimensional image area, and the first area is extracted based on the set area of interest.
  • the CPU 23 acquires a three-dimensional image from which a specific structure is to be detected (S1).
  • the CPU 23 sets a target position within the image area of the three-dimensional image (S41).
  • the CPU 23 sets the attention position according to an instruction input by the user via the operation unit 27 (that is, at the position instructed by the user).
  • the CPU 23 can detect a specific part in the three-dimensional image and set the detected specific part as the position of interest.
  • FIG. 15 shows a two-dimensional front image 70 when the imaging region of the three-dimensional image is viewed from the direction along the optical axis of the OCT measurement light.
  • the macula which is a specific part, is detected from the fundus tissue of the subject's eye, and the target position 73 is set to the detected macula.
  • the CPU 23 sets extraction patterns for a plurality of two-dimensional images with reference to the position of interest (S42).
  • the CPU 23 extracts the two-dimensional image that matches the set extraction pattern as the first region to be subjected to structure detection by the mathematical model (S43).
  • the two-dimensional image extraction pattern set in S42 does not need to match each two-dimensional image 61 captured by the medical imaging apparatus 11B, and can be set arbitrarily. For example, in the example shown in FIG. 15, when the three-dimensional image is viewed from the direction along the optical axis of the OCT measurement light, the lines crossed by the two-dimensional image to be extracted are radially centered on the position of interest 73. , an extraction pattern 75 is set. As a result, a plurality of two-dimensional images centered on the position of interest 73 are extracted as the first region.
  • the CPU 23 executes the first structure detection process for the first region extracted in S43 (S44). Also, the second structure detection process is performed on the second region, which is the region other than the first region, in the three-dimensional image (S45).
  • the same processing as the processing described above can be employed, so detailed description thereof will be omitted.
  • a system configuration of a medical image processing system 100 which is a modification of the above embodiment, will be described with reference to FIG.
  • portions that can employ the same configuration as in the above embodiment for example, the medical image processing device 21 and the medical image capturing device 11B, etc. are assigned the same reference numerals as in the above embodiment. The description is omitted or simplified.
  • the medical image processing system 100 shown in FIG. 16 includes a medical image processing device 21 and a cloud server 91.
  • the medical image processing device 21 processes the three-dimensional image captured by the medical image capturing device 11B.
  • the medical image processing apparatus 21 performs the first structure detection processing (S5 in FIG. 6, S17 in FIG. 8, FIG. 10, S33 in FIG. 12, and S44 in FIG. 14).
  • a device different from the medical image processing device 21 may function as the first image processing device.
  • the cloud server 91 includes a control unit 92 and a communication I/F 95.
  • the control unit 92 includes a CPU 93 that is a controller for control, and a storage device 94 that can store programs, data, and the like.
  • a storage device 94 stores a program for realizing the mathematical model described above.
  • a communication I/F 95 connects the cloud server 91 and the medical image processing apparatus 21 via a network (for example, the Internet, etc.) 9 .
  • the cloud server 91 performs the first structure detection process (S5 in FIG. 6, S17 in FIG. 8, S24 in FIG. 10, S33 in FIG. 12, It functions as a second image processing device that executes S44) in FIG.
  • the cloud server 91 executes the first structure detection process described above.
  • the cloud server 91 also executes an output step of outputting the result detected by the first structure detection process to the medical image processing apparatus 21 .
  • the various processes described above are properly executed.
  • the first structure detection process (S24) for the first region and the second structure detection process (S25) for the other second regions are executed.
  • the image area may be detected from all the two-dimensional images 71 forming the three-dimensional image.
  • the second structure detection process (S25) may be omitted.
  • the second structure detection process for the second area other than the image area is omitted.
  • the process of acquiring a three-dimensional image in S1 of FIGS. 6, 8, 10, 12, and 14 is an example of the "image acquisition step.”
  • the process of extracting the first region in S4 of FIG. 6, S15 of FIG. 8, S23 of FIG. 10, S32 of FIG. 12, and S43 of FIG. 14 is an example of the "extraction step.”
  • the first structure detection process shown in S5 of FIG. 6, S17 of FIG. 8, S24 of FIG. 10, S33 of FIG. 12, and S44 of FIG. 14 is an example of the "first structure detection step”.
  • the second structure detection process shown in S45 of FIG. 6, S25 of FIG. 10, S34 of FIG. 12, and S45 of FIG. 14 is an example of the "second structure detection step".
  • the process of aligning the images within the two-dimensional image in S16 of FIG. 8 and S21 of FIG. 10 is an example of the "alignment step within the two-dimensional image".
  • the process of aligning a plurality of two-dimensional images in S22 of FIG. 10 is an example of the "step of aligning two-dimensional images.”

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Computer Graphics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Veterinary Medicine (AREA)
  • Public Health (AREA)
  • Animal Behavior & Ethology (AREA)
  • Surgery (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Ophthalmology & Optometry (AREA)
  • Biophysics (AREA)
  • Eye Examination Apparatus (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
PCT/JP2022/009329 2021-03-31 2022-03-04 医療画像処理装置、医療画像処理プログラム、および医療画像処理方法 Ceased WO2022209574A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2023510719A JP7439990B2 (ja) 2021-03-31 2022-03-04 医療画像処理装置、医療画像処理プログラム、および医療画像処理方法
US18/477,067 US20240020839A1 (en) 2021-03-31 2023-09-28 Medical image processing device, medical image processing program, and medical image processing method

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2021-059329 2021-03-31
JP2021059329 2021-03-31

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/477,067 Continuation US20240020839A1 (en) 2021-03-31 2023-09-28 Medical image processing device, medical image processing program, and medical image processing method

Publications (1)

Publication Number Publication Date
WO2022209574A1 true WO2022209574A1 (ja) 2022-10-06

Family

ID=83458589

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/009329 Ceased WO2022209574A1 (ja) 2021-03-31 2022-03-04 医療画像処理装置、医療画像処理プログラム、および医療画像処理方法

Country Status (3)

Country Link
US (1) US20240020839A1 (https=)
JP (1) JP7439990B2 (https=)
WO (1) WO2022209574A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116071350A (zh) * 2023-03-06 2023-05-05 同心智医科技(北京)有限公司 基于深度学习的大脑微出血识别方法、装置及存储介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020026535A1 (ja) * 2018-08-03 2020-02-06 株式会社ニデック 眼科画像処理装置、ｏｃｔ装置、および眼科画像処理プログラム
WO2020036182A1 (ja) * 2018-08-14 2020-02-20 キヤノン株式会社 医用画像処理装置、医用画像処理方法及びプログラム
WO2020137678A1 (ja) * 2018-12-27 2020-07-02 キヤノン株式会社 画像処理装置、画像処理方法及びプログラム
JP2020121037A (ja) * 2019-01-31 2020-08-13 株式会社ニデック 眼科画像処理装置、および眼科画像処理プログラム
US20200327663A1 (en) * 2019-04-11 2020-10-15 HONGBOG Inc. Method of analyzing iris image for diagnosing dementia in artificial intelligence
JP2021037177A (ja) * 2019-09-04 2021-03-11 株式会社ニデック 眼科画像処理プログラムおよび眼科画像処理装置
JP2021037239A (ja) * 2019-09-05 2021-03-11 キヤノン株式会社 領域分類方法

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020026535A1 (ja) * 2018-08-03 2020-02-06 株式会社ニデック 眼科画像処理装置、ｏｃｔ装置、および眼科画像処理プログラム
WO2020036182A1 (ja) * 2018-08-14 2020-02-20 キヤノン株式会社 医用画像処理装置、医用画像処理方法及びプログラム
WO2020137678A1 (ja) * 2018-12-27 2020-07-02 キヤノン株式会社 画像処理装置、画像処理方法及びプログラム
JP2020121037A (ja) * 2019-01-31 2020-08-13 株式会社ニデック 眼科画像処理装置、および眼科画像処理プログラム
US20200327663A1 (en) * 2019-04-11 2020-10-15 HONGBOG Inc. Method of analyzing iris image for diagnosing dementia in artificial intelligence
JP2021037177A (ja) * 2019-09-04 2021-03-11 株式会社ニデック 眼科画像処理プログラムおよび眼科画像処理装置
JP2021037239A (ja) * 2019-09-05 2021-03-11 キヤノン株式会社 領域分類方法

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116071350A (zh) * 2023-03-06 2023-05-05 同心智医科技(北京)有限公司 基于深度学习的大脑微出血识别方法、装置及存储介质

Also Published As

Publication number Publication date
JPWO2022209574A1 (https=) 2022-10-06
JP7439990B2 (ja) 2024-02-28
US20240020839A1 (en) 2024-01-18

Similar Documents

Publication Publication Date Title
US12096981B2 (en) Ophthalmologic image processing device and non-transitory computer-readable storage medium storing computer-readable instructions
CN112533526B (zh) 眼科图像处理装置、oct装置及眼科图像处理程序
JP7521575B2 (ja) 眼科画像処理装置、ｏｃｔ装置、および眼科画像処理プログラム
JP7183590B2 (ja) 眼科画像処理装置、ｏｃｔ装置、および眼科画像処理プログラム
JP7328489B2 (ja) 眼科画像処理装置、および眼科撮影装置
JP2023115058A (ja) 眼科画像処理装置、および眼科画像処理プログラム
JP7302184B2 (ja) 眼科画像処理装置、および眼科画像処理プログラム
JP7439990B2 (ja) 医療画像処理装置、医療画像処理プログラム、および医療画像処理方法
JP7563384B2 (ja) 医療画像処理装置および医療画像処理プログラム
JP7147888B2 (ja) 眼科画像処理装置、ｏｃｔ装置、および眼科画像処理プログラム
JP7597026B2 (ja) 眼科画像処理装置、眼科画像処理プログラム、および眼科画像処理システム
CN113365547B (zh) 眼科图像处理装置、oct装置、眼科图像处理程序及数学模型构建方法
JP7612990B2 (ja) 眼科画像処理装置および眼科画像処理プログラム
JP6747617B2 (ja) 眼科画像処理装置、およびｏｃｔ装置
JP7845366B2 (ja) 医療画像処理装置、および医療画像処理プログラム
US20250111645A1 (en) Oct image processing device, storage medium storing oct image processing program and oct image processing method
JP7180187B2 (ja) 眼科画像処理装置、ｏｃｔ装置、および眼科画像処理プログラム
JP7468163B2 (ja) 眼科画像処理プログラムおよび眼科画像処理装置
JP2023149095A (ja) 眼底画像処理装置および眼底画像処理プログラム
JP2023149742A (ja) 眼底画像処理装置および眼底画像処理プログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22779799

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2023510719

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22779799

Country of ref document: EP

Kind code of ref document: A1