WO2023032480A1 - 医療画像処理装置、肝区域分割方法およびプログラム - Google Patents

医療画像処理装置、肝区域分割方法およびプログラム Download PDF

Info

Publication number
WO2023032480A1
WO2023032480A1 PCT/JP2022/027537 JP2022027537W WO2023032480A1 WO 2023032480 A1 WO2023032480 A1 WO 2023032480A1 JP 2022027537 W JP2022027537 W JP 2022027537W WO 2023032480 A1 WO2023032480 A1 WO 2023032480A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
liver
portal vein
region
input data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/027537
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
潔 長谷川
由祐 風見
順一 金子
ディーパック ケシュワニ
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fujifilm Corp
Original Assignee
Fujifilm Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fujifilm Corp filed Critical Fujifilm Corp
Priority to JP2023545138A priority Critical patent/JP7812864B2/ja
Publication of WO2023032480A1 publication Critical patent/WO2023032480A1/ja
Priority to US18/587,853 priority patent/US20240193785A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/02—Arrangements for diagnosis sequentially in different planes; Stereoscopic radiation diagnosis
    • A61B6/03—Computed tomography [CT]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/776—Validation; Performance evaluation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/70—Labelling scene content, e.g. deriving syntactic or semantic representations
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/04—Indexing scheme for image data processing or generation, in general involving 3D image data
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10081—Computed x-ray tomography [CT]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10088—Magnetic resonance imaging [MRI]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30056—Liver; Hepatic
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30101—Blood vessel; Artery; Vein; Vascular
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/03—Recognition of patterns in medical or anatomical images
    • G06V2201/031—Recognition of patterns in medical or anatomical images of internal organs

Definitions

  • the present disclosure relates to a medical image processing apparatus, a liver segmentation method and a program, and particularly to machine learning technology and image processing technology for handling medical images in which a region including the liver is captured.
  • the liver is divided into eight segments, S1 to S8, using the branched portal vein as an index. That is, S1 is the caudate lobe, S2 is the left lateral lobe posterior segment (dorsolateral segment), S3 is the left lobe lateral anterior segment (ventrolateral segment), S4 is the left medial lobe segment (quatrate lobe), and S5 is the right lobe anterior segment. S6 is the right lobe posterior inferior segment, S7 is the right lobe posterior superior segment, and S8 is the right lobe anterior superior segment.
  • liver segmentation into S1 to S8 segments is required in various situations. For example, when reporting an area in which an abnormal tumor exists in an interpretation report, it is required to appropriately specify the liver area on the medical image.
  • Patent Document 2 describes a convolution neural network (CNN) that uses deep learning to perform the task of classifying blood vessel branches in the liver.
  • CNN convolution neural network
  • the segments S1 to S8 in the liver do not physically or anatomically have a clear boundary surface, and there are large individual differences in the positions at which the liver segments are divided. Therefore, a method for automatically and uniquely segmenting liver segments from medical images is desired.
  • the method of dividing the liver region into segments S1 to S8 is basically based on each labeled portal vein branch (partial portal vein) in the liver. Specifically, a portal vein region is extracted from a medical image obtained by imaging a region including the liver, and the portal vein branches are labeled. After that, Voronoi division is performed based on the distance from the labeled portal vein branch, and the dominant region is set based on the obtained result.
  • the portal vein is classified and labeled into portal vein branches S1 to S8 corresponding to liver segments S1 to S8.
  • the governing region of the S1 portal vein branch has a relationship with the S1 liver segment, and the label of the portal vein branch and the label of the liver segment can be in one-to-one correspondence.
  • the liver segment to which each voxel belongs is determined based on the criteria of which portal branch label region each voxel in the three-dimensional image is closest to.
  • the portal vein branch label is a label for classifying (dividing) a predetermined image region into eight regions in association with the eight portal vein branches S1 to S8.
  • the portal vein branch label classifies the portal vein region as a predetermined image region into eight portal vein branch regions, and the liver segment to which each voxel belongs is determined based on the distance from the eight portal vein branch regions. be done.
  • each liver segment from S1 to S8 is not simple. Therefore, it is difficult for a doctor, who is a user, to uniquely set an appropriate boundary surface of the liver segment. Moreover, it is also difficult to automate complicated processing by a doctor as it is.
  • some images taken by modalities have low voxel density values in the portal vein region, or the portal vein region is not properly captured in the image.
  • the method using Voronoi segmentation based on the identified portal vein region may fail to accurately segment the liver segment.
  • the accuracy of liver segment segmentation changes depending on how blood vessels (portal veins) appear in the image (see FIGS. 12 and 13).
  • a possible solution to this problem is to use machine learning to generate a learning model that performs the liver segmentation task. That is, as data for learning, a large number of data sets of input images and data labeled with the correct labels for each liver segment from S1 to S8 are prepared for the images, and these data sets are used to Perform supervised learning. This produces a trained model that outputs the results of liver segment segmentation.
  • the present disclosure has been made in view of such circumstances, and aims to provide a medical image processing apparatus, a liver segmentation method, and a program capable of accurately segmenting the liver from a medical image.
  • a medical image processing apparatus includes a processor and a storage device storing a program executed by the processor, the program storing first input data including a first image of the liver and , portal vein branch labeling data in which each portal vein branch corresponding to the liver segment is labeled for each portal vein region in the liver in the first image, and machine learning using learning data including
  • the trained model receives input of the first input data and generates portal vein branches for each image unit element of the first image region of the first image.
  • an image unit element in a three-dimensional image may be understood as a voxel, and an image unit element in a secondary image may be understood as a pixel.
  • the first input data is a CT (Computed Tomography) image in which a region including the liver is captured, and a portal vein mask image in which the portal vein region is specified.
  • CT Computer Tomography
  • At least one of the first images may be a CT image or a portal vein mask image.
  • the first input data may include a CT image and a portal vein mask image.
  • the first input data further includes a liver mask image with a liver region identified, a vein mask image with a vein region identified, and an inferior vena cava region identified.
  • the configuration may include at least one of the masked inferior vena cava images.
  • the first input data may be configured to include a portal vein mask image, a liver mask image, and a vein mask image.
  • the first image area may be the entire area of the first image
  • the second image area may be the entire area of the second image.
  • the portal vein branch labels may be labels that classify portal vein branches into eight classes corresponding to eight types of liver segments S1 to S8.
  • the trained model may be configured using a convolutional neural network.
  • the machine learning process for generating the learned model includes the score map indicating the likelihood of the portal branch label output from the learned model, the first input data calculating losses only for portal regions labeled with portal branch labels in the portal branch labeling data corresponding to and updating parameters of the learning model based on the calculated losses.
  • each of the first image and the second image may be a three-dimensional image.
  • the processor labels the liver segment with a liver segment label based on the portal vein branch label assigned to each image unit element of the second image region. It may be a configuration.
  • the second input data includes a CT image obtained by imaging a region including the liver
  • the processor extracts the liver region from the CT image included in the second input data. is extracted, and the label information labeled with respect to the region other than the extracted liver region in the second image region may be invalidated.
  • Invalidating label information includes concepts such as deleting label information or ignoring label information, for example.
  • the processor converts the portal vein branch label assigned to each image unit element of the second image region into a liver segment label, thereby converting the region into the liver segment. It may be configured to generate divided liver segment images.
  • a liver segmentation method is a liver segmentation method in which a computer divides a liver region in an image into liver segments, comprising: first input data including a first image of the liver; Machine learning is performed using learning data including portal vein branch labeling data in which the portal vein branch label is attached to each portal vein branch corresponding to the liver segment for the portal vein region in the liver in the first image.
  • generating a learning model by performing, for each image unit element of the first image region of the first image, based on the labeling result of the portal vein branch label output by the learning model, parameters of the learning model to generate a trained model, receiving second input data that is the same type of input data as the first input data and includes a second image of the liver, and using the trained model giving a portal vein branch label to each image unit element of the second image region of the second image; and based on the portal vein branch label given to each image unit element of the second image region, and dividing a liver region included in the second input data into a plurality of liver segments.
  • a program according to another aspect of the present disclosure is a program that causes a computer to operate as a medical image processing apparatus, and includes first input data including a first image of the liver, and an intrahepatic portal in the first image.
  • first input data including a first image of the liver, and an intrahepatic portal in the first image.
  • a learned model generated by performing machine learning using learning data including portal vein branch labeling data labeled for each portal vein branch corresponding to the liver segment and the trained model receives the input of the first input data, and labels the portal vein branch label for each image unit element of the first image region including at least the liver region of the first image. It is a learning model trained to output.
  • the program causes the computer to accept second input data, which is the same type of input data as the first input data and includes a second image of the liver, and to use the trained model to generate the second input
  • Second input data which is the same type of input data as the first input data and includes a second image of the liver
  • the program causes the computer to accept second input data, which is the same type of input data as the first input data and includes a second image of the liver, and to use the trained model to generate the second input
  • FIG. 1 is a block diagram showing an example of an image processing device that performs processing for generating learning data.
  • FIG. 2 is a block diagram showing an example of an information processing apparatus that labels a portal vein region with respect to portal vein branches.
  • FIG. 3 is a conceptual diagram showing an example of a learning data set stored in a learning data storage unit.
  • FIG. 4 is a conceptual diagram showing an outline of a learning phase when generating a trained model applied to the medical image processing apparatus according to the first embodiment.
  • FIG. 5 is a block diagram showing a configuration example of a learning device.
  • FIG. 6 is a flowchart showing the flow of learning processing by the learning device.
  • FIG. 7 is a conceptual diagram showing an outline of processing in the inference phase using the trained model of the first embodiment.
  • FIG. 8 is a block diagram showing the configuration of the medical image processing apparatus according to the first embodiment.
  • FIG. 9 is a flow chart showing an example of a liver segmentation method using the medical image processing apparatus according to the first embodiment.
  • FIG. 10 is a conceptual diagram showing an overview of the learning phase in the second embodiment.
  • FIG. 11 is a block diagram showing an overview of the inference phase using a trained model generated by the learning method of the second embodiment.
  • FIG. 12 is an image example showing an example of a liver segment segmentation method using Voronoi segmentation according to a comparative example.
  • FIG. 13 is a diagram showing a comparison between a processing result of liver segmentation based on Voronoi segmentation according to a comparative example and a result of appropriate liver segmentation.
  • a CT image obtained by imaging a region including a patient's liver using a CT apparatus will be described as an example.
  • Creating a label for each segment of the liver (correct label for the liver segment) by a doctor on a CT image in which a region including the liver is captured poses a problem that the burden on the doctor is heavy, and there are large individual differences in labeling.
  • the labeling of the portal vein regions by type relative to the intrahepatic portal vein region does not pose a great burden on the physician compared to the task of labeling the liver segments. There is little variation in judgments due to
  • machine learning is performed using image data in which the portal region is labeled according to the type of portal vein branch as one of the data for learning.
  • segmentation of the liver is realized using a trained model obtained as a result of machine learning. That is, the data for learning is data used for machine-learning the learning model 50 described later.
  • a trained model 650 is generated by subjecting the learning model 50 to machine learning using learning data. That is, the learned model 650 is a model obtained by optimizing the parameters of the learning model 50 .
  • the trained model 650 is applied to the medical image processing apparatus 70 according to the first embodiment.
  • FIG. 1 and 2 are block diagrams illustrating examples of methods for generating training data.
  • FIG. 1 shows an example of an image processing apparatus 10 that performs processing for generating a liver mask image LM, a portal vein mask image PM, and a vein mask image HM from a CT image IM.
  • FIG. 2 shows an example of an information processing device 40 that generates a portal vein branch label map PLM from a CT image IM, a liver mask image LM, a portal vein mask image PM, and a vein mask image HM.
  • the data for learning includes the CT image IM, liver mask image LM, portal vein mask image PM, and vein mask image HM, as well as the portal vein branch label map PLM (see FIG. 3).
  • a CT image IM is a three-dimensional image reconstructed from three-dimensional data obtained by continuously capturing two-dimensional slice tomographic images.
  • Each of the liver mask image LM, the portal vein mask image PM, and the vein mask image HM is also a three-dimensional image. Note that the term "image" includes the meaning of image data.
  • the image processing device 10 is implemented using computer hardware and software. Software is synonymous with program.
  • the image processing device 10 includes a processor 12 and a non-transitory, tangible computer-readable medium 14 .
  • the form of the image processing apparatus 10 is not particularly limited, and may be a server, a workstation, or a personal computer.
  • the processor 12 includes a CPU (Central Processing Unit).
  • the processor 12 may include a GPU (Graphics Processing Unit).
  • the computer-readable medium 14 includes memory, which is the main memory, and storage, which is the auxiliary memory.
  • Computer-readable medium 14 may be, for example, a semiconductor memory, a hard disk drive (HDD) device, a solid state drive (SSD) device, or a combination thereof.
  • a computer-readable medium 14 stores a plurality of programs including an image processing program, data, and the like.
  • Processor 12 functions as liver extraction processor 15 , portal vein extraction processor 16 , and vein extraction processor 17 by executing program instructions stored in computer-readable medium 14 .
  • the liver extraction processing unit 15 performs processing for extracting the liver region from the input CT image IM.
  • a liver mask image LM is generated by the liver extraction processing unit 15 .
  • the liver mask image LM is an image in which the liver region is specified. For example, the voxel value of the liver region in the CT image IM is set to “1”, and the voxel value of the other region (non-liver region) is set to “0”. It may be a binary image that
  • the portal vein extraction processing unit 16 performs processing for extracting the portal vein region from the input CT image IM.
  • a portal vein mask image PM is generated by the portal vein extraction processing unit 16 .
  • the portal vein mask image PM is an image in which the portal vein region is specified. It may be a binary image with "0".
  • the vein extraction processing unit 17 performs processing for extracting a vein region from the input CT image IM.
  • a vein mask image HM is generated by the vein extraction processing unit 17 .
  • the vein mask image HM is an image in which a vein region is specified. It may be a binary image that
  • Each of the liver extraction processing unit 15, the portal vein extraction processing unit 16, and the vein extraction processing unit 17 is a trained model trained to generate a mask image from an input image by, for example, machine learning represented by deep learning. may be used to extract regions of the liver, portal vein, or vein.
  • a model for performing such an image recognition task is implemented using, for example, a CNN represented by V-net.
  • the image processing device 10 acquires the CT image IM from the image storage unit 20, and generates a liver mask image LM, a portal vein mask image PM, and a vein mask image HM corresponding to the CT image IM.
  • the images generated by the image processing apparatus 10 are stored in the learning data storage unit 30 in association with the original CT images IM.
  • FIG. 1 shows an example in which the image processing apparatus 10 generates three types of mask images, a liver mask image LM, a portal vein mask image PM, and a vein mask image HM.
  • Mask images are not limited to these.
  • the image processing apparatus 10 may generate other mask images such as the inferior vena cava mask image in which the inferior vena cava region is specified.
  • the image processing apparatus 10 may be configured to generate only some of the mask images of the plurality of types illustrated in FIG. 1, for example, to generate only the portal vein mask image PM. may
  • the image storage unit 20 includes a large-capacity storage in which a large number of images including CT images IM are stored.
  • the image storage unit 20 may be, for example, a DICOM (Digital Imaging and Communication in Medicine) server on a medical institution network.
  • a DICOM server is a server that operates according to the DICOM specification.
  • a DICOM server is a computer that stores and manages various data including images taken using a CT apparatus and other modalities, and is equipped with a large-capacity external storage device and a database management program.
  • the image processing apparatus 10 can acquire a plurality of CT images IM from the image storage unit 20 via a communication line (not shown).
  • the learning data storage unit 30 includes a large-capacity storage that stores data used for learning.
  • the learning data storage unit 30 may be included in the image processing device 10 . Also, part of the storage area of the image storage unit 20 may be used as the learning data storage unit 30 .
  • the information processing device 40 may be a computer including a processor 42 and a non-transitory, tangible computer-readable medium 44 .
  • the hardware configuration of processor 42 and computer readable medium 44 may be similar to the corresponding elements of processor 12 and computer readable medium 14 described in FIG.
  • the form of the information processing device 40 may be a server, a personal computer, a workstation, a tablet terminal, or the like.
  • the information processing device 40 may be a viewer terminal for interpretation.
  • An input device 47 and a display device 48 are connected to the information processing device 40 .
  • the input device 47 is configured by, for example, a keyboard, mouse, multi-touch panel, other pointing device, voice input device, or an appropriate combination thereof.
  • the display device 48 is configured by, for example, a liquid crystal display, an organic electro-luminescence (OEL) display, a projector, or an appropriate combination thereof.
  • OEL organic electro-luminescence
  • the information processing device 40 can acquire the data stored in the learning data storage unit 30 and display it on the display device 48 .
  • the information processing device 40 causes the display device 48 to display the portal vein mask image PM, and receives an input of a portal vein branch label from the input device 47 .
  • the information processing device 40 can acquire not only the portal vein mask image PM, but also a CT image IM, a liver mask image LM, a vein mask image HM, and the like, and display them on the display device 48 .
  • the computer-readable medium 44 stores a plurality of programs, data, etc., including a program for labeling the portal vein branches in the portal vein region of the portal vein mask image PM.
  • Processor 42 functions as portal vein branch labeling processor 46 by executing program instructions stored in computer-readable medium 44 .
  • the portal branch labeling processing unit 46 generates a portal branch label map PLM based on the information (label information) on the portal branch label input to the input device 47 by the doctor Dr.
  • the portal vein branch label is a label for classifying a predetermined image region into eight regions. Again, the predetermined image area is the portal vein area. Therefore, the label information is information specifying which portal vein branch corresponds to each of a plurality of portal vein branch regions, which are a plurality of partial regions included in the intrahepatic portal vein region.
  • the portal branch labeling processing unit 46 receives input of label information, and generates a portal branch label map PLM as teacher data based on the input label information.
  • the portal vein is classified into 8 classes of portal vein branches, S1 to S8, corresponding to the liver segments S1 to S8, respectively. That is, portal veins belonging to the S1 hepatic segment are classified into S1 portal vein branches, portal veins belonging to the S2 hepatic segment into S2 portal vein branches, and so on. Therefore, the label information defines portal vein branch labels S1 to S8 for classifying the portal vein region into eight portal vein branch regions corresponding to liver segments.
  • a table may be created that defines the correspondence between portal vein branch labels and liver segment labels. It is also possible to interpret the portal vein branch label as it is by replacing it with the liver segment label.
  • the doctor Dr who is the user uses the input device 47 to check the portal vein region in the image for each region of the portal vein branch. to label the portal vein branches.
  • the doctor Dr uses the input device 47 to specify the correspondence between the region of each portal vein branch and the portal vein branch label.
  • the portal vein branch labeling processing unit 46 assigns a portal vein branch label to each portal vein branch region included in the portal vein region, and generates a portal vein branch label map PLMj. That is, the portal vein branch labeling processing unit 46 assigns one of eight classes S1 to S8 to the portal vein branch region, which is a partial region of the portal vein region, according to the information input via the input device 47.
  • a portal branch label map PLM with classification labels (portal branch labels) is generated.
  • the portal vein regions are classified into eight classes according to the portal vein branch labels, and the portal vein branch regions are colored according to the portal vein branch labels.
  • the portal branch label map PLMj may be understood to be an image such as a portal branch segmentation image.
  • the portal vein branch label map PLM is linked to the portal vein mask image PM that is the source of generation, and stored in the learning data storage unit 30.
  • the portal vein branch label map PLM is also associated with the original CT image IM and stored in the learning data storage unit 30 .
  • the image processing device 10 and the information processing device 40 are separate devices. It is also possible to implement it with a single computer.
  • FIG. 3 is a conceptual diagram showing an example of a learning data set stored in the learning data storage unit 30.
  • the learning data storage unit 30 stores a plurality of data sets in which CT images IMj, liver mask images LMj, portal vein mask images PMj, vein mask images HMj, and portal vein branch label maps PLMj are linked.
  • the subscript “j” represents an index number for distinguishing multiple data sets.
  • a CT image IMj, a liver mask image LMj, a portal vein mask image PMj, and a vein mask image HMj are prepared as data for input.
  • a map PLMj is provided.
  • a plurality of data sets in which input data and teacher data are linked are shown as learning data sets.
  • the learning data set is a collection of data including a plurality of data sets in which input data and portal vein branch label maps PLMj corresponding to the input data are linked.
  • a CT image IMj a CT image IMj
  • a liver mask image LMj a portal vein mask image PMj
  • a vein mask image HMj a vein mask image HMj
  • the input data should include at least one of the CT image IMj and the portal vein mask image PMj.
  • FIG. 4 is a conceptual diagram showing an overview of the learning phase.
  • machine learning of the learning model 50 is performed based on the input image data, and a trained model 650 is generated.
  • the trained model 650 is applied to the medical image processing apparatus 70 according to the first embodiment.
  • the learning model 50 is constructed using CNN.
  • the learning model 50 may be configured using a neural network based on V-net architecture, for example.
  • the learning model 50 is trained to output a portal vein branch label for a predetermined image region based on input image data (input image).
  • the portal vein branch labels are labels for classifying a predetermined image region into eight regions in association with the eight portal vein branches S1 to S8.
  • all regions of the input image (all image regions) are classified into eight classes as predetermined image regions according to portal vein branch labels.
  • the learning model 50 illustrated in FIG. 4 accepts a CT image IMj, a liver mask image LMj, a portal vein mask image PMj, and a vein mask image HMj as input images.
  • the learning model 50 is trained to output a portal vein branch label for each voxel in the entire image area of the input image.
  • the learning model 50 outputs a score indicating the probability of the portal vein branch label for each voxel over the image area of the input image. That is, the learning model 50 outputs portal vein branch labels and scores for each of all voxels included in the entire image area of the portal vein mask image PMj.
  • a voxel is an example of an "image unit element" in this disclosure.
  • the learning model 50 outputs a prediction map 52 showing portal vein branch labels and scores.
  • the prediction map 52 is a portal branch label score map in which a score indicating the probability of the portal branch label is assigned to each voxel in the entire image area.
  • This score map is a probability map indicating which portal vein branch label of S1 to S8 each voxel has a high probability, and the portal vein branch label is predicted for the entire area of the image (entire image area). map.
  • the entire image area is classified into 8 classes from the S1 portal vein branch to the S8 portal vein branch. Therefore, the prediction map 52 output from the learning model 50 is a probability map for each portal branch label of the S1 portal branch to the S8 portal branch.
  • each image is shown as a two-dimensional slice cross-sectional image for convenience of illustration, but the image actually handled is a three-dimensional image.
  • a portal vein branch label is attached to a partial region of the portal vein region.
  • the learning model 50 assigns a score indicating the probability of a portal vein branch label to each voxel in the entire image region including not only the portal region but also regions other than the portal region in the input image.
  • the loss is calculated by ignoring the area other than the portal vein area, and the information other than the portal vein area is not reflected in the loss.
  • the correct label is attached to the portal vein region. Therefore, in the prediction map 52, only the score predicted for voxels in the portal region is reflected in the loss. On the other hand, in the prediction map 52, the score predicted for voxels other than the portal vein region is ignored without calculating the loss. In this way, the loss between the prediction map 52 and the portal branch label map PLMj is calculated by limiting the target only to the portal vein region, and the parameters of the learning model 50 are updated based on the calculated loss. . It should be noted that the loss may be rephrased as an error.
  • the learning model 50 learn using a plurality of learning data sets, the parameters of the learning model 50 are optimized, and a trained model is obtained as a result of learning.
  • the loss calculation target area is limited to the portal vein area in the image.
  • images containing various forms of portal vein regions are learned.
  • learning that can cover the entire liver region is performed, and the labeling prediction accuracy for each voxel is improved.
  • the input data combining the CT image IMj, liver mask image LMj, portal vein mask image PMj and vein mask image HMj is an example of "first input data" in the present disclosure.
  • the portal vein mask image PMj is an example of the "first image” in the present disclosure.
  • the entire image area of the portal vein mask image PMj is an example of the "first image area” in the present disclosure.
  • the portal branch label map PLMj is an example of "portal branch labeling data" in the present disclosure.
  • a data set including CT images IMj, liver mask images LMj, portal vein mask images PMj, vein mask images HMj, and portal vein branch label maps PLMj is an example of "learning data" in the present disclosure.
  • FIG. 5 is a block diagram showing a configuration example of the learning device 60.
  • the learning device 60 includes a processor 602 , a non-transitory tangible computer-readable medium 604 , a communication interface 606 , and an input/output interface 608 .
  • the hardware configuration of processor 602 and computer-readable medium 604 may be similar to the corresponding elements of processor 12 and computer-readable medium 14 described in FIG.
  • the form of the learning device 60 may be a server, a personal computer, or a workstation.
  • Processor 602 is coupled to computer-readable media 604 , communication interface 606 and input/output interface 608 via bus 610 .
  • Input device 614 and display device 616 are connected to bus 610 via input/output interface 608 .
  • the hardware configuration of the input device 614 and display device 616 may be similar to the corresponding elements of the input device 47 and display device 48 described in FIG.
  • the learning device 60 is connected to a communication line (not shown) via a communication interface 606 and is communicably connected to an external device such as the learning data storage unit 30 .
  • a computer-readable medium 604 stores a plurality of programs including a learning processing program 630 and a display control program 640, data, and the like.
  • the processor 602 functions as each processing unit of the data acquisition unit 632 , the learning model 50 , the loss calculation unit 634 and the optimizer 635 by executing the instruction of the learning processing program 630 .
  • the data acquisition unit 632 acquires learning data from the learning data storage unit 30.
  • a loss calculator 634 calculates a loss between the prediction map 52 and the portal branch label map PLM.
  • the portal vein branch label map PLM is teacher data corresponding to the input data used to generate the prediction map 52 .
  • the loss calculation unit 634 performs loss calculation by limiting the target to the portal vein region where the correct label exists in the portal vein branch label map PLM, and ignores the score value for voxels in regions other than the portal vein region, Not included in loss calculation. Note that the loss calculation by the loss calculator 634 is performed using, for example, a loss function.
  • the optimizer 635 determines the update amount of the parameters of the learning model 50 based on the loss calculated by the loss calculation unit 634, and updates the parameters of the learning model 50.
  • the optimizer 635 updates parameters based on algorithms such as gradient descent.
  • the parameters of the learning model 50 include filter coefficients (weights of connections between nodes) and node biases used for processing each layer of the CNN.
  • the learning device 60 acquires data from the learning data storage unit 30 and executes machine learning of the learning model 50 .
  • the learning device 60 can acquire (read) data and update parameters in units of mini-batches in which a plurality of learning data sets are collected.
  • learning device 60 generates trained model 650 .
  • FIG. 6 is a flowchart showing the flow of learning processing by the learning device 60.
  • the processor 602 acquires data from the learning data storage unit 30 .
  • the processor 602 receives an input of learning data and acquires a learning data set from the learning data storage unit 30 .
  • the processor 602 uses the learning model 50 to generate a prediction map 52 of portal vein branch labels. Specifically, the processor 602 inputs an image (see FIG. 3) included in the input data to the learning model 50, and uses the learning model 50 to generate a portal vein branch label prediction map 52 corresponding to the input data. to generate
  • step S106 the processor 602 computes a loss between the prediction map 52 and the portal branch label map PLM, limiting the target to voxels in the portal region.
  • step S108 the processor 602 updates the parameters of the learning model 50 based on the calculated loss.
  • the operations from step S102 to step S108 may be performed in mini-batch units.
  • the processor 602 determines whether or not to end learning.
  • the learning end condition may be determined based on the value of the loss, or may be determined based on the number of parameter updates. As a method based on the loss value, for example, the learning end condition may be that the loss converges within a specified range. As a method based on the number of updates, for example, the fact that the number of updates reaches a specified number may be used as the learning termination condition.
  • step S110 determines whether the determination result in step S110 is No. If the determination result in step S110 is No, the processor 602 returns to step S102 and continues the learning process. On the other hand, if the determination result of step S110 is Yes determination, the processor 602 ends the flowchart of FIG.
  • a trained model is generated by implementing the learning method shown in the flowchart of FIG.
  • a learning method implemented using the learning device 60 is understood as a method of generating a trained model.
  • FIG. 7 is a conceptual diagram showing an outline of processing in the inference phase using the trained model 650 of the first embodiment.
  • the trained model 650 is a model obtained by updating the parameters of the learning model 50 as a result of learning.
  • the inference phase is the phase of inferring liver segments in newly input image data. Specifically, in the inference phase, liver segmentation images LSs are generated for newly input CT images IMs. A liver segmentation image LSs is generated based on the probability map of portal vein branch labels.
  • the probability map is a map similar to the prediction map 52 output by the learning model 50 .
  • the probability map is also a portal branch label score map, and is a map in which each voxel included in the entire image area is assigned a score indicating the likelihood of the portal branch label.
  • a probability map is output from the trained model 650 . Therefore, the probability map has improved accuracy compared to the prediction map 52 .
  • the liver segmentation image LSs is a segmentation image obtained by dividing the liver region of the newly input data into eight liver segments.
  • a liver segment segment image LSs is generated based on the probability map.
  • the trained model 650 generated by the learning method of the first embodiment receives input of unknown input data of the same kind as the input data used for learning, and calculates the portal vein branch label likelihood score of each voxel in the image. Generate.
  • the likelihood of a portal branch label is synonymous with the likelihood of a portal branch label.
  • the input data of the same type as the input data used for learning is image data obtained by imaging a region including the liver, and is data of a CT image and a plurality of types of mask images (input data in FIG. 3). data).
  • unknown input data means new image data that has not been used for learning.
  • FIG. 7 shows an example of the same type of input data as the input data used for learning (see FIG. 4).
  • a trained model 650 is input with a combination of four types of images: CT images IMs, liver mask images LMs, portal vein mask images PMs, and vein mask images HMs.
  • the suffix “s” is attached to new image data that has not been used for learning and image data obtained as a result of inputting the new image data to the trained model 650 .
  • the liver mask image LMs, portal vein mask image PMs, and vein mask image HMs can be generated by performing liver extraction processing, portal vein extraction processing, and vein extraction processing on CT images IMs, respectively. These extraction processes can be performed by a processing unit similar to the liver extraction processing unit 15, portal vein extraction processing unit 16, and vein extraction processing unit 17 described in FIG.
  • the portal branch label with the highest score among the portal branch labels assigned to each voxel is adopted. That is, each voxel can be output with multiple portal branch labels. A score is also output for each of the multiple portal branch labels.
  • the portal branch label with the highest score among the plurality of portal branch labels is adopted as the portal branch label of the voxel.
  • the trained model 650 performs label conversion, etc., for converting the portal vein branch label into a liver segment label corresponding to the portal vein branch label according to the corresponding relationship between the portal vein branch label and the liver segment label. In this way, the liver region can be divided into liver segments based on the map of portal branch labels.
  • the label conversion includes the concept of replacing the portal branch label with the liver segment label, or regarding (interpreting) the portal vein branch label as the liver segment label.
  • liver segmentation image LSs is a segmentation image in which the liver region is segmented by liver segment labels, or a segmentation image in which the liver region is segmented by portal vein branch labels that can be interpreted as liver segment labels.
  • Input data combining CT images IMs, liver mask images LMs, portal vein mask images PMs, and vein mask images HMs is an example of "second input data" in the present disclosure.
  • the portal vein mask image PMs is an example of the "second image” in the present disclosure.
  • the entire image area of the portal vein mask image PMs is an example of the "second image area” in the present disclosure.
  • the learning model 50 is configured to calculate the score indicating the likeness of the portal vein branch label only for voxels in the liver region, and not to calculate the score indicating the likeness of the portal vein branch label for voxels other than the liver region.
  • the prediction map 52 that is output from the learning model 50 may be a map that includes a portal vein branch label likelihood score for at least each voxel in the liver region in the image, and the score for each voxel is calculated for all voxels in the entire image region. No calculation is required.
  • FIG. 8 is a block diagram showing the configuration of the medical image processing apparatus 70 according to the first embodiment.
  • the medical imaging device 70 includes a processor 702 , a non-transitory tangible computer readable medium 704 , a communication interface 706 , an input/output interface 708 , and a bus 710 .
  • Input device 714 and display device 716 are also connected to bus 710 via input/output interface 708 .
  • Each of these elements may be similar to the corresponding elements of processor 702, computer readable medium 704, communication interface 706, input/output interface 708, bus 710, input device 714, and display device 716 described in FIG.
  • the form of the medical image processing apparatus 70 may be a server, a personal computer, a workstation, a tablet terminal, or the like.
  • the medical image processing apparatus 70 is connected to a communication line (not shown) via a communication interface 706, and is communicably connected to an external device such as a DICOM server.
  • a computer-readable medium 704 stores a plurality of programs including a liver segmentation program 720 and a display control program 750, data, and the like.
  • the processor 702 functions as each processing unit of the trained model 650 and the label conversion unit 724 by executing the instructions of the liver segmentation program 720 .
  • the label conversion unit 724 performs a process of converting portal branch labels into liver segment labels. That is, the label conversion unit 724 performs labeling of liver segment labels based on portal vein branch labels.
  • the label conversion unit 724 may include a liver extraction processing unit 725 that extracts the liver region in the image and a label deletion processing unit 726 that deletes label information attached to voxels other than the liver region.
  • a processing algorithm of the liver extraction processing unit 725 may be the same as that of the liver extraction processing unit 15 described with reference to FIG.
  • the label information for the region other than the liver region is deleted to invalidate the label. It is also possible to use a processing form such as
  • the computer-readable medium 704 may further include at least one of an organ recognition program 740, a disease detection program 742, and a report generation support program 744.
  • the organ recognition program 740 includes a processing module that performs organ segmentation.
  • Organ recognition programs may include a lung segment labeling program, a blood vessel region extraction program, a bone labeling program, and the like.
  • the disease detection program 742 includes detection processing modules corresponding to specific diseases.
  • the disease detection program 742 for example, at least one of a lung nodule detection program, a lung nodule property analysis program, a pneumonia CAD (Computer Aided Diagnosis, Computer Aided Detection) program, a mammary gland CAD program, a liver CAD program, a brain CAD program, and a large intestine CAD program.
  • a lung nodule detection program for example, at least one of a lung nodule detection program, a lung nodule property analysis program, a pneumonia CAD (Computer Aided Diagnosis, Computer Aided Detection) program, a mammary gland CAD program, a liver CAD program, a brain CAD program, and a large intestine CAD program.
  • a lung nodule detection program for example, at least one of a lung nodule detection program, a lung nodule property analysis program, a pneumonia CAD (
  • the report creation support program 744 includes a trained document generation model that generates finding sentence candidates corresponding to target medical images.
  • Various processing programs such as the organ recognition program 740, the disease detection program 742, and the report creation support program 744 apply machine learning such as deep learning to obtain a trained model that has been trained to obtain the desired task output. It may be an AI processing module comprising:
  • AI models for CAD can be constructed using, for example, various CNNs with convolutional layers.
  • Input data for the AI model includes, for example, medical images such as two-dimensional images, three-dimensional images, or moving images, and output from the AI model is, for example, information indicating the position of a diseased area (lesion site) in the image, or It may be information indicating class classification such as a disease name, or a combination thereof.
  • AI models that handle time-series data, document data, etc. can be constructed using, for example, various recurrent neural networks (RNN).
  • the time-series data includes, for example, electrocardiogram waveform data.
  • the document data includes, for example, observation statements created by doctors.
  • the computer-readable medium 704 may further include a program that causes the processor 702 to function as the liver extraction processing unit 15, portal vein extraction processing unit 16, and vein extraction processing unit 17 described in FIG.
  • the processing functions of the medical image processing apparatus 70 may be realized by a plurality of computers. Also, part or all of the processing functions of the medical image processing apparatus 70 may be incorporated into the image processing apparatus 10 described with reference to FIG.
  • FIG. 9 is a flow chart showing an example of a liver segmentation method using the medical image processing apparatus 70 according to the first embodiment.
  • the processor 702 receives input of data including an image to be processed. Once the data is input, at step S204 the processor 702 generates a segmentation image of the portal branch labels.
  • the trained model 560 outputs a probability map of portal branch labels.
  • a probability map is a map in which portal vein branch labels and scores are attached to predetermined image regions, as described above.
  • processor 702 uses trained model 650 to label and score each voxel of the entire image region of the input image, or at least the image region including the liver region.
  • the liver region becomes the predetermined image region described above.
  • the total image area includes both portal and non-portal areas. Furthermore, based on the portal branch label and the score, it is determined to which portal branch label the voxel corresponding to the predetermined image region belongs, and the portal branch label is given to each voxel. As a result, a segmentation image is obtained in which a predetermined image region is classified by the portal vein branch label.
  • step S206 the processor 702 performs label conversion processing and divides the liver region into liver segments based on the portal vein branch label assigned to each voxel.
  • the processor 702 generates a liver segment segmented image LSs. Specifically, the processor 702 performs visualization processing such as clearly indicating the divided liver segment regions by color-coding each region, and generates the liver segment segmented image LSs.
  • the generated liver segment segment image LSs can be displayed on the display device 716, a viewer terminal (not shown), or the like.
  • step S208 the processor 702 ends the flowchart of FIG.
  • the liver region in the CT images IMs can be segmented with high accuracy regardless of how the blood vessels appear in the CT images IMs.
  • Input data used for learning may have various forms.
  • the input data used during learning may be a combination of three types of masks: a liver mask image LM, a portal vein mask image PM, and a vein mask image HM.
  • it may be a combination of two types of masks including at least the liver mask image LM.
  • CT images IM may be used as input data used during learning.
  • the CT image is the "first image” in the present disclosure.
  • the "first image area” is the entire image area of the portal vein mask image.
  • the entire image area of the portal vein mask image generated from the CT image and having the same image area as that of the CT image is the "first image area”.
  • only the portal vein mask image PM may be used as the input data used during learning. An example in which only the portal vein mask image PM is used as input data in the second embodiment will be described below.
  • FIG. 10 is a conceptual diagram showing an overview of the learning phase in the second embodiment. 10, elements identical or similar to those shown in FIGS. 4 and 5 are denoted by the same reference numerals, and redundant description is omitted.
  • the portal vein mask image PMj is used as data for input to the learning model 50 .
  • Other processing is the same as in the first embodiment.
  • ⁇ Preparing data for training ⁇ Learning data used in the second embodiment is prepared, for example, as follows.
  • a portal vein region extraction process is performed on the CT image IMj, and a portal vein mask image PMj is generated as the extraction result.
  • the doctor Dr labels the portal vein regions of the same CT image IMj with a portal vein branch label to label each portal vein branch region.
  • a portal vein branch label map PLMj is generated as teacher data.
  • the portal mask image PMj and the portal branch label map PLMj are linked to obtain a data set of the portal mask image PMj and the portal branch label map PLMj.
  • a sufficient number of data sets for learning data sets are prepared by performing similar processing on a large number of CT images.
  • the portal vein mask image PMj in the second embodiment is an example of "first input data" and "first image” in the present disclosure.
  • FIG. 11 is a block diagram showing an overview of the inference phase using the trained model 650 generated by the learning method of the second embodiment.
  • the same or similar elements as those shown in FIGS. 7 and 8 are denoted by the same reference numerals, and overlapping descriptions are omitted.
  • the configuration of the medical image processing apparatus according to the second embodiment may be the same as the configuration of the medical image processing apparatus 70 described with reference to FIG.
  • the trained model 650 is used, for example, as follows.
  • Processor 702 first extracts a portal vein region from CT images IMs including the liver obtained by imaging a patient using a CT apparatus, and generates portal vein mask images PMs as extraction results. Generate.
  • the processor 702 inputs the portal vein mask image PMs to the trained model 650 .
  • the processor 702 uses the learned model 650 to attach a portal vein branch label to each voxel for the entire region of the input portal vein mask image PMs. Also in this embodiment, as in the first embodiment, the trained model 650 determines the portal branch label with the highest score among the eight classes of portal vein branch labels as the portal branch label of the voxel. . Specifically, the trained model 650 can assign a plurality of portal vein branch labels to each voxel included in the entire image area of the portal vein mask image PMs. A score is also output for each of the multiple portal branch labels. The trained model 650 selects the portal branch label with the highest probability score among the eight classes, no matter how low the score indicating the predicted probability of each voxel is.
  • portal branch label segmentation image 652 A map showing the labeling results of the portal branch labels generated by the trained model 650 is called a portal branch label segmentation image 652 .
  • the processor 702 Based on the portal vein branch label segmentation image 652 generated by the trained model 650, the processor 702 attaches liver segment labels corresponding to the portal vein branch labels on the original CT images IMs.
  • the processor 702 extracts the liver region from the original CT images IMs. Labeling of regions other than the portal vein region is unnecessary. Therefore, the processor 702 removes the labels attached to the regions other than the portal vein region. As a result, only the label attached to the portal vein area remains among the portal vein branch labels attached to the entire image area of the portal vein mask image PMs.
  • the processor 702 performs post-processing such as fine correction of the inference result as necessary.
  • the post-processing here includes, for example, a process of filling an isolated small area with a label of a surrounding large area, ie, a so-called hole-filling process.
  • a definition of a small region may be, for example, a region having a predetermined volume or less.
  • the medical image processing apparatus 70 has a configuration of a processing unit that performs fine correction of labeling results.
  • the liver region is divided into 8 classes of liver segments from S1 to S8, and a liver segment segmented image LSs is generated.
  • the liver segment image LSs can be a segmentation image in which liver regions are classified by liver segment labels.
  • FIG. 12 is an image example showing an example of a liver segment segmentation method using Voronoi segmentation according to a comparative example.
  • the image shown on the left side of FIG. 12 is an example of a CT image from which the portal vein region has been extracted.
  • the image shown in the center of FIG. 12 is an example of a blood vessel labeling diagram showing portal veins labeled by the user specifying branch points of portal vein branches.
  • the image shown on the right side of FIG. 12 is an example of an image showing the result of segmentation of the liver region by Voronoi segmentation based on blood vessel labeling.
  • FIG. 13 is a diagram showing a comparison between the processing result of liver segmentation based on Voronoi segmentation according to the comparative example and the result of proper liver segmentation.
  • the image shown on the left side of FIG. 13 is an example of an image showing the processing result of liver segmentation based on Voronoi division according to the comparative example, and the image shown on the right side of FIG. 13 is the result of correct (correct) liver segmentation.
  • liver segmentation can be performed with high accuracy regardless of how blood vessels are captured. It can be carried out.
  • a program that causes a computer to implement the processing functions of the image processing apparatus 10, the information processing apparatus 40, the learning apparatus 60, and the medical image processing apparatus 70 is stored in a non-temporary device such as an optical disk, a magnetic disk, or a semiconductor memory or other tangible object.
  • the program can be recorded on a computer-readable medium, which is an information storage medium, and the program can be provided through this information storage medium.
  • Liver extraction processing unit 15 portal vein extraction processing unit 16 and vein extraction processing unit 17 in image processing device 10
  • portal vein branch labeling processing unit 46 in information processing device 40
  • data acquisition unit 632 and loss calculation unit 634 in learning device 60
  • the optimizer 635 and the hardware structure of the processing unit (processing unit) that performs various processes such as the label conversion unit 724, the liver extraction processing unit 725, and the label deletion processing unit 726 in the medical image processing apparatus 70 are, for example, Various processors, such as:
  • processors include CPUs, which are general-purpose processors that run programs and function as various processing units, GPUs, which are processors specialized for image processing, and FPGAs (Field Programmable Gate Arrays).
  • PLD Programmable Logic Device
  • ASIC Application Specific Integrated Circuit
  • a single processing unit may be composed of one of these various processors, or may be composed of two or more processors of the same type or different types.
  • one processing unit may be configured by a plurality of FPGAs, a combination of CPU and FPGA, or a combination of CPU and GPU.
  • a plurality of processing units may be configured by one processor.
  • a single processor is configured by combining one or more CPUs and software. There is a form in which a processor functions as multiple processing units.
  • SoC System On Chip
  • the various processing units are configured by using one or more of the above various processors as a hardware structure.
  • the hardware structure of these various processors is, more specifically, an electrical circuit that combines circuit elements such as semiconductor elements.
  • the technology of the present disclosure is not limited to CT images, and can be applied to various medical images captured by various medical devices (modalities).
  • Various medical images include MR images captured using a Magnetic Resonance Imaging (MRI) apparatus, ultrasound images that project human body information, and Positron Emission Tomography (PET) images captured using a PET apparatus. PET images, endoscopic images captured using an endoscope apparatus, and the like are included.
  • the image targeted by the technology of the present disclosure is not limited to a three-dimensional image, and may be a two-dimensional image. In the case of a configuration that handles two-dimensional images, the "voxels" in the contents described in each of the above embodiments are replaced with "pixels".
  • image processing device 12 processor 14 computer readable medium 15 liver extraction processing unit 16 portal vein extraction processing unit 17 vein extraction processing unit 20 image storage unit 30 learning data storage unit 40 information processing device 42 processor 44 computer readable medium 46 portal vein branch Labeling processing unit 47 input device 48 display device 50 learning model 52 prediction map 60 learning device 70 medical image processing device 602 processor 604 computer readable medium 606 communication interface 608 input/output interface 610 bus 614 input device 616 display device 630 learning processing program 632 data Acquisition unit 634 Loss calculation unit 635 Optimizer 640 Display control program 650 Trained model 652 Portal branch label segmentation image 702 Processor 704 Computer readable medium 706 Communication interface 708 Input/output interface 710 Bus 714 Input device 716 Display device 720 Liver segment segmentation program 724 label conversion unit 725 liver extraction processing unit 726 label deletion processing unit 740 organ recognition program 742 disease detection program 744 report creation support program 750 display control program Dr doctor IM, IMj, IMs CT images HM, HMj, HMs vein mask image PM, PMj

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • Public Health (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computational Linguistics (AREA)
  • Primary Health Care (AREA)
  • Epidemiology (AREA)
  • Optics & Photonics (AREA)
  • High Energy & Nuclear Physics (AREA)
  • Biophysics (AREA)
  • Pathology (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Veterinary Medicine (AREA)
  • Apparatus For Radiation Diagnosis (AREA)
PCT/JP2022/027537 2021-08-31 2022-07-13 医療画像処理装置、肝区域分割方法およびプログラム Ceased WO2023032480A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2023545138A JP7812864B2 (ja) 2021-08-31 2022-07-13 医療画像処理装置、肝区域分割方法およびプログラム
US18/587,853 US20240193785A1 (en) 2021-08-31 2024-02-26 Medical image processing apparatus, hepatic segment division method, and program

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2021-141653 2021-08-31
JP2021141653 2021-08-31

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/587,853 Continuation US20240193785A1 (en) 2021-08-31 2024-02-26 Medical image processing apparatus, hepatic segment division method, and program

Publications (1)

Publication Number Publication Date
WO2023032480A1 true WO2023032480A1 (ja) 2023-03-09

Family

ID=85412132

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/027537 Ceased WO2023032480A1 (ja) 2021-08-31 2022-07-13 医療画像処理装置、肝区域分割方法およびプログラム

Country Status (3)

Country Link
US (1) US20240193785A1 (https=)
JP (1) JP7812864B2 (https=)
WO (1) WO2023032480A1 (https=)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP4128140A1 (fr) * 2020-03-24 2023-02-08 Biocellvia Système et procédé pour produire un indicateur à partir d'une image d'une coupe histologique
JP7304475B1 (ja) * 2022-07-11 2023-07-06 アクタピオ，インコーポレイテッド 情報処理方法、情報処理装置および情報処理プログラム

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003033349A (ja) * 2001-07-23 2003-02-04 Hitachi Medical Corp 臓器の特定領域抽出表示方法及び装置
US20080103385A1 (en) * 2005-11-01 2008-05-01 Feng Ma Method and system for liver lobe segmentation and pre-operative surgical planning
JP2013090799A (ja) * 2011-10-26 2013-05-16 Fujifilm Corp 画像処理装置および方法並びにプログラム
JP2015208385A (ja) * 2014-04-24 2015-11-24 株式会社日立製作所 医用画像情報システム、医用画像情報処理方法及びプログラム
JP2017148283A (ja) * 2016-02-25 2017-08-31 富士フイルム株式会社 評価装置、方法およびプログラム
CN111145206A (zh) * 2019-12-27 2020-05-12 联想(北京)有限公司 肝脏图像分割质量评估方法、装置及计算机设备
JP2020120828A (ja) * 2019-01-29 2020-08-13 ザイオソフト株式会社 医用画像処理装置、医用画像処理方法、及び医用画像処理プログラム
WO2020203552A1 (ja) * 2019-03-29 2020-10-08 富士フイルム株式会社 線構造抽出装置及び方法、プログラム並びに学習済みモデル
JP2020170408A (ja) * 2019-04-04 2020-10-15 キヤノン株式会社 画像処理装置、画像処理方法、プログラム
CN112258486A (zh) * 2020-10-28 2021-01-22 汕头大学 基于进化神经架构搜索的眼底图像视网膜血管分割方法
CN112561917A (zh) * 2020-12-22 2021-03-26 上海联影智能医疗科技有限公司 图像的分段方法、系统、电子设备及可读存储介质

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4582974B2 (ja) * 2001-09-07 2010-11-17 株式会社日立メディコ 画像処理装置
JP5559642B2 (ja) * 2010-08-30 2014-07-23 富士フイルム株式会社 手術支援装置、手術支援方法および手術支援プログラム
CN102693540B (zh) * 2012-04-24 2016-12-14 深圳市旭东数字医学影像技术有限公司 一种肝脏分段的方法及其系统
CN111161241B (zh) * 2019-12-27 2024-04-23 联想(北京)有限公司 一种肝脏图像识别方法、电子设备及存储介质
KR20230092947A (ko) * 2020-10-22 2023-06-26 비져블 페이션트 의료용 이미지 내에서 적어도 하나의 관상 구조체를 세그멘트화하고 식별하기 위한 방법 및 시스템
CN112733708A (zh) * 2021-01-08 2021-04-30 山东交通学院 一种基于半监督学习的肝门静脉检测定位方法与系统
CN112842371A (zh) * 2021-01-29 2021-05-28 上海商汤智能科技有限公司 图像处理方法、装置、电子设备及存储介质
CN113658186A (zh) * 2021-07-21 2021-11-16 杭州深睿博联科技有限公司 一种基于深度学习的肝段分割方法及装置

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003033349A (ja) * 2001-07-23 2003-02-04 Hitachi Medical Corp 臓器の特定領域抽出表示方法及び装置
US20080103385A1 (en) * 2005-11-01 2008-05-01 Feng Ma Method and system for liver lobe segmentation and pre-operative surgical planning
JP2013090799A (ja) * 2011-10-26 2013-05-16 Fujifilm Corp 画像処理装置および方法並びにプログラム
JP2015208385A (ja) * 2014-04-24 2015-11-24 株式会社日立製作所 医用画像情報システム、医用画像情報処理方法及びプログラム
JP2017148283A (ja) * 2016-02-25 2017-08-31 富士フイルム株式会社 評価装置、方法およびプログラム
JP2020120828A (ja) * 2019-01-29 2020-08-13 ザイオソフト株式会社 医用画像処理装置、医用画像処理方法、及び医用画像処理プログラム
WO2020203552A1 (ja) * 2019-03-29 2020-10-08 富士フイルム株式会社 線構造抽出装置及び方法、プログラム並びに学習済みモデル
JP2020170408A (ja) * 2019-04-04 2020-10-15 キヤノン株式会社 画像処理装置、画像処理方法、プログラム
CN111145206A (zh) * 2019-12-27 2020-05-12 联想(北京)有限公司 肝脏图像分割质量评估方法、装置及计算机设备
CN112258486A (zh) * 2020-10-28 2021-01-22 汕头大学 基于进化神经架构搜索的眼底图像视网膜血管分割方法
CN112561917A (zh) * 2020-12-22 2021-03-26 上海联影智能医疗科技有限公司 图像的分段方法、系统、电子设备及可读存储介质

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
CIECHOLEWSKI MARCIN, KASSJAŃSKI MICHAŁ: "Computational Methods for Liver Vessel Segmentation in Medical Imaging: A Review", SENSORS, vol. 21, no. 6, pages 2027, XP093043190, DOI: 10.3390/s21062027 *
HIROYUKI YAMAMOTO, MASATOSHI HORI, YOSHINOBU SATO, TAKAMICHI MURAKAMI, HIRONOBU NAKAMURA, SHINICHI TAMURA : "Identification of the liver segmental anatomy based on global location of peripheral portal venous branches using 3D CT data", IEICE TECHNICAL REPORT, MI, IEICE, JP, vol. 104, no. 580 (MI2004-111), 1 January 2005 (2005-01-01), JP, pages 169 - 174, XP009544130 *

Also Published As

Publication number Publication date
JP7812864B2 (ja) 2026-02-10
JPWO2023032480A1 (https=) 2023-03-09
US20240193785A1 (en) 2024-06-13

Similar Documents

Publication Publication Date Title
Wong et al. Scribbleprompt: fast and flexible interactive segmentation for any biomedical image
JP6947759B2 (ja) 解剖学的対象物を自動的に検出、位置特定、及びセマンティックセグメンテーションするシステム及び方法
Carvalho et al. 3D segmentation algorithms for computerized tomographic imaging: a systematic literature review
CN112686899B (zh) 医学图像分析方法和装置、计算机设备及存储介质
JP7187680B2 (ja) 線構造抽出装置及び方法、プログラム並びに学習済みモデル
US12125208B2 (en) Method and arrangement for automatically localizing organ segments in a three-dimensional image
US20240193785A1 (en) Medical image processing apparatus, hepatic segment division method, and program
US20230207106A1 (en) Image segmentation for sets of objects
US12573062B2 (en) Image processing method, image processing device, program, and trained model
JP2024054748A (ja) 言語特徴抽出モデルの生成方法、情報処理装置、情報処理方法及びプログラム
EP3248172A1 (en) Atlas-based determination of tumour growth direction
US20240005498A1 (en) Method of generating trained model, machine learning system, program, and medical image processing apparatus
La Rosa A deep learning approach to bone segmentation in CT scans
El Hadramy et al. Towards real-time vessel guided augmented reality for liver surgery
JP5992853B2 (ja) 手術支援装置、方法およびプログラム
WO2014155916A1 (ja) 手術支援装置、方法およびプログラム
Wu et al. Clinical tooth segmentation based on local enhancement
CN119150993A (zh) 多模态生成模型训练方法及装置、文本生成方法及装置
Sharma et al. Importance of deep learning models to perform segmentation on medical imaging modalities
Feng et al. Segmenting computed tomograms for cardiac ablation using machine learning leveraged by domain knowledge encoding
US11501442B2 (en) Comparison of a region of interest along a time series of images
Lahreche et al. Medical Image Semantic Segmentation Using Deep Learning: A Survey
Guachi et al. 3D printing of prototypes starting from medical imaging: A liver case study
Vidhya et al. A systematic approach for constructing 3D MRI brain image over 2D images
US20250226095A1 (en) Label generation method, label generation device, trained model generation method, machine learning device, image processing method, image processing device, and program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22864068

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2023545138

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22864068

Country of ref document: EP

Kind code of ref document: A1