WO2023119866A1 - 情報処理装置、情報処理装置の作動方法、情報処理装置の作動プログラム、予測モデル、学習装置、および学習方法 - Google Patents

情報処理装置、情報処理装置の作動方法、情報処理装置の作動プログラム、予測モデル、学習装置、および学習方法 Download PDF

Info

Publication number
WO2023119866A1
WO2023119866A1 PCT/JP2022/040266 JP2022040266W WO2023119866A1 WO 2023119866 A1 WO2023119866 A1 WO 2023119866A1 JP 2022040266 W JP2022040266 W JP 2022040266W WO 2023119866 A1 WO2023119866 A1 WO 2023119866A1
Authority
WO
WIPO (PCT)
Prior art keywords
disease
data
related data
image
patch
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/040266
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
彩華 王
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fujifilm Corp
Original Assignee
Fujifilm Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fujifilm Corp filed Critical Fujifilm Corp
Priority to JP2023569121A priority Critical patent/JPWO2023119866A1/ja
Publication of WO2023119866A1 publication Critical patent/WO2023119866A1/ja
Priority to US18/673,322 priority patent/US20240312011A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/05—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves
    • A61B5/055—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/70—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for mining of medical data, e.g. analysing previous cases of other patients
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10088—Magnetic resonance imaging [MRI]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30016—Brain

Definitions

  • the technology of the present disclosure relates to an information processing device, an information processing device operating method, an information processing device operating program, a prediction model, a learning device, and a learning method.
  • Document 1 includes a magnetic resonance imaging (MRI) tomographic image (hereinafter referred to as an MRI image) of the brain of a subject who predicts the progression of dementia, Dementia-related data such as the subject's age, gender, genetic test data, and cognitive function test data (cognitive ability test score) are input, and the results of predicting the progression of dementia are output.
  • MRI magnetic resonance imaging
  • Dementia-related data such as the subject's age, gender, genetic test data, and cognitive function test data (cognitive ability test score) are input, and the results of predicting the progression of dementia are output.
  • a predictive model is described.
  • the brain has various anatomical areas such as the hippocampus, parahippocampal gyrus, amygdala, frontal lobe, temporal lobe, and occipital lobe. And the relationship between each anatomical segment and cognitive performance is different.
  • the prediction model described in Document 1 deals with MRI images of the entire brain and does not consider anatomical regions.
  • a method can be considered in which the MRI image is subdivided into a plurality of patch images and input to the prediction model, and the feature amount of each of the plurality of patch images is extracted by the prediction model.
  • this method is adopted, in the prediction model described in Document 1, correlation information between multiple patch images and correlation information between multiple patch images and dementia-related data (dementia-related data).
  • dementia-related data it is not possible to use the correlation information between multiple dementia-related data for prediction due to structural reasons, and the accuracy of predicting the progression of dementia could not be significantly improved. rice field.
  • One embodiment of the technology of the present disclosure provides an information processing device capable of increasing the prediction accuracy of a prediction result regarding a disease by a prediction model, an operation method of the information processing device, an operation program of the information processing device, a prediction model, learning, Apparatus and learning methods are provided.
  • An information processing apparatus of the present disclosure includes a processor, the processor acquires a medical image showing organs of a subject and disease-related data of the subject, subdivides the medical image into a plurality of patch images, A feature quantity extraction unit for extracting a feature quantity from disease-related data; and a correlation information extraction unit for extracting at least correlation information between the plurality of patch images and correlation information between the plurality of patch images and the disease-related data.
  • a predictive model is used, patch images and disease-related data are input to the predictive model, and disease-related predictive results are output from the predictive model.
  • the prediction model preferably has a transformer encoder that takes in input data in which patch images and disease-related data are mixed and extracts feature values.
  • the feature amount extraction unit includes a self-attention mechanism layer of a transformer encoder
  • the correlation information extraction unit includes a linear transformation layer that linearly transforms input data to the self-attention mechanism layer into first transformation data, and a linear transformation layer that transforms input data to the self-attention mechanism layer into first transformation data an activation function application layer that applies an activation function to generate second transformation data; and a computing unit that computes the product of each element of the output data from the self-attention mechanism layer and the second transformation data as correlation information. is preferably included.
  • the disease is dementia
  • the medical image is an image of the subject's brain
  • the processor extracts from the medical image a first segment image including the hippocampus, amygdala, and entorhinal cortex and a second segment image including the temporal lobe and the frontal lobe. It is preferable to extract two area images and subdivide the first area image and the second area image into a plurality of patch images.
  • the disease is dementia
  • the medical image is morphological image test data
  • the disease-related data is at least one of subject's age, sex, blood/cerebrospinal fluid test data, genetic test data, and cognitive function test data preferably include one.
  • the morphological imaging data is preferably a tomographic image obtained by nuclear magnetic resonance imaging.
  • a method of operating an information processing apparatus includes acquiring a medical image showing organs of a subject and disease-related data of the subject, subdividing the medical image into a plurality of patch images, and analyzing the patch images and the disease. Prediction including a feature quantity extraction unit for extracting a feature quantity from related data, and a correlation information extraction unit for extracting at least correlation information between a plurality of patch images and correlation information between a plurality of patch images and disease-related data Using the model, and inputting the patch image and disease-related data into the predictive model and having the predictive model output predictive results about the disease.
  • the operating program of the information processing apparatus of the present disclosure acquires a medical image showing the organs of a subject and disease-related data of the subject, subdivides the medical image into a plurality of patch images, Prediction including a feature quantity extraction unit for extracting a feature quantity from related data, and a correlation information extraction unit for extracting at least correlation information between a plurality of patch images and correlation information between a plurality of patch images and disease-related data
  • Prediction including a feature quantity extraction unit for extracting a feature quantity from related data, and a correlation information extraction unit for extracting at least correlation information between a plurality of patch images and correlation information between a plurality of patch images and disease-related data
  • a computer is caused to perform a process including using the model and inputting the patch image and disease-related data into the prediction model and causing the prediction model to output a prediction result regarding the disease.
  • the prediction model of the present disclosure includes a feature amount extraction unit that extracts a feature amount from a plurality of patch images obtained by subdividing a medical image showing the organs of a subject and disease-related data of the subject, and between the plurality of patch images a correlation information extraction unit that extracts at least correlation information and correlation information between the plurality of patch images and the disease-related data, and outputs a prediction result regarding the disease according to the input of the patch images and the disease-related data.
  • the learning device of the present disclosure provides a prediction model with medical images for learning and disease-related data for learning as learning data, and inputs patch images obtained by subdividing medical images showing organs of a subject and disease-related data of the subject.
  • the prediction model is a learning device that learns a prediction model so as to obtain a prediction result related to a disease as an output according to the condition.
  • the prediction model includes a feature extraction unit that extracts features from patch images and disease-related data a correlation information extraction unit for extracting at least correlation information between images and correlation information between the plurality of patch images and the disease-related data.
  • medical images for learning and disease-related data for learning are given to a prediction model as learning data, and patch images obtained by subdividing medical images showing organs of a subject and disease-related data of the subject are input.
  • This is a learning method that trains a prediction model so as to obtain a prediction result related to a disease as an output, depending on the a correlation information extraction unit for extracting at least correlation information between images and correlation information between the plurality of patch images and the disease-related data.
  • an information processing device an operation method of the information processing device, an operation program of the information processing device, a prediction model, a learning device, and a learning device capable of increasing the prediction accuracy of a prediction result regarding a disease by a prediction model can provide a method.
  • FIG. 3 is a block diagram showing a computer that constitutes an information processing server;
  • FIG. It is a block diagram which shows the process part of CPU of an information processing server.
  • FIG. 4 is a diagram conceptually showing processing of a patch image generation unit; It is a block diagram which shows the detailed structure of a prediction model.
  • FIG. 3 is a diagram showing a detailed configuration of a transformer encoder;
  • FIG. 4 is a diagram showing the detailed configuration of the first structural section;
  • FIG. 4 is a diagram showing an outline of processing in a prediction model learning phase;
  • 4 is a flow chart showing a processing procedure of an information processing server;
  • FIG. 11 is a block diagram showing a processing unit of a CPU of an information processing server of the second embodiment and an outline of processing;
  • an information processing server 10 is connected to user terminals 11 via a network 12 .
  • the information processing server 10 is an example of an “information processing device” according to the technology of the present disclosure.
  • the user terminal 11 is installed in, for example, a medical facility and operated by a doctor who diagnoses dementia, particularly Alzheimer's dementia, at the medical facility.
  • Dementia is an example of a "disease” related to the technology of the present disclosure.
  • Dementia includes Alzheimer's dementia, Lewy body dementia, vascular dementia, and the like.
  • the content of the diagnosis may be used for Alzheimer's disease other than Alzheimer's dementia.
  • PAD Preclinical Alzheimer's disease
  • MCI Mild Cognitive Impairment
  • the disease is preferably a cranial nerve disease such as dementia as an example.
  • Data related to the diagnostic criteria for dementia include cognitive function test data, morphological image test data, brain function image test data, blood/cerebrospinal fluid test data, and genetic test data.
  • Cognitive function test data includes clinical dementia evaluation method (hereinafter abbreviated as CDR-SOB (Clinical Dementia Rating-Sum of Boxes)) score, mini-mental state examination (hereinafter abbreviated as MMSE (Mini-Mental State Examination) ) and Alzheimer's disease assessment scale (hereinafter abbreviated as ADAS-Cog (Alzheimer's Disease Assessment Scale-cognitive subscale)).
  • the morphological imaging data include the MRI image 16, a brain tomographic image (hereinafter referred to as a CT image) obtained by computed tomography (CT), and the like.
  • Brain functional imaging test data includes brain tomographic images (hereinafter referred to as PET images) by positron emission tomography (PET), brain tomography by single photon emission tomography (SPECT) images (hereinafter referred to as SPECT images) and the like.
  • Blood and cerebrospinal fluid test data include the amount of p-tau (phosphorylated tau protein) 181 in cerebrospinal fluid (hereinafter abbreviated as CSF (Cerebrospinal Fluid)).
  • CSF Cerebrospinal Fluid
  • the genetic test data includes the genotype test results of the ApoE gene.
  • the user terminal 11 has a display 13 and input devices 14 such as a keyboard and a mouse.
  • the network 12 is, for example, a WAN (Wide Area Network) such as the Internet or a public communication network. Note that although only one user terminal 11 is connected to the information processing server 10 in FIG.
  • the user terminal 11 transmits a prediction request 15 to the information processing server 10 .
  • the prediction request 15 is a request for causing the information processing server 10 to predict the progression of dementia using the prediction model 41 (see FIG. 5).
  • Prediction request 15 includes MRI images 16 and dementia-related data 17 .
  • the MRI image 16 and dementia-related data 17 are data on the transmission date of the prediction request 15 .
  • the MRI image 16 and dementia-related data 17 may be data immediately before the date of transmission of the prediction request 15, for example, data from three days to one week before the date of transmission of the prediction request 15.
  • the MRI image 16 is an image of the subject's brain for predicting the progression of dementia.
  • the MRI image 16 is voxel data representing the three-dimensional shape of the subject's brain (see FIG. 6).
  • the MRI image 16 is an example of a “medical image” and “morphological imaging data” according to the technology of the present disclosure.
  • the brain is an example of an “organ” according to the technology of the present disclosure.
  • the dementia-related data 17 is data related to the subject's dementia.
  • the MRI image 16 is obtained, for example, from a PACS (Picture Archiving and Communication System) server.
  • the dementia-related data 17 is obtained, for example, from an electronic medical record server.
  • the dementia-related data 17 is input by operating the input device 14 by a doctor.
  • the dementia-related data 17 is an example of "disease-related data" according to the technology of the present disclosure.
  • the prediction request 15 also includes a terminal ID (Identification Data) and the like for uniquely identifying the user terminal 11 from which the prediction request 15 is transmitted.
  • the information processing server 10 uses the prediction model 41 to predict the progression of dementia of the subject and derives the prediction result 18.
  • the information processing server 10 distributes the prediction result 18 to the user terminal 11 that sent the prediction request 15 .
  • the user terminal 11 displays the prediction result 18 on the display 13 for viewing by the doctor.
  • the dementia-related data 17 includes the subject's age, gender, genetic test data, cognitive function test data, and CSF test data.
  • Genetic test data is, for example, genotype test results of the ApoE gene.
  • the genotype of the ApoE gene is a combination of two of the three ApoE genes ⁇ 2, ⁇ 3, and ⁇ 4 ( ⁇ 2 and ⁇ 3, ⁇ 3 and ⁇ 4, etc.).
  • the risk of developing type dementia is approximately 3 to 12 times higher.
  • Cognitive function test data are, for example, CDR-SOB scores.
  • CSF test data is, for example, the amount of p-tau (phosphorylated tau protein) 181 in CSF.
  • CSF test data is an example of "blood/cerebrospinal fluid test data" according to the technology of the present disclosure.
  • the prediction result 18 indicates whether the subject will or will not develop Alzheimer's disease within two years.
  • the computer that configures the information processing server 10 includes a storage 30, a memory 31, a CPU (Central Processing Unit) 32, a communication section 33, a display 34, and an input device 35. These are interconnected via bus lines 36 .
  • a storage 30, a memory 31, a CPU (Central Processing Unit) 32, a communication section 33, a display 34, and an input device 35 are interconnected via bus lines 36 .
  • CPU Central Processing Unit
  • the storage 30 is a hard disk drive built into the computer that constitutes the information processing server 10 or connected via a cable or network.
  • the storage 30 is a disk array in which a plurality of hard disk drives are connected.
  • the storage 30 stores a control program such as an operating system, various application programs, various data associated with these programs, and the like.
  • a solid state drive may be used instead of the hard disk drive.
  • the memory 31 is a work memory for the CPU 32 to execute processing.
  • the CPU 32 loads a program stored in the storage 30 into the memory 31 and executes processing according to the program. Thereby, the CPU 32 comprehensively controls each part of the computer.
  • the CPU 32 is an example of a "processor" according to the technology of the present disclosure. Note that the memory 31 may be built in the CPU 32 .
  • the communication unit 33 controls transmission of various information with external devices such as the user terminal 11.
  • the display 34 displays various screens. Various screens are provided with operation functions by GUI (Graphical User Interface).
  • the computer that configures the information processing server 10 receives input of operation instructions from the input device 35 through various screens.
  • the input device 35 is a keyboard, mouse, touch panel, microphone for voice input, and the like.
  • the storage 30 of the information processing server 10 stores an operating program 40 .
  • the operating program 40 is an application program for causing the computer to function as the information processing server 10 . That is, the operating program 40 is an example of the "information processing device operating program" according to the technology of the present disclosure.
  • a prediction model 41 is also stored in the storage 30 .
  • the CPU 32 of the computer that constitutes the information processing server 10 cooperates with the memory 31 and the like to operate a reception unit 45 and a read/write (hereinafter abbreviated as RW (Read Write)) control unit 46. , a patch image generation unit 47 , a prediction unit 48 , and a distribution control unit 49 .
  • RW Read Write
  • the reception unit 45 receives the prediction request 15 from the user terminal 11.
  • Prediction request 15 includes MRI images 16 and dementia-related data 17 as previously described. Therefore, the receiving unit 45 acquires the MRI image 16 and the dementia-related data 17 by receiving the prediction request 15 .
  • the reception unit 45 outputs the acquired MRI image 16 and dementia-related data 17 to the RW control unit 46 .
  • the receiving unit 45 also outputs the terminal ID of the user terminal 11 (not shown) to the distribution control unit 49 .
  • the RW control unit 46 controls storage of various data in the storage 30 and reading of various data in the storage 30 .
  • the RW control unit 46 stores the MRI image 16 and the dementia-related data 17 from the reception unit 45 in the storage 30 .
  • the RW control unit 46 also reads the MRI image 16 and the dementia-related data 17 from the storage 30 , outputs the MRI image 16 to the patch image generation unit 47 , and outputs the dementia-related data 17 to the prediction unit 48 .
  • the RW control unit 46 reads the prediction model 41 from the storage 30 and outputs the prediction model 41 to the prediction unit 48 .
  • the patch image generator 47 subdivides the MRI image 16 into a plurality of patch images 55 .
  • the patch image 55 has a size of 8 pixels ⁇ 8 pixels ⁇ 8 pixels, for example.
  • the patch image generation unit 47 outputs a patch image group 55G, which is a set of multiple patch images 55, to the prediction unit 48.
  • the prediction unit 48 inputs the patch image group 55G and the dementia-related data 17 to the prediction model 41, and outputs the prediction result 18 from the prediction model 41.
  • the prediction section 48 outputs the prediction result 18 to the distribution control section 49 .
  • the distribution control unit 49 controls distribution of the prediction result 18 to the user terminal 11 that sent the prediction request 15 . At this time, the distribution control unit 49 identifies the user terminal 11 that is the transmission source of the prediction request 15 based on the terminal ID from the reception unit 45 .
  • the prediction model 41 includes a patch image linear projection unit 60, a dementia-related data linear projection unit 61, a transformer encoder 62, a sequence pooling unit 63, and a multi-layer perceptron (MLP: Multi Layer Perceptron) head 64.
  • the patch image linear projection unit 60 converts each of the plurality of patch images 55 forming the patch image group 55G into sequence data and linearly projects the sequence data. Specifically, the patch image linear projection unit 60 first converts each patch image 55 into a one-dimensional vector. Then, each one-dimensional patch image 55 is linearly projected onto a multi-dimensional, for example, 64-dimensional tensor through a filter.
  • a filter for linear projection is learned in the learning phase of the prediction model 41 (see FIG. 10).
  • the patch image linear projection unit 60 thus outputs a plurality of tensor data (referred to as patch embedding) 70 obtained by linearly projecting each patch image 55 to the transformer encoder 62 .
  • position information 71 is added to the tensor data 70 (called position embedding).
  • the position information 71 is information for identifying where in the MRI image 16 the patch image 55 is located.
  • the dementia-related data linear projection unit 61 converts each of the subject's age, sex, genetic test data, cognitive function test data, and CSF test data that constitute the dementia-related data 17 into sequence data, and then performs linear projection. do. Specifically, the dementia-related data linear projection unit 61 first converts each of the dementia-related data 17 into a one-dimensional vector. Then, each of the one-dimensional dementia-related data 17 is linearly projected onto a multidimensional, for example, 64-dimensional tensor through a filter. Similar to the patch image linear projection unit 60 , the linear projection filter is learned in the learning phase of the prediction model 41 .
  • the dementia-related data linear projection unit 61 thus outputs tensor data 72 obtained by linearly projecting each of the dementia-related data 17 to the transformer encoder 62 . That is, tensor data 70 based on the patch image 55 and tensor data 72 based on the dementia-related data 17 are simultaneously input to the transformer encoder 62 .
  • a set of tensor data 70, position information 71, and tensor data 72 is hereinafter referred to as first input data 73_1.
  • the first input data 73_1 is an example of "input data in which a patch image and dementia-related data are mixed" according to the technology of the present disclosure.
  • the transformer encoder 62 extracts the feature quantity 74 from the first input data 73_1.
  • the feature quantity 74 is a set of numerical values, for example, thousands to hundreds of thousands.
  • the transformer encoder 62 outputs the feature quantity 74 to the sequence pooling section 63 .
  • Transformer encoder 62 is trained during the training phase of predictive model 41 .
  • the sequence pooling unit 63 obtains the statistic of the feature quantity 74, here the average value, and outputs the obtained average value to the multi-layer perceptron head 64 as an aggregated feature quantity 74G.
  • the statistic is not limited to the average value, and may be the maximum value or the like.
  • the multi-layer perceptron head 64 converts the aggregate feature quantity 74G into the prediction result 18. Multilayer perceptron head 64 is trained in the training phase of predictive model 41 .
  • the transformer encoder 62 includes a first structure portion 80_1, a second structure portion 80_2, . including. These plurality of structural portions 80 have the same structure.
  • the first input data 73_1 is input to the first structural section 80_1.
  • the first structure unit 80_1 outputs first output data 81_1 based on the first input data 73_1.
  • the first output data 81 is input to the second structure section 80_2. That is, the first output data 81_1 is also the second input data 73_2 of the second structure section 80_2.
  • the second structure unit 80_2 outputs second output data 81_2 based on the second input data 73_2.
  • the second output data 81_2 is input to a third structural section (not shown). That is, the second output data 81_2 is also the third input data 73_3 of the third structural section.
  • the output data 81 of the structure section 80 at the front stage is repeatedly input as the input data 73 to the structure section 80 at the rear stage.
  • the Nth output data 81_N is output from the Nth structure section 80_N.
  • This Nth output data 81_N is nothing but the feature quantity 74 that is the final output of the transformer encoder 62 .
  • the first structure section 80_1 includes a feature amount extraction section 85, a correlation information extraction section 86, a multi-layer perceptron 87, and an addition section 88.
  • Feature extractor 85 includes self-attention mechanism layer 90 .
  • Correlation information extraction unit 86 includes linear transformation layer 91 , activation function application layer 92 , and calculation unit 93 .
  • the first structure portion 80_1 will be described below as a representative.
  • the first input data 73_1 is input to the self-attention mechanism layer 90 .
  • the self-attention mechanism layer 90 acquires the query, key, and value of each tensor data 70 and 72 of the first input data 73_1, and calculates the similarity between the query and the key.
  • the self-attention mechanism layer 90 generates an attention weight map showing the corresponding relationship between each patch image 55 and the dementia-related data 17 .
  • the attention weight map is a set of numerical values between 0 and 1 indicating which of the first input data 73_1 should be paid attention to.
  • the self-attention mechanism layer 90 treats the numerical values of the attention weight map as probabilities and calculates the correspondence between the query and the value, thereby converting the first input data 73_1 into the intermediate output data 95 .
  • Self-attention mechanism layer 90 outputs intermediate output data 95 to arithmetic unit 93 .
  • the intermediate output data 95 is an example of "output data from the self-attention mechanism layer" according to the technology of the present disclosure.
  • the first input data 73_1 is also input to the linear transformation layer 91 .
  • the linear transformation layer 91 linearly transforms the first input data 73_1 into first transformation data 96 .
  • Linear transformation layer 91 outputs first transformation data 96 to activation function application layer 92 .
  • the activation function application layer 92 applies an activation function such as a sigmoid function to the first transformed data 96 to obtain second transformed data 97 .
  • the activation function application layer 92 outputs the second conversion data 97 to the calculation section 93 .
  • the computing unit 93 computes the product of each element of the intermediate output data 95 from the self-attention mechanism layer 90 and the second transformed data 97 from the activation function application layer 92 .
  • a calculation result 98 of the product of each element of the intermediate output data 95 and the second conversion data 97 is correlation information between the plurality of patch images 55 and correlation information between the plurality of patch images 55 and each of the dementia-related data 17. Correlation information and correlation information between each of the dementia-related data 17 .
  • the calculation unit 93 outputs the calculation result 98 to the multi-layer perceptron 87 .
  • the multi-layer perceptron 87 linearly transforms the computation result 98 and outputs it to the adding section 88 .
  • the adder 88 adds the first input data 73_1 and the operation result 98 after the linear conversion to obtain first output data 81_1. As described above, the first output data 81_1 is input to the second structure section 80_2 as the second input data 73_2.
  • the prediction model 41 extracts the feature quantity 74 from the plurality of patch images 55 obtained by subdividing the MRI image 16 of the subject's brain and the dementia-related data 17 of the subject.
  • a correlation information extraction unit 86 that extracts the feature amount extraction processing by 85, the correlation information between the plurality of patch images 55, and the calculation result 98 as the correlation information between the plurality of patch images 55 and the dementia-related data 17.
  • a prediction result output process by a multi-layer perceptron head 64 that outputs a prediction result 18 related to dementia according to the input of the patch image 55 and the dementia-related data 17.
  • the predictive model 41 is learned in the learning phase given learning data (also called teacher data or training data) 100 .
  • the learning data 100 is a set of MRI images for learning 16L, dementia-related data for learning 17L, and correct data 18CA.
  • the MRI images for learning 16L and the dementia-related data for learning 17L are, for example, MRI images 16 and dementia-related data of certain sample subjects (including patients) accumulated in a database such as ADNI (Alzheimer's Disease Neuroimaging Initiative). 17.
  • the correct data 18CA is the diagnosis result of Alzheimer's type dementia that the doctor actually gave to the sample subject.
  • the prediction model 41 is input with learning MRI images 16L and learning dementia-related data 17L.
  • the prediction model 41 outputs learning prediction results 18L for learning MRI images 16L and learning dementia-related data 17L.
  • a loss calculation of the prediction model 41 is performed based on the learning prediction result 18L and the correct data 18CA.
  • Various coefficients of the prediction model 41 are updated according to the result of the loss calculation, and the prediction model 41 is updated according to the update settings.
  • the learning phase input to the prediction model 41 of the MRI image 16L for learning and dementia-related data 17L for learning, the output of the prediction result 18L for learning from the prediction model 41, the loss calculation, the update setting, and the prediction model 41
  • the above series of updating processes are repeated while the learning data 100 are exchanged at least two times. Repetition of the above series of processes ends when the prediction accuracy of the learning prediction result 18L with respect to the correct data 18CA reaches a predetermined set level.
  • the prediction model 41 whose prediction accuracy reaches the set level in this manner is stored in the storage 30 and used by the prediction unit 48 . It should be noted that regardless of the prediction accuracy of the learning prediction result 18L for the correct data 18CA, the learning may be terminated when the above series of processes are repeated a set number of times.
  • the reception unit 45 receives the prediction request 15 from the user terminal 11, thereby acquiring the MRI image 16 and the dementia-related data 17 (step ST100).
  • the MRI image 16 and the dementia-related data 17 are output from the reception unit 45 to the RW control unit 46 and stored in the storage 30 under the control of the RW control unit 46 .
  • the MRI image 16 and dementia-related data 17 are read from the storage 30 by the RW control unit 46 .
  • the MRI image 16 is output from the RW control section 46 to the patch image generation section 47 .
  • the dementia-related data 17 is output from the RW control section 46 to the prediction section 48 .
  • the patch image generator 47 subdivides the MRI image 16 into a plurality of patch images 55 (step ST110).
  • a patch image group 55 ⁇ /b>G which is a set of a plurality of patch images 55 , is output from the patch image generation section 47 to the prediction section 48 .
  • the prediction unit 48 inputs the patch image group 55G and the dementia-related data 17 to the prediction model 41, and outputs the prediction result 18 from the prediction model 41 (step ST120).
  • the prediction result 18 is output from the prediction section 48 to the distribution control section 49, and is distributed to the user terminal 11 that transmitted the prediction request 15 under the control of the distribution control section 49 (step ST130).
  • the prediction result 18 is displayed on the display 13, and the prediction result 18 is provided for viewing by the doctor.
  • the CPU 32 of the information processing server 10 includes the reception unit 45, the patch image generation unit 47, and the prediction unit 48.
  • the reception unit 45 acquires the MRI image 16 of the subject's brain for predicting the progression of dementia and the dementia-related data 17 regarding the subject's dementia.
  • the patch image generator 47 subdivides the MRI image 16 into a plurality of patch images 55 .
  • the prediction section 48 uses the prediction model 41 including the feature amount extraction section 85 and the correlation information extraction section 86 .
  • the feature quantity extraction unit 85 extracts the feature quantity 74 from the patch image 55 and the dementia related data 17 .
  • Correlation information extraction unit 86 extracts calculation result 98 as correlation information between multiple patch images 55 and correlation information between multiple patch images 55 and each of dementia-related data 17 .
  • the prediction unit 48 inputs the patch image 55 and the dementia-related data 17 to the prediction model 41 and causes the prediction model 41 to output the prediction result 18 of progression of dementia.
  • Correlation information between the multiple patch images 55 and correlation information between the multiple patch images 55 and each of the dementia-related data 17 can be effectively used to predict the progression of dementia. Therefore, it becomes possible to improve the prediction accuracy of the prediction result 18 regarding dementia by the prediction model 41 .
  • the Transformer Encoder is a model that has achieved the highest performance (SOA: State of the Art) in many fields of natural language processing, and has recently been applied not only to natural language processing but also to image processing.
  • a transformer encoder applied to image processing is called a Vision Transformer (ViT) encoder.
  • the Vision Transformer encoder treats patch images, which are subdivided images, in the same way as words in natural language processing.
  • the Vision Transformer encoder can significantly reduce the computational cost in training over conventional models using, for example, convolutional neural networks, and has higher prediction accuracy than conventional models.
  • the first input data 73_1 in which the patch image 55 and the dementia-related data 17 are mixed is taken into the transformer encoder 62 having the mechanism of this vision transformer encoder, and the feature amount 74 is extracted into the transformer encoder 62. I am letting For this reason, learning can be performed using a larger amount of learning data 100 in a short time, and the prediction accuracy of the prediction result 18 regarding dementia by the prediction model 41 can be further improved.
  • the feature extraction unit 85 includes the self-attention mechanism layer 90 of the transformer encoder 62.
  • the correlation information extraction unit 86 includes a linear transformation layer 91 , an activation function application layer 92 and a calculation unit 93 .
  • the linear transformation layer 91 linearly transforms the input data 73 to the self-attention mechanism layer 90 into first transformed data 96 .
  • the activation function application layer 92 applies an activation function to the first transformation data 96 to obtain second transformation data 97 .
  • the computing unit 93 computes the product of each element of the intermediate output data 95 from the self-attention mechanism layer 90 and the second transformed data 97 . Therefore, the correlation information between the plurality of patch images 55, the correlation information between the plurality of patch images 55 and each of the dementia-related data 17, and the correlation information between each of the dementia-related data 17 The result 98 can easily be obtained.
  • Morphological image test data such as MRI image 16 is taken by almost all dementia patients. Therefore, if the morphological image test data such as the MRI image 16 is used as the medical image, the learning data 100 of the prediction model 41 is sufficient and the learning of the prediction model 41 progresses.
  • the progression of dementia varies depending on age, gender, blood/cerebrospinal fluid test data (CSF test data in this example), and genetic test data.
  • Cognitive function test data also serve as good indicators for predicting the progression of dementia. Therefore, if the subject's age, sex, blood/cerebrospinal fluid test data, genetic test data, and cognitive function test data are included in the dementia-related data 17, the prediction result 18 related to dementia by the prediction model 41 is predicted. Accuracy can be further improved.
  • the dementia-related data 17 may include at least one of age, sex, blood/cerebrospinal fluid test data, genetic test data, and cognitive function test data of the subject.
  • the CPU of the information processing server of the second embodiment includes, in addition to the processing units 45 to 49 of the first embodiment (only the patch image generation unit 47 is shown in FIG. 12), It functions as an area image extraction unit 110 .
  • the area image extraction unit 110 is provided in the front stage of the patch image generation unit 47 .
  • the MRI image 16 is input from the RW control unit 46 to the area image extraction unit 110 .
  • the segmental image extraction unit 110 extracts a first segmental image 111 and a second segmental image 112 from the MRI image 16 using, for example, a semantic segmentation model that class labels each anatomical segment of the brain.
  • the first area image 111 is an image of an area of the brain centered primarily on the hippocampus, including the hippocampus, amygdala, and entorhinal cortex.
  • the second segmental image 112 is an image of a segment of the brain centered primarily on the temporal lobe, including the temporal lobe and the frontal lobe.
  • the area image extractor 110 outputs the first area image 111 and the second area image 112 to the patch image generator 47 .
  • the patch image generator 47 subdivides the first area image 111 into a plurality of first patch images 113 . Also, the patch image generator 47 subdivides the second area image 112 into a plurality of second patch images 114 . Therefore, the patch image group 115G in this case is composed of a first patch image group 113G that is a set of a plurality of first patch images 113 and a second patch image group 114G that is a set of a plurality of second patch images 114. be done.
  • the patch image generation section 47 outputs the patch image group 115G to the prediction section 48 . Since subsequent processing is the same as that of the first embodiment, description thereof is omitted.
  • the hippocampus is involved in memory and spatial learning ability.
  • the amygdala plays a major role in forming and storing memories associated with emotional events.
  • the entorhinal cortex is a region necessary for normal functioning of episodic memory.
  • the temporal lobe is an area essential for auditory perception, language reception, visual memory, verbal memory, and emotion.
  • lesions in the right temporal lobe generally result in an inability to interpret nonverbal auditory stimuli (eg, music).
  • lesions in the left temporal lobe significantly impair speech recognition, memory, and organization.
  • the frontal lobe is responsible for initiating or inhibiting human behavior.
  • the frontal lobe also plays a role in organizing, planning, processing, and judging the information necessary for living. In addition, it is the functioning of the frontal lobe that allows us to see our objectively, to have emotions, and even to speak.
  • the segment image extraction unit 110 extracts from the MRI image 16 a first segment image 111 including the hippocampus, amygdala, and entorhinal cortex, and a second segment image 112 including the temporal lobe and the frontal lobe. .
  • the patch image generator 47 subdivides the first area image 111 into a plurality of first patch images 113 and subdivides the second area image 112 into a plurality of second patch images 114 .
  • the first patch image 113 and the second patch image 114 include anatomical areas important in predicting the progression of dementia, such as the hippocampus, amygdala, entorhinal cortex, temporal lobe, and frontal lobe. For this reason, the prediction accuracy of the prediction result 18 regarding dementia by the prediction model 41 can be further improved.
  • the medical image is not limited to the MRI image 16.
  • other morphological imaging data such as a CT image, brain function imaging data such as a PET image, or a SPECT image may be used.
  • the cognitive function test data may be the scores of the Rivermead Behavioral Memory Test (RBMT), the scores of activities of daily living (ADL: Activities of Daily Living), and the like. Further, the cognitive function test data may be an ADAS-Cog score, an MMSE score, or the like. Multiple types of cognitive function test data may be included in the dementia-related data 17 .
  • RBMT Rivermead Behavioral Memory Test
  • ADL Activities of Daily Living
  • ADAS-Cog score an MMSE score
  • Multiple types of cognitive function test data may be included in the dementia-related data 17 .
  • the CSF test data is not limited to the amount of p-tau181 shown in the example. It may be the amount of t-tau (total tau protein) or the amount of A ⁇ 42 (amyloid ⁇ protein).
  • the prediction result 18 is not limited to the content that the exemplary subject will/won't develop Alzheimer's disease within two years.
  • the content may be that the degree of progression of Alzheimer's dementia in the subject three years later is fast/slow.
  • Each probability of normal/mild cognitive impairment/Alzheimer's dementia may be used. It may be the amount of change in cognitive function test data.
  • the prediction result 18 is not limited to Alzheimer's dementia, but more generally, it may be content that the subject is normal/pre-onset stage/mild cognitive impairment/dementia.
  • Subjective cognitive impairment SCI; Subjective Cognitive Impairment
  • SCD Subjective Cognitive Decline
  • the content may be whether or not the subject progresses from normal or pre-onset stage to MCI, or whether the subject progresses from normal, pre-onset stage or MCI to Alzheimer's dementia. .
  • Prediction includes predicting cognitive function, such as how much the subject's cognitive function will decline in two years, and predicting the risk of developing dementia, such as the degree of risk of developing dementia. .
  • Screen data including the prediction result 18 may be distributed from the information processing server 10 to the user terminal 11 instead of distributing the prediction result 18 itself from the information processing server 10 to the user terminal 11 .
  • the manner in which the prediction result 18 is provided for viewing by the doctor is not limited to the manner in which the prediction result 18 is delivered to the user terminal 11 .
  • a printed matter of the prediction result 18 may be provided to the doctor, or an e-mail attached with the prediction result 18 may be sent to the doctor's mobile terminal.
  • the learning of the prediction model 41 shown in FIG. 10 may be performed in the information processing server 10, or may be performed in a device other than the information processing server 10. Further, the learning of the prediction model 41 may be continued even after operation.
  • the information processing server 10 is an example of a “learning device” according to the technology of the present disclosure.
  • the device other than the information processing server 10 is an example of a "learning device” according to the technology of the present disclosure.
  • the information processing server 10 may be installed in each medical facility, or may be installed in a data center independent of the medical facility. Also, the user terminal 11 may take on part or all of the functions of the processing units 45 to 49 of the information processing server 10 .
  • Dementia was exemplified as a disease, but it is not limited to this.
  • the disease may be, for example, cerebral infarction.
  • CT images or MRI images of the subject's brain and disease-related data such as the subject's age and gender are input into the prediction model, and the stroke rating scale (NIHSS: National Institutes of Health Stroke Scale) score or the amount of change in the score of the Japanese Stroke Scale (JSS) is output from the prediction model as a prediction result.
  • the disease is preferably dementia and cerebral infarction as exemplified, or neurodegenerative diseases such as Parkinson's disease and cranial nerve diseases including cerebrovascular diseases.
  • prediction includes prediction of disease progression and/or prediction to aid diagnosis of disease.
  • dementia has become a social problem with the advent of an aging society. Therefore, it can be said that this example, in which the disease is dementia, is a form that matches the current social problem.
  • processors include, as described above, in addition to the CPU 32, which is a general-purpose processor that executes software (operation program 40) and functions as various processing units, FPGAs (Field Programmable Gate Arrays), etc.
  • Programmable Logic Device PLD
  • ASIC Application Specific Integrated Circuit
  • One processing unit may be configured with one of these various processors, or a combination of two or more processors of the same or different type (for example, a combination of a plurality of FPGAs and/or a CPU and combination with FPGA). Also, a plurality of processing units may be configured by one processor.
  • a single processor is configured by combining one or more CPUs and software.
  • a processor functions as multiple processing units.
  • SoC System On Chip
  • a processor that realizes the functions of the entire system including multiple processing units with a single IC (Integrated Circuit) chip. be.
  • various processing units are configured using one or more of the above various processors as a hardware structure.
  • an electric circuit combining circuit elements such as semiconductor elements can be used.
  • the processor obtaining a medical image showing organs of a subject and disease-related data of the subject; segmenting the medical image into a plurality of patch images; a feature amount extraction unit for extracting feature amounts from the patch images and the disease-related data; and extracting at least correlation information between the plurality of patch images and correlation information between the plurality of patch images and the disease-related data.
  • a prediction model that includes a correlation information extraction unit that inputting the patch image and the disease-related data into the prediction model, and causing the prediction model to output a prediction result regarding the disease; Information processing equipment.
  • the information processing apparatus includes a transformer encoder that takes in input data in which the patch image and the disease-related data are mixed and extracts the feature amount.
  • the feature extraction unit includes a self-attention mechanism layer of the transformer encoder, The correlation information extraction unit a linear transformation layer that linearly transforms data input to the self-attention mechanism layer into first transformed data; an activation function application layer that applies an activation function to the first transformed data to obtain second transformed data; 3.
  • the information processing apparatus according to item 2 further comprising: as the correlation information, output data from the self-attention mechanism layer and a computing unit that computes a product of each element with the second conversion data.
  • the disease is dementia
  • the medical image is an image of the subject's brain
  • the processor extracting a first segment image including hippocampus, amygdala, and entorhinal cortex and a second segment image including temporal lobe and frontal lobe from the medical image; 3.
  • the information processing apparatus according to any one of additional items 1 to 3, wherein the first area image and the second area image are subdivided into the plurality of patch images.
  • the disease is dementia
  • the medical image is morphological imaging data; Any one of additional items 1 to 4, wherein the disease-related data includes at least one of age, sex, blood/cerebrospinal fluid test data, genetic test data, and cognitive function test data of the subject.
  • the information processing device according to .
  • the technology of the present disclosure can also appropriately combine various embodiments and/or various modifications described above. Moreover, it is needless to say that various configurations can be employed without departing from the scope of the present invention without being limited to the above embodiments. Furthermore, the technology of the present disclosure extends to storage media that non-temporarily store programs in addition to programs.
  • a and/or B is synonymous with “at least one of A and B.” That is, “A and/or B” means that only A, only B, or a combination of A and B may be used.
  • a and/or B means that only A, only B, or a combination of A and B may be used.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Medical Informatics (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Public Health (AREA)
  • Biomedical Technology (AREA)
  • Data Mining & Analysis (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Primary Health Care (AREA)
  • Epidemiology (AREA)
  • Databases & Information Systems (AREA)
  • Pathology (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Quality & Reliability (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Business, Economics & Management (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Resources & Organizations (AREA)
  • Strategic Management (AREA)
  • Biophysics (AREA)
  • Economics (AREA)
  • Mathematical Physics (AREA)
  • Operations Research (AREA)
  • Development Economics (AREA)
  • Tourism & Hospitality (AREA)
  • Game Theory and Decision Science (AREA)
  • Marketing (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Computational Linguistics (AREA)
  • General Business, Economics & Management (AREA)
PCT/JP2022/040266 2021-12-21 2022-10-27 情報処理装置、情報処理装置の作動方法、情報処理装置の作動プログラム、予測モデル、学習装置、および学習方法 Ceased WO2023119866A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2023569121A JPWO2023119866A1 (https=) 2021-12-21 2022-10-27
US18/673,322 US20240312011A1 (en) 2021-12-21 2024-05-24 Information processing apparatus, operation method of information processing apparatus, operation program of information processing apparatus, prediction model, learning apparatus, and learning method

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
JP2021-206988 2021-12-21
JP2021206988 2021-12-21
JP2022-119116 2022-07-26
JP2022119116 2022-07-26

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/673,322 Continuation US20240312011A1 (en) 2021-12-21 2024-05-24 Information processing apparatus, operation method of information processing apparatus, operation program of information processing apparatus, prediction model, learning apparatus, and learning method

Publications (1)

Publication Number Publication Date
WO2023119866A1 true WO2023119866A1 (ja) 2023-06-29

Family

ID=86901988

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/040266 Ceased WO2023119866A1 (ja) 2021-12-21 2022-10-27 情報処理装置、情報処理装置の作動方法、情報処理装置の作動プログラム、予測モデル、学習装置、および学習方法

Country Status (3)

Country Link
US (1) US20240312011A1 (https=)
JP (1) JPWO2023119866A1 (https=)
WO (1) WO2023119866A1 (https=)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116705306A (zh) * 2023-08-03 2023-09-05 首都医科大学附属北京天坛医院 监测脑卒中的方法、监测脑卒中的装置和存储介质
JP2025505416A (ja) * 2022-01-28 2025-02-26 グーグル エルエルシー ゲーテッドアテンションユニットを備えたアテンションニューラルネットワーク

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010017274A (ja) * 2008-07-09 2010-01-28 Fuji Xerox Co Ltd 画像処理装置及び画像処理プログラム
CN113781390A (zh) * 2021-07-28 2021-12-10 杭州深睿博联科技有限公司 一种基于半监督学习的胰腺囊肿鉴别方法和系统

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010017274A (ja) * 2008-07-09 2010-01-28 Fuji Xerox Co Ltd 画像処理装置及び画像処理プログラム
CN113781390A (zh) * 2021-07-28 2021-12-10 杭州深睿博联科技有限公司 一种基于半监督学习的胰腺囊肿鉴别方法和系统

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
ALEXEY DOSOVITSKIY; LUCAS BEYER; ALEXANDER KOLESNIKOV; DIRK WEISSENBORN; XIAOHUA ZHAI; THOMAS UNTERTHINER; MOSTAFA DEHGHANI; MATTH: "An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 3 June 2021 (2021-06-03), 201 Olin Library Cornell University Ithaca, NY 14853 , XP081976531 *
GOTO TSUBASA; WANG CAIHUA; LI YUANZHONG; TSUBOSHITA YUKIHIRO: "Multi-modal deep learning for predicting progression of Alzheimer's disease using bi-linear shake fusion", PROGRESS IN BIOMEDICAL OPTICS AND IMAGING, SPIE - INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING, BELLINGHAM, WA, US, vol. 11314, 16 March 2020 (2020-03-16), BELLINGHAM, WA, US , pages 113141X - 113141X-6, XP060131388, ISSN: 1605-7422, ISBN: 978-1-5106-0027-0, DOI: 10.1117/12.2549483 *
XIAOSONG WANG; ZIYUE XU; LEO TAM; DONG YANG; DAGUANG XU: "Self-supervised Image-text Pre-training With Mixed Data In Chest X-rays", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 30 March 2021 (2021-03-30), 201 Olin Library Cornell University Ithaca, NY 14853 , XP081919343 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2025505416A (ja) * 2022-01-28 2025-02-26 グーグル エルエルシー ゲーテッドアテンションユニットを備えたアテンションニューラルネットワーク
CN116705306A (zh) * 2023-08-03 2023-09-05 首都医科大学附属北京天坛医院 监测脑卒中的方法、监测脑卒中的装置和存储介质
CN116705306B (zh) * 2023-08-03 2023-10-31 首都医科大学附属北京天坛医院 监测脑卒中的方法、监测脑卒中的装置和存储介质

Also Published As

Publication number Publication date
JPWO2023119866A1 (https=) 2023-06-29
US20240312011A1 (en) 2024-09-19

Similar Documents

Publication Publication Date Title
JP7357927B2 (ja) 診断支援システムおよび方法
US11069056B2 (en) Multi-modal computer-aided diagnosis systems and methods for prostate cancer
CN108784655B (zh) 针对医疗患者的快速评估和后果分析
CN109447183B (zh) 预测模型训练方法、装置、设备以及介质
Yang et al. Large-scale brain functional network integration for discrimination of autism using a 3-D deep learning model
US12198343B2 (en) Multi-modal computer-aided diagnosis systems and methods for prostate cancer
US20240312011A1 (en) Information processing apparatus, operation method of information processing apparatus, operation program of information processing apparatus, prediction model, learning apparatus, and learning method
JP7382306B2 (ja) 診断支援装置、プログラム、学習済みモデル、および学習装置
JP7114347B2 (ja) 断層画像予測装置および断層画像予測方法
EP3918987B1 (en) Trained model
JP2021140769A (ja) 医用情報処理装置、医用情報処理方法及び医用情報処理プログラム
JP7457292B2 (ja) 脳画像解析装置、制御方法、及びプログラム
JP2024546831A (ja) コンピュータ実施方法及びシステム
Li et al. Generalizing MRI subcortical segmentation to neurodegeneration
WO2022071158A1 (ja) 診断支援装置、診断支援装置の作動方法、診断支援装置の作動プログラム、認知症診断支援方法、並びに学習済み認知症所見導出モデル
US20230260630A1 (en) Diagnosis support device, operation method of diagnosis support device, operation program of diagnosis support device, and dementia diagnosis support method
WO2023276977A1 (ja) 医療支援装置、医療支援装置の作動方法、医療支援装置の作動プログラム
Chkifa et al. Exploratory abide dataset analysis for autism detection
US20240153637A1 (en) Medical support device, operation method of medical support device, operation program of medical support device, learning device, and learning method
JP7852212B2 (ja) 診断支援装置、コンピュータプログラム及び医療機器の作動方法
US20250069745A1 (en) Diagnosis support device, recording medium, and diagnosis support method
Romano et al. Deep learning-driven risk-based subtyping of cognitively impaired individuals
US20230335283A1 (en) Information processing apparatus, operation method of information processing apparatus, operation program of information processing apparatus
KR102809490B1 (ko) 기계학습 방법 및 장치
Tchaleu et al. A Real-Time Monitoring System for Neurodegenerative Disease Progression Utilizing Generative Adversarial Networks

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22910587

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2023569121

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22910587

Country of ref document: EP

Kind code of ref document: A1