WO2022181154A1 - 判定システム、判定システムの制御方法、および制御プログラム - Google Patents

判定システム、判定システムの制御方法、および制御プログラム Download PDF

Info

Publication number
WO2022181154A1
WO2022181154A1 PCT/JP2022/002489 JP2022002489W WO2022181154A1 WO 2022181154 A1 WO2022181154 A1 WO 2022181154A1 JP 2022002489 W JP2022002489 W JP 2022002489W WO 2022181154 A1 WO2022181154 A1 WO 2022181154A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
image
determination
feature
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/002489
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
健一 渡辺
政之 京本
昌彦 橋田
信太郎 本多
直哉 和田
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Kyocera Corp
Original Assignee
Kyocera Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Kyocera Corp filed Critical Kyocera Corp
Priority to CN202280016760.4A priority Critical patent/CN116887756A/zh
Priority to US18/278,779 priority patent/US20240122565A1/en
Priority to JP2023502183A priority patent/JPWO2022181154A1/ja
Priority to AU2022225282A priority patent/AU2022225282A1/en
Priority to EP22759196.3A priority patent/EP4299008A4/en
Publication of WO2022181154A1 publication Critical patent/WO2022181154A1/ja
Anticipated expiration legal-status Critical
Priority to JP2025132670A priority patent/JP2025166099A/ja
Ceased legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/52—Devices using data or image processing specially adapted for radiation diagnosis
    • A61B6/5211—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data
    • A61B6/5217—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data extracting a diagnostic or physiological parameter from medical diagnostic data
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/52—Devices using data or image processing specially adapted for radiation diagnosis
    • A61B6/5211—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/46—Arrangements for interfacing with the operator or the patient
    • A61B6/461—Displaying means of special interest
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—Two-dimensional [2D] image generation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/7715—Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
    • G16H40/60—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices
    • G16H40/63—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices for local operation
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/05—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves
    • A61B5/055—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/56—Details of data transmission or power supply, e.g. use of slip rings
    • A61B6/563—Details of data transmission or power supply, e.g. use of slip rings involving image data transmission via a network
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/56—Details of data transmission or power supply, e.g. use of slip rings
    • A61B6/566—Details of data transmission or power supply, e.g. use of slip rings involving communication between diagnostic systems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30008—Bone
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30061—Lung
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2210/00—Indexing scheme for image generation or computer graphics
    • G06T2210/41—Medical

Definitions

  • the present disclosure relates to a determination system that presents determination results regarding the current or future state of a target part of the human body, a control method for the determination system, and a control program.
  • a determination system includes an acquisition unit that acquires a target image showing a target part of a subject's body, and a first determination information that indicates the state of the target part from the target image. a first generation unit, a second generation unit that generates feature information indicating a feature related to the physical state of the subject from the target image used to generate the first determination information, the first determination information, and a display control unit that acquires the characteristic information and causes a display device to display the first determination information and the characteristic information.
  • a method for controlling a determination system includes a target image acquisition step of acquiring a target image showing a target part of a subject's body; a first generation step of generating determination information; a second generation step of generating feature information indicating features related to the physical state of the subject from the target image used to generate the first determination information; a display control step of acquiring the first determination information and the characteristic information and displaying the first determination information and the characteristic information on a display device.
  • a determination system may be implemented by a computer.
  • a control program for a determination system realized in , and a computer-readable recording medium recording it are also included in the scope of the present disclosure.
  • FIG. 1 is a block diagram showing an example of a configuration of a determination system according to one aspect of the present disclosure
  • FIG. It is a functional block diagram which shows an example of a structure of a determination system. It is a figure which shows an example of the data structure of a display information database.
  • FIG. 10 is a diagram showing an example of first determination information generated from a target image showing lungs of a target person; It is a figure which shows the example of the 1st determination information produced
  • 4 is a flow chart showing an example of the flow of processing executed by the information processing device of the determination system; It is a figure which shows an example of a display screen. It is a figure which shows an example of a display screen.
  • FIG. 9 is a flow chart showing another example of the flow of processing executed by the determination system; It is a functional block diagram which shows an example of a structure of a determination system. It is a figure which shows an example of the data structure of a display information database. It is a functional block diagram which shows an example of a structure of a determination system.
  • FIG. 1 is a block diagram showing an example of a configuration of a determination system according to one aspect of the present disclosure
  • the determination system 100 acquires a target image showing a target part of the body of a subject, and generates first determination information indicating the state of the target part from the target image.
  • the determination system 100 also generates feature information indicating features related to the physical condition of the subject from the target image used to generate the first determination information.
  • the determination system 100 displays the first determination information and the feature information on the display device 5 .
  • the determination system 100 can present the user with the first determination information and feature information generated from the same target image showing the target part of the body of the target person.
  • the feature information is useful information for the user to judge the reliability of the first determination information. By referring to the feature information, the user can understand the reason why the first determination information was generated from the target image, and can judge the validity and reliability of the first determination information.
  • the target part may be any part of the subject's body, such as the whole body, head, neck, arms, torso, waist, buttocks, legs, and feet. can be any of them.
  • the target image may be a medical image showing the target part of the target person.
  • the target image may be a medical image obtained by imaging a target portion of a target person in order to examine the target person.
  • the medical images are X-ray images of subjects, CT (Computed Tomography) images, MRI (Magnetic Resonance Imaging) images, PET (Positron Emission Tomography) images, RI (Radio Isotope) images, mammography images, ultra It may be one of an acoustic image, an endoscopic image, and an angiographic image.
  • the target image may be an image obtained by imaging any of bones, joints, muscles, fat (subcutaneous fat and visceral fat), organs, blood vessels, etc. of the target region.
  • the first determination information may be information that determines the state of the target part at the first point in time when the target image is captured.
  • the first determination information may include information that determines the state of the target site at a second point in time when a predetermined period has passed since the first point in time when the target image was captured (see FIG. 3).
  • the first point in time may be, for example, the point in time when a medical image of the subject's target region is acquired.
  • the first point in time may typically be the point in time at which a subject image of the subject's current target site condition is acquired. That is, the first point in time may substantially mean the present point in time.
  • the predetermined period may be any period that has passed since the first time point, and may be half a year, one year, five years, or ten years. It may be 50 years. That is, the second point in time may be intended to be substantially any point in time in the future.
  • the predetermined period is not limited to one period, and may include multiple periods.
  • the first determination information is an artificial intelligence (AI, for example, a first determination information generation device 2 described later) provided in the determination system 100, etc., based on the target image, the current or future state of the target part of the subject. It may be information indicating the determination result of the determination.
  • AI artificial intelligence
  • the first determination information may be diagnostic information including a diagnostic result regarding whether or not the subject has a disease.
  • the diagnostic information may include the presence or absence of disease and the degree of progression and severity of disease.
  • the diagnostic information may include a numerical value used for determining whether or not a person has a disease. It may also include the risk of developing a disease in the future and the risk of having symptoms such as pain in the future. If there is no disease, the diagnostic result may be normal or no abnormality. It may also include information indicating a non-disease state, such as being within the normal range but showing slight signs of disease. It may also include information indicating that a detailed inspection is necessary, or information indicating that determination was impossible.
  • the feature information may be information indicating features detected from the target image by artificial intelligence (AI, for example, the feature information generation device 3 to be described later) or the like provided in the determination system 100 .
  • the feature information may include identification information including the name of each feature detected from the target image, and position information indicating the position of each feature in the target image.
  • the feature information indicates information indicating the presence of any abnormality (symptom) detected in the target image and the position where the abnormality was detected when the target image is compared with an image in which a normal target site is imaged. It may be location information.
  • the feature information may include features that are unrelated to the target part of the subject. Also, the feature information may include features unrelated to determination of the state of the target part.
  • the feature may be any feature on the image obtained by analyzing the target image.
  • the characteristics are malignant tumor image, benign tumor image, thrombus image, angiosclerosis image, ground glass opacity, calcified focal opacity, melon skin-like finding, patchy opacity, interstitial change image, pulmonary cyst image, pulmonary emphysema image.
  • at least one of vascular compression image, nerve compression image, intervertebral disc degeneration image, meniscal injury image, ligament injury image, osteosclerosis image, fracture image, bone deformation image, osteophyte formation image, and joint space reduction image may be
  • the determination system 100 may further generate region-of-interest information indicating a region of interest related to the first determination information, which is a partial region of the target image.
  • the determination system 100 displays the first determination information, the feature information generated from the target image used to generate the first determination information, and the region-of-interest image on the display device 5 .
  • the region-of-interest information may be information indicating the position of the region of interest focused on by the first determination information generation device 2 in the process of generating the first determination information. That is, the region of interest may be a region focused on in the process of generating the first determination information from the target image, and the region-of-interest information may be information indicating the position of the region of interest in the target image.
  • the region-of-interest information may be information indicating the surroundings of the ground-glass shadow.
  • the region-of-interest information is the narrow part of the joint space, the bone sclerosis, and the like. may be information indicating
  • the determination system 100 can more effectively improve the user's understanding of the first determination information.
  • the determination system 100 can generate the first determination information, the feature information, and the region-of-interest image as an example.
  • the configuration for generating the region-of-interest image is not essential.
  • FIG. 1 is a block diagram showing a configuration example of the determination system 100.
  • the subject may be a patient who undergoes examination and treatment regarding the condition of the target site at the medical facility 6 .
  • the target image may be a medical image obtained by imaging a target part of a subject. A case where the target regions are the lungs and knees of the subject will be described below as an example, but the target regions are not limited to these.
  • the determination system 100 may include an information processing device 1, a first determination information generation device 2, and a characteristic information generation device 3.
  • the information processing device 1, the first determination information generation device 2, and the characteristic information generation device 3 are all computers.
  • the first determination information generation device 2 and the characteristic information generation device 3 are connected to the information processing device 1 so as to be communicable.
  • the information processing device 1, the first determination information generation device 2, and the characteristic information generation device 3 may be connected to a LAN (local area network) of the medical facility 6, as shown in FIG.
  • LAN local area network
  • the medical facility 6 may be equipped with the target image management device 4 and one or more display devices 5 in addition to the determination system 100 .
  • the target image management device 4 and the display device 5 may also be connected to the LAN of the medical facility 6 as shown in FIG.
  • the first determination information generation device 2 is a computer capable of generating first determination information from the target image.
  • the first determination information generation device 2 may be capable of generating region-of-interest information from the target image.
  • the first determination information generating device 2 will be explained later.
  • the feature information generation device 3 is a computer capable of generating feature information from a target image.
  • the feature information generation device 3 will be described later.
  • the information processing device 1 acquires target images from the target image management device 4 .
  • the information processing device 1 transmits the target image to the first determination information generation device 2 and instructs it to generate the first determination information and the region-of-interest information.
  • the information processing device 1 also transmits the same target image to the feature information generating device 3 and instructs it to generate feature information.
  • the information processing device 1 receives first determination information and region-of-interest information from the first determination information generation device 2 and receives feature information from the feature information generation device 3 .
  • the display device 5 receives the first determination information, the region-of-interest information, and the feature information from the information processing device 1, and includes a display unit 51 (see, for example, FIG. 2) capable of displaying this information.
  • the display device 5 may be a computer used by medical personnel such as doctors belonging to the medical facility 6 .
  • the display device 5 may be, for example, a personal computer, a tablet terminal, a smartphone, or the like, and may include a communication section for transmitting and receiving data with other devices, an input section such as a keyboard and a microphone, and the like. In this case, the display device 5 can receive input of various instructions from medical personnel.
  • the first determination information generation device 2 generates the first determination information and the region-of-interest information, but is not limited to this.
  • the first determination information generation device 2 may generate only the first determination information, and another device (not shown) different from the first determination information generation device 2 may generate the region of interest information.
  • the information processing device 1 may transmit the target image to the other device and instruct it to generate the region-of-interest information.
  • the target image management device 4 is a computer that functions as a server for managing target images.
  • the target image management device 4 may transmit the target image specified in the instruction input to the display device 5 to the information processing device 1 in response to an instruction from the medical staff.
  • the instructions from the medical personnel include identification information unique to the subject (subject ID described later), identification information unique to the target image (image ID described later), the MAC address of the information processing apparatus 1, and the like. may be included.
  • the electronic medical record management device 9 is a computer that functions as a server for managing electronic medical record information of subjects who have been examined at the medical facility 6 .
  • each device of the determination system 100 may acquire basic information related to the subject from the electronic medical record management device 9 .
  • the basic information is information included in the subject's electronic medical record information, and indicates the subject's sex, age, height, weight, and the subject's current (first time point) health condition. At least one of the information may be included.
  • the basic information may include a subject ID, which is unique identification information for each subject and given to each subject.
  • an information processing device 1 a first determination information generation device 2, a characteristic information generation device 3, a target image management device 4, and an electronic medical record management device 9 are connected to a LAN provided in a medical facility 6.
  • the network within the medical facility 6 may employ the Internet, a telephone communication network, an optical fiber communication network, a cable communication network, a satellite communication network, or the like.
  • the LAN within the medical facility 6 may be communicably connected to an external communication network.
  • the information processing device 1 and at least one of the display device 5, the first determination information generation device 2, the feature information generation device 3, the target image management device 4, and the electronic medical record management device 9 are connected via a LAN. may be connected directly without Further, even if the number of the display device 5, the first determination information generation device 2, the characteristic information generation device 3, the target image management device 4, and the electronic medical record management device 9 that can communicate with the information processing device 1 is plural, good. Furthermore, in the determination system 100, a plurality of information processing apparatuses 1 may be introduced.
  • FIG. 2 is a functional block diagram showing an example of the configuration of the determination system 100. As shown in FIG. FIG. 2 also shows target image management device 4 and display device 5 communicably connected to determination system 100 .
  • the information processing device 1, the first determination information generation device 2, and the characteristic information generation device 3 will be described below.
  • the information processing device 1 includes a control unit 11 that controls each unit of the information processing device 1, a storage unit 12 that stores various data used by the control unit 11, and a communication unit 13 that performs various data communications.
  • the control unit 11 includes an acquisition unit 111, an instruction generation unit 112, a display information generation unit 113, and a display control unit 114.
  • the acquisition unit 111 acquires the target image from the target image management device 4, as shown in FIG.
  • the target image may be acquired from a computer (eg, display device 5) used by the doctor.
  • the acquiring unit 111 may acquire an image ID, which is identification information unique to each target image and is assigned to each target image, together with the target image.
  • the acquisition unit 111 may acquire one or a plurality of target images corresponding to each of a plurality of subjects.
  • the acquiring unit 111 may acquire the subject ID in addition to the image ID.
  • the instruction generation unit 112 acquires the target image and the image ID and generates various instructions.
  • the instruction generation unit 112 generates a first generation instruction and a second generation instruction to be transmitted to the first determination information generation device 2 and a third generation instruction to be transmitted to the characteristic information generation device 3 .
  • the first generation instruction is an instruction to cause the first determination information generation device 2 to generate the first determination information from the target image
  • the second generation instruction is an instruction to generate the region-of-interest information from the target image. be.
  • the third generation instruction is an instruction for causing the feature information generation device 3 to generate feature information from the target image.
  • the first generation instruction and the second generation instruction may include the MAC address of the first determination information generation device 2 as the transmission destination, the MAC address of the information processing device 1 as the transmission source, and the like.
  • the third generation instruction may include the MAC address of the characteristic information generation device 3 as the transmission destination, the MAC address of the information processing device 1 as the transmission source, and the like.
  • the instruction generation unit 112 may assign a unique instruction ID to each generated instruction.
  • each of the first generation instruction, the second generation instruction, and the third generation instruction transmitted together with the target image to which the same image ID is assigned may be assigned an instruction ID associated with each other.
  • the first generation instruction and the second generation instruction generated by the instruction generation unit 112 are transmitted to the first determination information generation device 2 via the communication unit 13 together with the instruction ID and the target image.
  • the third generation instruction generated by the instruction generation unit 112 is transmitted to the feature information generation device 3 via the communication unit 13 together with the instruction ID and the target image.
  • the display information generation unit 113 For each target image, the display information generation unit 113 generates (1) first determination information and region-of-interest information generated by the first determination information generation device 2, and (2) feature information generated by the feature information generation device 3. and get The display information generation unit 113 refers to the instruction ID or the like associated with each acquired information, associates the first determination information, the region-of-interest information, and the feature information, and generates display information.
  • the display information generation unit 113 may store the generated display information together with the instruction ID in the display information database 121 of the storage unit 12 .
  • FIG. 3 is a diagram showing an example of the data structure of the display information database 121. As shown in FIG.
  • the following information is associated and recorded in the display information database 121 .
  • Instruction ID The first determination information and the region-of-interest information generated by the first determination information generation device 2 in response to the first generation instruction and the second generation instruction to which each instruction ID is assigned.
  • Characteristic information generated by the characteristic information generation device 3 according to the third generation instruction to which each instruction ID is assigned.
  • the display information database 121 may further record the following information in association with each other.
  • the image ID of the target image transmitted to the first determination information generation device 2 and the feature information generation device 3 together with the first generation instruction, second generation instruction, and third generation instruction to which each instruction ID is assigned.
  • - Electronic medical record information of a subject which may include the subject ID appearing in the target image to which each image ID is assigned.
  • the display control unit 114 causes the display unit 51 of the display device 5 to display using each information associated with each instruction ID from the display information database 121 .
  • the first determination information generation device 2 includes a control unit 21 that comprehensively controls each unit of the first determination information generation device 2, a storage unit 22 that stores various data used by the control unit 21, and performs various data communications.
  • a communication unit 23 is provided.
  • the control unit 21 includes a first generation unit 211 that generates first determination information based on the target image, and a third generation unit 212 that generates region-of-interest information.
  • the first generation unit 211 generates, from the target image, first determination information regarding the target site at a first time point or at a second time point after a predetermined period has elapsed from the first time point.
  • FIG. 4 is a diagram showing an example of the first determination information generated from the target image showing the lungs of the target person.
  • FIG. 4 shows a case where "determination result regarding current lung condition" is generated as an example of the first determination information.
  • FIG. 5 is a diagram showing an example of the first determination information generated from the target image showing the knee joint of the target person.
  • FIG. 5 shows a case (example 1) where "determination result regarding the current knee joint state" is generated as the first determination information, and "determination result regarding the future knee joint state” is generated as the first determination information. (Example 2) is shown.
  • the first generation unit 211 may have a determination information generation model capable of generating first determination information indicating the state of the target part of the subject using the target image of the subject.
  • the judgment information generation model may be a trained first neural network that has been trained using patient information about multiple patients with diseases of the target site as teacher data.
  • the patient information may include diagnostic information and medical images indicating the diagnostic result of the condition of the target part of each patient.
  • the patient information may be further associated with information indicating when the medical image was captured.
  • This trained first neural network may be used as a determination information generation model capable of outputting the first determination information from the target image of the subject.
  • the disease may be at least one of cancer, heart disease, pneumonia, emphysema, cerebral infarction, dementia, osteoarthritis, spondylosis, fracture and osteoporosis.
  • the osteoarthritis may be hip osteoarthritis or knee osteoarthritis.
  • the fracture may be a vertebral compression fracture or a proximal femur fracture.
  • the first determination information may be the degree of progression or severity of the disease, and when the disease is cancer, heart disease, or the like, the first determination information indicates the degree of progression or severity of the disease. Stage classification and the like may be included.
  • the first determination information may include a KL (Kellgren-Lawrence) classification or the like indicating the degree of progression or severity of the disease.
  • the first determination information may include Garden Stage or the like indicating the degree of progress or severity.
  • the first determination information may be a numerical value used for determining whether or not a person has a disease. For example, if the disease is osteoporosis, the first determination information may be a value indicating bone density.
  • the first generating unit 211 performs calculation based on the determination information generation model in response to input of the target image to the input layer, and outputs the first determination information from the output layer.
  • the first generation unit 211 may be configured to extract a predetermined feature amount from the target image and use it as input data.
  • a known algorithm such as the following can be applied to extract the feature quantity.
  • CNN Convolutional neural network
  • RNN Recurrent neural network
  • LSTM Long Short-Term Memory
  • the judgment information generation model when generating a determination information generation model that outputs first determination information related to lung disease, the following learning data and teacher data (correct labels) may be used.
  • the judgment information generation model outputs accurate judgment information on lung disease for an arbitrary subject's lung X-ray image. It becomes possible to • Training data: X-ray images of the lungs of multiple patients. ⁇ Training data: Presence or absence of lung disease or the name of the disease (no abnormality, lung cancer, pneumonia, etc.) shown in each patient's lung X-ray image.
  • the judgment information generation model when generating a determination information generation model that outputs determination information related to knee disease, the following learning data and teacher data (correct labels) may be used.
  • the judgment information generation model outputs accurate judgment information on knee disease for any subject's knee X-ray image. It becomes possible to • Training data: x-ray images of the knees of multiple patients. ⁇ Training data: Presence or absence of knee disease or the name of the disease (no abnormality, osteoarthritis, Kellgren-Laurence classification, etc.) shown in each patient's X-ray image.
  • the third generation unit 212 generates region-of-interest information indicating the position of the region of interest focused on in the process of generating the first determination information by the first generation unit 211 .
  • the region-of-interest information may be output as region-of-interest information indicating the position of a partial region of the target image to which a certain layer of the first neural network reacted strongly in the process of outputting the first determination information.
  • the third generation unit 212 extracts a feature map for the entire target image when the target image is input to the input layer of the first neural network and processed up to the convolution layers. Also, the third generator 212 performs segmentation on the target image.
  • the segmentation is a process of evaluating how each pixel of the target image affects the first determination information and outputting the evaluation result.
  • the third generating unit 212 may identify important positions with respect to the first determination information based on the magnitude of change in output that occurs when a gradient change is applied to a certain position on the feature map. good. In this case, a position having a large influence on the first determination information has a large change in gradient, and a position having a small influence on the first judgment information has a small change in gradient.
  • the third generation unit 212 outputs a region of the target image corresponding to a position that greatly affects the first determination information in the feature map as a region of interest, and generates region-of-interest information indicating the region of interest.
  • ⁇ CAM Class Activation Mapping
  • Grad-CAM Gradient-based Class Activation Mapping
  • Attention Branch Network ⁇ Attention Guided CNN ⁇ Residual Attention Network
  • the first determination information generated by the first generation unit 211 and the region-of-interest information generated by the third generation unit 212 are transmitted to the information processing apparatus 1 via the communication unit 23 together with the instruction ID.
  • the characteristic information generation device 3 includes a control unit 31 that controls each unit of the characteristic information generation device 3, a storage unit 32 that stores various data used by the control unit 31, and a communication unit 33 that performs various data communications. I have.
  • the control unit 31 includes a second generation unit 311 that generates feature information based on the target image.
  • the second generator 311 generates feature information from the target image.
  • the second generation unit 311 may have a feature generation model capable of generating feature information related to the state of the body of the subject from the target image of the subject.
  • the determination information generation model may be a second neural network that is learned using patient images showing the target site of each of a plurality of patients with disease of the target site as teacher data.
  • a patient image may be associated with identification information including the name and annotation of each feature detected from the patient image, and location information indicating the location of each feature in the patient image.
  • This trained second neural network may be used as a feature information generation model capable of outputting feature information from the target image of the subject.
  • the disease may be at least one of cancer, heart disease, pneumonia, emphysema, cerebral infarction, dementia, osteoarthritis, spondylosis, fracture and osteoporosis.
  • the second generation unit 311 performs calculation based on the feature information generation model in response to the input of the target image to the input layer, and outputs feature information from the output layer.
  • the second generating unit 311 may be configured to extract a predetermined feature amount from the target image and use it as input data.
  • a known algorithm such as the following can be applied to extract the feature quantity.
  • ⁇ R-CNN Region-based convolutional neural network
  • ⁇ YOLO You Only Look Once
  • SDD Single Shot Detector
  • the feature information generated by the second generation unit 311 is transmitted to the information processing device 1 via the communication unit 33 together with the instruction ID.
  • the determination system 100 having the above configuration acquires first determination information, region-of-interest information, and feature information generated from the same target image, and causes the display device 5 to display this information. Thereby, the determination system 100 can present the region-of-interest information and the characteristic information that assist the understanding of the first determination information to the user (for example, medical personnel) together with the first determination information. Therefore, the user can correctly understand the first determination information generated from the target image.
  • a feature information generation model that outputs feature information about X-ray images of lungs
  • X-ray images of the lungs of a plurality of patients are used as learning data
  • X-ray images of the lungs of each patient are used as training data.
  • Pulmonary features ground-glass opacities, calcific focal opacities, etc.
  • their locations can be used.
  • the feature information generation model can generate accurate features related to lung feature information for an arbitrary subject's lung X-ray image. and position can be output.
  • a feature information generation model that outputs feature information about X-ray images of knees
  • X-ray images of knees of a plurality of patients are used as learning data
  • X-ray images of knees of each patient are used as training data.
  • Features of the knee that are in contact can be used.
  • the feature information generation model can generate accurate feature information related to knee feature information for an arbitrary subject's knee X-ray image. and position can be output.
  • the feature information can include information that is highly relevant to the first determination information, information that is not highly relevant to the first determination information, and information that is different from the state of the target site.
  • the determination system 100 presents the user with feature information in addition to the first determination information and region-of-interest information generated from the target image. Thereby, the determination system 100 can inform the user of not only the condition of the subject's target part but also the presence of features related to the condition of the subject's body. Therefore, the user understands the condition of the subject's target site and the subject's physical condition, which is not limited to the subject's target site, and can determine applicable treatments and interventions for the subject. .
  • FIG. 6 is a flowchart showing an example of the flow of processing executed by the information processing device 1 of the determination system 100. As shown in FIG.
  • the information processing device 1 acquires a target image from the target image management device 4 or the display device 5 (step S1: target image acquisition step).
  • the information processing device 1 transmits the target image, the first generation instruction, and the second generation instruction to the first determination information generation device 2 (step S2).
  • the first determination information generation device 2 In response to the first generation instruction and the second generation instruction, the first determination information generation device 2 generates the first determination information and the region-of-interest information from the target image (first generation step, third generation step).
  • the information processing device 1 acquires the first determination information and the region-of-interest information from the first determination information generation device 2 (step S3).
  • the information processing device 1 transmits the target image and the third generation instruction to the feature information generation device 3 (step S4).
  • the feature information generation device 3 generates feature information from the target image (second generation step).
  • the information processing device 1 acquires feature information from the feature information generation device 3 (step S5).
  • the information processing device 1 transmits the first determination information and the region-of-interest information acquired in step S3 and the feature information acquired in step S5 to the display device 5, and causes the display device 5 to display the information (step S6: display control step).
  • the processes of steps S2 and S3 may be executed before or after the processes of steps S4 and S5.
  • FIG. 7 to 12 are diagrams showing examples of display screens.
  • a case where an X-ray image of a subject's lungs or knee joints is used as the subject image will be described as an example.
  • 7 to 12 show examples of display screens that respectively display the first determination information, the region-of-interest information, and the feature information.
  • the display screens shown in FIGS. 7 and 8 include a region R1 for displaying the subject ID, a region R2 for displaying the first determination information, and a region R5 for displaying the target image and the image ID used to generate the first determination information. have.
  • the first determination information "Pneumonia progresses" is displayed in the area R2.
  • the region R2 displays the first determination information "You are at risk of knee osteoarthritis. There is a possibility that you will experience pain in your knee joint in three years.”
  • the display screens shown in FIGS. 7 and 8 include a region R6 for receiving an instruction to transition to the display screen displaying the region-of-interest information, and a region R6 for receiving an instruction to transition to the display screen displaying the characteristic information. has R7.
  • the display screen changes to that shown in FIG.
  • the display screen changes to that shown in FIG.
  • the display screens shown in FIGS. 9 and 10 have a region R1 for displaying a subject ID, a region R3 for displaying region-of-interest information, and a region R5 for displaying a target image and image ID.
  • the area of interest information "upper right lung field” and “lower left lung field” are displayed in the region R3.
  • the region R3 displays region R3 of interest region information of "inner side of the knee joint” and "femoral surface shape of the outer side of the knee joint".
  • the display screens shown in FIGS. 9 and 10 include an area R8 for receiving an instruction to transition to the display screen displaying the first determination information, and an area R8 for receiving an instruction to transition to the display screen displaying the feature information. It has a region R7. For example, when the user selects the region R8 shown in FIG. 9, the display screen changes to that shown in FIG. On the other hand, when the user selects the area R7 shown in FIG. 10, the display screen changes to that shown in FIG.
  • the display screens shown in FIGS. 11 and 12 have a region R1 for displaying the subject ID, a region R4 for displaying feature information, and a region R5 for displaying the target image and image ID.
  • characteristic information such as "upper right lung field: frosted glass shadow”, “lower left lung field: calcified focus shadow”, etc. is displayed in the region R4.
  • characteristic information such as "inside knee joint: osteosclerosis”, “femoral surface outside knee joint: osteophyte”, and "femoral fragility" is displayed. .
  • the display screens shown in FIGS. 11 and 12 include an area R8 for receiving an instruction to transition to the display screen displaying the first determination information, and an instruction to transition to the display screen displaying the target area information. It has a receiving region R6. For example, when the user selects the region R8 shown in FIG. 11, the display screen changes to that shown in FIG. On the other hand, when the user selects the region R6 shown in FIG. 11, the display screen changes to that shown in FIG.
  • the first determination information, the region-of-interest information, and the characteristic information may be displayed on the display unit 51, respectively, but the present invention is not limited to this.
  • the first determination information, the region-of-interest information, and the characteristic information may be collectively displayed on the display unit 51 .
  • An example of a display screen that simultaneously displays the first determination information, the region-of-interest information, and the feature information will be described with reference to FIGS. 13 and 14.
  • FIG. 13 and 14 are diagrams showing examples of display screens.
  • the display screens shown in FIGS. 13 and 14 include a region R1 for displaying a subject ID, a region R2 for displaying first determination information, a region R3 for displaying target region information, a region R4 for displaying characteristic information, a first determination It has a region R5 for displaying the target image and the image ID used to generate the information.
  • the first determination information "Pneumonia is progressing” is displayed in the region R2, and the target region information "upper right lung field” and “lower left lung field” are displayed in the region R3. is displayed. Further, the feature information “upper right lung field: ground glass shadow” and “lower left lung field: calcified focus shadow” are displayed in the area R4.
  • the region R2 displays the first determination information "I am at risk of knee osteoarthritis” and "There is a possibility that I will have pain in my knee joint in 3 years”.
  • the target region information of "knee joint inner side” and “knee joint outer side femoral surface shape” are displayed. Further, in the region R4, characteristic information such as “inside knee joint: bone sclerosis”, “femoral surface outside knee joint: osteophyte”, and "fragile femur” is displayed.
  • the user browses a plurality of screens. No need. According to this configuration, the convenience of the determination system 100 can be improved.
  • the first determination information, the region-of-interest information, and the feature information are displayed simultaneously on one display unit 51, but the present invention is not limited to this.
  • the first determination information, the region-of-interest information, and the feature information may be divided into a plurality of display units 51 designated by the user and displayed simultaneously.
  • the first generation unit 211 may be configured to consider the feature information generated by the feature information generation device 3 when generating the first determination information from the target image.
  • the flow of processing performed by the determination system 100 having this configuration will be described with reference to FIG.
  • FIG. 15 is a flowchart showing another example of the flow of processing executed by the determination system 100. As shown in FIG. For convenience of explanation, the same reference numerals are given to the same processes as those explained in FIG.
  • the information processing device 1 acquires a target image from the target image management device 4 or the display device 5 (step S1: target image acquisition step).
  • the information processing device 1 transmits the target image and the third generation instruction to the feature information generation device 3 (step S4).
  • the feature information generation device 3 generates feature information from the target image (second generation step).
  • the information processing device 1 acquires feature information from the feature information generation device 3 (step S5).
  • the information processing device 1 transmits the target image, the feature information, the first generation instruction, and the second generation instruction to the first determination information generation device 2 (step S2a).
  • the first determination information generation device 2 In response to the first generation instruction and the second generation instruction, the first determination information generation device 2 generates the first determination information and the region-of-interest information from the target image and the feature information (first generation step, third generation step). .
  • the information processing device 1 acquires the first determination information and the region-of-interest information from the first determination information generation device 2 (step S3).
  • the information processing apparatus 1 transmits the feature information acquired in step S5 and the first determination information and the region-of-interest information acquired in step S3 to the display device 5, and causes the display device 5 to display them (step S6). .
  • the determination system 100 can generate and output first determination information with higher reliability.
  • the region-of-interest information may be information indicating the position of the region focused on in the process of generating the first determination information from the target image. Therefore, the configuration may be such that an image indicating the region-of-interest information is generated.
  • a determination system 100a including a first determination information generation device 2a that generates a target image indicating target region information will be described with reference to FIG.
  • FIG. 16 is a block diagram showing an example of the configuration of the determination system 100a.
  • members having the same functions as those of the members described in the above embodiments are denoted by the same reference numerals, and description thereof will not be repeated.
  • the determination system 100a includes an information processing device 1a, a first determination information generation device 2a, and a feature information generation device 3.
  • the first determination information generation device 2a includes a control unit 21a that comprehensively controls each unit of the first determination information generation device 2a, a storage unit 22 that stores various data used by the control unit 21a, and various data communications.
  • a communication unit 23 is provided.
  • the control unit 21a includes a first generation unit 211 that generates first determination information based on a target image, a third generation unit 212 that generates attention area information, and a attention image generation unit 213.
  • the image-of-interest generation unit 213 generates the image of interest according to the second generation instruction.
  • the image-of-interest generation unit 213 may generate an image of interest in which the region of interest indicated by the region-of-interest information generated by the third generation unit 212 is superimposed on the target image.
  • the first determination information generated by the first generation unit 211, the region-of-interest information generated by the third generation unit 212, and the image of interest generated by the image-of-interest generation unit 213 are transmitted together with the instruction ID via the communication unit 23. is transmitted to the information processing apparatus 1a.
  • the display information generation unit 113a of the information processing device 1a For each target image, the display information generation unit 113a of the information processing device 1a generates (1) the first determination information generated by the first determination information generation device 2a, the region-of-interest information, and the image of interest, and (2) the feature information. and the feature information generated by the generation device 3 are acquired.
  • the display information generating unit 113a refers to the instruction ID or the like associated with each acquired information, associates the first determination information, the focused area information, the focused image, and the feature information, and generates display information. do.
  • the display information generation unit 113a may store the generated display information together with the instruction ID in the display information database 121a of the storage unit 12a.
  • FIG. 17 is a diagram showing an example of the data structure of the display information database 121a.
  • the following information is associated and recorded in the display information database 121a.
  • Instruction ID The first determination information, the region-of-interest information, and the image of interest generated by the first determination information generation device 2 in response to the first generation instruction and the second generation instruction to which each instruction ID is assigned.
  • Characteristic information generated by the characteristic information generation device 3 according to the third generation instruction to which each instruction ID is assigned.
  • the display control unit 114 of the information processing device 1a causes the display unit 51 of the display device 5 to display information associated with each instruction ID from the display information database 121a.
  • the determination system 100a can clearly indicate to the user which area of the target image is the area of interest. Thereby, the determination system 100a can improve the convenience of the region-of-interest information.
  • the feature information is information indicating the position of the feature detected from the target image. Therefore, the configuration may be such that the first feature image is generated by superimposing the feature information on the target image.
  • a determination system 100b including a feature information generation device 3a that generates a feature image in which feature information is superimposed on a target image will be described with reference to FIG.
  • FIG. 18 is a block diagram showing an example of the configuration of the determination system 100b.
  • members having the same functions as those of the members described in the above embodiments are denoted by the same reference numerals, and description thereof will not be repeated.
  • the determination system 100b includes an information processing device 1b, a first determination information generation device 2, and a characteristic information generation device 3a.
  • the feature information generation device 3a includes a control unit 31a that controls all the parts of the feature information generation device 3a, a storage unit 32 that stores various data used by the control unit 31a, and a communication unit 33 that performs various data communications. I have.
  • the control unit 31a includes a second generating unit 311 and a characteristic image generating unit 312 that generate characteristic information based on the target image.
  • the characteristic image generation unit 312 generates the first characteristic image according to the third generation instruction.
  • the characteristic image generating section 312 may generate the first characteristic image by superimposing the characteristic information generated by the second generating section 311 on the target image.
  • the feature information generated by the second generation unit 311 and the feature image generated by the feature image generation unit 312 are transmitted to the information processing device 1b via the communication unit 33 together with the instruction ID.
  • the display information generation unit 113b of the information processing device 1b For each target image, the display information generation unit 113b of the information processing device 1b generates (1) the first determination information and the region-of-interest information generated by the first determination information generation device 2, and (2) the feature information generation device 3a. Acquire the generated feature information and feature image.
  • the display information generation unit 113b refers to the instruction ID or the like associated with each acquired information, associates the first determination information and the region-of-interest information with the characteristic information and the characteristic image, and generates display information. .
  • the display information generation unit 113b may store the generated display information together with the instruction ID in the display information database 121b of the storage unit 12b.
  • the following information is associated and recorded in the display information database 121b.
  • Instruction ID The first determination information and the region-of-interest information generated by the first determination information generation device 2 in response to the first generation instruction and the second generation instruction to which each instruction ID is assigned.
  • the display control unit 114 of the information processing device 1b causes the display unit 51 of the display device 5 to display information associated with each instruction ID from the display information database 121b.
  • the determination system 100b can clearly indicate to the user in which area of the target image the feature was detected. Thereby, the determination system 100b can improve the convenience of the feature information.
  • FIG. 19 is a block diagram showing an example of the configuration of the determination system 100c.
  • members having the same functions as those of the members described in the above embodiments are denoted by the same reference numerals, and description thereof will not be repeated.
  • the determination system 100c includes an information processing device 1c, a first determination information generation device 2a, and a characteristic information generation device 3a.
  • the display information generation unit 113c of the information processing device 1c For each target image, the display information generation unit 113c of the information processing device 1c generates (1) the first determination information generated by the first determination information generation device 2a, the region-of-interest information, and the image of interest, and (2) the feature information.
  • the feature information and the feature image generated by the generation device 3a are acquired.
  • the display information generating unit 113c refers to the instruction ID or the like associated with each acquired information, associates the first determination information, the focused area information, and the focused image with the characteristic information and the characteristic image, and displays the Generate information.
  • the display information generation unit 113c may store the generated display information together with the instruction ID in the display information database 121c of the storage unit 12c.
  • the display control unit 114 of the information processing device 1c causes the display unit 51 of the display device 5 to display information associated with each instruction ID from the display information database 121c.
  • FIG. 20 is a diagram showing an example of the data structure of the display information database 121c.
  • the following information is associated and recorded in the display information database 121c.
  • Instruction ID The first determination information, the region-of-interest information, and the image of interest generated by the first determination information generation device 2a in response to the first generation instruction and the second generation instruction to which each instruction ID is assigned.
  • the display control unit 114 of the information processing device 1c causes the display unit 51 of the display device 5 to display information associated with each instruction ID from the display information database 121c.
  • the determination system 100c can clearly indicate to the user which region of the target image is the region of interest and from where the feature was detected. Thereby, the determination system 100c can improve the convenience of the region-of-interest information and the feature information.
  • the information processing device 1c may have the function of the feature image generation unit 312.
  • the information processing device 1c can generate the second feature image from the region-of-interest information or the image of interest generated by the first determination information generation device 2a and the feature information acquired from the feature information generation device 3a. can.
  • the second feature image is an image obtained by superimposing the feature information and the region-of-interest information on the target image.
  • the second feature image may be an image in which feature information is superimposed on the image of interest (see FIGS. 21 and 22).
  • FIG. 21 and 22 are diagrams showing examples of display screens.
  • an X-ray image of the subject's lungs is used as the target image
  • the target image is displayed instead of the target region information
  • the second feature image is displayed instead of the feature information.
  • an example will be described.
  • an X-ray image of a subject's knee joint is used as the subject image.
  • the display screens shown in FIGS. 21 and 22 include a region R1 for displaying the subject ID, a region R2 for displaying the first determination information, and a region R5 for displaying the target image and the image ID used to generate the first determination information. have.
  • the display screens shown in FIGS. 21 and 22 also have a region R9 for displaying the image of interest and a region R10 for displaying the second feature image.
  • the configuration may be such that the first characteristic image is displayed in the region R10 instead of the second characteristic image.
  • the target image may be represented as a heat map on the target image.
  • the feature image may be, for example, an image in which the position of the detected feature and the name of each feature are displayed on the target image or the image of interest.
  • the determination system 100c may display the image of interest and the second characteristic image side by side on the display device.
  • the user can comprehensively grasp the first determination information, the target area information, and the feature information generated from the same target image.
  • the determination system 100c can make the user understand the first determination information and effectively encourage the user to utilize the first determination information.
  • the second determination information indicating the method of medical intervention for the subject and the effect of the medical intervention may be generated from the first determination information.
  • a configuration of a determination system 100d capable of generating second determination information will be described using FIG.
  • FIG. 23 is a functional block diagram showing an example of the configuration of the determination system 100d.
  • members having the same functions as those of the members described in the above embodiments are denoted by the same reference numerals, and description thereof will not be repeated.
  • the determination system 100d includes an information processing device 1d, a first determination information generation device 2d, and a feature information generation device 3.
  • the first determination information generation device 2d includes a control unit 21d that controls all units of the first determination information generation device 2d, a storage unit 22 that stores various data used by the control unit 21d, and various data communications.
  • a communication unit 23 is provided.
  • the fourth generation unit 214 From the first determination information generated by the first generation unit 211, the fourth generation unit 214 generates second determination information indicating the method of medical intervention for the subject and the effect of the medical intervention. In one example, the fourth generator 214 may generate the second determination information in response to the first generation instruction.
  • the fourth generation unit 214 From the first determination information generated by the first generation unit 211, the fourth generation unit 214 generates second determination information regarding the effect of applying medical intervention to the subject.
  • the fourth generation unit 214 has an intervention effect determination model that outputs second determination information indicating the method of intervention for the subject and the effect of the intervention with the first determination information of the subject as input. good.
  • the intervention effect determination model may be a third neural network that has been trained using, as teacher data, effect information for each of multiple patients who have undergone intervention for the disease of the target site.
  • the effect information may be information in which an intervention applied to the target site of each patient and intervention effect information indicating the effect of the intervention are associated with each patient.
  • This learned third neural network may be used as an intervention effect determination model capable of outputting the second determination information from the first determination information.
  • the interventions included diet therapy, exercise therapy, drug therapy, stimulation therapy, manual therapy, ultrasound therapy, use of walking aids, use of braces, orthotics, osteotomies, intra-articular injections, joint-preserving surgery, At least one of joint replacement surgery and spinal instrumentation surgery may be included.
  • the first determination information generated by the first generation unit 211, the region-of-interest information generated by the third generation unit 212, and the second determination information generated by the fourth generation unit 214 are sent together with the instruction ID by the communication unit 23. is transmitted to the information processing device 1d via the
  • the display information generation unit 113d of the information processing device 1d For each target image, the display information generation unit 113d of the information processing device 1d generates (1) the first determination information, the region-of-interest information, and the second determination information generated by the first determination information generation device 2d, and (2) and the feature information generated by the feature information generation device 3 are acquired.
  • the display information generating unit 113d refers to the instruction ID or the like associated with each acquired information, associates the first determination information, the focused area information, the second determination information, and the feature information, and generates the display information. to generate
  • the display information generation unit 113d may store the generated display information together with the instruction ID in the display information database 121d of the storage unit 12d.
  • the display control unit 114 of the information processing device 1d causes the display unit 51 of the display device 5 to display information associated with each instruction ID from the display information database 121d.
  • the determination system 100d obtains, from the target image of the subject, the first determination information regarding the state of the target part of the subject, as well as the effect of applying medical intervention to the subject.
  • the second determination information can be presented to the user.
  • the user of the determination system 100d can appropriately determine whether medical intervention is necessary for the subject.
  • the fourth generation unit 214 relating to the lung X-ray image is generated by learning to which the error backpropagation method is applied in the LSTM using the learning data and teacher data described below.
  • Such a fourth generation unit 214 is capable of predicting an accurate intervention effect on the lungs for an X-ray image of the lungs of an arbitrary subject.
  • Training data a combination of x-ray images of the lungs of multiple patients and the interventions performed on those patients.
  • ⁇ Training data Information about the prognostic effect of the lungs shown in each of the lung X-ray images of a plurality of patients (whether or not the effect is present, changes in length of stay, etc.).
  • the fourth generation unit 214 relating to the X-ray image of the knee is generated by learning to which the error backpropagation method is applied in the LSTM using the learning data and teacher data described below.
  • Such a fourth generating unit 214 is capable of predicting an accurate knee intervention effect for any subject's knee X-ray image.
  • Training data a combination of x-ray images of the knees of multiple patients and the interventions performed on those patients.
  • Training data Information about the prognostic effect of the knee (change in pain, improvement in walking ability, etc.) in each of the knee X-ray images of a plurality of patients.
  • FIG. 24 is a block diagram showing a configuration example of a determination system 100 according to another aspect of the present disclosure.
  • the information processing device 1 is communicably connected to each device of the medical facilities 6a and 6b via the communication network 7.
  • the medical facility 6a includes a display device 5a, a target image management device 4a, and an electronic medical record management device 9a, which are communicably connected.
  • the medical facility 6b includes a display device 5b, a target image management device 4b, and an electronic chart management device 9b, which are communicably connected.
  • FIG. 24 shows an example in which the LANs of the medical facility 6a and the medical facility 6b are connected to the communication network 7.
  • the information processing apparatus 1 is not limited to the configuration shown in FIG. 24 as long as it is communicably connected to the apparatuses in the medical facilities 6a and 6b via the communication network 7.
  • FIG. 24 shows an example in which the LANs of the medical facility 6a and the medical facility 6b are connected to the communication network 7.
  • the information processing apparatus 1 is not limited to the configuration shown in FIG. 24 as long as it is communicably connected to the apparatuses in the medical facilities 6a and 6b via the communication network 7.
  • the information processing device 1 can acquire a target image of the subject Pa from the medical facility 6a and acquire a target image of the subject Pb from the medical facility 6b. .
  • the target image of each subject includes identification information specific to each medical facility 6a, 5b assigned to each medical facility 6 that examines each subject, and the target image assigned to each subject. It is sufficient if the identification information unique to the person is included.
  • the identification information unique to each medical facility 6a, 6b may be, for example, a facility ID. Further, the identification information unique to each subject may be, for example, a patient ID.
  • the information processing device 1 transmits various kinds of information acquired from the first determination information generation device 2 and the characteristic information generation device 3 to the display devices 5a and 5b installed in the medical facilities 6a and 6b that are the transmission sources of the target images. can be done.
  • the information processing apparatus 1 transmits the first determination information and region-of-interest information indicating the state of the target site of the subject, and the feature information to each medical facility 6a, 6b can be correctly transmitted to the display devices 5a, 5b.
  • the information processing device 1, the first determination information generating device 2, and the feature information generating device 3 may each be connected to the communication network 7. Alternatively, at least one of the first determination information generation device 2 and the characteristic information generation device 3 may be arranged in either of the medical facilities 6a and 6b.
  • the information processing devices 1a to 1d may be connected via the communication network 7 to the LANs provided in each of the medical facilities 6a and 6b so as to be communicable.
  • the information processing devices 1a to 1d, the first determination information generating devices 2 and 2a, and the characteristic information generating devices 3 and 3a may be connected to the communication network 7, respectively.
  • at least one of the first determination information generating devices 2, 2a and the characteristic information generating devices 3, 3a may be arranged in any one of the medical facilities 6a, 6b.
  • the acquisition unit 111 of the information processing device 1, 1a to 1d acquires the target image from the target image management device 4, and transmits the target image to the first determination information generation device 2, 2a, 2d and the feature
  • the configuration for transmitting to the information generating devices 3 and 3a has been described as an example. However, it is not limited to these.
  • the first determination information generation devices 2, 2a, and 2d and the feature information generation devices 3 and 3a may acquire target images from the target image management device 4.
  • An image ID may be included.
  • the first determination information generating devices 2, 2a, and 2d generate target images to which image IDs included in the first generation instructions and the second generation instructions received from the information processing devices 1, 1a to 1d are given by the target image management device. It can be obtained from 4.
  • the information processing devices 1, 1a to 1d may have the functions of the first generation unit 211 and the third generation unit 212 (and the target image generation unit 213).
  • the information processing device 1, 1a to 1d may have at least one of the function of the image-of-interest generating section 213 and the function of the fourth generating section 214.
  • the information processing devices 1, 1a to 1d may have the function of the second generation unit 311.
  • the information processing apparatuses 1, 1a to 1d may have the function of the characteristic image generating section 312. FIG.
  • the information processing devices 1, 1a to 1d may have the function of the second generation unit 311.
  • the information processing apparatuses 1, 1a to 1d may have the function of the characteristic image generating section 312. FIG.
  • control blocks especially the control units 11, 11a to 11d, 21, 21a, 21d, 31, 31a
  • the control blocks are implemented by logic circuits (hardware) formed in integrated circuits (IC chips) or the like. may be implemented by software.
  • the determination systems 100 to 100d are equipped with computers that execute program instructions, which are software that implements each function.
  • This computer includes, for example, one or more processors, and a computer-readable recording medium storing the program.
  • the processor reads the program from the recording medium and executes it, thereby achieving the object of the present disclosure.
  • a CPU Central Processing Unit
  • the recording medium a "non-temporary tangible medium" such as a ROM (Read Only Memory), a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used.
  • a RAM Random Access Memory
  • the program may be supplied to the computer via any transmission medium (communication network, broadcast wave, etc.) capable of transmitting the program.
  • Any transmission medium communication network, broadcast wave, etc.
  • One aspect of the present disclosure may also be embodied in the form of a data signal embedded in a carrier wave, with the program embodied by electronic transmission.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Pathology (AREA)
  • Primary Health Care (AREA)
  • Epidemiology (AREA)
  • Databases & Information Systems (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Optics & Photonics (AREA)
  • High Energy & Nuclear Physics (AREA)
  • Veterinary Medicine (AREA)
  • Biophysics (AREA)
  • Animal Behavior & Ethology (AREA)
  • Surgery (AREA)
  • Molecular Biology (AREA)
  • Data Mining & Analysis (AREA)
  • Software Systems (AREA)
  • Business, Economics & Management (AREA)
  • General Business, Economics & Management (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Evolutionary Computation (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Quality & Reliability (AREA)
  • Physiology (AREA)
  • Image Analysis (AREA)
  • Medical Treatment And Welfare Office Work (AREA)
  • Selective Calling Equipment (AREA)
PCT/JP2022/002489 2021-02-26 2022-01-25 判定システム、判定システムの制御方法、および制御プログラム Ceased WO2022181154A1 (ja)

Priority Applications (6)

Application Number Priority Date Filing Date Title
CN202280016760.4A CN116887756A (zh) 2021-02-26 2022-01-25 判定系统、判定系统的控制方法以及控制程序
US18/278,779 US20240122565A1 (en) 2021-02-26 2022-01-25 Determination system, method of controlling determination system, and control program
JP2023502183A JPWO2022181154A1 (https=) 2021-02-26 2022-01-25
AU2022225282A AU2022225282A1 (en) 2021-02-26 2022-01-25 Assessment system, assessment system control method, and control program
EP22759196.3A EP4299008A4 (en) 2021-02-26 2022-01-25 EVALUATION SYSTEM, EVALUATION SYSTEM CONTROL PROCEDURE AND CONTROL PROGRAM
JP2025132670A JP2025166099A (ja) 2021-02-26 2025-08-07 判定システム、判定システムの制御方法、および制御プログラム

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2021030858 2021-02-26
JP2021-030858 2021-02-26

Publications (1)

Publication Number Publication Date
WO2022181154A1 true WO2022181154A1 (ja) 2022-09-01

Family

ID=83048889

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/002489 Ceased WO2022181154A1 (ja) 2021-02-26 2022-01-25 判定システム、判定システムの制御方法、および制御プログラム

Country Status (6)

Country Link
US (1) US20240122565A1 (https=)
EP (1) EP4299008A4 (https=)
JP (2) JPWO2022181154A1 (https=)
CN (1) CN116887756A (https=)
AU (1) AU2022225282A1 (https=)
WO (1) WO2022181154A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPWO2024143476A1 (https=) * 2022-12-28 2024-07-04

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007135683A (ja) * 2005-11-15 2007-06-07 Gifu Univ 診断支援装置
JP2008036068A (ja) 2006-08-04 2008-02-21 Hiroshima Univ 骨粗鬆症診断支援装置および方法、骨粗鬆症診断支援プログラム、骨粗鬆症診断支援プログラムを記録したコンピュータ読み取り可能な記録媒体、骨粗鬆症診断支援用ｌｓｉ
JP2011212094A (ja) * 2010-03-31 2011-10-27 Fujifilm Corp 診断支援システム、診断支援装置、診断支援方法および診断支援プログラム
JP2014071592A (ja) * 2012-09-28 2014-04-21 Fujifilm Corp 投薬効果予測システム及びその制御方法、並びに制御プログラム
JP2019097805A (ja) * 2017-11-30 2019-06-24 キヤノン株式会社 情報処理装置、情報処理方法およびプログラム
WO2020116115A1 (ja) * 2018-12-04 2020-06-11 Hoya株式会社 情報処理装置およびモデルの生成方法
WO2020218460A1 (ja) * 2019-04-26 2020-10-29 エーザイ・アール・アンド・ディー・マネジメント株式会社 診断支援装置、推測装置、診断支援システム、診断支援方法、診断支援プログラム及び学習済みモデル

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4480508B2 (ja) * 2004-08-02 2010-06-16 富士通株式会社 診断支援プログラムおよび診断支援装置
JP2009153677A (ja) * 2007-12-26 2009-07-16 Konica Minolta Medical & Graphic Inc 動態画像処理システム
JP5083060B2 (ja) * 2008-06-18 2012-11-28 コニカミノルタエムジー株式会社 画像表示装置、画像表示方法及びプログラム
JP7015275B2 (ja) * 2018-12-04 2022-02-02 Hoya株式会社 モデルの生成方法、教師データの生成方法、および、プログラム
JP7161979B2 (ja) * 2019-07-26 2022-10-27 株式会社 日立産業制御ソリューションズ 説明支援装置、および、説明支援方法

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007135683A (ja) * 2005-11-15 2007-06-07 Gifu Univ 診断支援装置
JP2008036068A (ja) 2006-08-04 2008-02-21 Hiroshima Univ 骨粗鬆症診断支援装置および方法、骨粗鬆症診断支援プログラム、骨粗鬆症診断支援プログラムを記録したコンピュータ読み取り可能な記録媒体、骨粗鬆症診断支援用ｌｓｉ
JP2011212094A (ja) * 2010-03-31 2011-10-27 Fujifilm Corp 診断支援システム、診断支援装置、診断支援方法および診断支援プログラム
JP2014071592A (ja) * 2012-09-28 2014-04-21 Fujifilm Corp 投薬効果予測システム及びその制御方法、並びに制御プログラム
JP2019097805A (ja) * 2017-11-30 2019-06-24 キヤノン株式会社 情報処理装置、情報処理方法およびプログラム
WO2020116115A1 (ja) * 2018-12-04 2020-06-11 Hoya株式会社 情報処理装置およびモデルの生成方法
WO2020218460A1 (ja) * 2019-04-26 2020-10-29 エーザイ・アール・アンド・ディー・マネジメント株式会社 診断支援装置、推測装置、診断支援システム、診断支援方法、診断支援プログラム及び学習済みモデル

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
See also references of EP4299008A4
SELVARAJU ET AL.: "Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization", ARXIV:1610.02391V4[CS.CV, 2019

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPWO2024143476A1 (https=) * 2022-12-28 2024-07-04
JP7736949B2 (ja) 2022-12-28 2025-09-09 京セラ株式会社 情報処理システム、情報処理システムの制御方法、制御プログラム、記録媒体

Also Published As

Publication number Publication date
JP2025166099A (ja) 2025-11-05
CN116887756A (zh) 2023-10-13
EP4299008A1 (en) 2024-01-03
EP4299008A4 (en) 2025-01-15
JPWO2022181154A1 (https=) 2022-09-01
US20240122565A1 (en) 2024-04-18
AU2022225282A1 (en) 2023-09-14

Similar Documents

Publication Publication Date Title
US12256996B2 (en) Systems and methods for generating a three-dimensional model of a joint from two-dimensional images
Ludewig et al. Three-dimensional clavicular motion during arm elevation: reliability and descriptive data
US8059878B2 (en) Registering MR patient data on the basis of generic models
US11672614B1 (en) Robotic surgical systems with multi-modality imaging for performing surgical steps
US8685093B2 (en) Methods and systems for diagnosing, treating, or tracking spinal disorders
JP7051307B2 (ja) 医用画像診断装置
KR20230165284A (ko) 진단 또는 중재 이용을 위해 전자 의료 이미지들을 처리하기 위한 시스템들 및 방법들
US20140031668A1 (en) Surgical and Medical Instrument Tracking Using a Depth-Sensing Device
US20080235052A1 (en) System and method for sharing medical information between image-guided surgery systems
CN108324246A (zh) 医疗诊断辅助系统及方法
US20240023809A1 (en) Systems and Methods for Detection of Musculoskeletal Anomalies
JP6967983B2 (ja) 画像処理装置、画像処理方法、及びプログラム
JP2023526458A (ja) 骨折整復のための術中撮像、並びに仮想モデリング方法、システム、及び手段
WO2012033739A2 (en) Surgical and medical instrument tracking using a depth-sensing device
US20240206973A1 (en) Systems and methods for a spinal anatomy registration framework
JP2025166099A (ja) 判定システム、判定システムの制御方法、および制御プログラム
JP2024540039A (ja) 超音波プローブの複合現実ガイダンス
da Silva et al. Advances in 3D fusion of multimodal medical images: 3D reconstruction of bone, muscle, and ligament structures under load from radiographs and magnetic resonance imaging
US20250235265A1 (en) Systems, devices, and methods for predicting total knee arthroplasty and partial knee arthroplasty procedures
US20240268966A1 (en) Apparatus, system, and method for lapidus correction
US11816821B2 (en) Method and system for generating an enriched image of a target object and corresponding computer program and computer-readable storage medium
WO2022018942A1 (ja) 医用画像処理装置、方法およびプログラム
US20240282221A1 (en) Apparatus, system, and method for patient-specific models
CN114305689B (zh) 一种手术导航定位方法、装置及手术台车
US20250245831A1 (en) Automated Determination Of Bone Mineral Density From A Medical Image

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22759196

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2023502183

Country of ref document: JP

WWE Wipo information: entry into national phase

Ref document number: 202280016760.4

Country of ref document: CN

WWE Wipo information: entry into national phase

Ref document number: 18278779

Country of ref document: US

WWE Wipo information: entry into national phase

Ref document number: 2022225282

Country of ref document: AU

ENP Entry into the national phase

Ref document number: 2022225282

Country of ref document: AU

Date of ref document: 20220125

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 2022759196

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2022759196

Country of ref document: EP

Effective date: 20230926

WWW Wipo information: withdrawn in national office

Ref document number: 2022759196

Country of ref document: EP