Classifications

• • G—PHYSICS
  • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
  • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
  • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
  • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/0002—Inspection of images, e.g. flaw detection
  • G06T7/0012—Biomedical image inspection
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V10/00—Arrangements for image or video recognition or understanding
  • G06V10/20—Image preprocessing
  • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
• • G—PHYSICS
  • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
  • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
  • G16H15/00—ICT specially adapted for medical reports, e.g. generation or transmission thereof
• • G—PHYSICS
  • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
  • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
  • G16H30/00—ICT specially adapted for the handling or processing of medical images
  • G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
• • G—PHYSICS
  • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
  • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
  • G16H30/00—ICT specially adapted for the handling or processing of medical images
  • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
• • G—PHYSICS
  • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
  • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
  • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
  • G16H50/70—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for mining of medical data, e.g. analysing previous cases of other patients
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/30—Subject of image; Context of image processing
  • G06T2207/30004—Biomedical image processing
  • G06T2207/30096—Tumor; Lesion

Definitions

• This disclosure relates to an information processing device, an information processing method, and an information processing program.
• image diagnosis has been performed using medical images obtained by imaging devices such as CT (Computed Tomography) devices and MRI (Magnetic Resonance Imaging) devices.
• medical images are analyzed by CAD (Computer Aided Detection/Diagnosis) using classifiers trained by deep learning, etc., to detect and/or diagnose regions of interest, including structures and lesions, contained in the medical images.
• CAD Computer Aided Detection/Diagnosis
• the medical images and the results of the CAD analysis are sent to the terminals of medical professionals, such as radiologists, who interpret the medical images.
• the medical professionals, such as radiologists use their own terminals to refer to the medical images and the analysis results to interpret the medical images and create interpretation reports.
• JP 2019-153250 A discloses a technology for creating radiology reports based on keywords entered by a radiologist and the results of analysis of medical images.
• a recurrent neural network that has been trained to generate sentences from input characters is used to create sentences to be included in the radiology report.
• Japanese Patent Application Publication No. 2008-052544 discloses a technique for searching a database for similar images and related reports based on search reference image data.
• character information constituting an image reading report on image data is added as search metadata and stored in a database.
• the image reading physician specifies the search reference image data, and the search is performed using the character information already added to the search reference image data as keywords.
• the way in which text is written in an image reading report may differ depending on the creator of the image reading report (e.g., the image reading physician). For example, one image reading physician may express something as "20 mm nodule,” whereas another may express it as "nodule (20 mm).”
• image reading report creation support technology even if there is no problem with the content of the text generated by the computer, it may take time to correct the expression. Therefore, there is a demand for technology that can support the creation of image reading reports based on the images to be read, without the time required to correct the expression of the text.
• the present disclosure provides an information processing device, an information processing method, and an information processing program that can assist in creating radiology reports.
• a first aspect of the present disclosure is an information processing device that includes at least one processor that acquires an image, generates at least one image finding information that is information indicating findings in the image based on the image, and searches for at least one finding statement related to the image finding information from a database in which multiple finding statements are registered in advance.
• the processor may receive input of at least one piece of narrowed-finding information related to an image, and generate, as image finding information, information indicating an image finding related to the narrowed-finding information.
• the processor may search the database for a finding sentence related to at least one of the image finding information and the narrowed-down finding information.
• the processor may accept input of a partial finding sentence that is part of a finding sentence related to an image and includes narrowed-down finding information, and identify the narrowed-down finding information included in the partial finding sentence.
• the processor may modify the findings obtained by the search based on the image.
• a sixth aspect of the present disclosure is any one of the first to fifth aspects above, in which the processor acquires at least one of a past image obtained by previously photographing the subject of the image and at least one piece of past finding information that is information indicating findings in the past image, and generates image finding information indicating changes in findings over time based on the image and at least one of the past image and past finding information.
• a seventh aspect of the present disclosure is any one of the first to sixth aspects above, in which the processor may extract at least one region of interest included in the image, and generate information indicating findings of the extracted region of interest as image finding information.
• the processor when a processor extracts multiple regions of interest contained in an image, the processor may generate image finding information for each region of interest and search a database for finding text related to the multiple image finding information for each region of interest.
• a ninth aspect of the present disclosure is any one of the first to sixth aspects above, in which the processor may accept designation of at least one region of interest included in the image, and generate information indicating findings of the designated region of interest as image finding information.
• the processor when the processor receives a designation of multiple regions of interest included in an image, the processor may generate image finding information for each region of interest and search the database for finding statements related to the multiple image finding information for each region of interest.
• An eleventh aspect of the present disclosure is any one of the first to tenth aspects above, in which the processor may identify, for each of a plurality of finding sentences registered in the database, finding information related to the finding sentence and assign it to the finding sentence, and may search the database for finding sentences to which finding information identical or similar to the image finding information has been assigned.
• the processor may assign the same finding information to each of two or more finding sentences registered in the database when each of the two or more finding sentences includes a synonym.
• a thirteenth aspect of the present disclosure is any one of the first to twelfth aspects above, in which a plurality of finding sentences and key images related to the finding sentences are registered in advance in the database in association with each other, and the processor may display the finding sentences obtained by searching and the key images related to the finding sentences on the display in association with each other.
• a fourteenth aspect of the present disclosure is any one of the first to thirteenth aspects above, in which the processor calculates a degree of match between the image finding information and the finding information included in the searched finding text, and may determine a display method for the searched finding text based on the degree of match.
• a fifteenth aspect of the present disclosure is any one of the first to fourteenth aspects above, in which, if a finding statement for a previous image obtained by previously photographing the subject in the image is registered in the database, the processor may search for the finding statement with priority.
• a sixteenth aspect of the present disclosure is any one of the first to fifteenth aspects above, in which the multiple observation statements registered in the database are provided with creator information indicating the creator of the observation statement, and the processor may receive a designation of the creator of the observation statement to be searched, and if an observation statement to which creator information indicating the designated creator is attached is registered in the database, the processor may search the observation statement with priority.
• a seventeenth aspect of the present disclosure is any one of the first to sixteenth aspects above, in which the image is a medical image, and the image finding information may indicate at least one of the type, characteristics, location, measurement value, and presumed disease name of a lesion contained in the medical image.
• An 18th aspect of the present disclosure is any one of the second to fourth aspects above, in which the image is a medical image, and the narrowed-down finding information may indicate a presumed disease name of a lesion contained in the medical image.
• a 19th aspect of the present disclosure is an information processing method in which a computer executes a process of acquiring an image, generating at least one image finding information that is information indicating the findings of the image based on the image, and searching for at least one finding statement related to the image finding information from a database in which a plurality of finding statements are registered in advance.
• a twentieth aspect of the present disclosure is an information processing program that causes a computer to execute a process of acquiring an image, generating at least one image finding information that is information indicating the findings of the image based on the image, and searching for at least one finding statement related to the image finding information from a database in which a plurality of finding statements are registered in advance.
• the information processing device, information processing method, and information processing program disclosed herein can assist in creating an image interpretation report.
• FIG. 1 is a diagram illustrating an example of a schematic configuration of an information processing system.
• FIG. 1 is a diagram illustrating an example of a medical image.
• FIG. 1 is a diagram illustrating an example of a medical image.
• FIG. 2 is a block diagram showing an example of a hardware configuration of an information processing device.
• FIG. 2 is a block diagram showing an example of a functional configuration of an information processing device.
• FIG. 11 is a diagram illustrating an example of information registered in a report DB.
• FIG. 2 is a diagram illustrating an example of a dictionary.
• FIG. 13 is a diagram showing an example of a screen displayed on a display.
• FIG. 13 is a diagram showing an example of a screen displayed on a display.
• FIG. 10 is a flowchart illustrating an example of a first information process.
• FIG. 13 is a diagram showing an example of a screen displayed on a display.
• FIG. 13 is a diagram illustrating an example of a table.
• FIG. 13 is a diagram showing an example of a screen displayed on a display.
• 10 is a flowchart illustrating an example of a second information process.
• FIG. 1 is a diagram showing a schematic configuration of the information processing system 1.
• the information processing system 1 shown in FIG. 1 photographs the subject's examination target area and stores the medical images acquired by photographing, based on an examination order from a doctor of a medical department using a known ordering system.
• the medical images are interpreted by a radiologist and an interpretation report is created, and the interpretation report is viewed by a doctor of the requesting medical department.
• the information processing system 1 includes an imaging device 2, an image reading WS (WorkStation) 3 which is an image reading terminal, a medical treatment WS 4, an image server 5, an image DB (DataBase) 6, a report server 7, and a report DB 8.
• the imaging device 2, image reading WS 3, medical treatment WS 4, image server 5, image DB 6, report server 7, and report DB 8 are connected to each other via a wired or wireless network 9 in a state in which they can communicate with each other.
• Each device is a computer installed with an application program that enables the device to function as a component of the information processing system 1.
• the application program may be recorded on a recording medium such as a DVD-ROM (Digital Versatile Disc Read Only Memory) or a CD-ROM (Compact Disc Read Only Memory) and distributed, and then installed on the computer from the recording medium.
• the application program may be stored in a storage device or network storage of a server computer connected to the network 9 in a state accessible from the outside, and downloaded to the computer upon request and installed.
• the imaging device 2 is a device (modality) that generates a medical image T representing the diagnostic target part by photographing the diagnostic target part of the subject.
• Examples of the imaging device 2 include a plain X-ray device, a CT (Computed Tomography) device, an MRI (Magnetic Resonance Imaging) device, a PET (Positron Emission Tomography) device, an ultrasound diagnostic device, an endoscope, and a fundus camera.
• the medical images generated by the imaging device 2 are transmitted to the image server 5 and stored in the image DB 6.
• FIG. 2 is a schematic diagram showing an example of a medical image acquired by the imaging device 2.
• the medical image T shown in FIG. 2 is, for example, a CT image consisting of multiple tomographic images T1 to Tm (m is 2 or more), each of which represents a cross-sectional plane from the head to the waist of a single subject (human body).
• Medical image T is an example of an image disclosed herein.
• FIG. 3 is a schematic diagram showing an example of one of the multiple tomographic images Tx.
• the tomographic image Tx shown in FIG. 3 shows a tomographic plane including the lungs.
• Each of the tomographic images T1 to Tm may include a structure area SA showing various organs and organs of the human body (e.g., lungs and liver), and various tissues that constitute the various organs and organs (e.g., blood vessels, nerves, muscles, etc.).
• Each tomographic image may also include a lesion area AA such as a nodule, tumor, injury, defect, or inflammation.
• the lung area is the structure area SA
• the nodule area is the lesion area AA.
• one tomographic image may include multiple structure areas SA and/or lesion areas AA.
• at least one of the structure area SA included in the medical image and the lesion area AA included in the medical image is referred to as a "region of interest.”
• the image reading WS 3 is a computer used by medical professionals, such as radiologists, to read medical images and create image reading reports, and includes the information processing device 10 according to this exemplary embodiment.
• the image reading WS 3 requests the image server 5 to view medical images, performs various image processing on medical images received from the image server 5, displays the medical images, and accepts input of text related to the medical images.
• the image reading WS 3 also performs analysis processing on medical images, supports the creation of image reading reports based on the analysis results, requests the report server 7 to register and view image reading reports, and displays image reading reports received from the report server 7. These processes are performed by the image reading WS 3 by executing software programs for each process.
• the clinical WS4 is a computer used by medical staff such as doctors in a medical department to perform detailed observations of medical images, view interpretation reports, and create electronic medical records, and is composed of a processing device, a display device such as a monitor, and an input device such as a keyboard and a mouse.
• the clinical WS4 makes requests to the image server 5 to view medical images, displays medical images received from the image server 5, makes requests to the report server 7 to view interpretation reports, and displays interpretation reports received from the report server 7. These processes are performed by the clinical WS4 executing software programs for each process.
• the image server 5 is a general-purpose computer with a software program installed that provides the functions of a database management system (DBMS).
• DBMS database management system
• the image server 5 is connected to the image DB 6.
• the connection between the image server 5 and the image DB 6 is not particularly limited, and may be via a data bus or via a network such as a NAS (Network Attached Storage) or a SAN (Storage Area Network).
• the image DB 6 is realized by a storage medium such as a hard disk drive (HDD), a solid state drive (SSD), or a flash memory.
• the image DB 6 registers medical images acquired by the imaging device 2 and associated information attached to the medical images.
• the additional information may include, for example, identification information such as an image ID (identification) for identifying the medical image, a cross-sectional ID assigned to each cross-sectional image included in the medical image, a subject ID for identifying the subject, and an examination ID for identifying the examination.
• the additional information may also include information related to the medical image capture, such as the imaging method, imaging conditions, imaging purpose, imaging location, and imaging date and time.
• Imaging method and imaging conditions refer to, for example, the type of imaging device 2, the imaging site, imaging protocol, imaging sequence, imaging technique, whether or not a contrast agent was used, and slice thickness in tomography.
• the additional information may also include information related to the subject, such as the subject's name, date of birth, age, and sex.
• the image server 5 when the image server 5 receives a request to register a medical image from the imaging device 2, it converts the medical image into a database format and registers it in the image DB 6. In addition, when the image server 5 receives a request to view from the image interpretation WS 3 and the medical treatment WS 4, it searches for medical images registered in the image DB 6 and transmits the searched medical images to the image interpretation WS 3 and the medical treatment WS 4 that originated the view request.
• the report server 7 is a general-purpose computer on which a software program is installed that provides the functions of a database management system.
• the report server 7 is connected to the report DB 8.
• the connection between the report server 7 and the report DB 8 is not particularly limited, and may be via a data bus or via a network such as a NAS or SAN.
• the report DB8 is realized by a storage medium such as a HDD, SSD, or flash memory.
• the report DB8 registers the image reading reports created in the image reading WS3 (described in detail below).
• the report DB8 is an example of a database of the present disclosure.
• the report server 7 when the report server 7 receives a request to register an image reading report from the image reading WS 3, it formats the image reading report for a database and registers it in the report DB 8. In addition, when the report server 7 receives a request to view an image reading report from the image reading WS 3 and the medical treatment WS 4, it searches for the image reading report registered in the report DB 8 and transmits the searched image reading report to the image reading WS 3 and the medical treatment WS 4 that made the view request.
• the network 9 is, for example, a network such as a LAN (Local Area Network) or a WAN (Wide Area Network).
• the imaging device 2, image interpretation WS 3, medical treatment WS 4, image server 5, image DB 6, report server 7, and report DB 8 included in the information processing system 1 may be located in the same medical institution, or in different medical institutions.
• the number of devices, i.e., the imaging device 2, image interpretation WS 3, medical treatment WS 4, image server 5, image DB 6, report server 7, and report DB 8 is not limited to the number shown in FIG. 1, and each device may be composed of multiple devices having similar functions.
• the way in which text is written in an image reading report may differ depending on the creator of the image reading report (e.g., image reading physician). For example, one image reading physician may express an image as "20 mm nodule,” whereas another may express it as "nodule (20 mm)."
• the information processing device 10 supports the creation of an image reading report based on the image of the image reading target, without requiring the user to modify the expression of such text.
• the information processing device 10 will be described below. As described above, the information processing device 10 is included in the image reading WS 3.
• the information processing device 10 includes a CPU (Central Processing Unit) 21, a non-volatile storage unit 22, and a memory 23 as a temporary storage area.
• the information processing device 10 also includes a display 24 such as a liquid crystal display, an input unit 25 such as a keyboard and a mouse, and a network I/F (Interface) 26.
• the network I/F 26 is connected to the network 9 and performs wired and/or wireless communication.
• the CPU 21, the storage unit 22, the memory 23, the display 24, the input unit 25, and the network I/F 26 are connected to each other via a bus 28 such as a system bus and a control bus so that various information can be exchanged between them.
• the storage unit 22 is realized by a storage medium such as an HDD, SSD, or flash memory.
• An information processing program 27 for the information processing device 10 is stored in the storage unit 22.
• the CPU 21 reads the information processing program 27 from the storage unit 22, expands it in the memory 23, and executes the expanded information processing program 27.
• the CPU 21 is an example of a processor of the present disclosure.
• a dictionary 29 is also stored in the storage unit 22 (described in detail below).
• a personal computer, a server computer, a smartphone, a tablet terminal, a wearable terminal, etc. can be appropriately applied as the information processing device 10.
• the information processing device 10 includes a registration unit 30, an acquisition unit 32, a generation unit 34, a search unit 36, and a control unit 38.
• the CPU 21 executes the information processing program 27, the CPU 21 functions as each of the functional units of the registration unit 30, the acquisition unit 32, the generation unit 34, the search unit 36, and the control unit 38.
• an image interpretation report that has already been created is registered in the report DB 8.
• Fig. 6 shows an example of information included in an image interpretation report registered in the report DB 8.
• a plurality of finding statements, creator information indicating the creator of the finding statements, key images related to the finding statements, and finding information related to the finding statements are registered in association with each other in the report DB 8.
• the findings, creator information, and key image are registered in the report DB 8 by the report server 7 when the radiology report is created.
• FIG. 6 shows the key image registered in the report DB 8, but the key image may be registered in another database such as the image DB 6. In this case, an address indicating the registration destination of the key image may be registered in the report DB 8 instead of the key image itself.
• the registration unit 30 identifies the finding information for each of the multiple finding statements registered in the report DB 8 and assigns it to the finding statement.
• FIG. 6 shows the state in which the finding information has been assigned to each finding statement.
• the finding information is information that indicates at least one of various findings, such as the type (name), characteristics, position, measurement value, and presumed disease name of the region of interest contained in the medical image.
• Examples of types include the names of structures such as “lung” and “liver” and the names of lesions such as “nodule”. Properties mainly refer to the characteristics of the lesion. For example, in the case of a pulmonary nodule, examples include absorption values such as “solid” and “ground glass”, margin shapes such as “clear/unclear”, “smooth/irregular”, “spicule”, “lobulated” and “serrated”, and findings indicating the overall shape such as “near-round” and “irregular”. Other examples include the relationship with surrounding tissues such as “pleural contact” and “pleural indentation”, and findings regarding the presence or absence of contrast and washout.
• the location means the anatomical location, the location in the medical image, and the relative positional relationship with other regions of interest such as "inside”, “periphery”, and “surroundings".
• the anatomical location may be indicated by the organ name such as “lung” and “liver”, or may be expressed by subdividing the lung into the "right lung", “upper lobe", and apical region ("S1").
• the measurement value is a value that can be quantitatively measured from the medical image, and is, for example, at least one of the size and signal value of the region of interest.
• the size is expressed, for example, by the major axis, minor axis, area, and volume of the region of interest.
• the signal value is expressed, for example, by the pixel value of the region of interest, and the CT value in units of HU.
• the estimated disease name is an evaluation result estimated based on the lesion, and examples include disease names such as “cancer” and “inflammation”, and evaluation results regarding the disease name and characteristics such as “negative/positive”, “benign/malignant”, and "mild/severe”.
• the respective finding sentences may contain synonyms. For example, one radiology doctor may use the word “spicules," while another may use a different word with the same meaning, "fuzziness.”
• the registration unit 30 assigns the same finding information to each of the two or more finding sentences (so-called normalization). For example, in FIG. 6, "spicules" included in the finding sentence No. 1 and “fuzziness” included in the finding sentence No. 2 are synonymous, so the same finding information, "spicules,” is assigned to each finding sentence.
• a dictionary 29 may be stored in advance in the storage unit 22, which defines the correspondence between words (so-called named entities) indicating finding information that may be included in a finding sentence and the finding information, in which synonymous named entities are associated with the same finding information.
• FIG. 7 shows an example of the dictionary 29.
• the synonymous words “spicules,” “fuzziness,” and “spinous processes” are each associated with the same finding information, "spicules.”
• the registration unit 30 may extract named entities from the finding sentences registered in the report DB 8 and identify the finding information included in the finding sentences by referring to the dictionary 29.
• a known named entity extraction method using a natural language processing model such as BERT (Bidirectional Encoder Representations from Transformers) may be appropriately applied.
• the registration unit 30 may also identify the finding information based on a medical image related to the finding statement, such as a key image and a medical image registered in the image DB 6 or the like. Specifically, the registration unit 30 may identify the finding information not included in the finding statement based on the medical image.
• the method of identifying the finding information based on the medical image is the same as the method of generating the image finding information 62 by the generation unit 34 described later.
• the size of the region of interest is small, it is of little importance to mention it or it may not be measurable from the medical image, but if it is large, it can be seen from the medical image, so it may be omitted from the findings regardless of its size.
• the size of the region of interest may be related to characteristics, etc., and even if it is omitted from the findings, it may implicitly affect the content of the findings. For example, the larger the lung cancer tumor, the more likely it is that internal necrosis will occur. Therefore, it is preferable for the registration unit 30 to add finding information that is not included in the findings but can be identified based on the medical image to the findings and register it in the report DB 8.
• the registration unit 30 when one finding statement registered in the report DB 8 includes finding information related to multiple regions of interest (lesions), it is preferable for the registration unit 30 to identify the finding information for each region of interest.
• the finding statement No. 3 includes a description of two nodules (regions of interest), and the finding information is also identified and registered for each nodule (region of interest).
• the registration unit 30 also preferably identifies the factuality of the identified finding information.
• "Factuality” is information indicating whether a finding is observed or not, as well as the degree of certainty. This is because an image interpretation report may deliberately include not only findings that are clearly observed in medical images, but also findings that are not observed in medical images, and findings that are suspected but have a low degree of certainty. For example, the presence or absence of "calcification” may be used in the diagnosis of pulmonary nodules, and the image interpretation report may deliberately state that "calcification is not observed” (see column No. 4 in Figure 6).
• the finding information may be such that it modifies other finding information, in which case it is preferable that the registration unit 30 also identifies the modifying relationship between the finding information.
• “calcification” which is an example of a characteristic of a pulmonary nodule, may be described in detail, such as “microcalcification is observed in the center.”
• the registration unit 30 may identify the finding information "center” and "micro” as other finding information that modifies the finding information "calcification.”
• finding information that modifies "calcification” include "micro,” “coarse,” “scattered,” “center,” “ring-shaped,” and “complete.”
• renal cell carcinoma which is an example of a suspected disease name based on kidney nodules, may also include descriptions of tissue types such as “clear cell,” “papillary,” “chromophobe,” and “multilocular cystic” in the findings.
• the registration unit 30 may identify findings indicating such tissue types as other findings that modify the findings information "renal cell carcinoma.”
• the information processing device 10 searches for finding statements using the finding information registered in the report DB 8, so it is desirable to register as much finding information as possible in the report DB 8.
• the registration unit 30 identifies finding information based on at least one of the finding statements and medical images, registers it in the report DB 8, and makes it available for searches, thereby contributing to improved search accuracy.
• the report DB 8 registers the finding statements, preparer information, and key image registered when the image interpretation report was created, in association with the finding information assigned by the registration unit 30. Using this information, the information processing device 10 supports the creation of the image interpretation report. The function of the information processing device 10 in supporting the creation of the image interpretation report will be described below with reference to Figs. 8 to 9.
• Fig. 8 shows an example of a screen D1 displayed on the display 24 by the control unit 38 when creating an image interpretation report.
• the acquisition unit 32 acquires the medical image T10 to be interpreted from the image server 5.
• the medical image acquired by the acquisition unit 32 may consist of a single image, or may include multiple images, such as the medical image T consisting of the tomographic images T1 to Tm in FIG. 2.
• the generating unit 34 generates at least one image finding information 62, which is information indicating the findings of the medical image T10, based on the medical image T10 acquired by the acquiring unit 32.
• the image finding information 62 is, for example, information indicating at least one of the type (name), characteristics, position, measurement value, and estimated disease name of the lesion contained in the medical image T10. Details of the type (name), characteristics, position, measurement value, and estimated disease name are the same as those of the finding information assigned to the above-mentioned finding text, so description will be omitted.
• the generation unit 34 first extracts at least one region of interest (lesion region A10 in FIG. 8) contained in the medical image T10 acquired by the acquisition unit 32.
• CAD Computer Aided Detection/Diagnosis
• AI Artificial Intelligence
• the generation unit 34 may extract a region of interest from a medical image using a learning model such as a CNN (Convolutional Neural Network) that is trained to receive a medical image as input and extract and output a region of interest contained in the medical image.
• a learning model such as a CNN (Convolutional Neural Network) that is trained to receive a medical image as input and extract and output a region of interest contained in the medical image.
• the generating unit 34 generates information indicating findings of the extracted region of interest (lesion region A10) as image finding information 62.
• the generating unit 34 may input the region of interest extracted from the medical image and generate image finding information of the region of interest using a learning model such as CNN that has been trained in advance to output image finding information of the region of interest.
• Figure 8 shows an example in which multiple image finding information 62 such as "left lung” and "spicula" are generated as image finding information 62 of the lesion region A10.
• the generating unit 34 may generate information indicating changes in the findings over time as image finding information.
• Information indicating changes in findings over time is, for example, information indicating changes in the size of the lesion, fluctuations in measured values such as signal values, the appearance or disappearance of the lesion, and improvements or deterioration of the condition.
• the acquisition unit 32 acquires at least one of a past image obtained by previously capturing an image of the subject of the medical image to be interpreted, and at least one piece of past finding information that is information indicating findings in the past image.
• the past image is, for example, registered in advance in the image DB 6.
• the past finding information is, for example, identified by the registration unit 30 based on an interpretation report created for the past image, and is registered in advance in the report DB 8.
• the generating unit 34 generates image finding information indicating changes in findings over time based on the medical image of the subject to be read acquired by the acquiring unit 32 and at least one of the past image and the past finding information. For example, the generating unit 34 may generate image finding information indicating changes in findings over time by comparing the medical image T10 of the subject to be read with the past image. For example, the generating unit 34 may extract a region of interest in the past image that corresponds to the region of interest extracted from the medical image of the subject to be read (hereinafter referred to as the "past region of interest"), and when the region of interest included in the medical image of the subject to be read is larger than the past region of interest, generate image finding information indicating that the size is tending to increase.
• the past region of interest a region of interest in the past image that corresponds to the region of interest extracted from the medical image of the subject to be read
• the generating unit 34 may generate image finding information indicating a change in findings over time by comparing image finding information generated for the medical image to be interpreted with past finding information. For example, when image finding information indicating the size of a region of interest extracted from the medical image to be interpreted (e.g., "23 mm") is larger than past finding information indicating the size of a previous region of interest (e.g., "20 mm"), the generating unit 34 may generate image finding information indicating that the size is tending to increase.
• image finding information indicating the size of a region of interest extracted from the medical image to be interpreted e.g., "23 mm”
• past finding information indicating the size of a previous region of interest e.g., "20 mm
• the search unit 36 searches the report DB 8 for at least one finding sentence that includes the image finding information 62 generated by the generation unit 34. Specifically, the search unit 36 searches the report DB 8 for a finding sentence that is assigned with finding information that is the same as or similar to at least one image finding information 62 generated by the generation unit 34.
• FIG. 8 shows an example in which multiple finding sentence candidates 81-84, each of which includes at least one of the multiple image finding information 62 generated by the generation unit 34, are output as search results.
• the finding sentence candidates 81-84 are examples of finding sentences that include the image finding information of the present disclosure.
• the search unit 36 may preferentially search for a finding statement related to the image finding information 62 indicating a predetermined content among the multiple image finding information 62 generated by the generation unit 34.
• the search unit 36 may preferentially search for a finding statement related to the image finding information 62 indicating the type, characteristics, position, measurement value, and estimated disease name of the lesion generated by the generation unit 34.
• the search unit 36 may not use the image finding information 62 indicating the position and measurement value in the search.
• the search unit 36 may allow ambiguity in the image finding information 62 used for the search. For example, for image finding information 62 indicating a position, a finding sentence to which finding information indicating a position within a predetermined range (e.g., the position of an anatomically adjacent area) is added may be output as a search result. For image finding information 62 indicating a measurement value, a finding sentence to which finding information indicating a measurement value including a difference in a predetermined amount or percentage is added may be output as a search result.
• a finding sentence to which finding information indicating a measurement value including a difference in a predetermined amount or percentage is added may be output as a search result.
• a finding sentence to which finding information indicating different suspected disease names belonging to the same classification (e.g., "primary lung cancer” and "lung cancer") is added may be output as a search result.
• finding information indicating different suspected disease names belonging to the same classification e.g., "primary lung cancer” and "lung cancer”
• ICD International Statistical Classification of Diseases and Related Health Problems
• the search unit 36 may search for the finding statement as a priority. For example, when a medical image accompanied by a subject ID indicating the subject of the medical image T10 to be interpreted is registered with reference to the image DB 6, the search unit 36 may search for an interpretation report for the medical image from the report DB 8. According to this embodiment, it becomes easier to standardize the expression method for each subject, so that an easy-to-read interpretation report can be created even when referring to a past interpretation report, for example, during follow-up observation.
• the search unit 36 may also accept the user's specification of the creator of the finding to be searched, and if a finding to which creator information indicating the specified creator is attached is registered in the report DB 8, the search unit 36 may search for that finding as a priority.
• the search unit 36 may search for that finding as a priority.
• this type of configuration for example, by specifying the user himself/herself, it is possible to search for findings created by the user in the past, i.e., findings that the user prefers. Also, for example, by specifying a representative creator such as a veteran radiologist, it becomes easier to standardize the way findings are expressed even when multiple radiologists share the work of reading images.
• the search unit 36 may also prioritize searching for findings written on medical images that have the same or similar information about the imaging (e.g., imaging method, imaging conditions, imaging purpose, imaging location, imaging date and time, etc.) as the medical image T10 to be interpreted. Similarly, the search unit 36 may prioritize searching for findings written on medical images that have the same or similar information about the subject (e.g., age, sex, and medical history) as the medical image T10 to be interpreted. This information can be identified, for example, based on additional information attached to the medical image.
• imaging e.g., imaging method, imaging conditions, imaging purpose, imaging location, imaging date and time, etc.
• the search unit 36 may prioritize searching for findings written on medical images that have the same or similar information about the subject (e.g., age, sex, and medical history) as the medical image T10 to be interpreted. This information can be identified, for example, based on additional information attached to the medical image.
• the search unit 36 may also calculate the degree of agreement between the imaging finding information 62 generated by the generation unit 34 and the finding information assigned to the searched finding sentence candidates 81 to 84. For example, the search unit 36 may calculate, as the degree of agreement, the proportion of finding information that is the same as or similar to the imaging finding information 62 contained in the searched finding sentences. In this case, the search unit 36 may also weight the imaging finding information 62 according to its contents, such as by weighting the imaging finding information 62 indicating the type, characteristics, and estimated disease name of the lesion higher than the imaging finding information 62 indicating the position and measurement value of the lesion.
• the finding sentences registered in the report DB 8 may be provided with finding information identified based on medical images, even if it is not included in the finding sentence.
• the search unit 36 may calculate the degree of match only with the finding information included in the finding sentence, or may calculate the degree of match including the finding information identified based on medical images that is not included in the finding sentence. The search unit 36 may also calculate both of these degrees of match.
• the control unit 38 controls the display 24 to display the finding sentence candidates 81-84 obtained by the search unit 36. It is also preferable that the control unit 38 displays the finding sentence candidates 81-84 obtained by the search and the key images related to the finding sentence candidates 81-84 on the display 24 in association with each other. For example, as shown in screen D1, the control unit 38 may display the finding sentence candidates 81-84 obtained by the search unit 36 and thumbnails of the key images registered in the report DB 8 in association with each of the finding sentence candidates 81-84. For example, when a thumbnail of a key image is selected by the user operating the mouse pointer 92 via the input unit 25, the control unit 38 may enlarge and display the selected key image on the display 24.
• the control unit 38 may highlight the region of interest extracted by the generation unit 34.
• FIG. 8 shows an example in which the lesion region A10 is highlighted by a bounding box 90 in the medical image T10.
• the control unit 38 may determine a display method for the searched finding sentence candidates 81-84 based on the degree of agreement. In FIG. 8, the degree of agreement is displayed to the left of each of the finding sentence candidates 81-84. As shown in FIG. 8, the control unit 38 may display the finding sentence candidates 81-84 in order of the degree of agreement. In addition, for example, the control unit 38 may highlight the finding sentences whose degree of agreement is equal to or exceeds a predetermined threshold. In another example, the control unit 38 may display the number of pieces of finding information that are the same as or similar to the image finding information 62 and that are assigned to the searched finding sentence candidates 81-84.
• the control unit 38 may also control the display 24 to display the portions of the finding sentence candidates 81-84 that correspond to the image finding information 62 in a identifiable manner.
• Identity means, for example, changing the thickness, size, italics, font, character color, and background color of the characters, underlining, strikethrough, and boxing, or displaying in different columns.
• the portions of the finding sentence candidates 81-84 that correspond to the image finding information 62 are underlined.
• the control unit 38 may also accept a selection of a finding sentence obtained by the search unit 36 to be used as the basis for a finding sentence for the medical image T10 to be interpreted, and accept modifications to the selected finding sentence.
• the user operates the mouse pointer 92 via the input unit 25 to select the finding sentence candidate 81-84 displayed on the screen D1 to be used.
• Figure 9 shows an example of a screen D2 displayed on the display 24 by the control unit 38 when the finding sentence candidate No. 2 82 is selected on the screen D1 of Figure 8.
• the control unit 38 may also modify the searched-for finding sentences based on the medical image T10 acquired by the acquisition unit 32. Specifically, the control unit 38 may use the image finding information 62 generated by the generation unit 34 based on the medical image T10 to modify the portions of the searched-for finding sentences that do not match the image finding information 62. In this case, the control unit 38 may perform control to display the portions of the finding sentences that do not match the image finding information 62 on the display 24 in an identifiable manner. As an example, FIG. 9 shows a finding sentence 82R from among the finding sentence candidates 82 in which the descriptions of the position and measurement values have been modified to content corresponding to the image finding information 62. In the finding sentence 82R, the portions that do not match the image finding information 62 (the descriptions of the position and measurement values) are enclosed in a box.
• the control unit 38 may also control the display 24 to identifiably display, among the image finding information 62 generated by the generation unit 34, image finding information 62 that is not included in the finding statement 82R.
• the image finding information 62 that is not included in the finding statement 82R is marked with a strikethrough.
• the control unit 38 may also update the presence or absence of a strikethrough in the image finding information 62 when the finding statement 82R is corrected.
• the user checks the finding statement 82R displayed on screen D2, modifies the finding statement 82R via the input unit 25 as necessary, and then selects the registration button 94.
• the control unit 38 transmits a request to the report server 7 to register an image interpretation report including the modified finding statement 82R.
• the control unit 38 may also request that the image finding information 62 generated by the generation unit 34 be registered as well.
• the CPU 21 executes the information processing program 27, thereby executing the first information processing shown in FIG. 10.
• the first information processing is executed, for example, when a command to start execution is given by the user via the input unit 25.
• step S10 the acquisition unit 32 acquires the medical image to be interpreted from the image server 5.
• step S12 the generation unit 34 generates at least one image finding information, which is information indicating the findings of the medical image, based on the medical image acquired in step S10.
• step S14 the search unit 36 searches the report DB 8 for at least one finding statement to which the image finding information generated in step S12 has been assigned.
• step S16 the control unit 38 controls the display 24 to display the finding statement obtained by the search in step S14, and ends this information processing.
• the information processing device 10 includes at least one processor, which acquires an image, generates at least one image finding information, which is information indicating the findings of the image, based on the image, and searches for at least one finding sentence related to the image finding information from a database in which multiple finding sentences are registered in advance.
• the information processing device 10 can search for findings that correspond to the medical image to be interpreted and that have already been registered in the report DB 8. Therefore, the method of expressing findings can be reused as is, and the creation of an interpretation report can be assisted without requiring the user to go through the trouble of modifying the expression of the sentences.
• a medical image may include multiple lesion areas.
• the generating unit 34 When multiple regions of interest are extracted from a medical image, it is preferable for the generating unit 34 to generate image finding information for each region of interest. That is, it is preferable for the generating unit 34 to generate image finding information indicating at least one of the type, characteristics, position, measurement value, and estimated disease name for each region of interest, as shown in No. 3 of FIG. 8.
• the search unit 36 searches the report DB 8 for a finding statement related to multiple image finding information for each region of interest.
• the search unit 36 searches for a finding statement that includes a description of multiple regions of interest, as shown in No. 3 of FIG. 8, and that is provided with finding information that is as identical or similar as possible to the multiple image finding information for each region of interest.
• the control unit 38 modifies the selected finding sentence candidate 82 from among the finding sentence candidates 81 to 84 obtained by the search unit 36, but this is not limiting.
• the control unit 38 may perform control to modify each of the finding sentence candidates 81 to 84 on the screen D1 and then display them on the display 24.
• the control unit 38 may cause the display 24 to be able to switch between displaying the finding sentence obtained by the search (i.e., the original sentence) and the modified finding sentence.
• the generating unit 34 extracts at least one region of interest included in a medical image and generates image finding information indicating findings in the extracted region of interest, but this is not limited to the above.
• the generating unit 34 may accept designation of at least one region of interest included in a medical image.
• the generating unit 34 may display a medical image to be interpreted on the display 24, and accept designation of the position and range of the region of interest in the medical image by the user via the input unit 25.
• the generating unit 34 may generate information indicating findings in the specified region of interest as image finding information.
• the region of interest that is accepted is not limited to one, and the specification of multiple regions of interest contained in the medical image may be accepted.
• the generation unit 34 generates image finding information for each region of interest.
• the search unit 36 searches the report DB 8 for finding statements related to the multiple image finding information for each region of interest.
• a search is performed using image finding information generated by the generating unit 34 based on a medical image.
• various variations of finding sentences such as finding sentence candidates 81 and 82 having the same content but different expression methods, and finding sentence candidates 83 and 84 having different content from the finding sentence candidates 81 and 82, may be output as search results.
• finding sentences such as finding sentence candidates 81 and 82 having the same content but different expression methods, and finding sentence candidates 83 and 84 having different content from the finding sentence candidates 81 and 82, may be output as search results.
• finding sentences such as finding sentence candidates 81 and 82 having the same content but different expression methods, and finding sentence candidates 83 and 84 having different content from the finding sentence candidates 81 and 82, may be output as search results.
• finding sentences such as finding sentence candidates 81 and 82 having the same content but different expression methods, and finding sentence candidates 83 and 84 having different content from the finding sentence candidates 81 and 82, may be output as search results.
• finding sentences that differ not only in expression methods but also in content
• the information processing device 10 according to the second exemplary embodiment has a function to narrow down the image finding information used in the search, in addition to the function described in the first exemplary embodiment.
• the functions of the information processing device 10 according to the second exemplary embodiment will be described with reference to the drawings, but some of the explanations that overlap with the first exemplary embodiment will be omitted.
• the registration unit 30 assigns each of the multiple finding statements registered in the report DB 8 with finding information related to the finding statement (see FIG. 6).
• the acquisition unit 32 acquires the medical image T10 to be interpreted from the image server 5.
• the control unit 38 controls the display of the medical image T10 acquired by the acquisition unit 32 on the display 24.
• the control unit 38 accepts input of at least one piece of narrowed-finding information related to the medical image T10 to be interpreted acquired by the acquisition unit 32.
• the control unit 38 may present candidates for narrowed-finding information and accept selection of at least one of them.
• the narrowed-finding information is, for example, information indicating the estimated disease name of a lesion contained in the medical image T10.
• FIG. 11 shows an example of a screen D3 for accepting input of narrowed finding information 64, which is displayed on the display 24 by the control unit 38.
• narrowed finding information 64 which is displayed on the display 24 by the control unit 38.
• the control unit 38 may accept the input of narrowed finding information, for example, by a pull-down menu that can specify various predefined suspected disease names (see FIG. 12).
• the user interprets the medical image T10 displayed on the display 24 and inputs the narrowed finding information 64 via the input unit 25.
• the generation unit 34 generates information indicating findings in the medical image T10 related to the narrowed finding information 64 as image finding information 62R of the medical image T10 to be interpreted.
• FIG. 12 shows an example of a table 96 in which the narrowed finding information is associated with image finding information related to each narrowed finding information.
• the table 96 is stored in advance in the storage unit 22, for example.
• the generating unit 34 generates image finding information 62R that can be generated based on the medical image T10 to be interpreted and that is determined in the table 96 to be related to the narrowed-down finding information 64.
• image finding information 62R For example, as shown in FIG. 8, based on the medical image T10, three types of image finding information 62 indicating the characteristics of the lesion area A10 can be generated: "spicules,” “irregular,” and "partially solid.”
• the image finding information corresponding to "lung cancer” includes “spicules” and “irregular,” but does not include “partially solid.”
• the generating unit 34 generates information indicating "spicules” and “irregular,” but does not generate information indicating "partially solid,” as the image finding information 62R of the medical image T10.
• the search unit 36 searches the report DB 8 for finding sentences to which at least one of the image finding information 62R and the narrowed finding information 64 has been assigned.
• FIG. 13 shows an example of a screen D4 on which the finding sentence candidates 81-83 including at least one of the image finding information 62R indicating "spicules" and “irregular shape” and the narrowed finding information 64 indicating "lung cancer” are output as search results.
• Screen D4 does not include the finding sentence candidate 84 in FIG. 8. This is because the finding sentence candidate 84 does not include the image finding information 62R indicating "spicules" and "irregular shape", but only includes the finding information indicating "partially solid type” that was excluded based on the narrowed finding information 64.
• the CPU 21 executes the information processing program 27, thereby executing the second information processing shown in FIG. 14.
• the second information processing is executed, for example, when a command to start execution is given by the user via the input unit 25.
• step S50 the acquisition unit 32 acquires the medical image to be interpreted from the image server 5.
• step S52 the control unit 38 controls the display 24 to display the medical image acquired in step S50, and accepts input of at least one piece of narrowed-finding information related to the medical image.
• step S54 the generation unit 34 generates information indicating the findings of the medical image related to the narrowed-finding information accepted in step S52 as image finding information of the medical image acquired in step S50.
• step S56 the search unit 36 searches the report DB 8 for a finding statement to which at least one of the image finding information generated in step S54 and the narrowed finding information received in step S52 has been added.
• step S58 the control unit 38 controls the display 24 to display the finding statement obtained by the search in step S56, and ends this information processing.
• the information processing device 10 includes at least one processor, which acquires an image, generates at least one image finding information that is information indicating findings in the image based on the image, and searches for at least one finding statement related to the image finding information from a database in which multiple finding statements are registered in advance.
• the processor also accepts input of at least one narrowed-down finding information related to the image, and generates, as image finding information, information indicating findings in the image related to the narrowed-down finding information.
• the information processing device 10 can search for a finding statement that corresponds to both the medical image to be interpreted (image finding information) and the contents of the interpretation by the radiologist (narrowed finding information) and that has already been registered in the report DB 8. Therefore, the method of expressing the finding statement can be reused as is while further improving the accuracy of the search, thereby supporting the creation of radiology reports without requiring the user to go through the trouble of modifying the expression of the sentences.
• the narrowed finding information 64 is input using a pull-down menu, but the present invention is not limited to this.
• the control unit 38 may accept an input of a partial finding sentence that is a part of a finding sentence related to a medical image and includes narrowed finding information, and identify the narrowed finding information included in the partial finding sentence. For example, when an input of a partial finding sentence "Lung cancer is suspected" is accepted, the control unit 38 may perform a named entity extraction process and identify "lung cancer", which is one of the narrowed finding information previously defined in table 96 (see FIG. 12).
• a known named entity extraction method using a natural language processing model such as BERT may be appropriately applied.
• the narrowed-down finding information is finding information indicating a presumed disease name (see FIG. 12), but this is not limited to the above.
• finding information indicating at least one of the type (name), characteristics, position, measurement value, and presumed disease name of the lesion contained in the medical image T10 can be appropriately applied.
• the narrowed finding information in FIG. 12 includes presumed disease names (e.g., "lung cancer” and “liver cancer”) that differ in anatomical location (e.g., "lung” and “liver”).
• presumed disease names e.g., "lung cancer” and "liver cancer
• information indicating a presumed disease name determined in advance for each anatomical location of a lesion may be used as the narrowed finding information.
• the control unit 38 may identify the anatomical location of a lesion based on the image finding information generated by the generation unit 34.
• control unit 38 may accept input of the narrowed finding information via a pull-down menu that allows the user to specify a presumed disease name corresponding to the identified anatomical location (e.g., "lung cancer,” “pneumonia,” and “atelectasis” for "lung”).
• a presumed disease name corresponding to the identified anatomical location (e.g., "lung cancer,” “pneumonia,” and “atelectasis” for "lung”).
• the control unit 38 may also accept input of multiple pieces of narrowed finding information.
• the search unit 36 searches the report DB 8 for image finding information and a finding statement to which at least one of the multiple pieces of narrowed finding information has been assigned.
• the means for accepting input of multiple pieces of narrowed finding information is not particularly limited. For example, this may be realized by displaying on the display 24 a GUI (Graphical User Interface) capable of specifying multiple items such as a multi-select list and check boxes. For example, this may be realized by displaying on the display 24 a GUI such as a free-form text box in which the user can input multiple pieces of narrowed finding information or a partial finding statement including multiple pieces of narrowed finding information. For example, this may be realized by displaying on the display 24 a GUI such as a slider in which the size of the region of interest and the range of measurement values such as signal values (i.e. maximum and minimum values).
• the control unit 38 may also accept input of multiple narrowed finding information in stages. Specifically, the control unit 38 may accept input of first narrowed finding information, and then accept input of second narrowed finding information related to the first narrowed finding information. In this case, the control unit 38 may vary the content presented as candidates for the second narrowed finding information depending on the first narrowed finding information. For example, when "renal cell carcinoma" is input as the first narrowed finding information, the control unit 38 may accept input of at least one of "clear cell type,” "papillary,” “chromophobe,” and "multilocular cystic” as second narrowed finding information indicating the tissue type of "renal cell carcinoma.”
• the candidate for the narrowed finding information is predefined in the table 96, but the present invention is not limited to this.
• the control unit 38 may search the report DB 8 for the finding sentence to which the image finding information generated by the generation unit 34 is added, and may compile the finding information added to the finding sentence obtained as the search result, and set the candidate for the narrowed finding information in descending order of frequency.
• the control unit 38 may search the report DB 8 for the finding sentence to which the first narrowed finding information is added, and may compile the finding information added to the finding sentence obtained as the search result, and set the candidate for the second narrowed finding information in descending order of frequency.
• the type of finding information to be compiled i.e., the finding information that can be a candidate for the narrowed finding information
• the finding information that can be a candidate for the narrowed finding information may be limited. For example, only the finding information indicating the type (name) of the lesion and the estimated disease name may be compiled, and the finding information indicating the characteristics, position, and measurement value may not be compiled.
• the control unit 38 may present information that assists the user in selecting the narrowed finding information. For example, the control unit 38 may present the number of candidate finding sentences output as a search result when a finding sentence is searched for using the narrowed finding information. For example, the control unit 38 may present the proportion of the finding sentences to which the narrowed finding information is assigned among the finding sentences registered in the report DB 8. For example, when the input of the first narrowed finding information and the second narrowed finding information is accepted in stages, the control unit 38 may present at least one of the number and the proportion of the finding sentence candidates to which the second narrowed finding information is assigned among the finding sentence candidates to which the first narrowed finding information is assigned.
• the generating unit 34 generates image finding information 62R related to the narrowed finding information 64, the searching unit 36 searches for a finding sentence based on the image finding information 62R, and the control unit 38 displays the image finding information 62R on the display 24 (see FIG. 13).
• the present invention is not limited to this.
• the generating unit 34 may generate both all image finding information 62 (see FIG. 8) that can be generated based on the medical image T10 of the image reading target similar to the first exemplary embodiment, and image finding information 62R related to the narrowed finding information 64.
• the control unit 38 may control the display 24 to identifiably display the image finding information 62R related to the narrowed finding information 64 among all image finding information 62 that can be generated based on the medical image T10 of the image reading target.
• "Make it identifiable” means, for example, changing the thickness, size, italics, font, text color, and background color of the characters, underlining, strikethrough, and boxing, or displaying it in a different column, etc.
• the control unit 38 may also control the display 24 to display a list of image finding information that is defined in the table 96 (see FIG. 12) as being related to the inputted narrowed finding information 64. For example, when "lung cancer" is inputted as the narrowed finding information 64, the control unit 38 may control the display 24 to display "irregular, solid, lobulated, spicula, pleural indentation, calcification, cavity" that is defined in the table 96 as image finding information corresponding to "lung cancer". The control unit 38 may also control the display 24 to identifiably display the image finding information 62R generated by the generation unit 34 from this list.
• the user may be able to narrow down the image finding information and the narrowed-down finding information used in the search.
• the search unit 36 may redo the search for the finding text based on the selected image finding information 62 and 62R.
• the control unit 38 may request that the image finding information 62 and 62R selected by the user be registered (i.e., that the image finding information 62 and 62R that was not selected be deleted and registered).
• the user may be allowed to specify the image finding information and narrowed finding information to be excluded from the search results. That is, the search unit 36 may not output, as a search result, the image finding information and the finding sentence to which the narrowed finding information specified to be excluded has been added. For example, when it is specified to exclude the image finding information indicating "spicules", the search unit 36 may not output, as a search result, the finding sentence to which the finding information indicating "spicules" has been added, regardless of other finding information and the degree of match, etc.
• the information processing device 10 of the present disclosure is applicable to various images obtained by photographing a subject and including a region of interest.
• the information processing device 10 may be applied to images obtained from equipment, buildings, pipes, welds, etc. as subjects in non-destructive testing such as radiographic testing and ultrasonic flaw detection testing.
• the region of interest may indicate, for example, cracks, flaws, bubbles, foreign objects, etc.
• the various processors shown below can be used as the hardware structure of the processing units that execute various processes, such as the registration unit 30, acquisition unit 32, generation unit 34, search unit 36, and control unit 38.
• the various processors include a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units, as well as programmable logic devices (PLDs), which are processors whose circuit configuration can be changed after manufacture, such as FPGAs (Field Programmable Gate Arrays), and dedicated electrical circuits, such as ASICs (Application Specific Integrated Circuits), which are processors with a circuit configuration designed specifically to execute specific processes.
• a CPU which is a general-purpose processor that executes software (programs) and functions as various processing units, as well as programmable logic devices (PLDs), which are processors whose circuit configuration can be changed after manufacture, such as FPGAs (Field Programmable Gate Arrays), and dedicated electrical circuits, such as ASICs (Application Specific Integrated Circuits), which are processor
• a single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (e.g., a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with a single processor.
• Examples of configuring multiple processing units with a single processor include, first, a form in which one processor is configured with a combination of one or more CPUs and software, as typified by client and server computers, and this processor functions as multiple processing units. Secondly, a form in which a processor is used to realize the functions of the entire system, including multiple processing units, with a single IC (Integrated Circuit) chip, as typified by system on chip (SoC). In this way, the various processing units are configured as a hardware structure using one or more of the various processors listed above.
• SoC system on chip
• the hardware structure of these various processors can be an electrical circuit that combines circuit elements such as semiconductor elements.
• the information processing program 27 is described as being pre-stored (installed) in the storage unit 22, but this is not limited to the above.
• the information processing program 27 may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a USB (Universal Serial Bus) memory.
• the information processing program 27 may also be downloaded from an external device via a network.
• the technology disclosed herein extends to a storage medium that non-temporarily stores an information processing program, in addition to the information processing program.
• the technology of the present disclosure can also be appropriately combined with the above exemplary embodiments and examples.
• the above description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure, and are merely one example of the technology of the present disclosure.
• the above explanation of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or substitutions may be made to the description and illustrations shown above, within the scope of the gist of the technology of the present disclosure.

Landscapes

• Engineering & Computer Science (AREA)
• Health & Medical Sciences (AREA)
• Medical Informatics (AREA)
• Public Health (AREA)
• General Health & Medical Sciences (AREA)
• Epidemiology (AREA)
• Primary Health Care (AREA)
• Data Mining & Analysis (AREA)
• Biomedical Technology (AREA)
• Radiology & Medical Imaging (AREA)
• Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
• General Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Physics & Mathematics (AREA)
• Pathology (AREA)
• Databases & Information Systems (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Multimedia (AREA)
• Quality & Reliability (AREA)
• Medical Treatment And Welfare Office Work (AREA)
• Measuring And Recording Apparatus For Diagnosis (AREA)

Priority Applications (2)

Applications Claiming Priority (4)

Related Child Applications (1)

Publications (1)

Family

ID=90477955

Family Applications (1)

Country Status (3)

Citations (6)

• 2023
  • 2023-09-27 JP JP2024550417A patent/JPWO2024071246A1/ja active Pending
  • 2023-09-27 WO PCT/JP2023/035274 patent/WO2024071246A1/ja not_active Ceased
• 2025
  • 2025-03-14 US US19/079,471 patent/US20250210182A1/en active Pending

Patent Citations (6)

Also Published As

Similar Documents

Legal Events