US20220199230A1 - Context driven summary view of radiology findings - Google Patents

Context driven summary view of radiology findings Download PDF

Info

Publication number
US20220199230A1
US20220199230A1 US17/692,566 US202217692566A US2022199230A1 US 20220199230 A1 US20220199230 A1 US 20220199230A1 US 202217692566 A US202217692566 A US 202217692566A US 2022199230 A1 US2022199230 A1 US 2022199230A1
Authority
US
United States
Prior art keywords
context information
current
prior studies
summary view
studies
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US17/692,566
Inventor
Thusitha Danajaya De Silva MABOTUWANA
Ye Xu
Yuechen Qian
Gabriel Ryan Mankovich
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Koninklijke Philips NV
Original Assignee
Koninklijke Philips NV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Koninklijke Philips NV filed Critical Koninklijke Philips NV
Priority to US17/692,566 priority Critical patent/US20220199230A1/en
Publication of US20220199230A1 publication Critical patent/US20220199230A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00ICT specially adapted for the handling or processing of medical images
    • G16H30/20ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0012Biomedical image inspection
    • G06T7/0014Biomedical image inspection using an image reference approach
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/768Arrangements for image or video recognition or understanding using pattern recognition or machine learning using context analysis, e.g. recognition aided by known co-occurring patterns
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H15/00ICT specially adapted for medical reports, e.g. generation or transmission thereof
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00ICT specially adapted for the handling or processing of medical images
    • G16H30/40ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/70ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for mining of medical data, e.g. analysing previous cases of other patients
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing

Definitions

  • Radiologists are required to work with an increasing number of images to diagnose and treat patients in an optimal manner. Patients such as, for example, cancer patients, frequently undergo numerous imaging exams, accumulating many studies in their medical records pertaining to the same anatomical region. Each time a new study needs to be read, the radiologist opens the current order to understand why the study has been performed and what prior findings the patient has. The imaging order, however, contains limited information related to findings such that the radiologist would need to open the results of the most relevant prior study to better understand the status of the patient's findings. This prior report, however, may not contain sufficient information about all the relevant findings, providing an incomplete overview of the patient's finding history.
  • the radiologist is often required to open multiple prior reports and/or images and build mental links between related findings to obtain a complete picture of the patient's history. This process is difficult and time consuming. In addition, there are currently no quality assurance tools to ensure consistency and longevity of findings such that some previously identified findings may be overlooked. If findings are not followed-up on, as required, there may be potentially adverse effects on the patient outcome.
  • a method for generating a context driven summary view of medical findings by retrieving a current study including at least one image to be analyzed, extracting current context information from the current study and storing the current context information in a database and matching the current context information with prior context information from prior studies to return a set of relevant prior studies.
  • a system for generating a context driven summary view of medical findings having a processor retrieving a current study including at least one image to be analyzed, extracting current context information from the current study and matching the current context information with prior context information from prior studies to return a set of relevant prior studies; and a memory storing the current and prior context information in a database.
  • the steps including retrieving a current study including at least one image to be analyzed, extracting current context information from the current study and store the current context information in a database and matching the current context information with prior context information from prior studies to return a set of relevant prior studies.
  • FIG. 1 shows a schematic drawing of a system according to a first exemplary embodiment.
  • FIG. 2 shows another schematic drawing of the system of FIG. 1 .
  • FIG. 3 shows a flow diagram of a method according to the first exemplary embodiment.
  • FIG. 4 shows an exemplary screenshot of a summary view displayed according to the method of FIG. 3 .
  • FIG. 5 shows another exemplary screenshot of the summary view of FIG. 4 .
  • FIG. 6 shows an exemplary screenshot of key images displayed according to the method of FIG. 3 .
  • FIG. 7 shows an alternate exemplary screenshot of a summary view displayed according to the method of FIG. 3 .
  • FIG. 8 shows a schematic drawing of a system and method according to a second exemplary embodiment.
  • FIG. 9 shows a flow diagram according to the method of FIG. 8 .
  • FIG. 10 shows a schematic diagram of a system according to a third exemplary embodiment.
  • FIG. 11 shows a flow diagram of a method according to the third exemplary embodiment.
  • the exemplary embodiments may be further understood with reference to the following description and the appended drawings wherein like elements are referred to with the same reference numerals.
  • the exemplary embodiments relate to a system and method for reviewing a medical image.
  • the exemplary embodiments describe extracting the context of a current imaging study to aid in reporting the results of the current imaging study.
  • the context of the current imaging study may be used to generate a context-driven summary of prior studies, suggest templates/macros related corresponding to the current study and/or identify all related findings which should be reported on in a final report.
  • a system 100 generates a context driven summary view of prior related studies.
  • the system 100 comprises a processor 102 , a user interface 104 , a display 106 and a memory 108 .
  • the processor 102 generates the summary view for a current study 120 including at least one image to be analyzed by extracting context information of the current study 120 using a context extraction module 110 .
  • the context extraction module 110 may also extract context information regarding the patient's clinical indications from an order of the current study 120 .
  • the context information may include data such as, for example, a modality (e.g., MRI, mammogram) and/or a body part (e.g., breast, head) imaged in the images of the current study 120 .
  • a modality e.g., MRI, mammogram
  • a body part e.g., breast, head
  • the context extraction module 110 may also extract context information including the patient's clinical indications from the image exam order of the current study.
  • This context information is stored in a database 118 of the memory 108 , which also stores finding (e.g., tumor) related information for current and/or prior studies of the patient.
  • the database 118 may also store for example, annotations (i.e., mark-ups on an image) relating to the finding.
  • the processor 102 further includes a capture module 112 , which captures and stores information to the database 118 whenever a new annotation is created on the image by a user, and a Natural Language Processing (NLP) module 122 which extracts text data from the current study and/or prior reports.
  • NLP Natural Language Processing
  • the processor 102 also includes a context matching module 114 which matches the context extracted from the images of the current study 120 with all prior studies to determine which prior studies are related to the current study 120 .
  • current and/or prior study information may be displayed on the display 106 in a summary view.
  • the summary view may, for example, show observations related to each of the findings and/or annotations on the images related to each of the findings.
  • a navigation/visualization module 116 of the processor 102 permits a user to select a displayed observation and/or annotation to display further details of the selected finding.
  • relevant prior studies may also be filtered to display a minimum set of findings that should be included in a report of the current study to maintain consistency and longevity of reports.
  • the context matching module 114 may also compare the context information of the current/prior studies with context information of templates and/or macros available for use by the user to return a set of relevant templates/macros. Thus, the user may easily select the templates/macros to cut down on reporting time. User selections may be made via the user interface 104 , which may include input devices such as, for example, a keyboard, mouse, and/or touch display on the display 106 .
  • FIG. 3 shows a method 200 by which the summary view is generated.
  • the system 100 retrieves the images to be analyzed in the current study 120 .
  • the images may be stored and viewed in, for example, a Picture Archiving and Communications System (PACS) within a Radiology Information System (RIS).
  • the context extraction module 110 extracts content information from the images to be analyzed in the current study.
  • the context information may include, for example, modality and/or body part.
  • Images stored on the RIS/PACS system for example, are stored and viewed in a DICOM (Digital Imaging and Communications in Medicine) file format, which includes a header containing information related to the modality and body part.
  • DICOM Digital Imaging and Communications in Medicine
  • the context information may also include the patient's clinical indications from the imaging order of the current study.
  • the extracted context information is then stored in the database 118 , in a step 230 .
  • the database 118 also includes context information for images of prior studies, which may have been previously extracted and stored.
  • a step 240 images of prior studies are analyzed by the capture module 114 to identify and store finding related information in the database 118 .
  • findings may include a tumor or mass identified in the images. Observations may include, for example, a description of the mass and/or a result of a biopsy of the tumor. Annotations for each of the observations and findings may be created on the image. Each time a new finding is created, the finding related information, including observations and annotations, are stored in the database 118 . Each finding may have multiple observations associated with it. For example, the finding may be a first mass on a patient's left breast.
  • the radiologist may input one or more observations regarding the first mass such as, for example, a designation of the mass as either malignant or benign and/or a description of the mass. Each observation may have one or more annotations on the image associated with it.
  • the radiologist may also indicate that the annotations are for a new finding (e.g., a second mass) so that a new finding and observation are created. Alternatively, the radiologist may indicate that the annotation is associated with a new observation for an existing finding.
  • Prior finding related information may also be stored by processing prior reports using a natural language processing (NLP) module 122 such as, for example, MEDLEE, so that findings from prior reports/studies may be quickly extracted. NLP modules alone provide only textual information of findings and will not show image annotations.
  • NLP natural language processing
  • the NLP module 122 may thus be combined with the above-described capture module 112 to extract both text and image data from prior studies.
  • Finding related information of the current study is similarly generated and stored in the database 118 . It will be understood by those of skill in the art that the finding related information of each study may be created and stored in the database 118 as each study is conducted and reviewed such that the finding related information may be stored in the database and later recalled when necessary.
  • the context matching module 114 compares the context information of the current study 120 with the context information of prior studies to return a set of matching prior studies.
  • Context matching can be achieved using one of variety of matching techniques.
  • the context matching module 114 may utilize a rule-based approach in which the context matching module 114 first matches the studies to determine whether the modalities and body parts correspond.
  • the context matching module 114 extracts and compares protocol information of the current study and prior studies. Protocol information may include information such as, for example, whether contrast was applied, whether the study was lateral bilateral, left or right and, for cases in which the studies include MR images, the acquisition type (i.e., 2D/3D) and/or T1/T2.
  • the system 100 may extract information such as patient position and orientation from DICOM to determine whether it matches the position/orientation of the patient in prior studies.
  • the context matching module 114 may utilize an ontology based approach in which a comprehensive ontology is used to determine whether context information of the current study matches context information of prior studies.
  • the ontology may include anatomy and modality related information.
  • the anatomy may include body regions such as “Head and Neck” or “Abdomen and Pelvis” under which specific organs belonging to each region may be listed.
  • kidney and liver would be included under the “Abdomen and Pelvis” region.
  • the context matching module 114 may determine that a prior study matches the current study if the organs match one another. For example, if the context of the current study is the kidney, any prior studies having context information including the kidney, left kidney and/or right kidney would be returned as relevant.
  • the context matching module 114 may utilize a data driven approach in which suitable contexts are determined by prior data. As described above in regard to the context extraction module 110 and the database 118 , whenever a user selects or changes a context in the navigation/visualization module 116 , these changes are stored and recorded in the database 118 . For example, for a patient having lung cancer with brain metastasis, the current context may be the brain. However, prior lung related studies would also be relevant. Thus, a radiologist may select these other related studies when analyzing an image.
  • the system 100 stores these selections in the database 118 such that once the system 100 has accumulated sufficient data, the context matching module 114 may automatically run data mining algorithms such as, for example, k-Nearest Neighbor and Mutual Information, to determine relevant studies. These algorithms may determine which contexts are most relevant given the current context.
  • data mining algorithms such as, for example, k-Nearest Neighbor and Mutual Information
  • the context matching module 114 may utilize a hybrid approach in which any of the approaches described above may be combined.
  • the context matching module 114 may combine the rule based approach with the ontology based approach such that a semantic relationship between concepts is considered in addition to the matching of context such as modality and body part.
  • four specific approaches are described above, it will be understood by those of skill in the art that the context matching module 114 may utilize any of a variety of matching techniques so long as the context matching module 114 is able to match the context information of the current study 120 with the context information of prior studies.
  • the navigation/visualization module 116 generates a summary view of the returned set of matching prior studies and displays the summary view on the display 106 , in a step 260 .
  • the summary view may, for example, show findings and the correlating observations for each of the related prior studies.
  • the patient had a mammogram in 2009, which resulted in a finding in the left breast.
  • an MRI was performed with and without contrast, resulting in two observations for this particular finding.
  • the current context is set to show modalities including mammograms and MRIs, as shown in FIG. 4 , and the body part set to “breast,” as shown in FIG. 5 .
  • the summary view provides a quick overview of the context-sensitive patient's findings, allowing the user to view the most recent and/or important related prior studies without having to read through multiple prior reports.
  • the summary view also allows the user to visualize a progression of findings. For example, the user may easily determine how many new findings have been identified in the past two years, which would provide an indication of how rapidly a particular cancer is spreading.
  • the user may input user preferences via the user interface 104 .
  • the user may, for example, filter the prior studies shown in the summary view or indicate any desired changes to the current context via the user interface 104 .
  • the user may expand or narrow the current context (e.g., narrow the current context to include only MRIs), indicate a number of prior studies to display in the summary view or indicate a desired period during which all displayed prior studies should fall.
  • Other filters may include, for example, a type of lesion, which may include mass, symmetry and enhancement.
  • the user may also navigate the prior studies to obtain more detailed information regarding that study. For example, the user may select an observation via a single mouse click to open the report corresponding to that particular study. Hovering the mouse over an observation may automatically show the corresponding key images for that study, as shown in FIG. 6 . Double clicking one of these key images may automatically open a key images window such that the user may view these key images in their original size and/or resolution.
  • the summary view may show images associated with each of the relevant prior studies.
  • the images may also be shown with the annotations thereof such that selecting an image or annotation would open report and/or observation correlating to the selected image.
  • the desired display e.g., observations or images
  • the user may also indicate other preferences such as, for example, whether observations related to each of the findings should be displayed longitudinally, as shown in FIG. 4 , or horizontally, as shown in FIG. 7 . It will be understood by those of skill in the art that a variety of other user preferences for displaying the summary view, although not specifically described, are also possible.
  • the system 100 may be utilized for quality assurance purposes by determining a minimum set of prior related findings which are required to be included in a report of the current study, as will be described in further detail below in regard to the method 300 .
  • the context extraction module 110 extracts context information from the current study to determine relevant prior findings via, for example, a prior finding determination module, which may include the capture module 112 for identifying and storing prior finding information to the database 118 where a structured reporting system has been utilized, the NLP module 122 for extracting finding information from text reports (e.g., from a non-structured reporting system) of prior studies and the context matching module 114 for comparing current context information to prior context information to identify the minimum set of findings required to be reported in the current study.
  • the processor 102 may also determine macros used by the relevant prior findings so that macros may be reused in the report for the current study.
  • a method 300 for identifying the minimum set of findings from the most recent, relevant prior study may be substantially similar to the method 200 , described above.
  • steps 310 - 340 are substantially similar to the steps 210 - 240 of the method 200 .
  • the system 100 retrieves images of a current study to be analyzed and extracts context information from the current study using the context extraction module 110 in step 320 .
  • the context of the current study may be extracted in a manner substantially similar to the context extraction described in regard to the method 200 .
  • This context information is stored in the database 118 , in step 330 .
  • a step 340 finding related information of prior studies is identified and stored in the database 118 , as described above in regard to the step 240 .
  • a full report of the prior study may be stored in the database 118 .
  • this information may be stored in a standard PACS format.
  • the NLP module 122 may extract a set of findings from an impressions section of a free-text radiology report of the prior studies. For example, if the prior report states, “There is a 16 ⁇ 11 mm diameter mass in the left cerebellopontine angle cisterns bulging into the ostium of the left internal auditory canal which is slightly hyperintense to brain on T2-weighted images and bright on the T1-weighted postcontrast images presumably due to homogeneous contrast enhancement,” the NLP module 122 will determine that there is a finding of a ‘mass in the left cerebellopontine’ with an attribute diameter set to 16 ⁇ 11 mm. The NLP module 122 may be used in conjunction with the prior reports retrieved in the step 340 .
  • the matching module 114 compares the current study context with contexts of the prior study, substantially as described above in regard to the step 250 of the method 200 , to return a minimum set of findings required to be included in the report of the current study.
  • Context matching may be performed using any of the approaches described above.
  • the matching module 114 may utilize any of a rule based approach, an ontology based approach, a data driven approach or a hybrid approach utilizing any combination of different matching processes.
  • the processor 102 returns the minimum set of findings that should be included in the report of the current study.
  • the navigation/visualization module 116 may display the minimum set of findings on the display via, for example, a graphical user interface, which also permits the user to filter and/or specify the findings to be returned in the minimum set.
  • the default may be set to include findings of the most recent, relevant prior study. For example, if the current study is a liver study, the minimum set of findings will be based on the most recent liver study. This default setting may be easily changed by the user using the displayed graphical user interface. For example, where findings have been assigned a clinical significance score, the user may desire to report only the prior findings that have a minimum clinical significance score of 3. In another embodiment, in addition to or instead of displaying the returned relevant prior findings, an alert may be displayed to warn the user that the current report has been saved without including all prior findings that were deemed relevant in the step 370 .
  • macros e.g., predefined text which may be inserted into the report via a predefined name of the macro
  • text fragments from the prior study may be utilized for the current study.
  • macros/sentences that are associated with the minimum set of findings are listed on the display 106 for the user to review and use, as desired. Reusing macros/text provides consistency between reports and preserves the longevity of the findings.
  • the processor 102 may automatically populate relevant fields (e.g., shape of the finding) of the finding in the current study based on the prior finding.
  • the macros used in the relevant prior studies are stored in the database 118 . Within a free text reporting environment, the text that contains the finding is used as the macro.
  • Macros may be particularly useful when using dictation systems in radiology reports. Radiologists use macros and templates (predefined document structures) to report observations quickly and efficiently. During dictation, the user speaks voice commands, which are matched to definitions of templates and macros so that the processor 102 may insert the predefined documents structure or text into the report. For example, when the user writes a report, the user may include five sections: procedure, clinical information, comparison, findings and impression. The Radiological Society of North America (RSNA) recommends information to be reported in each section according to modality, anatomy and clinical problems under investigation. For example, a template for a chest x-ray study may be as follows:
  • a method 400 generates a list of relevant templates/macros that the user may utilize for a current study using the system 100 .
  • the context extraction module 110 and the NLP module 122 are used to extract context information from a current study (e.g., to extract information such as modality and body part), an image order for the current study (e.g., to extract clinical information such as signs and symptoms) and prior reports (e.g., to extract patient's recommendations).
  • the extracted information may then be matched by the context matching module 114 to suggest relevant templates and/or macros.
  • the memory 108 may further include available templates and macros in the database 118 .
  • Each template has a name and defines a set of sections in which findings will be inserted.
  • Each macro has a name and defines a piece of text for insertion into the report.
  • the NLP module 122 extracts context information from the narrative content of each reporting template/macro. For example, anatomy and diagnosis terms may be extracted from the templates/macros so that the system 100 associates a list of attributes to each template/macro. These attributes are stored in the memory 108 so that the templates/macros may be later recalled.
  • the context information is extracted from a current study and/or relevant prior studies. The context information may be extracted substantially as described above in the step 220 of the method 200 and the step 320 of the method 300 .
  • the context extraction module 110 may extract DICOM attributes if the study including body parts and modality.
  • the NLP module 122 may further extract patient signs and symptoms from the reasons of exam section of the image order and/or patient's recommendation from relevant prior reports including: the date reported at each report, protocol/modality used in each report (e.g., MRI, Ultrasound), protocol/modality and date of the follow up exams (e.g., MRI 6 months) and findings of the image in each report (e.g., liver mass).
  • relevant prior reports may be identified, for example, by context matching between the current study and prior studies, as described above in regard to the methods 200 and 300 .
  • the context matching module 114 compares the context information of the templates/macros obtained in step 410 with the context information of the current and/or prior relevant studies obtained in step 420 to generate a set of relevant templates/macros.
  • the processor 102 may use any combination of rule-based, ontology and data driven (e.g., statistical) approaches, as described above in regard to the method 200 .
  • the processor 102 may match context information from the current/prior studies (e.g., words related to clinical signs, anatomy and protocol) with the context information from the template/macro definitions.
  • the system 100 may also use synonyms and acronyms or words in matching.
  • the processor 102 matches, for example, an abdomen CT study with templates/macros related to the stomach, colon, liver, spleen, pancreas, kidney, bladder, etc.
  • the processor 102 may match the follow-up with templates/macros related to follow-up using, for example, the NLP module 122 .
  • the processor 102 may also compare the date and/or protocol of the current study with the patient's recommendation information from prior reports. If there is a match, related templates/macros with the same findings may be ranked higher than others.
  • the navigation/display module 116 displays the relevant templates/macros to the user on the display 106 .
  • the relevant templates/macros are displayed.
  • templates/macros are listed in order of relevance with the most relevant templates/macros, for example, on the top of the list.
  • the list of available templates/macros may be displayed in, for example, alphabetical order, with the most relevant ones highlighted. It will be understood by those of skill in the art that the user may indicate his/her preferences for displaying the templates/macros via the user interface 104 .
  • the context extraction module 110 , the capture module 112 , the context matching module 114 , the NLP module 122 and the navigation/visualization module 116 may be programs containing lines of code that, when compiled, may be executed on a processor.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Public Health (AREA)
  • General Health & Medical Sciences (AREA)
  • Epidemiology (AREA)
  • Primary Health Care (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Pathology (AREA)
  • Biomedical Technology (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Quality & Reliability (AREA)
  • Measuring And Recording Apparatus For Diagnosis (AREA)
  • Medical Treatment And Welfare Office Work (AREA)
  • Apparatus For Radiation Diagnosis (AREA)
  • Image Processing (AREA)
  • Magnetic Resonance Imaging Apparatus (AREA)
  • Computing Systems (AREA)
  • Multimedia (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)

Abstract

A system, method and computer readable storage medium for generating a context driven summary view of medical findings by retrieving a current study including at least one image to be analyzed, extracting current context information from the current study and storing the current context information in a database and matching the current context information with prior context information from prior studies to return a set of relevant prior studies.

Description

    CROSS REFERENCE TO RELATED APPLICATIONS
  • This application is a Continuation of U.S. application Ser. No. 14/769,480 filed Aug. 21, 2015, which is the U.S. National Phase application under 35 U.S.C. § 371 of International Application No. PCT/IB2014/060096 filed Mar. 24, 2014, which claims the benefit of U.S. Provisional Patent Application No. 61/806,550. These applications are hereby incorporated by reference herein.
  • BACKGROUND
  • Radiologists are required to work with an increasing number of images to diagnose and treat patients in an optimal manner. Patients such as, for example, cancer patients, frequently undergo numerous imaging exams, accumulating many studies in their medical records pertaining to the same anatomical region. Each time a new study needs to be read, the radiologist opens the current order to understand why the study has been performed and what prior findings the patient has. The imaging order, however, contains limited information related to findings such that the radiologist would need to open the results of the most relevant prior study to better understand the status of the patient's findings. This prior report, however, may not contain sufficient information about all the relevant findings, providing an incomplete overview of the patient's finding history. Thus, the radiologist is often required to open multiple prior reports and/or images and build mental links between related findings to obtain a complete picture of the patient's history. This process is difficult and time consuming. In addition, there are currently no quality assurance tools to ensure consistency and longevity of findings such that some previously identified findings may be overlooked. If findings are not followed-up on, as required, there may be potentially adverse effects on the patient outcome.
  • SUMMARY
  • A method for generating a context driven summary view of medical findings by retrieving a current study including at least one image to be analyzed, extracting current context information from the current study and storing the current context information in a database and matching the current context information with prior context information from prior studies to return a set of relevant prior studies.
  • A system for generating a context driven summary view of medical findings having a processor retrieving a current study including at least one image to be analyzed, extracting current context information from the current study and matching the current context information with prior context information from prior studies to return a set of relevant prior studies; and a memory storing the current and prior context information in a database.
  • A non-transitory computer readable storage medium with an executable program stored thereon, wherein the program instructs a processor to perform steps to generate a context driven summary view of medical findings. The steps including retrieving a current study including at least one image to be analyzed, extracting current context information from the current study and store the current context information in a database and matching the current context information with prior context information from prior studies to return a set of relevant prior studies.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • FIG. 1 shows a schematic drawing of a system according to a first exemplary embodiment.
  • FIG. 2 shows another schematic drawing of the system of FIG. 1.
  • FIG. 3 shows a flow diagram of a method according to the first exemplary embodiment.
  • FIG. 4 shows an exemplary screenshot of a summary view displayed according to the method of FIG. 3.
  • FIG. 5 shows another exemplary screenshot of the summary view of FIG. 4.
  • FIG. 6 shows an exemplary screenshot of key images displayed according to the method of FIG. 3.
  • FIG. 7 shows an alternate exemplary screenshot of a summary view displayed according to the method of FIG. 3.
  • FIG. 8 shows a schematic drawing of a system and method according to a second exemplary embodiment.
  • FIG. 9 shows a flow diagram according to the method of FIG. 8.
  • FIG. 10 shows a schematic diagram of a system according to a third exemplary embodiment.
  • FIG. 11 shows a flow diagram of a method according to the third exemplary embodiment.
  • DETAILED DESCRIPTION
  • The exemplary embodiments may be further understood with reference to the following description and the appended drawings wherein like elements are referred to with the same reference numerals. The exemplary embodiments relate to a system and method for reviewing a medical image. In particular, the exemplary embodiments describe extracting the context of a current imaging study to aid in reporting the results of the current imaging study. The context of the current imaging study may be used to generate a context-driven summary of prior studies, suggest templates/macros related corresponding to the current study and/or identify all related findings which should be reported on in a final report. Although the exemplary embodiments are specifically described in regard to reading images of cancer patients within a radiology department, it will be understood by those of skill in the art that the system and method of the present disclosure may be used for patients having any of a variety of diseases or conditions within any of a variety of hospital departments.
  • As shown in FIGS. 1 and 2, a system 100 according to an exemplary embodiment of the present disclosure generates a context driven summary view of prior related studies. The system 100 comprises a processor 102, a user interface 104, a display 106 and a memory 108. The processor 102 generates the summary view for a current study 120 including at least one image to be analyzed by extracting context information of the current study 120 using a context extraction module 110. The context extraction module 110 may also extract context information regarding the patient's clinical indications from an order of the current study 120. The context information may include data such as, for example, a modality (e.g., MRI, mammogram) and/or a body part (e.g., breast, head) imaged in the images of the current study 120. When the radiologist is rendering a diagnosis (i.e., the current study has not yet been reported), the context extraction module 110 may also extract context information including the patient's clinical indications from the image exam order of the current study. This context information is stored in a database 118 of the memory 108, which also stores finding (e.g., tumor) related information for current and/or prior studies of the patient. The database 118 may also store for example, annotations (i.e., mark-ups on an image) relating to the finding. The processor 102 further includes a capture module 112, which captures and stores information to the database 118 whenever a new annotation is created on the image by a user, and a Natural Language Processing (NLP) module 122 which extracts text data from the current study and/or prior reports.
  • The processor 102 also includes a context matching module 114 which matches the context extracted from the images of the current study 120 with all prior studies to determine which prior studies are related to the current study 120. In one example, current and/or prior study information may be displayed on the display 106 in a summary view. The summary view may, for example, show observations related to each of the findings and/or annotations on the images related to each of the findings. Once the summary view has been displayed, a navigation/visualization module 116 of the processor 102 permits a user to select a displayed observation and/or annotation to display further details of the selected finding. In another example, relevant prior studies may also be filtered to display a minimum set of findings that should be included in a report of the current study to maintain consistency and longevity of reports. In another exemplary embodiment, the context matching module 114 may also compare the context information of the current/prior studies with context information of templates and/or macros available for use by the user to return a set of relevant templates/macros. Thus, the user may easily select the templates/macros to cut down on reporting time. User selections may be made via the user interface 104, which may include input devices such as, for example, a keyboard, mouse, and/or touch display on the display 106.
  • FIG. 3 shows a method 200 by which the summary view is generated. In a step 210, the system 100 retrieves the images to be analyzed in the current study 120. The images may be stored and viewed in, for example, a Picture Archiving and Communications System (PACS) within a Radiology Information System (RIS). In a step 220, the context extraction module 110 extracts content information from the images to be analyzed in the current study. As described above, the context information may include, for example, modality and/or body part. Images stored on the RIS/PACS system, for example, are stored and viewed in a DICOM (Digital Imaging and Communications in Medicine) file format, which includes a header containing information related to the modality and body part. When the radiologist or other user is rendering a diagnosis for the current study in which the image to be analyzed has not yet been reported, the context information may also include the patient's clinical indications from the imaging order of the current study. The extracted context information is then stored in the database 118, in a step 230. The database 118 also includes context information for images of prior studies, which may have been previously extracted and stored.
  • In a step 240, images of prior studies are analyzed by the capture module 114 to identify and store finding related information in the database 118. In the case of cancer patients, findings may include a tumor or mass identified in the images. Observations may include, for example, a description of the mass and/or a result of a biopsy of the tumor. Annotations for each of the observations and findings may be created on the image. Each time a new finding is created, the finding related information, including observations and annotations, are stored in the database 118. Each finding may have multiple observations associated with it. For example, the finding may be a first mass on a patient's left breast. The radiologist may input one or more observations regarding the first mass such as, for example, a designation of the mass as either malignant or benign and/or a description of the mass. Each observation may have one or more annotations on the image associated with it. The radiologist may also indicate that the annotations are for a new finding (e.g., a second mass) so that a new finding and observation are created. Alternatively, the radiologist may indicate that the annotation is associated with a new observation for an existing finding. Prior finding related information may also be stored by processing prior reports using a natural language processing (NLP) module 122 such as, for example, MEDLEE, so that findings from prior reports/studies may be quickly extracted. NLP modules alone provide only textual information of findings and will not show image annotations. The NLP module 122 may thus be combined with the above-described capture module 112 to extract both text and image data from prior studies. Finding related information of the current study is similarly generated and stored in the database 118. It will be understood by those of skill in the art that the finding related information of each study may be created and stored in the database 118 as each study is conducted and reviewed such that the finding related information may be stored in the database and later recalled when necessary.
  • In a step 250, the context matching module 114 compares the context information of the current study 120 with the context information of prior studies to return a set of matching prior studies. Context matching can be achieved using one of variety of matching techniques. In a first example, the context matching module 114 may utilize a rule-based approach in which the context matching module 114 first matches the studies to determine whether the modalities and body parts correspond. Second, the context matching module 114 extracts and compares protocol information of the current study and prior studies. Protocol information may include information such as, for example, whether contrast was applied, whether the study was lateral bilateral, left or right and, for cases in which the studies include MR images, the acquisition type (i.e., 2D/3D) and/or T1/T2. Third, the system 100 may extract information such as patient position and orientation from DICOM to determine whether it matches the position/orientation of the patient in prior studies.
  • In a second example, the context matching module 114 may utilize an ontology based approach in which a comprehensive ontology is used to determine whether context information of the current study matches context information of prior studies. The ontology may include anatomy and modality related information. For example, the anatomy may include body regions such as “Head and Neck” or “Abdomen and Pelvis” under which specific organs belonging to each region may be listed. For example, kidney and liver would be included under the “Abdomen and Pelvis” region. Using the ontology approach, the context matching module 114 may determine that a prior study matches the current study if the organs match one another. For example, if the context of the current study is the kidney, any prior studies having context information including the kidney, left kidney and/or right kidney would be returned as relevant.
  • In a third example, the context matching module 114 may utilize a data driven approach in which suitable contexts are determined by prior data. As described above in regard to the context extraction module 110 and the database 118, whenever a user selects or changes a context in the navigation/visualization module 116, these changes are stored and recorded in the database 118. For example, for a patient having lung cancer with brain metastasis, the current context may be the brain. However, prior lung related studies would also be relevant. Thus, a radiologist may select these other related studies when analyzing an image. The system 100 stores these selections in the database 118 such that once the system 100 has accumulated sufficient data, the context matching module 114 may automatically run data mining algorithms such as, for example, k-Nearest Neighbor and Mutual Information, to determine relevant studies. These algorithms may determine which contexts are most relevant given the current context.
  • In a fourth example, the context matching module 114 may utilize a hybrid approach in which any of the approaches described above may be combined. For example, the context matching module 114 may combine the rule based approach with the ontology based approach such that a semantic relationship between concepts is considered in addition to the matching of context such as modality and body part. Although four specific approaches are described above, it will be understood by those of skill in the art that the context matching module 114 may utilize any of a variety of matching techniques so long as the context matching module 114 is able to match the context information of the current study 120 with the context information of prior studies.
  • Once the context matching module 114 returns a set of matching prior studies, the navigation/visualization module 116 generates a summary view of the returned set of matching prior studies and displays the summary view on the display 106, in a step 260. As shown in FIG. 4, the summary view may, for example, show findings and the correlating observations for each of the related prior studies. In the exemplary case shown in FIG. 4, the patient had a mammogram in 2009, which resulted in a finding in the left breast. In 2010, an MRI was performed with and without contrast, resulting in two observations for this particular finding. Two additional findings, as shown in the second and third columns, were also discovered at this time. The current context is set to show modalities including mammograms and MRIs, as shown in FIG. 4, and the body part set to “breast,” as shown in FIG. 5. The summary view provides a quick overview of the context-sensitive patient's findings, allowing the user to view the most recent and/or important related prior studies without having to read through multiple prior reports. The summary view also allows the user to visualize a progression of findings. For example, the user may easily determine how many new findings have been identified in the past two years, which would provide an indication of how rapidly a particular cancer is spreading.
  • In a step 270, the user may input user preferences via the user interface 104. The user may, for example, filter the prior studies shown in the summary view or indicate any desired changes to the current context via the user interface 104. For example, the user may expand or narrow the current context (e.g., narrow the current context to include only MRIs), indicate a number of prior studies to display in the summary view or indicate a desired period during which all displayed prior studies should fall. Other filters may include, for example, a type of lesion, which may include mass, symmetry and enhancement. The user may also navigate the prior studies to obtain more detailed information regarding that study. For example, the user may select an observation via a single mouse click to open the report corresponding to that particular study. Hovering the mouse over an observation may automatically show the corresponding key images for that study, as shown in FIG. 6. Double clicking one of these key images may automatically open a key images window such that the user may view these key images in their original size and/or resolution.
  • In an alternate embodiment, as shown in FIG. 7, rather than displaying the observations of relevant prior studies, the summary view may show images associated with each of the relevant prior studies. The images may also be shown with the annotations thereof such that selecting an image or annotation would open report and/or observation correlating to the selected image. It will be understood by those of skill in the art that the desired display (e.g., observations or images) may be indicated by the user as a user preference. The user may also indicate other preferences such as, for example, whether observations related to each of the findings should be displayed longitudinally, as shown in FIG. 4, or horizontally, as shown in FIG. 7. It will be understood by those of skill in the art that a variety of other user preferences for displaying the summary view, although not specifically described, are also possible.
  • According to another exemplary embodiment shown in FIG. 8, the system 100 may be utilized for quality assurance purposes by determining a minimum set of prior related findings which are required to be included in a report of the current study, as will be described in further detail below in regard to the method 300. Upon opening a report creating environment for the current study, the context extraction module 110 extracts context information from the current study to determine relevant prior findings via, for example, a prior finding determination module, which may include the capture module 112 for identifying and storing prior finding information to the database 118 where a structured reporting system has been utilized, the NLP module 122 for extracting finding information from text reports (e.g., from a non-structured reporting system) of prior studies and the context matching module 114 for comparing current context information to prior context information to identify the minimum set of findings required to be reported in the current study. Once the minimum set of findings has been identified, the processor 102 may also determine macros used by the relevant prior findings so that macros may be reused in the report for the current study.
  • As shown in FIG. 9, a method 300 for identifying the minimum set of findings from the most recent, relevant prior study may be substantially similar to the method 200, described above. In particular, steps 310-340 are substantially similar to the steps 210-240 of the method 200. In step 310, the system 100 retrieves images of a current study to be analyzed and extracts context information from the current study using the context extraction module 110 in step 320. The context of the current study may be extracted in a manner substantially similar to the context extraction described in regard to the method 200. This context information is stored in the database 118, in step 330. In a step 340, finding related information of prior studies is identified and stored in the database 118, as described above in regard to the step 240. In the absence of a structured reporting environment, a full report of the prior study may be stored in the database 118. Alternatively, this information may be stored in a standard PACS format.
  • In a step 350, the NLP module 122 may extract a set of findings from an impressions section of a free-text radiology report of the prior studies. For example, if the prior report states, “There is a 16×11 mm diameter mass in the left cerebellopontine angle cisterns bulging into the ostium of the left internal auditory canal which is slightly hyperintense to brain on T2-weighted images and bright on the T1-weighted postcontrast images presumably due to homogeneous contrast enhancement,” the NLP module 122 will determine that there is a finding of a ‘mass in the left cerebellopontine’ with an attribute diameter set to 16×11 mm. The NLP module 122 may be used in conjunction with the prior reports retrieved in the step 340.
  • In a step 360, the matching module 114 compares the current study context with contexts of the prior study, substantially as described above in regard to the step 250 of the method 200, to return a minimum set of findings required to be included in the report of the current study. Context matching may be performed using any of the approaches described above. For example, the matching module 114 may utilize any of a rule based approach, an ontology based approach, a data driven approach or a hybrid approach utilizing any combination of different matching processes. In a step 370, the processor 102 returns the minimum set of findings that should be included in the report of the current study. The navigation/visualization module 116 may display the minimum set of findings on the display via, for example, a graphical user interface, which also permits the user to filter and/or specify the findings to be returned in the minimum set. The default may be set to include findings of the most recent, relevant prior study. For example, if the current study is a liver study, the minimum set of findings will be based on the most recent liver study. This default setting may be easily changed by the user using the displayed graphical user interface. For example, where findings have been assigned a clinical significance score, the user may desire to report only the prior findings that have a minimum clinical significance score of 3. In another embodiment, in addition to or instead of displaying the returned relevant prior findings, an alert may be displayed to warn the user that the current report has been saved without including all prior findings that were deemed relevant in the step 370.
  • Once prior relevant studies have been identified, macros (e.g., predefined text which may be inserted into the report via a predefined name of the macro) or text fragments from the prior study may be utilized for the current study. In a step 380, macros/sentences that are associated with the minimum set of findings are listed on the display 106 for the user to review and use, as desired. Reusing macros/text provides consistency between reports and preserves the longevity of the findings. When a prior macro is reused, the processor 102 may automatically populate relevant fields (e.g., shape of the finding) of the finding in the current study based on the prior finding. The macros used in the relevant prior studies are stored in the database 118. Within a free text reporting environment, the text that contains the finding is used as the macro.
  • Macros may be particularly useful when using dictation systems in radiology reports. Radiologists use macros and templates (predefined document structures) to report observations quickly and efficiently. During dictation, the user speaks voice commands, which are matched to definitions of templates and macros so that the processor 102 may insert the predefined documents structure or text into the report. For example, when the user writes a report, the user may include five sections: procedure, clinical information, comparison, findings and impression. The Radiological Society of North America (RSNA) recommends information to be reported in each section according to modality, anatomy and clinical problems under investigation. For example, a template for a chest x-ray study may be as follows:
      • Procedure
      • Clinical Information
        • Cough
        • Fever
        • Shortness of Breath
        • Pre-operative Exam
      • Comparison
      • Findings
        • Heart
        • Lungs
        • Bones
      • Impression
        • Normal
        • No acute disease
          When using macros, the user defines the content and name of the macro. The content is a piece of text and the name is the voice command to insert the content to the report. For example, a macro describing gall bladder wall calcification could be: “Gall bladder wall calcification is present and consistent with a porcelain gall bladder, which is associated with an increased risk for development of gall bladder carcinoma.” The name associated with this macro may be “porcelain gall bladder,” such that when the user speaks “porcelain gall bladder,” the processor 102 automatically inserts the above-defined text into the report.
  • According to another exemplary embodiment, as shown in FIGS. 10 and 11, a method 400 generates a list of relevant templates/macros that the user may utilize for a current study using the system 100. As shown in FIG. 10, the context extraction module 110 and the NLP module 122 are used to extract context information from a current study (e.g., to extract information such as modality and body part), an image order for the current study (e.g., to extract clinical information such as signs and symptoms) and prior reports (e.g., to extract patient's recommendations). The extracted information may then be matched by the context matching module 114 to suggest relevant templates and/or macros. The memory 108 may further include available templates and macros in the database 118. Each template has a name and defines a set of sections in which findings will be inserted. Each macro has a name and defines a piece of text for insertion into the report.
  • In a step 410, as shown in FIG. 11, the NLP module 122 extracts context information from the narrative content of each reporting template/macro. For example, anatomy and diagnosis terms may be extracted from the templates/macros so that the system 100 associates a list of attributes to each template/macro. These attributes are stored in the memory 108 so that the templates/macros may be later recalled. In a step 420, the context information is extracted from a current study and/or relevant prior studies. The context information may be extracted substantially as described above in the step 220 of the method 200 and the step 320 of the method 300. For example, the context extraction module 110 may extract DICOM attributes if the study including body parts and modality. The NLP module 122 may further extract patient signs and symptoms from the reasons of exam section of the image order and/or patient's recommendation from relevant prior reports including: the date reported at each report, protocol/modality used in each report (e.g., MRI, Ultrasound), protocol/modality and date of the follow up exams (e.g., MRI 6 months) and findings of the image in each report (e.g., liver mass). The relevant prior reports may be identified, for example, by context matching between the current study and prior studies, as described above in regard to the methods 200 and 300.
  • In a step 430, the context matching module 114 compares the context information of the templates/macros obtained in step 410 with the context information of the current and/or prior relevant studies obtained in step 420 to generate a set of relevant templates/macros. The processor 102 may use any combination of rule-based, ontology and data driven (e.g., statistical) approaches, as described above in regard to the method 200. For example, the processor 102 may match context information from the current/prior studies (e.g., words related to clinical signs, anatomy and protocol) with the context information from the template/macro definitions. The system 100 may also use synonyms and acronyms or words in matching. For example, “MRI” is equivalent to both “MR” and “Magnetic Resonance Imaging.” Using ontology based reasoning, the processor 102 matches, for example, an abdomen CT study with templates/macros related to the stomach, colon, liver, spleen, pancreas, kidney, bladder, etc. In another example, if the exam order includes a “follow-up,” the processor 102 may match the follow-up with templates/macros related to follow-up using, for example, the NLP module 122. The processor 102 may also compare the date and/or protocol of the current study with the patient's recommendation information from prior reports. If there is a match, related templates/macros with the same findings may be ranked higher than others.
  • In a step 440, the navigation/display module 116 displays the relevant templates/macros to the user on the display 106. In one exemplary embodiment, only the relevant templates/macros are displayed. In another exemplary embodiment, templates/macros are listed in order of relevance with the most relevant templates/macros, for example, on the top of the list. In yet another exemplary embodiment, the list of available templates/macros may be displayed in, for example, alphabetical order, with the most relevant ones highlighted. It will be understood by those of skill in the art that the user may indicate his/her preferences for displaying the templates/macros via the user interface 104.
  • It is noted that the claims may include reference signs/numerals in accordance with PCT Rule 6.2(b). However, the present claims should not be considered to be limited to the exemplary embodiments corresponding to the reference signs/numerals.
  • Those skilled in the art will understand that the above-described exemplary embodiments may be implemented in any number of manners, including, as a separate software module, as a combination of hardware and software, etc. For example, the context extraction module 110, the capture module 112, the context matching module 114, the NLP module 122 and the navigation/visualization module 116 may be programs containing lines of code that, when compiled, may be executed on a processor.
  • It will be apparent to those skilled in the art that various modifications may be made to the disclosed exemplary embodiments and methods and alternatives without departing from the spirit or scope of the disclosure. Thus, it is intended that the present disclosure cover the modifications and variations provided that they come within the scope of the appended claims and their equivalents.

Claims (20)

What is claimed is:
1. A method, comprising:
retrieving a current medical study of a patient including at least one image to be analyzed;
extracting current context information from the current medical study;
matching the current context information with prior context information from prior studies to identify a minimum set of findings to be reported in the current medical study, the minimum set of findings including most recent prior studies of the patient, wherein the matching the current context information with the prior context information includes an ontological approach;
displaying a summary view of a set of relevant prior studies comprising finding related information for the relevant prior studies.
2. The method of claim 1, wherein the summary view further comprises images related to the set of relevant prior studies.
3. The method of claim 2, wherein the summary view further comprises annotations on the images related to the set of relevant prior studies.
4. The method of claim 1, further comprising:
filtering the summary view based on a user preference.
5. The method of claim 4, wherein filtering the summary view comprises one of modifying the current context information, identifying a type of lesion, identifying a time period for relevant prior studies, or indicating a number of relevant prior studies to be shown.
6. The method of claim 1, wherein the current context information comprises one of a modality or a body part in the at least one image to be analyzed.
7. The method of claim 1, wherein the current context information comprises clinical indications from an image exam order of the current medical study.
8. A system, comprising:
a memory storing a current medical study of a patient including at least one image to be analyzed and one or more prior medical studies of the patient;
a processor configured to:
retrieve the current medical study including the at least one image to be analyzed;
extract current context information from the current medical study;
match the current context information with prior context information from the prior medical studies to identify a minimum set of findings to be reported in the current medical study, the minimum set of findings including most recent prior studies of the patient, wherein matching the current context information with the prior context information includes an ontological approach; and
display a summary view of a set of relevant prior studies comprising finding related information for the relevant prior studies.
9. The system of claim 8, wherein the summary view further comprises images related to the set of relevant prior studies.
10. The system of claim 9, wherein the summary view further comprises annotations on the images related to the set of relevant prior studies.
11. The system of claim 1, wherein the processor is further configured to:
filter the summary view based on a user preference.
12. The system of claim 11, wherein filtering the summary view comprises one of modifying the current context information, identifying a type of lesion, identifying a time period for relevant prior studies, or indicating a number of relevant prior studies to be shown.
13. The system of claim 8, wherein the current context information comprises one of a modality or a body part in the at least one image to be analyzed.
14. The system of claim 8, wherein the current context information comprises clinical indications from an image exam order of the current medical study.
15. A non-transitory computer readable storage medium comprising a set of instructions that when executed by a processor causes the processor to perform operations, comprising:
retrieving a current medical study of a patient including at least one image to be analyzed;
extracting current context information from the current medical study;
matching the current context information with prior context information from prior studies to identify a minimum set of findings to be reported in the current medical study, the minimum set of findings including most recent prior studies of the patient, wherein the matching the current context information with the prior context information includes an ontological approach;
displaying a summary view of a set of relevant prior studies comprising finding related information for the relevant prior studies.
16. The non-transitory computer readable storage medium of claim 15, wherein the summary view further comprises images related to the set of relevant prior studies.
17. The non-transitory computer readable storage medium of claim 16, wherein the summary view further comprises annotations on the images related to the set of relevant prior studies.
18. The non-transitory computer readable storage medium of claim 15, further comprising:
filtering the summary view based on a user preference, wherein filtering the summary view comprises one of modifying the current context information, identifying a type of lesion, identifying a time period for relevant prior studies, or indicating a number of relevant prior studies to be shown.
19. The non-transitory computer readable storage medium of claim 15, wherein the current context information comprises one of a modality or a body part in the at least one image to be analyzed.
20. The non-transitory computer readable storage medium of claim 15, wherein the current context information comprises clinical indications from an image exam order of the current medical study.
US17/692,566 2013-03-29 2022-03-11 Context driven summary view of radiology findings Abandoned US20220199230A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/692,566 US20220199230A1 (en) 2013-03-29 2022-03-11 Context driven summary view of radiology findings

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US201361806550P 2013-03-29 2013-03-29
PCT/IB2014/060096 WO2014155273A1 (en) 2013-03-29 2014-03-24 A context driven summary view of radiology findings
US201514769480A 2015-08-21 2015-08-21
US17/692,566 US20220199230A1 (en) 2013-03-29 2022-03-11 Context driven summary view of radiology findings

Related Parent Applications (2)

Application Number Title Priority Date Filing Date
US14/769,480 Continuation US11289188B2 (en) 2013-03-29 2014-03-24 Context driven summary view of radiology findings
PCT/IB2014/060096 Continuation WO2014155273A1 (en) 2013-03-29 2014-03-24 A context driven summary view of radiology findings

Publications (1)

Publication Number Publication Date
US20220199230A1 true US20220199230A1 (en) 2022-06-23

Family

ID=50486925

Family Applications (2)

Application Number Title Priority Date Filing Date
US14/769,480 Active 2034-11-08 US11289188B2 (en) 2013-03-29 2014-03-24 Context driven summary view of radiology findings
US17/692,566 Abandoned US20220199230A1 (en) 2013-03-29 2022-03-11 Context driven summary view of radiology findings

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US14/769,480 Active 2034-11-08 US11289188B2 (en) 2013-03-29 2014-03-24 Context driven summary view of radiology findings

Country Status (5)

Country Link
US (2) US11289188B2 (en)
EP (1) EP2979210A1 (en)
JP (1) JP6461909B2 (en)
CN (1) CN105074708B (en)
WO (1) WO2014155273A1 (en)

Families Citing this family (60)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11871901B2 (en) 2012-05-20 2024-01-16 Cilag Gmbh International Method for situational awareness for surgical network or surgical network connected device capable of adjusting function based on a sensed situation or usage
US10340041B2 (en) * 2014-05-09 2019-07-02 Acupath Laboratories, Inc. Biopsy mapping tools
ES2734390T3 (en) * 2015-02-25 2019-12-05 Koninklijke Philips Nv Detection of missing findings for automatic creation of a view of longitudinal findings
US11127494B2 (en) 2015-08-26 2021-09-21 International Business Machines Corporation Context-specific vocabulary selection for image reporting
EP3356971B1 (en) 2015-10-02 2024-03-13 Koninklijke Philips N.V. System for mapping findings to pertinent echocardiogram loops
EP3371727A1 (en) * 2015-11-05 2018-09-12 Koninklijke Philips N.V. Longitudinal health patient profile for incidental findings
US20170337329A1 (en) * 2016-05-18 2017-11-23 Siemens Healthcare Gmbh Automatic generation of radiology reports from images and automatic rule out of images without findings
EP3475857A1 (en) * 2016-06-23 2019-05-01 Koninklijke Philips N.V. Facilitated structured measurement management tool progress and compliance analytics solution
EP3475860B1 (en) * 2016-06-28 2024-08-07 Koninklijke Philips N.V. System and architecture for seamless workflow integration and orchestration of clinical intelligence
WO2018077776A1 (en) * 2016-10-25 2018-05-03 Koninklijke Philips N.V. Device, system, and method for optimizing usage of prior studies
WO2019020717A1 (en) * 2017-07-25 2019-01-31 Koninklijke Philips N.V. Device, system, and method for optimizing image acquisition workflows
US11244746B2 (en) * 2017-08-04 2022-02-08 International Business Machines Corporation Automatically associating user input with sections of an electronic report using machine learning
US11510741B2 (en) 2017-10-30 2022-11-29 Cilag Gmbh International Method for producing a surgical instrument comprising a smart electrical system
US11291510B2 (en) 2017-10-30 2022-04-05 Cilag Gmbh International Method of hub communication with surgical instrument systems
US11925373B2 (en) 2017-10-30 2024-03-12 Cilag Gmbh International Surgical suturing instrument comprising a non-circular needle
US11801098B2 (en) 2017-10-30 2023-10-31 Cilag Gmbh International Method of hub communication with surgical instrument systems
US12396806B2 (en) 2017-12-28 2025-08-26 Cilag Gmbh International Adjustment of a surgical device function based on situational awareness
US11864728B2 (en) 2017-12-28 2024-01-09 Cilag Gmbh International Characterization of tissue irregularities through the use of mono-chromatic light refractivity
US11179175B2 (en) 2017-12-28 2021-11-23 Cilag Gmbh International Controlling an ultrasonic surgical instrument according to tissue location
US11696760B2 (en) 2017-12-28 2023-07-11 Cilag Gmbh International Safety systems for smart powered surgical stapling
US11013563B2 (en) 2017-12-28 2021-05-25 Ethicon Llc Drive arrangements for robot-assisted surgical platforms
US11202570B2 (en) 2017-12-28 2021-12-21 Cilag Gmbh International Communication hub and storage device for storing parameters and status of a surgical device to be shared with cloud based analytics systems
US11076921B2 (en) 2017-12-28 2021-08-03 Cilag Gmbh International Adaptive control program updates for surgical hubs
US11304763B2 (en) 2017-12-28 2022-04-19 Cilag Gmbh International Image capturing of the areas outside the abdomen to improve placement and control of a surgical device in use
US12127729B2 (en) 2017-12-28 2024-10-29 Cilag Gmbh International Method for smoke evacuation for surgical hub
US11304699B2 (en) 2017-12-28 2022-04-19 Cilag Gmbh International Method for adaptive control schemes for surgical network control and interaction
US11464559B2 (en) 2017-12-28 2022-10-11 Cilag Gmbh International Estimating state of ultrasonic end effector and control system therefor
US20190206569A1 (en) 2017-12-28 2019-07-04 Ethicon Llc Method of cloud based data analytics for use with the hub
US11832899B2 (en) 2017-12-28 2023-12-05 Cilag Gmbh International Surgical systems with autonomously adjustable control programs
US11026751B2 (en) 2017-12-28 2021-06-08 Cilag Gmbh International Display of alignment of staple cartridge to prior linear staple line
US11389164B2 (en) 2017-12-28 2022-07-19 Cilag Gmbh International Method of using reinforced flexible circuits with multiple sensors to optimize performance of radio frequency devices
US11896322B2 (en) 2017-12-28 2024-02-13 Cilag Gmbh International Sensing the patient position and contact utilizing the mono-polar return pad electrode to provide situational awareness to the hub
US20190201112A1 (en) 2017-12-28 2019-07-04 Ethicon Llc Computer implemented interactive surgical systems
US12062442B2 (en) 2017-12-28 2024-08-13 Cilag Gmbh International Method for operating surgical instrument systems
US11324557B2 (en) 2017-12-28 2022-05-10 Cilag Gmbh International Surgical instrument with a sensing array
US12096916B2 (en) 2017-12-28 2024-09-24 Cilag Gmbh International Method of sensing particulate from smoke evacuated from a patient, adjusting the pump speed based on the sensed information, and communicating the functional parameters of the system to the hub
US11633237B2 (en) 2017-12-28 2023-04-25 Cilag Gmbh International Usage and technique analysis of surgeon / staff performance against a baseline to optimize device utilization and performance for both current and future procedures
WO2019133144A1 (en) 2017-12-28 2019-07-04 Ethicon Llc Detection and escalation of security responses of surgical instruments to increasing severity threats
US11998193B2 (en) 2017-12-28 2024-06-04 Cilag Gmbh International Method for usage of the shroud as an aspect of sensing or controlling a powered surgical device, and a control algorithm to adjust its default operation
US11969216B2 (en) 2017-12-28 2024-04-30 Cilag Gmbh International Surgical network recommendations from real time analysis of procedure variables against a baseline highlighting differences from the optimal solution
US11896443B2 (en) 2017-12-28 2024-02-13 Cilag Gmbh International Control of a surgical system through a surgical barrier
US11612444B2 (en) 2017-12-28 2023-03-28 Cilag Gmbh International Adjustment of a surgical device function based on situational awareness
US11257589B2 (en) 2017-12-28 2022-02-22 Cilag Gmbh International Real-time analysis of comprehensive cost of all instrumentation used in surgery utilizing data fluidity to track instruments through stocking and in-house processes
US11109866B2 (en) 2017-12-28 2021-09-07 Cilag Gmbh International Method for circular stapler control algorithm adjustment based on situational awareness
US11969142B2 (en) 2017-12-28 2024-04-30 Cilag Gmbh International Method of compressing tissue within a stapling device and simultaneously displaying the location of the tissue within the jaws
US11311306B2 (en) 2017-12-28 2022-04-26 Cilag Gmbh International Surgical systems for detecting end effector tissue distribution irregularities
US20190201090A1 (en) 2017-12-28 2019-07-04 Ethicon Llc Capacitive coupled return path pad with separable array elements
US11857152B2 (en) 2017-12-28 2024-01-02 Cilag Gmbh International Surgical hub spatial awareness to determine devices in operating theater
US20190201130A1 (en) * 2017-12-28 2019-07-04 Ethicon Llc Communication of data where a surgical network is using context of the data and requirements of a receiving system / user to influence inclusion or linkage of data and metadata to establish continuity
US11259830B2 (en) 2018-03-08 2022-03-01 Cilag Gmbh International Methods for controlling temperature in ultrasonic device
US11534196B2 (en) 2018-03-08 2022-12-27 Cilag Gmbh International Using spectroscopy to determine device use state in combo instrument
US11986233B2 (en) 2018-03-08 2024-05-21 Cilag Gmbh International Adjustment of complex impedance to compensate for lost power in an articulating ultrasonic device
US11090047B2 (en) 2018-03-28 2021-08-17 Cilag Gmbh International Surgical instrument comprising an adaptive control system
EP3779997A4 (en) * 2018-04-04 2021-05-19 FUJIFILM Corporation Medical document display control device, medical document display control method, and medical document display control program
US11756691B2 (en) * 2018-08-01 2023-09-12 Martin Reimann Brain health comparison system
CN112655050B (en) * 2018-08-28 2025-01-14 皇家飞利浦有限公司 Advanced loop selection system and method for supporting efficient echo comparison
US12347538B2 (en) * 2019-02-06 2025-07-01 OptimDosing, LLC Smart multidosing
US11331100B2 (en) 2019-02-19 2022-05-17 Cilag Gmbh International Staple cartridge retainer system with authentication keys
CN112420167A (en) * 2019-08-20 2021-02-26 阿里巴巴集团控股有限公司 Image report generation method, device and equipment
JP7426908B2 (en) * 2020-06-22 2024-02-02 キヤノン株式会社 Medical information processing device, medical information processing system, medical information processing method, and program

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080104116A1 (en) * 2004-12-02 2008-05-01 Lieven Van Hoe Image Evaluation System, Methods and Database
WO2014169164A1 (en) * 2013-04-13 2014-10-16 The Trustees Of The University Of Pennsylvania System and method for medical image analysis and probabilistic diagnosis
US11127494B2 (en) * 2015-08-26 2021-09-21 International Business Machines Corporation Context-specific vocabulary selection for image reporting

Family Cites Families (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003108664A (en) * 2001-09-27 2003-04-11 Yokogawa Electric Corp Finding creation system
JP2004305551A (en) * 2003-04-09 2004-11-04 Konica Minolta Medical & Graphic Inc Medical image diagnostic reading system
JP4719408B2 (en) 2003-07-09 2011-07-06 富士通株式会社 Medical information system
JP2005160502A (en) 2003-11-28 2005-06-23 Hitachi Medical Corp Image diagnosis support apparatus
JP2007072649A (en) * 2005-09-06 2007-03-22 Fujifilm Corp Interpretation report creation device
US20070133736A1 (en) * 2005-10-17 2007-06-14 Siemens Corporate Research Inc Devices, systems, and methods for imaging
JP5128154B2 (en) * 2006-04-10 2013-01-23 富士フイルム株式会社 Report creation support apparatus, report creation support method, and program thereof
JP2008021267A (en) * 2006-07-14 2008-01-31 Fuji Xerox Co Ltd Document retrieval system, document retrieval processing method and document retrieval processing program
JP5098253B2 (en) * 2006-08-25 2012-12-12 コニカミノルタエムジー株式会社 Database system, program, and report search method
JP5172262B2 (en) * 2007-09-27 2013-03-27 富士フイルム株式会社 Report creation support system and report creation support method
US8046226B2 (en) * 2008-01-18 2011-10-25 Cyberpulse, L.L.C. System and methods for reporting
US8485042B2 (en) * 2008-08-05 2013-07-16 Endress + Hauser Gmbh + Co. Kg Method for manufacturing an elastic body of Al2O3 ceramic, measuring membrane for a pressure sensor and pressure sensor with such a membrane
JP5173700B2 (en) 2008-09-25 2013-04-03 キヤノン株式会社 Data search apparatus, control method therefor, and data search system
JP2011018111A (en) * 2009-07-07 2011-01-27 Konica Minolta Medical & Graphic Inc Information processing system and program
JP5744182B2 (en) 2010-04-19 2015-07-01 コーニンクレッカ フィリップス エヌ ヴェ Report viewer using radiation descriptors
JP2012053632A (en) 2010-08-31 2012-03-15 Fujifilm Corp Device, method and program for supporting preparation of medical report
JP5462414B2 (en) * 2011-06-30 2014-04-02 パナソニック株式会社 Similar case retrieval device, relevance degree database creation device, similar case retrieval method, and relevance degree database creation method
JP5475923B2 (en) 2011-08-04 2014-04-16 パナソニック株式会社 Similar case retrieval apparatus and similar case retrieval method
JP6143494B2 (en) * 2012-03-02 2017-06-07 東芝メディカルシステムズ株式会社 Interpretation report creation support system, interpretation report creation support device, and interpretation request device
CN103324823A (en) * 2012-03-05 2013-09-25 株式会社东芝 Radiogram interpretation report creation assistance device

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080104116A1 (en) * 2004-12-02 2008-05-01 Lieven Van Hoe Image Evaluation System, Methods and Database
WO2014169164A1 (en) * 2013-04-13 2014-10-16 The Trustees Of The University Of Pennsylvania System and method for medical image analysis and probabilistic diagnosis
US11127494B2 (en) * 2015-08-26 2021-09-21 International Business Machines Corporation Context-specific vocabulary selection for image reporting

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Hahn, Udo, Martin Romacker, and Stefan Schulz. "MEDSYNDIKATE—a natural language system for the extraction of medical information from findings reports." International journal of medical informatics 67.1-3 (2002): 63-74. *
Tsatsoulis, Costas, and Heather A. Amthauer. "Finding clusters of similar events within clinical incident reports: a novel methodology combining case based reasoning and information retrieval." BMJ Quality & Safety 12.suppl 2 (2003): ii24-ii32. *

Also Published As

Publication number Publication date
US20160012319A1 (en) 2016-01-14
JP6461909B2 (en) 2019-01-30
CN105074708B (en) 2019-09-27
JP2016521149A (en) 2016-07-21
CN105074708A (en) 2015-11-18
US11289188B2 (en) 2022-03-29
WO2014155273A1 (en) 2014-10-02
EP2979210A1 (en) 2016-02-03

Similar Documents

Publication Publication Date Title
US20220199230A1 (en) Context driven summary view of radiology findings
CN105940401B (en) System and method for providing executable annotations
CN108475538B (en) Structured discovery objects for integrating third party applications in an image interpretation workflow
US8625867B2 (en) Medical image display apparatus, method, and program
US7979383B2 (en) Atlas reporting
JP7258772B2 (en) holistic patient radiology viewer
US20120035963A1 (en) System that automatically retrieves report templates based on diagnostic information
CN106233289B (en) Method and system for visualization of patient history
US11062448B2 (en) Machine learning data generation support apparatus, operation method of machine learning data generation support apparatus, and machine learning data generation support program
US20110066635A1 (en) Medical image information display apparatus, medical image information display method, and recording medium on which medical image information display program is recorded
JP2015524107A (en) System and method for matching patient information to clinical criteria
WO2018073707A1 (en) System and method for workflow-sensitive structured finding object (sfo) recommendation for clinical care continuum
JP5816321B2 (en) Information processing apparatus, information processing system, information processing method, and program
US20220139512A1 (en) Mapping pathology and radiology entities
EP4014240A1 (en) System and method for reporting on medical images
JP2013506900A (en) Document identification using image-based queries
JP2022162368A (en) Diagnosis support device, diagnosis support method and diagnosis support program
JPWO2005122002A1 (en) Structured document creation method and apparatus
WO2023011891A1 (en) Automated alerting system for relevant examinations
JP2023114340A (en) Medical information processing apparatus, operating method and program for medical information processing apparatus

Legal Events

Date Code Title Description
STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: FINAL REJECTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: ADVISORY ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: FINAL REJECTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: ADVISORY ACTION MAILED

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION