WO2021107098A1 - 文書作成支援装置、文書作成支援方法及び文書作成支援プログラム - Google Patents

文書作成支援装置、文書作成支援方法及び文書作成支援プログラム Download PDF

Info

Publication number
WO2021107098A1
WO2021107098A1 PCT/JP2020/044238 JP2020044238W WO2021107098A1 WO 2021107098 A1 WO2021107098 A1 WO 2021107098A1 JP 2020044238 W JP2020044238 W JP 2020044238W WO 2021107098 A1 WO2021107098 A1 WO 2021107098A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
information
document creation
character string
specific area
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2020/044238
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
晶路 一ノ瀬
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fujifilm Corp
Original Assignee
Fujifilm Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fujifilm Corp filed Critical Fujifilm Corp
Priority to JP2021561545A priority Critical patent/JPWO2021107098A1/ja
Priority to DE112020005870.0T priority patent/DE112020005870T5/de
Publication of WO2021107098A1 publication Critical patent/WO2021107098A1/ja
Priority to US17/748,017 priority patent/US11978274B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/18—Extraction of features or characteristics of the image
    • G06V30/18143—Extracting features based on salient regional features, e.g. scale invariant feature transform [SIFT] keypoints
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/12—Detection or correction of errors, e.g. by rescanning the pattern
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/19—Recognition using electronic means
    • G06V30/19007—Matching; Proximity measures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/40—Document-oriented image-based pattern recognition
    • G06V30/41—Analysis of document content
    • G06V30/416—Extracting the logical structure, e.g. chapters, sections or page numbers; Identifying elements of the document, e.g. authors
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H15/00—ICT specially adapted for medical reports, e.g. generation or transmission thereof
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/03—Recognition of patterns in medical or anatomical images

Definitions

  • This disclosure relates to a document creation support device that supports the creation of documents such as an interpretation report, a document creation support method, and a document creation support program.
  • CT Computer Tomography
  • MRI Magnetic Resonance Imaging
  • CAD Computer-Aided Diagnosis
  • regions, positions, volumes, etc. of lesions, etc. included in the medical images are extracted, and these are extracted. It is also obtained as an analysis result (for example, “Pulmonary Nodules on Multi-Detector Row CT Scans: Performance Comparison of Radiologists and Computer-aided Detection”, Rubin GD et al., In Radiology 2005; 234: 274- 283 (hereinafter referred to as “Reference 1”)).
  • the analysis result generated by the analysis process is associated with the examination information such as the patient name, gender, age, and the imaging device that acquired the medical image, is stored in the database, and is used for diagnosis.
  • an engineer such as a radiologist who has acquired the medical image determines an image interpreter according to the medical image, and informs the determined image interpreter that the medical image and the analysis result by CAD exist.
  • the image interpreting doctor interprets the medical image by referring to the distributed medical image and the analysis result on the image interpretation terminal, and creates an image interpretation report.
  • the image interpreting doctor may interpret a plurality of tomographic images obtained by one imaging with an imaging device such as a CT device and an MRI device, and describe the findings obtained from each tomographic image in the interpretation report.
  • an imaging device such as a CT device and an MRI device
  • creating such an interpretation report manually by an interpretation doctor is a burden on the interpretation work.
  • This disclosure provides a document creation support device, a document creation support method, and a document creation support program that can support the creation of documents such as interpretation reports.
  • a first aspect of the present disclosure is a document creation support device including at least one processor, in which the processor acquires an image and a character string relating to the image, extracts at least one feature region included in the image, and then extracts the image. Among the feature areas, a specific area that corresponds to a phrase included in the character string is specified, and information for supporting the creation of a document containing the character string is presented based on the specific result.
  • the processor may embed information for accessing an image including the specific area in a phrase corresponding to the specific area in the character string and display it on the display unit. ..
  • the document creation support device may generate property information indicating the properties of a specific area by the processor and display the property information on the display unit.
  • the processor in the document creation support device, the processor generates property information indicating the properties of the specific area, and the words and phrases related to the specific area in the character string are consistent with the property information. If not, a warning may be given.
  • the document creation support device may generate an image in which the processor adds a mark indicating the position of the specific area to the image including the specific area.
  • the document creation support device extracts a feature region based on at least one of the position, type, and size of the structure contained in the image by the processor. May be good.
  • a sixth aspect of the present disclosure is a document creation support method, in which an image and a character string related to the image are acquired, at least one feature area included in the image is extracted, and the feature area is included in the character string. It identifies a specific area that corresponds to a phrase, and presents information to support the creation of a document containing a character string based on a specific result.
  • a seventh aspect of the present disclosure is a document creation support program, which acquires an image and a character string related to the image, extracts at least one feature area included in the image, and includes the feature area in the character string. It is a program for causing a computer to perform a process of identifying a specific area corresponding to a word and presenting information for supporting the creation of a document including a character string based on a specific result.
  • the document creation support device, the document creation support method, and the document creation support program of the present disclosure can support the creation of a document such as an interpretation report.
  • FIG. 1 is a diagram showing a schematic configuration of the medical information system 1.
  • the medical information system 1 shown in FIG. 1 is based on an examination order from a doctor in a clinical department using a known ordering system, photographs of a part to be inspected of a subject, storage of medical images obtained by photographing, and an image interpreter. It is a system for interpreting medical images and creating an interpretation report, viewing the interpretation report by the doctor of the requesting clinical department, and observing the details of the medical image to be interpreted.
  • the medical information system 1 includes a plurality of imaging devices 2, a plurality of image interpretation terminals WS (WorkStation) 3, medical care WS4, an image server 5, an image DB (DataBase) 6, a report server 7, and a report server 7.
  • the report DB 8 is configured to be connected to each other via a wired or wireless network 10 so as to be able to communicate with each other.
  • Each device is a computer on which an application program for functioning as a component of the medical information system 1 is installed.
  • the application program is recorded and distributed on a recording medium such as a DVD (Digital Versatile Disc) and a CD-ROM (Compact Disc Read Only Memory), and is installed on a computer from the recording medium.
  • a recording medium such as a DVD (Digital Versatile Disc) and a CD-ROM (Compact Disc Read Only Memory)
  • it is stored in the storage device of the server computer connected to the network 10 or in the network storage in a state of being accessible from the outside, and is downloaded and installed in the computer upon request.
  • the photographing device 2 is a device (modality) that generates a medical image representing the diagnosis target part by photographing the part to be diagnosed of the subject. Specifically, it is a simple X-ray imaging apparatus, a CT apparatus, an MRI apparatus, a PET (Positron Emission Tomography) apparatus, and the like.
  • the medical image generated by the imaging device 2 is transmitted to the image server 5 and stored in the image DB 6.
  • the image interpretation WS3 is a computer used by, for example, an image interpretation doctor in a radiology department to interpret a medical image and create an image interpretation report, and includes a document creation support device 20 (details will be described later) according to this exemplary embodiment.
  • a request for viewing a medical image to the image server 5 various image processing for the medical image received from the image server 5, a display of the medical image, and an input reception of a finding regarding the medical image are performed.
  • analysis processing for medical images and input findings support for creating an interpretation report based on the analysis results, a request for registration and viewing of an interpretation report for the report server 7, and an interpretation received from the report server 7 are performed. The report is displayed.
  • the medical care WS4 is a computer used by doctors in the clinical department for detailed observation of images, viewing of interpretation reports, creation of electronic medical records, etc., and is a processing device, a display device such as a display, and an input device such as a keyboard and a mouse. Consists of.
  • an image viewing request is made to the image server 5
  • an image received from the image server 5 is displayed
  • an image interpretation report viewing request is made to the report server 7
  • an image interpretation report received from the report server 7 is displayed.
  • the image server 5 is a general-purpose computer in which a software program that provides a database management system (DataBase Management System: DBMS) function is installed. Further, the image server 5 includes a storage in which the image DB 6 is configured. This storage may be a hard disk device connected by the image server 5 and the data bus, or a disk device connected to NAS (Network Attached Storage) and SAN (Storage Area Network) connected to the network 10. It may be.
  • NAS Network Attached Storage
  • SAN Storage Area Network
  • the image data and incidental information of the medical image acquired by the imaging device 2 are registered in the image DB 6.
  • the incidental information includes, for example, an image ID (identification) for identifying an individual medical image, a patient ID for identifying a subject, an examination ID for identifying an examination, and a unique ID assigned to each medical image ( UID: unique identification), examination date on which the medical image was generated, examination time, type of imaging device used in the examination to obtain the medical image, patient information such as patient name, age, gender, examination site (ie) Information such as imaging site), imaging information (for example, imaging protocol, imaging sequence, imaging method, imaging conditions, use of contrast medium, etc.), series number or collection number when multiple medical images are acquired in one examination. Is included.
  • the image server 5 when the image server 5 receives the viewing request from the image interpretation WS3 and the medical examination WS4 via the network 10, the image server 5 searches for the medical image registered in the image DB 6, and uses the searched medical image as the requesting image interpretation WS3 and the medical examination. Send to WS4.
  • the report server 7 incorporates a software program that provides the functions of a database management system to a general-purpose computer.
  • the report server 7 receives the image interpretation report registration request from the image interpretation WS3, the report server 7 prepares the image interpretation report in a database format and registers it in the report DB 8.
  • the image interpretation report includes, for example, a medical image to be interpreted, an image ID for identifying the medical image, an image interpretation doctor ID for identifying the image interpretation doctor who performed the image interpretation, a lesion name, a lesion position information, and a medical image including a specific area. It may include information for access (details will be described later), property information (details will be described later), and the like.
  • the report server 7 when the report server 7 receives a viewing request for the interpretation report from the interpretation WS3 and the medical treatment WS4 via the network 10, the report server 7 searches for the interpretation report registered in the report DB 8 and uses the searched interpretation report as the requester's interpretation. It is transmitted to WS3 and medical treatment WS4.
  • Network 10 is a wired or wireless local area network that connects various devices in the hospital.
  • the network 10 may be configured such that the local area networks of each hospital are connected to each other by the Internet or a dedicated line.
  • the document creation support device 20 includes a CPU (Central Processing Unit) 11, a non-volatile storage unit 13, and a memory 16 as a temporary storage area. Further, the document creation support device 20 includes a display unit 14 such as a liquid crystal display, an input unit 15 such as a keyboard and a mouse, and a network I / F (InterFace) 17 connected to the network 10.
  • the CPU 11, the storage unit 13, the display unit 14, the input unit 15, the memory 16, and the network I / F 17 are connected to the bus 18.
  • the CPU 11 is an example of the processor in the present disclosure.
  • the storage unit 13 is realized by an HDD (Hard Disk Drive), an SSD (Solid State Drive), a flash memory, or the like.
  • the document creation support program 12 is stored in the storage unit 13 as a storage medium.
  • the CPU 11 reads the document creation support program 12 from the storage unit 13, expands it into the memory 16, and executes the expanded document creation support program 12.
  • the document creation support device 20 includes an acquisition unit 21, an extraction unit 22, a specific unit 23, a generation unit 24, and a display control unit 25.
  • the CPU 11 executes the document creation support program 12, it functions as an acquisition unit 21, an extraction unit 22, a specific unit 23, a generation unit 24, and a display control unit 25.
  • the acquisition unit 21 acquires the medical image G0 as an example of the image from the image server 5 via the network I / F17. Further, the acquisition unit 21 acquires a finding sentence as an example of a character string relating to the medical image G0 input by the image interpreting doctor via the input unit 15.
  • FIG. 4 is a diagram schematically showing a medical image G0.
  • a CT image of a lung consisting of m tomographic images D1 to Dm (m is 1 or more) is used as a medical image G0.
  • the tomographic image DT1 contains the nodule shadow A1 and the tomographic image DT2 contains the infiltration shadow A2.
  • the extraction unit 22 extracts at least one feature region included in the medical image G0.
  • the extraction unit 22 extracts the feature region based on at least one of the position, type, and size of the structure included in the medical image G0.
  • the characteristic region refers to an image-characteristic region including, for example, an abnormal shadow such as a nodule shadow A1 and an infiltration shadow A2 as an example of a structure.
  • the extraction unit 22 analyzes the medical image G0 by CAD or the like, extracts the characteristic region containing the nodular shadow A1 from the tomographic image DT1, and extracts the characteristic region including the infiltration shadow A2 from the tomographic image DT2.
  • the method for extracting the feature region by the extraction unit 22 for example, the method described in Document 1 above can be used, but the method is not limited thereto.
  • the information on the size and position of the abnormal shadow is included as the information on the feature region extracted by the extraction unit 22.
  • the size of the abnormal shadow can be the vertical and horizontal sizes of each abnormal shadow. Further, the diameter when the abnormal shadow is approximated to a circle may be used as the size.
  • the position of the abnormal shadow can be, for example, the position of the center of gravity of the abnormal shadow in each tomographic image. Further, as the information on the size and position of the abnormal shadow, the coordinates of the four corners of the rectangle including the abnormal shadow may be used.
  • the specific unit 23 specifies a specific area, which is an area corresponding to a phrase included in the finding sentence, among the feature areas extracted by the extraction unit 22.
  • a method for specifying a specific area by the specific unit 23 for example, the methods described in "Stacked Cross Attention for Image-Text Matching", Kuang-Huei Lee et al., In European Conference on Computer Vision (ECCV), 2018 are used. Can be used.
  • the region in the image means by each word in the sentence.
  • the technology to identify is disclosed.
  • the method for specifying the specific area by the specific unit 23 is not limited to this.
  • the generation unit 24 generates information (hereinafter, referred to as "support information") for supporting the creation of the interpretation report based on the specific result in the specific unit 23.
  • the display control unit 25 displays the image interpretation report creation screen 30 on the display unit 14, and controls the display contents on the creation screen 30 based on the support information generated by the generation unit 24. The details of the support information will be described later.
  • FIG. 5 is a diagram showing an example of the image interpretation report creation screen 30 displayed on the display unit 14.
  • the creation screen 30 includes an image display area 40 in which the medical image G0 is displayed, and an input area 41 in which the content of the finding sentence input by the image interpreter via the input unit 15 is displayed. Is done.
  • the tomographic image DT1 including the feature area including the nodule shadow A1 is displayed in the image display area 40.
  • a tomographic image including each feature region may be displayed as a thumbnail on the creation screen 30.
  • the creation screen 30 of FIG. 5 includes a thumbnail 55 of the tomographic image DT1 including the feature area including the nodule shadow A1 and a thumbnail 56 of the tomographic image DT2 including the feature area including the infiltration shadow A2. ..
  • the image interpreting doctor selects a thumbnail of the tomographic image to be displayed in the image display area 40 via the input unit 15, and the display control unit 25 displays the tomographic image corresponding to the selected thumbnail in the image display area 40. You may control the display on.
  • FIG. 6 is a diagram showing a creation screen 30 in a state in which the content of the input finding sentence is displayed.
  • the display control unit 25 displays the character string "nodule shadow on the left upper lobe S3" in the input area 41. Control to do.
  • the specific unit 23 specifies a characteristic region including the nodule shadow A1 as a specific region corresponding to the phrase “nodule shadow”.
  • FIG. 7 is a diagram showing a creation screen 30 on which a finding sentence to which a hyperlink is added as an example of support information is displayed.
  • the generation unit 24 generates a hyperlink 43A for the tomographic image DT1 as an example of information for accessing the tomographic image DT1 including a specific region corresponding to the phrase “nodule shadow”.
  • the generation unit 24 embeds the hyperlink 43A for the generated tomographic image DT1 in the phrase "nodule shadow" in the findings.
  • the hyperlink 43A may include a URL (uniform resource locator) indicating a storage location of the tomographic image DT1.
  • the information for accessing the tomographic image DT1 is not limited to the hyperlink, and for example, the coordinate position of the tomographic image DT1 in the medical image G0 may be used.
  • the display control unit 25 controls the generation unit 24 to display the finding sentence in which the hyperlink 43A for the tomographic image DT1 is embedded in the phrase “nodule shadow” on the display unit 14.
  • the tomographic image DT1 can be accessed by selecting the phrase "nodule shadow" in the finding sentence.
  • FIG. 8 is a diagram showing a creation screen 30 when the interpretation doctor further adds a finding sentence.
  • the specific unit 23 corresponds to the phrase “infiltration shadow” and infiltrates A2.
  • the feature area including is specified as a specific area.
  • the generation unit 24 generates a hyperlink 43B for the tomographic image DT2 including a specific region corresponding to the phrase “infiltration shadow”. In addition, the generation unit 24 embeds the hyperlink 43B for the generated tomographic image DT2 in the phrase "infiltration shadow” in the findings.
  • the hyperlink 43B may include a URL indicating a storage location of the tomographic image DT2.
  • the display control unit 25 controls the generation unit 24 to display the finding sentence in which the hyperlink 43B for the tomographic image DT2 is embedded in the phrase “infiltration shadow” on the display unit 14. Thereby, in the example of FIG. 8, the tomographic image DT2 can be accessed by selecting the phrase "infiltration shadow” in the finding sentence. In this case, as shown in FIG. 8, the display control unit 25 controls to display the tomographic image DT2 in the image display area 40.
  • the information for accessing the image including the specific area generated by the generation unit 24 as described above is included in the interpretation report and registered in the report DB 8. Therefore, even when the interpretation report is viewed from the interpretation WS3 and the medical treatment WS4, the image including the specific area corresponding to the phrase can be accessed by selecting the phrase in the finding sentence.
  • the operation of the document creation support device 20 will be described.
  • the CPU 11 executes the document creation support program 12
  • the document creation support process shown in FIG. 9 is executed.
  • the document creation support process shown in FIG. 9 is executed, for example, when an instruction to start creating an interpretation report for the medical image G0 is input via the input unit 15.
  • step S10 of FIG. 9 the acquisition unit 21 acquires the medical image G0 from the image server 5 and acquires the findings regarding the medical image G0 input via the input unit 15.
  • step S12 the extraction unit 22 extracts at least one feature region included in the medical image G0 acquired in step S10.
  • step S14 the specific unit 23 identifies a specific area, which is an area corresponding to the phrase included in the finding sentence acquired in step S10, among the feature areas extracted in step S12.
  • step S16 the generation unit 24 generates support information for supporting the creation of the interpretation report based on the specific result of step S14.
  • step S18 the display control unit 25 controls the content to be displayed on the display unit 14 based on the support information generated in step S16, and ends the process.
  • FIG. 10 is a flowchart showing details of the support information generation process according to the first exemplary embodiment, which is performed in step S16 of FIG.
  • the generation unit 24 generates information for accessing the image including the specific area specified in step S14 as support information.
  • the generation unit 24 embeds the information for accessing the image including the specific area generated in step S20 in the phrase corresponding to the specific area included in the finding sentence.
  • the findings regarding the medical image G0 and the medical image G0 are acquired, and at least one feature region included in the medical image G0 is extracted.
  • the feature areas identify a specific area that corresponds to the phrase included in the finding sentence, and identify the finding sentence as information to support the creation of a document containing the finding sentence based on the specific result.
  • Information for accessing the medical image G0 including the specific area is embedded in the phrase corresponding to the area and displayed on the display unit. Therefore, it is possible to create an image interpretation report so that it is possible to know which tomographic image in the medical image G0 can be seen to recognize the findings described in the image interpretation report, regardless of the manual input of the image interpretation doctor. , Can support the creation of documents such as interpretation reports.
  • the finding sentence acquired by the acquisition unit 21 is input by the image interpreting doctor via the input unit 15, but the present invention is not limited to this.
  • CAD analysis is performed using a discriminator that has been learned to output findings such as lesion size, shape, and estimated disease name by inputting CAD analysis results.
  • a technique for generating a finding from the result is disclosed. This technique may be applied to this exemplary embodiment, and the extraction unit 22 may extract a feature region from the medical image G0 acquired by the acquisition unit 21 and generate a finding sentence from the extracted feature region. Further, a technique for generating a finding sentence based on a fixed phrase, which is disclosed in Japanese Patent Application Laid-Open No. 7-3-1591, may be applied. According to the form of automatically generating the findings in this way, it is possible to support the creation of a document such as an interpretation report, so that the burden on the interpretation doctor when creating the interpretation report can be reduced.
  • the phrase corresponding to the specific area in the finding sentence when there is another image corresponding to the medical image G0 including the specific area, the phrase corresponding to the specific area in the finding sentence includes information for accessing the other image. It may be embedded and displayed on the display unit. For example, when performing follow-up observation on the nodule shadow A1, to access a tomographic image taken in the past corresponding to the tomographic image DT1 in which the phrase "nodule shadow" includes the nodule shadow A1. Information may be embedded and displayed on the display unit. According to such a form, it is possible to support the creation of a document such as an interpretation report when performing follow-up observation.
  • the content of the support information generated by the generation unit 24 and the display content on the creation screen 30 displayed on the display unit 14 by the display control unit 25 based on the support information are the same as the first exemplary embodiment. different.
  • the configuration of the medical information system 1 (see FIG. 1), the hardware configuration of the document creation support device 20 (see FIG. 2), and the functional configuration (see FIG. 3) according to this exemplary embodiment are the first exemplary embodiments. Since it is the same as the embodiment, the description thereof will be omitted.
  • FIG. 11 is a diagram showing a creation screen 30 on which the property information 44 indicating the property of the specific area is displayed as an example of the support information.
  • the generation unit 24 generates the property information 44 indicating the property of the specific area corresponding to the word / phrase included in the character string.
  • the display control unit 25 controls to display the property information 44 generated by the generation unit 24 on the display unit 14.
  • the properties of the specific region indicate, for example, properties such as the position, size, and shape of the structure included in the specific region.
  • the generation unit 24 may generate property information 44 regarding a presumed disease name suspected in a specific region based on properties such as the position, size, and shape of the structure.
  • the generation unit 24 analyzes a specific region including the nodule shadow A1 in the tomographic image DT1 by CAD or the like in response to the phrase “nodule shadow”, and property information indicating the properties of the nodule shadow A1 included in the specific region.
  • Generate 44 As an example of the property information 44, the generation unit 24 has positive information 44A for " ⁇ 21 mm large”, “lobular +” and “solidity +", negative information 44B for "calcification-", and "suspected squamous cell carcinoma”. ”Disease name information 44C is generated.
  • the property information provided with "+” indicates that the property is positive
  • the property information provided with "-" indicates that the property is negative.
  • the method for generating the property information 44 by the generation unit 24 for example, the method described in Document 1 above can be used, but the method is not limited thereto.
  • FIG. 12 is a flowchart showing details of the support information generation process according to the second exemplary embodiment, which is performed in step S16 of FIG.
  • the generation unit 24 analyzes the specific region specified in step S14.
  • the generation unit 24 generates property information indicating the property of the specific region as support information based on the analysis.
  • the findings regarding the medical image G0 and the medical image G0 are acquired, and at least one feature region included in the medical image G0 is extracted.
  • the properties of the specific area are used.
  • the property information to be shown is generated, and the property information is displayed on the display unit. Therefore, the image interpreter can confirm the properties of the specific area before creating the findings regarding the properties of the specific area, so that it is possible to suppress the oversight of the findings and support the creation of documents such as the interpretation report. can do.
  • the display control unit 25 changes the display method of each of the property information 44 so that it can be distinguished from which of the positive information 44A, the negative information 44B, and the disease name information 44C. May be good.
  • the background color may be changed for each of the positive information 44A, the negative information 44B, and the disease name information 44C. According to such a form, the user can easily know whether the various property information 44 is a positive, a negative, or a presumed disease name.
  • the mode in which the generation unit 24 generates the property information 44 has been described, but the present invention is not limited to this.
  • the extraction unit 22 may extract the feature region from the medical image G0 and generate the property information 44 of the feature region in advance. In this case, after acquiring the words and phrases included in the character string, the property information 44 can be displayed on the display unit 14 without analyzing the specific area by CAD or the like.
  • the property information 44 is not limited to the positive information 44A, the negative information 44B, and the disease name information 44C as shown in FIG.
  • CAD analysis is performed using a discriminator that has been learned to output findings such as lesion size, shape, and estimated disease name by inputting CAD analysis results.
  • a technique for generating a finding from the result is disclosed. This technique may be applied to this exemplary embodiment to generate finding sentences indicating the properties of a specific region as the property information 44. Further, a technique for generating a finding sentence based on a fixed phrase, which is disclosed in Japanese Patent Application Laid-Open No. 7-3-1591, may be applied. According to the form of automatically generating the findings in this way, it is possible to support the creation of a document such as an interpretation report, so that the burden on the interpretation doctor when creating the interpretation report can be reduced.
  • the content of the support information generated by the generation unit 24 and the display content on the creation screen 30 displayed on the display unit 14 by the display control unit 25 based on the support information are the same as the first exemplary embodiment. different.
  • the configuration of the medical information system 1 (see FIG. 1), the hardware configuration of the document creation support device 20 (see FIG. 2), and the functional configuration (see FIG. 3) according to this exemplary embodiment are the first exemplary embodiments. Since it is the same as the embodiment, the description thereof will be omitted.
  • FIG. 13 is a diagram showing a creation screen 30 on which the property information 44 indicating the properties of the specific area and the warning 45 based on the property information 44 are displayed as an example of the support information. Since the property information 44 is the same as that of the second exemplary embodiment, the description thereof will be omitted. In the example of FIG. 13, the form in which the property information 44 is displayed on the creation screen 30 is shown, but the property information 44 may not be displayed on the creation screen 30.
  • the generation unit 24 generates property information 44 indicating the properties of the specific area corresponding to the word / phrase included in the character string, and warns 45 when the word / phrase related to the specific area in the character string does not match the property information 44. Generate warning information to do.
  • the display control unit 25 controls to display the warning 45 on the display unit 14 based on the warning information generated by the generation unit 24.
  • the property information 44 is the same information as that of the second exemplary embodiment, the description thereof will be omitted.
  • the generation unit 24 generates negative information 44B of "calcification-" as property information 44 of a specific region corresponding to the phrase "nodule shadow”.
  • the phrase "including calcification” related to the specific region acquired by the acquisition unit 21 is analyzed, and it is determined that the content does not match the negative information 44B of "calcification-”.
  • warning information for issuing a wavy warning 45 to the phrase "including calcification” is generated.
  • the method of warning 45 is not limited to drawing a wavy line on a phrase that does not match the property information 44, and is, for example, a method of changing the color of the phrase, displaying a pop-up screen, or sounding an alarm. May be used.
  • FIG. 14 is a flowchart showing details of the support information generation process according to the third exemplary embodiment, which is performed in step S16 of FIG.
  • the generation unit 24 analyzes the specific region specified in step S14.
  • the generation unit 24 generates property information indicating the properties of the specific region based on the analysis.
  • the generation unit 24 determines whether or not the phrase corresponding to the feature region included in the finding sentence and the property information generated in step S42 are consistent. When the generation unit 24 determines that the two are consistent, the generation unit 24 terminates this routine. On the other hand, when it is determined that the two are not consistent, the generation unit 24 generates warning information as support information for warning that the two are not consistent in step S46.
  • the findings regarding the medical image G0 and the medical image G0 are acquired, and at least one feature region included in the medical image G0 is extracted.
  • the properties of the specific area Generates the property information to be shown, and warns when the phrase related to a specific area in the character string does not match the property information. Therefore, it is possible to suppress misdiagnosis by an image interpretation doctor and support the creation of a document such as an image interpretation report.
  • the generation unit 24 may generate an image in which a mark indicating the position of the specific region is added to the medical image G0 including the specific region.
  • a mark indicating the position of the specific region is added to the medical image G0 including the specific region.
  • the specific region corresponding to the phrase “nodule shadow” specified by the specific unit 23 is surrounded by a broken line rectangle 50.
  • a specific region corresponding to the phrase “infiltration shadow” specified by the specific portion 23 is surrounded by a broken line rectangle 50.
  • the generation unit 24 may generate an image in which a mark indicating the position of the feature region is added to the medical image G0 including the feature region.
  • a mark indicating the position of the feature region is added to the medical image G0 including the feature region.
  • the feature region including the nodule shadow A1 extracted by the extraction unit 22 is surrounded by a broken line rectangle 50.
  • each image with a mark indicating the position of each feature region may be generated, or a mark indicating the position of each feature region may be generated.
  • One given image may be generated.
  • the position of the feature area can be displayed on the display unit 14 before the reading doctor inputs the finding sentence, so that the reading doctor can suppress the oversight of the abnormal shadow, and the reading report or the like can be displayed. Can assist in the creation of documents.
  • the mark indicating the position of the specific area and the feature area is not limited to the broken line rectangle 50, and may be various marks such as polygons, circles, and arrows, and the line type of the mark (for example, solid line). , Broken line and dotted line, etc.), line color, line thickness, etc. may be changed as appropriate.
  • each process of the extraction unit 22, the identification unit 23, and the generation unit 24 in the document creation support device 20 included in the image interpretation WS3 is performed by, for example, another analysis server connected to the network 10.
  • the external device receives the character string from the document creation support device 20, acquires the medical image G0 corresponding to the character string from the image server 5, extracts the feature area from the medical image G0, and extracts the character string and the feature area.
  • a specific area is specified from, and support information is generated based on the specific result.
  • the document creation support device 20 transmits the character string acquired by the acquisition unit 21 to the external device, and controls the display content displayed on the display unit 14 by the display control unit 25 based on the support information generated by the external device. ..
  • a known voice input system may be applied to the document creation support device 20 to input a character string related to the medical image G0 by voice.
  • the present disclosure is applied when creating an image interpretation report as a document, but medical documents other than the image interpretation report such as electronic medical records and diagnostic reports, and characters related to other images.
  • the present disclosure may be applied when creating a document containing columns.
  • the image interpretation report creation support process is performed using the medical image G0 whose diagnosis target is the lung, but the diagnosis target is not limited to the lung.
  • diagnosis target is not limited to the lung.
  • any part of the human body such as the heart, liver, brain, and limbs can be diagnosed.
  • a hardware-like processing unit that executes various processes such as an acquisition unit 21, an extraction unit 22, a specific unit 23, a generation unit 24, and a display control unit 25.
  • various processors processors shown below can be used.
  • the various processors include CPUs, which are general-purpose processors that execute software (programs) and function as various processing units, as well as circuits after manufacturing FPGAs (Field Programmable Gate Arrays) and the like.
  • Dedicated electricity which is a processor with a circuit configuration specially designed to execute specific processing such as programmable logic device (PLD), ASIC (Application Specific Integrated Circuit), which is a processor whose configuration can be changed. Circuits and the like are included.
  • One processing unit may be composed of one of these various processors, or a combination of two or more processors of the same type or different types (for example, a combination of a plurality of FPGAs or a combination of a CPU and an FPGA). ) May be configured. Further, a plurality of processing units may be configured by one processor. As an example of configuring a plurality of processing units with one processor, first, as represented by a computer such as a client and a server, one processor is configured by combining one or more CPUs and software. There is a form in which this processor functions as a plurality of processing units.
  • SoC System On Chip
  • the various processing units are configured by using one or more of the above-mentioned various processors as a hardware structure.
  • circuitry in which circuit elements such as semiconductor elements are combined can be used.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Epidemiology (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Primary Health Care (AREA)
  • Public Health (AREA)
  • Medical Treatment And Welfare Office Work (AREA)
  • Measuring And Recording Apparatus For Diagnosis (AREA)
PCT/JP2020/044238 2019-11-29 2020-11-27 文書作成支援装置、文書作成支援方法及び文書作成支援プログラム Ceased WO2021107098A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2021561545A JPWO2021107098A1 (https=) 2019-11-29 2020-11-27
DE112020005870.0T DE112020005870T5 (de) 2019-11-29 2020-11-27 Unterstützungsvorrichtung für dokumentenerstellung, unterstützungsverfahren für dokumentenerstellung und unterstützungsprogramm für dokumentenerstellung
US17/748,017 US11978274B2 (en) 2019-11-29 2022-05-18 Document creation support apparatus, document creation support method, and document creation support program

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2019217419 2019-11-29
JP2019-217419 2019-11-29

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/748,017 Continuation US11978274B2 (en) 2019-11-29 2022-05-18 Document creation support apparatus, document creation support method, and document creation support program

Publications (1)

Publication Number Publication Date
WO2021107098A1 true WO2021107098A1 (ja) 2021-06-03

Family

ID=76130585

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/044238 Ceased WO2021107098A1 (ja) 2019-11-29 2020-11-27 文書作成支援装置、文書作成支援方法及び文書作成支援プログラム

Country Status (4)

Country Link
US (1) US11978274B2 (https=)
JP (1) JPWO2021107098A1 (https=)
DE (1) DE112020005870T5 (https=)
WO (1) WO2021107098A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2023161267A (ja) * 2022-04-25 2023-11-07 株式会社湯山製作所 業務支援システム及び業務支援プログラム

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021107098A1 (ja) * 2019-11-29 2021-06-03 富士フイルム株式会社 文書作成支援装置、文書作成支援方法及び文書作成支援プログラム
WO2021157718A1 (ja) * 2020-02-07 2021-08-12 富士フイルム株式会社 文書作成支援装置、文書作成支援方法及びプログラム
JP7718915B2 (ja) * 2021-08-30 2025-08-05 富士フイルム株式会社 学習装置、方法およびプログラム、並びに情報処理装置、方法およびプログラム
US20240281979A1 (en) * 2023-02-22 2024-08-22 Canon Medical Systems Corporation Medical image processing apparatus and method

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2013039230A (ja) * 2011-08-16 2013-02-28 Canon Inc 医療診断支援装置及び医療診断支援方法
JP2019153250A (ja) * 2018-03-06 2019-09-12 富士フイルム株式会社 医療文書作成支援装置、方法およびプログラム
JP2019169049A (ja) * 2018-03-26 2019-10-03 富士フイルム株式会社 医用画像特定装置、方法およびプログラム
US20190325249A1 (en) * 2016-06-28 2019-10-24 Koninklijke Philips N.V. System and method for automatic detection of key images

Family Cites Families (43)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3332104B2 (ja) 1993-07-19 2002-10-07 株式会社東芝 読影レポート作成支援装置
JP2005148990A (ja) * 2003-11-13 2005-06-09 Konica Minolta Medical & Graphic Inc 医用画像読影システム及び読影レポート作成方法
JP5024981B2 (ja) * 2005-04-20 2012-09-12 株式会社東芝 医用画像データ変換装置
US7979383B2 (en) * 2005-06-06 2011-07-12 Atlas Reporting, Llc Atlas reporting
US7804990B2 (en) * 2006-01-25 2010-09-28 Siemens Medical Solutions Usa, Inc. System and method for labeling and identifying lymph nodes in medical images
JP2009086765A (ja) * 2007-09-27 2009-04-23 Fujifilm Corp 医用レポートシステム、医用レポート作成装置、及び医用レポート作成方法
US7949167B2 (en) * 2008-06-12 2011-05-24 Siemens Medical Solutions Usa, Inc. Automatic learning of image features to predict disease
JP5455470B2 (ja) * 2009-07-02 2014-03-26 株式会社東芝 医用画像読影システム
JP5523891B2 (ja) * 2009-09-30 2014-06-18 富士フイルム株式会社 病変領域抽出装置、その作動方法およびプログラム
CN103365952B (zh) * 2012-04-06 2017-04-12 东芝医疗系统株式会社 医疗信息检索装置
JP6595193B2 (ja) * 2014-03-11 2019-10-23 キヤノンメディカルシステムズ株式会社 読影レポート作成装置および読影レポート作成システム
JP2015215740A (ja) * 2014-05-09 2015-12-03 キヤノン株式会社 情報処理装置、情報処理方法及びプログラム
JP6180470B2 (ja) 2015-07-13 2017-08-16 株式会社ワイズ・リーディング 文章候補提示端末、文章候補提示システム、文章候補提示方法、及びプログラム
US10181187B2 (en) * 2015-09-09 2019-01-15 Canon Kabushiki Kaisha Information processing apparatus, method thereof, information processing system, and computer-readable storage medium that display a medical image with comment information
JP6711676B2 (ja) * 2016-04-13 2020-06-17 キヤノン株式会社 医用レポート作成装置及びその制御方法、医用レポート作成システム、並びに、プログラム
NL2019410B1 (en) * 2017-08-10 2019-02-21 Aidence B V Computer-aided diagnostics using deep neural networks
US10803581B2 (en) * 2017-11-06 2020-10-13 Beijing Keya Medical Technology Co., Ltd. System and method for generating and editing diagnosis reports based on medical images
US10679345B2 (en) * 2017-12-20 2020-06-09 International Business Machines Corporation Automatic contour annotation of medical images based on correlations with medical reports
EP3518245A1 (en) * 2018-01-29 2019-07-31 Siemens Healthcare GmbH Image generation from a medical text report
JP2019149005A (ja) * 2018-02-27 2019-09-05 富士フイルム株式会社 医療文書作成支援装置、方法およびプログラム
JP7000206B2 (ja) * 2018-03-06 2022-01-19 富士フイルム株式会社 医用画像処理装置、医用画像処理方法、及び医用画像処理プログラム
JPWO2019193983A1 (ja) * 2018-04-04 2021-05-13 富士フイルム株式会社 医療文書表示制御装置、医療文書表示制御方法、及び医療文書表示制御プログラム
WO2019193982A1 (ja) * 2018-04-04 2019-10-10 富士フイルム株式会社 医療文書作成支援装置、医療文書作成支援方法、及び医療文書作成支援プログラム
US10825173B2 (en) * 2018-04-25 2020-11-03 International Business Machines Corporation Automatically linking a description of pathology in a medical image report to an image
WO2020026033A2 (en) * 2018-08-02 2020-02-06 Imedis Al Ltd Systems and methods for improved analysis and generation of medical imaging reports
JP7212147B2 (ja) * 2019-04-04 2023-01-24 富士フイルム株式会社 医療文書作成支援装置、方法およびプログラム
US11791044B2 (en) * 2019-09-06 2023-10-17 RedNova Innovations, Inc. System for generating medical reports for imaging studies
CA3156519A1 (en) * 2019-10-01 2021-04-08 Sirona Medical, Inc. AI-ASSISTED MEDICAL IMAGE INTERPRETATION AND REPORT GENERATION
JP6847558B2 (ja) 2019-10-08 2021-03-24 株式会社ユニバーサルエンターテインメント 遊技機
US11923070B2 (en) * 2019-11-28 2024-03-05 Braid Health Inc. Automated visual reporting technique for medical imaging processing system
WO2021107098A1 (ja) * 2019-11-29 2021-06-03 富士フイルム株式会社 文書作成支援装置、文書作成支援方法及び文書作成支援プログラム
JP7371116B2 (ja) * 2019-11-29 2023-10-30 富士フイルム株式会社 文書作成支援装置、文書作成支援方法及びプログラム
WO2021112141A1 (ja) * 2019-12-03 2021-06-10 富士フイルム株式会社 文書作成支援装置、方法およびプログラム
CN114868193A (zh) * 2019-12-18 2022-08-05 皇家飞利浦有限公司 相互改善来自临床笔记和医学图像分类的概念提取的协同训练框架
JPWO2021157705A1 (https=) * 2020-02-07 2021-08-12
WO2021162008A1 (ja) * 2020-02-10 2021-08-19 富士フイルム株式会社 文書作成支援装置、文書作成支援方法及びプログラム
WO2021167018A1 (ja) * 2020-02-18 2021-08-26 富士フイルム株式会社 情報処理装置、情報処理方法及び情報処理プログラム
DE112021001190T5 (de) * 2020-02-20 2022-12-15 Fujifilm Corporation Informationsverarbeitungsvorrichtung, -verfahren und -programm
WO2021177357A1 (ja) * 2020-03-03 2021-09-10 富士フイルム株式会社 情報処理装置、情報処理方法及び情報処理プログラム
US11449717B2 (en) * 2020-03-12 2022-09-20 Fujifilm Business Innovation Corp. System and method for identification and localization of images using triplet loss and predicted regions
WO2021187483A1 (ja) * 2020-03-17 2021-09-23 富士フイルム株式会社 文書作成支援装置、方法およびプログラム
JP7436636B2 (ja) * 2020-03-23 2024-02-21 富士フイルム株式会社 文書作成支援装置、方法およびプログラム
US11244755B1 (en) * 2020-10-02 2022-02-08 International Business Machines Corporation Automatic generation of medical imaging reports based on fine grained finding labels

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2013039230A (ja) * 2011-08-16 2013-02-28 Canon Inc 医療診断支援装置及び医療診断支援方法
US20190325249A1 (en) * 2016-06-28 2019-10-24 Koninklijke Philips N.V. System and method for automatic detection of key images
JP2019153250A (ja) * 2018-03-06 2019-09-12 富士フイルム株式会社 医療文書作成支援装置、方法およびプログラム
JP2019169049A (ja) * 2018-03-26 2019-10-03 富士フイルム株式会社 医用画像特定装置、方法およびプログラム

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2023161267A (ja) * 2022-04-25 2023-11-07 株式会社湯山製作所 業務支援システム及び業務支援プログラム

Also Published As

Publication number Publication date
DE112020005870T5 (de) 2022-11-03
US20220277577A1 (en) 2022-09-01
US11978274B2 (en) 2024-05-07
JPWO2021107098A1 (https=) 2021-06-03

Similar Documents

Publication Publication Date Title
JP7618003B2 (ja) 文書作成支援装置、方法およびプログラム
JP2019153250A (ja) 医療文書作成支援装置、方法およびプログラム
US11978274B2 (en) Document creation support apparatus, document creation support method, and document creation support program
JP7701493B2 (ja) 医用画像処理装置、方法およびプログラム
JP7000206B2 (ja) 医用画像処理装置、医用画像処理方法、及び医用画像処理プログラム
WO2022215530A1 (ja) 医用画像装置、医用画像方法、及び医用画像プログラム
US12406755B2 (en) Document creation support apparatus, method, and program
US20230281810A1 (en) Image display apparatus, method, and program
JP7237089B2 (ja) 医療文書作成支援装置、方法およびプログラム
US12527528B2 (en) Image display apparatus, method, and program
JP7504987B2 (ja) 情報処理装置、情報処理方法及び情報処理プログラム
US20220028510A1 (en) Medical document creation apparatus, method, and program
JP7371116B2 (ja) 文書作成支援装置、文書作成支援方法及びプログラム
JP7371220B2 (ja) 情報処理装置、情報処理方法及び情報処理プログラム
JP7007469B2 (ja) 医療文書作成支援装置、方法およびプログラム、学習済みモデル、並びに学習装置、方法およびプログラム
WO2021193548A1 (ja) 文書作成支援装置、方法およびプログラム
WO2020241857A1 (ja) 医療文書作成装置、方法およびプログラム、学習装置、方法およびプログラム、並びに学習済みモデル
JP7420914B2 (ja) 情報処理装置、情報処理方法及び情報処理プログラム
WO2023054645A1 (ja) 情報処理装置、情報処理方法及び情報処理プログラム
WO2022230641A1 (ja) 文書作成支援装置、文書作成支援方法、及び文書作成支援プログラム
US12512210B2 (en) Medical image display apparatus, method, and program
JP2021175454A (ja) 医用画像処理装置、方法およびプログラム
WO2022215529A1 (ja) 医用画像解析装置、医用画像解析方法、及び医用画像解析プログラム
WO2022153702A1 (ja) 医用画像表示装置、方法およびプログラム
WO2021107142A1 (ja) 文書作成支援装置、方法およびプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20894472

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021561545

Country of ref document: JP

Kind code of ref document: A

122 Ep: pct application non-entry in european phase

Ref document number: 20894472

Country of ref document: EP

Kind code of ref document: A1