JP7828203B2 - Document creation device, document creation method, and program - Google Patents

Description

The present invention relates to a document creation device, a document creation method, and a program, and in particular to technology for creating documents related to images.

Medical images are analyzed using CAD (Computer-Aided Diagnosis) processing, which uses classifiers trained through deep learning and other methods, to extract the area, location, and volume of lesions and other features contained in the medical image, and obtain these as analysis results. The radiologist then interprets the medical image on their own radiology device, referring to the medical image and the analysis results, and creates a radiology report.

Patent Documents 1 and 2 also disclose methods for reducing the burden on operators when creating medical documents related to medical images.

Japanese Patent Application Laid-Open No. 2019-149005 JP 2009-070201 A

As such, instead of having radiologists manually input all of the findings in radiology reports, in recent years there has been a growing need for methods to automatically extract candidate findings, such as automatically inputting findings using templates or extracting candidate findings using machine learning.

However, with conventional technology, when users wanted to include a text with a large amount of information about lesions in an image in an interpretation report, the automatically generated text was of poor quality, and it was not possible to provide text that could be used.

The present invention was made in consideration of these circumstances, and aims to provide a document creation device, document creation method, and program that can create high-quality documents related to images, even if the images contain a large amount of information that needs to be included in the document.

One aspect of a document creation device for achieving the above-mentioned objective is a document creation device that includes at least one processor and at least one memory that stores instructions to be executed by the at least one processor, wherein the at least one processor analyzes an input image to acquire content, identifies a base sentence that includes at least a portion of the content from a database in which multiple documents are stored, and modifies at least a portion of the base sentence to create a first document related to the input image that is neither excessive nor deficient in content. According to this aspect, the first sentence related to the input image can be created based on a sentence that includes the content of the input image, making it possible to create a high-quality document related to the input image even for an input image that has a large amount of information to be included in the document.

Preferably, at least one processor identifies multiple base sentences and combines and modifies the multiple base sentences, thereby creating a first document that is complete in terms of content.

It is preferable that at least one processor determines the extent to which the base sentence is to be corrected based on whether the base sentence is in excess or deficiency relative to the content. It is also preferable that at least one processor determines the extent to which the base sentence is to be corrected on a sentence-by-sentence basis. This allows the extent to which the base sentence is to be corrected to be determined appropriately.

It is preferable that the database stores sentences contained in multiple documents in a structured manner. This allows the base sentence to be identified appropriately.

Preferably, the input image is a medical image of a first subject, and the first document is an interpretation report. This aspect is suitable for creating an interpretation report for a medical image.

The multiple documents preferably include radiology reports for medical images taken of a second subject different from the first subject. This makes it possible to identify base sentences for different subjects that contain similar content.

The content preferably includes at least one of the anatomical location, characteristics, size, and factuality of the lesion, which allows a first document to be created regarding the lesion.

It is preferable that at least one processor identifies a base sentence using at least one of the organ recognition results and the lesion recognition results for the input image. This allows a first document relating to at least one of the organ and the lesion to be created.

The modifications preferably include at least one of additions, changes, and deletions. This allows a first document to be created from the base text.

It is preferable that at least one processor performs at least one of additions and modifications using at least one of a pre-prepared template and a trained model. This allows at least one of additions and modifications to be performed appropriately.

It is preferable that at least one processor determines the location of the addition as the range of correction to the base text based on the type of content. This allows the location of the addition to be determined appropriately.

It is preferable that at least one processor displays the correction range of the base sentence on the display, distinguishing it from the non-correction range. This allows the user to recognize the correction range.

Preferably, the at least one processor allows a user to specify a region of interest in the input image and retrieves content based on the region of interest, thereby creating a first document about the user's desired region of interest.

It is preferable that at least one processor calculates the degree of match between the content and multiple documents and identifies a base sentence based on the degree of match. This makes it possible to identify a base sentence with a relatively high degree of match.

It is preferable that the processor uses all of the acquired content information and deletes unnecessary information from the base text to create a first document that is neither too much nor too little compared to the content. This allows for the appropriate creation of a first document that is neither too much nor too little compared to the content.

One aspect of a document creation method for achieving the above-mentioned objective is a document creation method comprising: an acquisition step for acquiring an input image; an analysis step for analyzing the input image to acquire content; an identification step for identifying a base sentence that includes at least a portion of the content from a database in which multiple documents are stored; and a creation step for modifying at least a portion of the base sentence to create a first document related to the input image that is neither too much nor too little compared to the content. According to this aspect, the first sentence related to the input image can be created based on a sentence that includes the content of the input image, making it possible to create a high-quality document related to the input image even for an input image that has a large amount of information to be included in the document.

One aspect of a program for achieving the above object is a program that causes a computer to execute the above document creation method. This aspect may also include a computer-readable non-transitory storage medium on which this program is recorded. According to this aspect, a first sentence related to an input image can be created based on a sentence containing the content of the input image, making it possible to create a high-quality document related to the input image even if the input image contains a large amount of information that needs to be included in the document.

This invention makes it possible to create high-quality documents related to images, even for images that contain a large amount of information.

FIG. 1 is a diagram showing the overall configuration of a document creation system. FIG. 2 is a block diagram showing the configuration of the document creation device. FIG. 3 is a flowchart showing a document creation method using the document creation system. FIG. 4 shows an image to be interpreted and its findings. FIG. 5 shows an example of an input observation sentence and the structured result thereof. FIG. 6 shows an image to be interpreted and its findings. FIG. 7 is a diagram illustrating an example of the characteristics of a pulmonary nodule analyzed by the trained model. FIG. 8 shows the results of characterization of a certain lesion and the findings. FIG. 9 shows an example of a finding statement from a past radiology report and the structuring result thereof. FIG. 10 shows an example of a finding statement from a past radiology report and the structuring result thereof.

A preferred embodiment of the present invention will now be described in detail with reference to the accompanying drawings.

<Conventional issues>
The following describes the conventional problems that arise when creating an image interpretation report by including text with a large amount of information. An example of text with a large amount of information is shown below.
"A nodule measuring 3.2 cm in diameter was found in the S1 of the right lung. It is solid with a clear border. The border is slightly irregular, with some spicules. Inside, a thin-walled cavity with an irregular border was found. No calcification or bronchial radiolucency was observed. Pleural indentation was also observed."
When providing such information-rich text using a template, if there is no template corresponding to that text in the system's database, the text cannot be provided. Also, when creating a template based on a past radiology report, if there is no template of the exact same format, i.e., one that can be reused exactly as is, the template cannot be prepared.

For example, if the only templates available were "L, M, and N not found" and "L and M found," it was not possible to create the sentence "L, M, and N found." Similarly, if the only template available was "Cavities and calcifications found internally," it was not possible to create the sentence "Cavities, calcifications, and bronchial radiographs found internally."

On the other hand, when providing text with a large amount of information using a trained model such as a neural network, it is difficult to generate text that mentions a large variety of characteristic information, and there is a risk that the findings desired by the radiologist will not be generated, and instead findings that the radiologist does not want will be provided. In other words, when providing information using a trained model, appropriateness and fluency become issues. Here, appropriateness refers to the absence of excessive or omissions, and fluency refers to the absence of unnatural wording as a natural language expression.

As such, it has traditionally been difficult to provide text with a large amount of information.

<Document Creation System>
The document creation system according to this embodiment is a system that captures medical images of a subject (patient) and creates an interpretation report on the captured medical images.

Figure 1 is an overall configuration diagram of a document creation system 10. As shown in Figure 1, the document creation system 10 is composed of medical image inspection equipment 12, a medical image database 14, a document creation device 16, an image interpretation report database 18, and a user terminal 20.

The medical image inspection equipment 12, medical image database 14, document creation device 16, radiology report database 18, and user terminal 20 are connected via network 22 so that they can send and receive data. Network 22 includes a wired or wireless LAN (Local Area Network) that connects various devices within the medical institution. Network 22 may also include a WAN (Wide Area Network) that connects the LANs of multiple medical institutions.

The medical imaging inspection equipment 12 is an imaging device that captures images of the target area of a subject and generates medical images. Examples of medical imaging inspection equipment 12 include an X-ray imaging device, a CT (Computed Tomography) device, an MRI (Magnetic Resonance Imaging) device, a PET (Positron Emission Tomography) device, an ultrasound device, and a CR (Computed Radiography) device that uses a flat panel X-ray detector.

The medical image database 14 is a database that manages medical images captured by the medical image inspection equipment 12. The medical image database 14 is implemented using a computer equipped with a large-capacity storage device for saving medical images. The computer is equipped with software that provides the functions of a database management system.

The Dicom (Digital Imaging and Communications in Medicine) standard can be applied to the format of medical images. Supplementary information (Dicom tag information) defined in the Dicom standard may be added to medical images. Note that the term "image" in this specification includes not only the image itself, such as a photograph, but also image data, which is a signal representing the image.

The document creation device 16 analyzes an input image to acquire content, identifies a base sentence containing at least part of the content from a database in which multiple documents are stored, and modifies at least part of the base sentence to create a first document related to the input image that is neither excessive nor insufficient compared to the content. Here, the input image is a current image of the subject acquired from the medical image inspection device 12 or the medical image database 14. The database in which multiple documents are stored is the radiology report database 18, and the first document is a radiology report related to the current image of the subject.

The document creation device 16 can be a personal computer or a workstation (an example of a "computer"). Figure 2 is a block diagram showing the configuration of the document creation device 16. As shown in Figure 2, the document creation device 16 includes a processor 16A, memory 16B, and a communication interface 16C.

Processor 16A executes instructions stored in memory 16B. The hardware structure of processor 16A is the following various processors. The various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) and functions as various functional units, a GPU (Graphics Processing Unit), which is a processor specialized for image processing, a PLD (Programmable Logic Device), which is a processor whose circuit configuration can be changed after manufacture such as an FPGA (Field Programmable Gate Array), and a dedicated electrical circuit, such as an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically to perform specific processing.

A single processing unit may be configured with one of these various processors, or with two or more processors of the same or different types (for example, multiple FPGAs, or a combination of a CPU and an FPGA, or a combination of a CPU and a GPU). Multiple functional units may also be configured with a single processor. Examples of multiple functional units configured with a single processor include, first, a configuration where a single processor is configured with a combination of one or more CPUs and software, as typified by client or server computers, and this processor operates as multiple functional units. Second, a configuration where a processor is used to realize the functions of an entire system including multiple functional units on a single IC (Integrated Circuit) chip, as typified by SoCs (System On Chips). In this way, the various functional units are configured with a hardware structure using one or more of the various processors listed above.

More specifically, the hardware structure of these various processors is an electrical circuit that combines circuit elements such as semiconductor devices.

Memory 16B stores instructions to be executed by processor 16A. Memory 16B includes RAM (Random Access Memory) and ROM (Read Only Memory), not shown. Processor 16A uses RAM as a working area and executes software using various programs and parameters, including the document creation program described below, stored in ROM, and also performs various processes of document creation device 16 by using parameters stored in ROM, etc.

The communication interface 16C controls communication with the medical image inspection equipment 12, medical image database 14, radiology report database 18, and user terminal 20 via the network 22 in accordance with a predetermined protocol.

The document creation device 16 may be a cloud server accessible from multiple medical institutions via the Internet. The processing performed by the document creation device 16 may be a cloud service that is pay-per-use or has a fixed fee.

Returning to the explanation of Figure 1, the radiology report database 18 is a database that manages past radiology reports created by users such as radiologists (radiologists) on the user terminal 20. The radiology reports include findings. The findings are not limited to sentences separated by periods, punctuation marks, etc., but may also be groups of words. The findings may be a single sentence, or may be a sentence made up of multiple sentences. The radiology report database 18 may store the findings of past radiology reports in a structured manner, or may store the findings in an unstructured manner.

The radiology report database 18 is implemented as a computer equipped with a large-capacity storage device for saving radiology reports. The computer is equipped with software that provides the functions of a database management system. The medical image database 14 and the radiology report database 18 may be configured on a single computer.

The user terminal 20 is a terminal device used by the radiologist to view and edit the radiology report. The user terminal 20 is, for example, a personal computer. The user terminal 20 may be a workstation or a tablet terminal. The user terminal 20 has an input device 20A and a display 20B. The radiologist uses the input device 20A to input instructions to the document creation system 10. The user terminal 20 also displays medical images and radiology reports on the display 20B. The radiologist then interprets the medical images displayed on the display 20B and inputs the findings, which are the results of the interpretation, using the input device 20A.

<Document creation method>
3 is a flowchart showing a document creation method using the document creation system 10. The document creation method is realized by the processor 16A executing a document creation program stored in the memory 16B. The document creation program may be provided by a computer-readable non-transitory storage medium or via the Internet.

Steps ST1 and ST2 are pre-processing steps performed to store the radiology report containing the findings in the radiology report database 18.

In step ST1, the radiologist specifies the image to be interpreted using the input device 20A of the user terminal 20. The document creation device 16 reads the image to be interpreted specified by the radiologist from among the medical images captured by the medical image inspection equipment 12 and stored in the medical image database 14, and displays it on the display 20B of the user terminal 20. Figure 4 is a diagram showing the image to be interpreted and its findings. F4A in Figure 4 indicates the image to be interpreted I1 specified in step ST1.

Next, the document creation device 16 extracts a lesion area (an example of a "region of interest") from the image to be interpreted displayed on the display 20B. The lesion area may be extracted manually by the radiologist using the input device 20A of the user terminal 20. Here, it is assumed that the lesion area L1 of the image to be interpreted I1 shown in F4A has been extracted.

In step ST2, the radiologist uses the input device 20A to input a finding regarding the lesion area L1 of the image I1 to be interpreted, which was displayed on the display 20B in step ST1. Here, it is assumed that the finding FS1 shown in F4B has been saved. The finding FS1 is written as follows: "There is a 3 cm mass in S3 of the left lung. The margins are unclear and the shape is irregular. Spicules are observed in some areas, and it is lobulated. A cavity is observed inside, and the cavity wall is thickened. It is in contact with the pleura."
Thereafter, the document creation device 16 stores the radiology report including the inputted findings in the radiology report database 18 .

By repeating the processes of steps ST1 and ST2, multiple radiology reports including findings are stored in the radiology report database 18. It is preferable that the processes of steps ST11 and ST2 are repeated for different subjects.

In this embodiment, the document creation device 16 structures the findings and stores the structured findings in the radiology report database 18. Figure 5 shows an example of an input finding and its structuring result. F5A shown in Figure 5 is the input finding. Here, the finding reads, "A 3 cm solid mass was observed in S4 of the right lung. The border was unclear, the margin was lobulated, and it exhibited spicules. Pleural indentation was also observed. No calcification, cavities, or bronchial radiolucency were observed internally."

F5B in Figure 5 is the structured result of structuring the findings in F5A using natural language processing. As shown in F5B, the structured result is that the findings written in free text are structured as information for each lesion.

Here, the "organ" category is "lung," the "location" category is "right lung" and "S4," the "lesion" category is "mass," the "factuality of lesion" category is "present," the "disease name" and "factuality of disease name" categories are left blank, the "size" category is "longest diameter 3 cm," and the "characteristics" category is "solid," "unclear border," "lobulated (+)," "pleural indentation (+)," "calcification (-)," "cavitation (-)," and "bronchial radiography (-)," with the relationships identified and classified. Note that factuality refers to the possibility of existence.

In this way, multiple findings are structured and stored in the radiology report database 18.

Steps ST3 and beyond are a process for automatically generating new findings based on the findings stored in the image interpretation report database 18. In step ST3 (an example of an "acquisition process"), the image interpretation physician uses the input device 20A of the user terminal 20 to specify an image of the subject to be interpreted from among the medical images captured by the medical image inspection equipment 12 and stored in the medical image database 14.

Step ST3 is a scene in which a medical image of a subject (an example of a "first subject") different from the subject of step ST1 (an example of a "second subject") is interpreted, and the image to be interpreted specified in step ST3 is an image different from the image to be interpreted I1 specified in step ST1. Figure 6 shows the image to be interpreted and its findings. F6A in Figure 6 indicates the image to be interpreted I2 specified in step ST3.

The document creation device 16 reads the image I2 to be interpreted from the medical image database 14 and displays it on the display 20B of the user terminal 20. The document creation device 16 also analyzes the image I2 to be interpreted and extracts the lesion area from the image I2 to be interpreted. The lesion area may be extracted manually by the radiologist using the input device 20A of the user terminal 20. The document creation device 16 also acquires the anatomical location and size of the lesion area. Here, it is assumed that the lesion area L2 shown in F5A has been extracted from the image I2 to be interpreted.

In step ST4 (an example of an "analysis process"), the document creation device 16 performs CAD processing (an example of "analysis") to analyze the characteristics of the image I2 to be interpreted, specified in step ST3, and obtains the results of the characteristic analysis (image features). CAD processing uses a trained model to which a convolutional neural network (CNN) is applied. Here, the document creation device 16 is capable of CAD processing to detect pulmonary nodules and CAD processing to detect liver tumors. It is assumed that the lesion area L2 specified in step ST3 is a pulmonary nodule.

In step ST5 (an example of an "identification process"), the document creation device 16 searches the structured data of multiple findings stored in the radiology report database 18 based on the content of the image I2 to extract past findings that serve as the basis (an example of a "base sentence"). The base findings are extracted from the findings stored in the radiology report database 18 that have a relatively high degree of match with the content of the image I2 to be interpreted. The content of the image I2 to be interpreted includes at least one of the anatomical location, characteristics, size, and factuality of the lesion. Here, the content of the image I2 to be interpreted is the anatomical location and size of the lesion area obtained in step ST3 and the results of the characteristic analysis in step ST4. The document creation device 16 may extract findings by performing a brute-force search of all pre-structured findings stored in the radiology report database 18.

The base finding statement may be extracted using the organ recognition results for the image to be interpreted. For example, if the lungs are the subject of interpretation, the base finding statement may be searched for in the interpretation report for the lungs. The finding statement may be extracted using the lesion recognition results for the image to be interpreted. For example, if the subject of interpretation is a nodule, inflammation, pleural effusion, etc., the base finding statement may be searched for in the corresponding interpretation report.

In this case, since the lesion area L2 has been determined to be a pulmonary nodule in step ST4, the document creation device 16 need only compare structured data of multiple finding sentences related to pulmonary nodules.

The base finding statement may be extracted manually by the radiologist. In this case, the document creation device 16 displays a list of multiple finding statements on the display 20B, and the radiologist can select the finding statement with the highest degree of match from the list using the input device 20A.

Here, we assume that the observation statement FS1 shown in F4B in Figure 4 was extracted as the base.

In step ST6, the document creation device 16 calculates the degree of match and difference between the content of the image I2 to be interpreted and the finding FS1 extracted as the base. Here, it is assumed that the following parts A1, A2, A3, and A5 of the finding FS1 are calculated to match the content of the image I2 to be interpreted, and part A4 is calculated to differ from the content of the image I2 to be interpreted.

A1: "There is a 3cm mass in the S3 region of the left lung."
A2: "The margins are unclear and the shape is irregular."
A3: "Some spicules are observed, and the lesion appears lobulated."
A4: "There is a cavity inside, and the cavity wall is thickened."
A5: "It is in contact with the pleura."

That is, the document creation device 16 has found the following differences (examples of "excess or deficiency with respect to the content").
Difference: "Inside: No cavity, Bronchial radiography: Present"
The degree of agreement and the degree of difference between the content of the image I2 to be interpreted and the findings extracted as the base may be manually extracted by the radiologist.

In step ST7 (an example of a "creation process"), the document creation device 16 generates a finding statement for the lesion area L2 of the image I2 to be interpreted based on the analysis results of step ST4 and the similarity and difference scores of step ST6, and displays it on the display 20B. F6B shown in Figure 6 is the finding statement FS2 generated and displayed in step ST7. The finding statement FS2 is written as follows: "There is a 3 cm mass in S3 of the left lung. The margins are unclear and the shape is irregular. Some spicules are observed, and it is lobulated. There is a cavity inside. It is △△. It is in contact with the pleura."

Here, in the observation statement FS2, the following sections B1, B2, B3, and B5 do not require correction, while section B4 does require correction.

B1: "There is a 3cm mass in the S3 region of the left lung."
B2: The margins are unclear and the shape is irregular.
B3: "Some spicules are observed, and the lesion appears lobulated."
B4: "There is a cavity inside that is ○○. It is △△."
B5: "It's touching the pleura."

That is, sections B1, B2, B3, and B5 in finding statement FS2 are common to sections A1, A2, A3, and A5 in finding statement FS1. The document creation device 16 may display on display 20B sections that require correction (an example of the "scope of correction"), distinguishing them from sections that do not require correction (an example of the "outside the scope of correction"). The document creation device 16 may automatically correct sections that require correction, or may display at least one or more correction options.

Note that corrections include at least one of adding, changing, and deleting. The extent of correction may be determined based on whether the content of the image to be interpreted is insufficient or excessive. The extent of correction may also be determined on a sentence-by-sentence basis.

Additions and changes can be made using text created from a pre-prepared template, or using text created using a trained model. Since the text to be added or changed is only part of the findings and contains little information, it can be corrected using the template method or the trained model method.

Here, the radiologist uses input device 20A to select the "XX, △△" section in B4 and input the appropriate term and sentence. This allows the radiologist to complete the observation statement for image I2 to be interpreted.

The document creation device 16 may also automatically perform the process of adding the following content to the differences extracted in step ST6 to create a new finding statement that corrects the finding statement FS1.

Additional information: "No cavity was found within the above nodule, and bronchial radiolucency was observed."

The findings prepared in this way are written as follows:

"There is a 3cm mass in the S3 region of the left lung. The margins are unclear and the shape is irregular. Some spicules are present, and the mass is lobulated. No cavity is found inside, and bronchial radiolucency is observed. It is in contact with the pleura."

In other words, this finding statement has the same parts A1, A2, A3, and A5 as part of finding statement FS1, but part A4 has been changed to read, "No internal cavities were observed, and a bronchial radiolucency image was observed."

In this way, the document creation device 16 uses all of the content acquired from the image I2 to be interpreted and deletes unnecessary information from the finding statement FS1 to create a finding statement that is neither excessive nor insufficient with the content. Here, finding statements FS1 from different subjects with the same disease are effectively utilized.

In step ST7, multiple findings requiring correction may be displayed on display 20B, allowing the radiologist to select one. When multiple findings requiring correction are displayed, the range of correction is displayed differently for each. The following is an example in which the parts requiring correction are displayed differently for candidates C1, C2, and C3. The parts requiring correction for candidate C1 are XX, for candidate C2 are XX and △△, and for candidate C3 are XX, △△, and □□.

Candidate C1: "A 32mm solid nodule is present in the left lower lobe, S6/10. The margin is XX, but lobulated areas are present."
Candidate C2: "A 32mm ○○ is observed in the left lower lobe S6/10. The margin is △△, but there is a lobulated area."
Candidate C3: "I see a 32mm ○○ in the left lower lobe S6/10. The margin is △△, but I see the □□ part."

The radiologist can select one of the candidates C1, C2, or C3 and correct any parts that need to be corrected to complete the findings.

As described above, the document creation method according to the first embodiment makes it possible to create a first document related to an input image that contains just the right amount of content acquired from the input image. Therefore, even for images that contain a large amount of information that needs to be included in an interpretation report, it is possible to create a high-quality document related to the image.

If the text to be corrected was created based on the radiology report database 18 in which structured findings are stored, the text may be corrected using the structured findings and the actual findings written in natural language.

On the other hand, if the sentence to be corrected was created before the introduction of the radiology report database 18 and is not saved in the radiology report database 18, the sentence to be corrected can be searched for on a word or synonym basis, and the inconsistencies can be extracted and displayed on display 20B.

Second Embodiment
7 is a diagram showing an example of the properties (an example of "content") of a pulmonary nodule analyzed in CAD processing using a trained model. As shown in FIG. 7, the properties include six items: "absorption value,""boundary,""shape,""marginalproperties,""internalproperties," and "relationship with surrounding tissue."

"Absorption values" are classified into "solid," "partially solid," and "ground-glass" types.

"Boundary" is classified as "clear," "slightly clear," "neither clear nor unclear," "slightly unclear," or "clear."

"Shape" is classified as either "near-circular" or "irregular."

"Marginal characteristics" are analyzed for each of "smoothness," "lobulated," "serrated," "spicules," and "straightness." Furthermore, "spicules" are also analyzed for "circumferential presence" and "partial presence."

"Internal characteristics" are analyzed for each of "bronchial radiograph," "calcification," "cavity," and "fat." "Calcification" is also analyzed for size, location, and concentration, while "cavity" is analyzed for specularity, thickness, and smoothness.

"Relationship with surrounding tissues" is analyzed for "pleural contact," "pleural indentation," "pleural thickening," "vascular/bronchial compression," and "vascular/bronchial convergence." If "pleural contact" is present, it is classified as either "extensive" or "only partial."

Figure 8 shows the characterization analysis results for a certain lesion and the accompanying findings. F8A in Figure 8 shows the characterization analysis results in bold. As shown in F8A, the "absorption value" is "solid type," the "boundary" is "clear," and the "shape" is "near-circular."

Furthermore, the "marginal characteristics" are "slightly irregular," "lobulated -," "serrated -," "spicules +" and "present in some areas," and "straight -." The "internal characteristics" are "bronchial radiolucency image -," "calcification -," "cavitation +" and "air-floor image -," "neither (thickness)," "irregular," and "fat -." Furthermore, the "relationship with surrounding tissues" are "pleural contact -," "pleural indentation +," "pleural thickening -," "neither (compression)," and "neither (convergence)."

F8B in Figure 8 is the finding statement FS3 that should be prepared by the radiologist in response to the characteristic analysis results shown in F8A. The finding statement FS3 is written as follows: "A nodule measuring 3.2 cm in long diameter was observed in S1 of the right lung. It is solid and clearly demarcated. The margin is slightly irregular, with some spicules present. A cavity with an irregular margin was observed internally. No calcification or bronchial radiolucency was observed. Pleural indentation was observed."

As such, the findings in FS3 do not include all of the results of the characterization analysis shown in F8A. F8C in Figure 8 bolds the characteristics from the characterization analysis results shown in F8A that should be included in the findings.

As shown in F8C, characteristics that should be described in the findings statement include "solid" for "absorption value," "clear" for "boundary," and "near-circular" for "shape." Additionally, characteristics that should be described in the findings statement include "slightly irregular," "spicules +," and "present in some areas" for "marginal characteristics," "calcification -," "cavity +," and "irregularity" for "internal characteristics," and "pleural indentation +" for "relationship with surrounding tissue."

The characteristics to be described in the findings may be determined using a trained model, prioritization may be rule-based, or they may be determined based on past statistics.

In this way, the document creation device 16 uses all of the information from the attribute analysis results that will be included in the finding statement, and deletes unnecessary information from the finding statement, thereby creating a finding statement that is neither excessive nor insufficient compared to the attribute analysis results.

Figure 9 shows an example of a finding statement from a past radiology report stored in the radiology report database 18, and its structuring results. F9A shown in Figure 9 is finding statement FS4, and F9B shown in Figure 9 is the structuring result of finding statement FS4. Finding statement FS4 states, "A 3.2 cm nodule was observed in S1 of the right lung. It is solid and has a clear border. The border is slightly irregular, with spicules around the entire circumference. A cavity with an irregular border was observed inside. No calcification or bronchial radiolucency was observed. Pleural indentation was observed."

Figure 10 shows an example of a finding statement from a past radiology report stored in the radiology report database 18, and the structured result thereof. F10A shown in Figure 10 is finding statement FS5, and F10B shown in Figure 10 is the structured result of finding statement FS5. Finding statement FS5 states, "A solid nodule measuring 2 cm in major axis was observed in S8 of the left lung. The boundary is somewhat unclear. The margin is somewhat irregular, with spicules observed in some areas."

When comparing the characteristics to be described as finding FS3 with finding FS4 and finding FS5, it can be seen that it is quite similar to finding FS4. In finding FS4, the "spicules" are described as being "all around the circumference," so it is necessary to find a finding that describes that the "spicules" are "present in part," and finding FS5 fits this description.

Therefore, by using finding sentence FS4 as a base and modifying only the "spicules" sentence, i.e., by rewriting it using finding sentence FS5, it is possible to describe all of the desired characteristics.

In this way, multiple base finding sentences may be identified and corrections may be made by combining the identified multiple finding sentences (an example of "multiple base sentences"). A finding sentence containing content that is missing from the first base finding sentence may be identified as the second base finding sentence. The desired finding sentence can be generated by deleting the excess parts of the first finding sentence and extracting the missing parts from the second finding sentence.

This example shows how to modify some of the text, but if you want to delete something, simply delete the unnecessary text. When adding something, the position at which you add it is important. The position at which to add it can be determined based on the category of the content to be added (absorption value, boundary, shape, marginal characteristics, internal characteristics, relationship with surrounding tissue). For example, in the case of "pleural indentation," it is customary to write "relationship with surrounding tissue" last, so it should be added at the end of the findings.

<Others>
The document creation according to this embodiment can be applied to documents other than medical documents and documents related to medical images. For example, when creating a diagnostic document for social infrastructure facilities such as transportation, electricity, gas, and water, an input image can be analyzed to acquire content, a base sentence containing at least a portion of the content can be identified from a database storing multiple documents, and at least a portion of the base sentence can be modified to create a diagnostic document related to the input image that is neither too much nor too little in terms of content.

The technical scope of the present invention is not limited to the scope described in the above embodiments. The configurations and other features of each embodiment can be combined as appropriate within the scope of the present invention.

DESCRIPTION OF SYMBOLS 10...Document creation system 12...Medical image inspection equipment 14...Medical image database 16...Document creation device 16A...Processor 16B...Memory 16C...Communication interface 18...Radiological interpretation report database 20...User terminal 20A...Input device 20B...Multiple displays 20B...Display 22...Network FS1...Findings FS2...Findings FS3...Findings FS4...Findings FS5...Findings I1...Image to be interpreted I2...Image to be interpreted L1...Lesion area L2...Lesion areas ST1 to ST7...Steps of document creation method

Claims (17)

at least one processor;
at least one memory storing instructions for execution by said at least one processor;
Equipped with
The at least one processor
Analyze the input image to obtain the content,
Identifying a base document that includes at least a portion of the content from a database in which a plurality of documents are stored;
modifying at least a portion of the base text to create a first document related to the input image, the first document being equivalent to or incomplete with the content;
determining a range of correction of the base sentence based on an excess or deficiency of the base sentence relative to the content;
Document creation device. The at least one processor
Identifying a plurality of said base sentences;
combining and modifying the plurality of base sentences;
The document creation device according to claim 1 . The at least one processor
determining a range of correction of the base sentence on a sentence-by-sentence basis;
3. The document creation device according to claim 1 . The database stores sentences contained in the plurality of documents in a structured manner.
The document creation device according to any one of claims 1 to 3 . the input image is a medical image of a first subject,
the first document is an image interpretation report;
The document creation device according to any one of claims 1 to 4 . the plurality of documents include an interpretation report for a medical image of a second subject different from the first subject;
The document creation device according to claim 5 . the content includes at least one of an anatomical location, a nature, a size, and a factuality of the lesion;
7. The document creation device according to claim 5 or 6 . The at least one processor
identifying the base sentence using at least one of an organ recognition result and a lesion recognition result of the input image;
The document creation device according to any one of claims 5 to 7 . The modification includes at least one of addition, modification, and deletion.
The document creation device according to any one of claims 1 to 8 . The at least one processor
Adding and/or modifying data using a pre-prepared template and/or a trained model.
The document creation device according to claim 9 . The at least one processor
determining a position to add the content based on the type of content as a range to be corrected in the base sentence ;
11. The document creation device according to claim 9 or 10 . The at least one processor
displaying the correction range of the base sentence on a display, distinguishing it from the range other than the correction range;
The document creation device according to any one of claims 1 to 11 . The at least one processor
prompting a user to specify a region of interest in the input image;
obtaining the content based on the region of interest;
The document creation device according to any one of claims 1 to 12 . The at least one processor
Calculating a degree of match between the content and the plurality of documents;
identifying the base sentence based on the degree of match;
The document creation device according to any one of claims 1 to 13 . the processor uses all of the information of the acquired content and deletes unnecessary information from the base sentence to create the first document so that it is neither too much nor too little compared to the content;
The document creation device according to any one of claims 1 to 14 . At least one processor
an acquisition step of acquiring an input image;
an analysis step of analyzing the input image to obtain content;
an identifying step of identifying a base document including at least a portion of the content from a database in which a plurality of documents are stored;
a creating step of modifying at least a portion of the base text to create a first document related to the input image, the first document being just the same as the content;
Run
The creating step determines a range of correction of the base sentence based on an excess or deficiency of the base sentence relative to the content.
How to create documentation . A program that causes a computer to execute the document creation method according to claim 16 .