WO2019176407A1

WO2019176407A1 - Learning assisting device, learning assisting method, learning assisting program, region-of-interest discriminating device, region-of-interest discriminating method, region-of-interest discriminating program, and learned model

Info

Publication number: WO2019176407A1
Application number: PCT/JP2019/004771
Authority: WO
Inventors: 晶路一ノ瀬; 佳児中村; 嘉郎北村
Original assignee: 富士フイルム株式会社
Priority date: 2018-03-16
Filing date: 2019-02-12
Publication date: 2019-09-19
Also published as: US11468659B2; JP7080304B2; JPWO2019176407A1; US20200387729A1

Abstract

Provided are a learning assisting device, a learning assisting method, a learning assisting program, a region-of-interest discriminating device, a region-of-interest discriminating method, a region-of-interest discriminating program and a learned model for easily acquiring the teacher data required for learning in a medical field from radiogram interpretation reports and generating a discriminant model. A learning assisting device 18 is composed of an acquisition unit 26, a registration unit 27, a storage device 28, a learning unit 29, and a control unit 31. The acquisition unit 26 analyzes a radiogram interpretation report 23 and thereby acquires an image of a region-of-interest and the name of the region of interest. The registration unit 27 registers teacher data comprising the image of the region-of-interest and the name of the region of interest acquired by the acquisition unit 26 in the storage device 28. The learning unit 29 performs, using a plurality of teacher data 33 registered in the storage device 28, learning for generating a discriminant model 34 that outputs the image of the region-of-interest and the name of the region of interest for the input of an inspection image 22 of a radiogram interpretation report 23.

Description

Learning support device, learning support method, learning support program, region of interest discrimination device, region of interest discrimination method, region of interest discrimination program, and learned model

The present invention relates to a learning support device, a learning support method, a learning support program, a region of interest discrimination device, a region of interest discrimination method, a region of interest discrimination program, and a learned model.

In the medical field, a region name in an anatomical organ name is specified and a disease name is diagnosed. For example, in Patent Document 1, a plurality of regions are specified for a three-dimensional medical image such as a brain (region segmentation). In addition, as segmentation of an area | region, as shown to the nonpatent literature 1, the semantic segmentation which grasps | ascertains an image at a pixel level may be used, for example.

JP 2016-202904 A

After learning the name of the region of interest such as an anatomical organ name, area name, disease name, and disease state and the image of the region of interest using machine learning such as deep learning, the image of the region of interest is extracted from the image of the region of interest. The name is being determined. In order to accurately determine the name of the region of interest, in deep learning, learning with a large amount of high-quality teacher data in line with the purpose is indispensable. It is considered that teacher data is acquired from a large amount of data such as an interpretation report stored in a medical information system such as PACS (Picture Archiving and Communication System). However, it is not reasonable to manually determine useful data for teacher data from a large amount of data.

When generating a discrimination model for determining the name of a region of interest from an image of the region of interest, the present invention can easily acquire a large amount of teacher data including the image of the region of interest and the name of the region of interest. It is an object to provide a learning support device, a learning support method, a learning support program, a region of interest discrimination device, a region of interest discrimination method, a region of interest discrimination program, and a learned model.

In order to achieve the above object, the learning support apparatus of the present invention includes a storage unit, an acquisition unit, a registration unit, and a learning unit. The storage unit stores an interpretation report including image and character information. The acquisition unit analyzes the image interpretation report to acquire the image of the region of interest included in the image and the name of the region of interest included in the character information. The registration unit registers teacher data including the image of the region of interest acquired by the acquisition unit and the name of the region of interest. The learning unit uses a plurality of teacher data registered in the registration unit to perform learning for generating a discrimination model that outputs an image of a region of interest and a name of the region of interest in response to input of an image of an interpretation report.

It is preferable that the acquisition unit acquires the position information of the region of interest by analyzing the interpretation report.

The image interpretation report has link information that associates the finding information included in the character information with the position information of the region of interest included in the image, and the acquisition unit preferably acquires the position information of the region of interest from the link information. .

It is preferable that the interpretation report has annotation information attached around the region of interest, and the acquisition unit acquires the position information of the region of interest from the annotation information.

It is preferable that the acquisition unit acquires the image of the region of interest and the position information of the region of interest by the region of interest determination unit that determines the region of interest from the image of the interpretation report.

When the learning unit acquires the names of a plurality of regions of interest including a first name and a second name different from the first name by analyzing the interpretation report, the learning unit uses the first name as the region of interest. It is preferable to perform learning using the first teacher data including the name and the second teacher data including the second name as the name of the region of interest.

When the names of a plurality of regions of interest are acquired by analyzing the interpretation report, the learning unit preferably performs learning using region position information related to the position information of the region of interest in addition to the teacher data. .

The acquisition unit refers to the hierarchical structure database that stores the names of the superordinate concepts or subordinate concepts corresponding to the superordinate concepts or subordinate concepts with respect to the name of the region of interest, and determines the names of the superordinate concepts or subordinate concepts from the names of the regions of interest. It is preferable that the learning unit performs learning using the teacher data including the image of the region of interest and the name of the superordinate concept or the subordinate concept.

The acquisition unit refers to a similar name database in which a plurality of similar names similar to each other for the name of the region of interest are stored in advance, determines a representative name from the plurality of similar names, and the learning unit It is preferable to perform learning using teacher data including an image of a region and a representative name.

The acquisition unit newly acquires the image of the region of interest and the name of the region of interest when the interpretation report is newly stored in the storage unit, and the registration unit acquires the image of the region of interest newly acquired by the acquisition unit and the interest When the new teacher data is registered, the learning unit is updated by re-learning using a plurality of teacher data including the new teacher data when the new teacher data is registered. It is preferable to generate a discriminant model.

The acquisition unit, when a newly stored interpretation report and a past interpretation report for the same patient are stored in the storage unit, an image of the past interpretation report and a newly stored interpretation report By aligning the image with the newly acquired image of the region of interest, the image of the region of interest included in the image of the past interpretation report is acquired, and the registration unit obtains the region of interest acquired based on the past image interpretation report. The past image teacher data consisting of the image and the name of the newly acquired region of interest is registered, and when the past image teacher data is registered, the learning unit uses a plurality of teacher data including the past image teacher data. It is preferable to generate an updated discrimination model by learning again.

The interpretation report preferably includes an electronic medical record.

It is preferable that the acquisition unit acquires an anatomical organ name, a zone name, a disease name, and a medical condition as the name of the region of interest. More specifically, the area name is preferably an area name of the lung, liver, or brain.

The learning support method of the present invention includes a storage unit, an acquisition unit, a registration unit, and a learning support method of a learning support device including a learning unit, and the acquisition unit includes the image by analyzing the interpretation report. Registration for registering teacher data consisting of the image of the region of interest and the name of the region of interest acquired by the acquisition unit, and an acquisition step of acquiring the region of interest image and the name of the region of interest included in the character information A learning step for performing learning to generate a discrimination model that outputs an image of a region of interest and a name of the region of interest with respect to input of an image of an interpretation report using a plurality of teacher data.

The learning support method of the present invention is a learning support program for causing a computer to function as a storage unit, an acquisition unit, a registration unit, and a learning unit. Another learning support apparatus of the present invention is a computer having a memory and a processor, which stores an image interpretation report including an image and character information, and analyzes the image interpretation report to analyze a region of interest included in the image. The image and the name of the region of interest included in the text information are acquired, the teacher data consisting of the acquired image of the region of interest and the name of the region of interest is registered, and the image of the interpretation report is obtained using a plurality of registered teacher data And a processor that performs learning to generate a discrimination model that outputs an image of the region of interest and a name of the region of interest.

The region-of-interest discrimination apparatus of the present invention includes a storage unit, an acquisition unit, a registration unit, a learning unit, and a discrimination unit. When the image of the interpretation report is input, the determination unit determines the image of the region of interest and the name of the region of interest using the determination model.

The region-of-interest discrimination method of the region-of-interest discrimination apparatus of the present invention includes an acquisition step, a registration step, and a region-of-interest discrimination device including a storage unit, an acquisition unit, a registration unit, a learning unit, and a discrimination unit. The learning step includes a determination step of determining the image of the region of interest and the name of the region of interest using a determination model when the image of the interpretation report is input.

The region-of-interest discriminating program of the region-of-interest discriminating apparatus according to the present invention uses a discrimination model when a computer, a storage unit, an acquisition unit, a registration unit, a learning unit, and an image of an interpretation report are input. It functions as a discriminating unit that discriminates the image of the region and the name of the region of interest.

The learned model of the present invention includes a computer, a storage unit, an acquisition unit, a registration unit, a learning unit, and an image of an interpretation report. It functions as a discriminating unit for discriminating the name of each.

According to the present invention, it is possible to provide a learning support apparatus, a learning support method, and a learning support program for easily acquiring teacher data necessary for learning in the medical field from an interpretation report and generating a discrimination model.

It is a figure showing schematic structure of a medical information system. It is a figure showing schematic structure of the learning assistance apparatus of this invention. It is explanatory drawing which represented each area of the lung typically. It is explanatory drawing explaining the method of acquiring teacher data from an interpretation report and producing | generating learning data. It is explanatory drawing explaining the method to acquire the name of a region of interest and the positional information on a region of interest from an interpretation report. It is a flowchart for demonstrating the effect | action of the learning assistance apparatus and region-of-interest discrimination | determination apparatus of this invention. It is explanatory drawing explaining the modification which acquires the positional information on a region of interest from annotation information. It is explanatory drawing explaining the modification which acquires the positional information on a region of interest using area | region positional information. It is explanatory drawing explaining the modification which acquires the positional information on a region of interest with reference to a similar name database. It is explanatory drawing explaining the modification which acquires the positional information on a region of interest from the past interpretation report. It is explanatory drawing which represented each area of the liver typically.

In FIG. 1, the medical information system 2 includes a modality 11, an interpretation doctor terminal 12, a clinician terminal 13, an image management server 14, an image database 15, an interpretation report server 16, an interpretation report database 17, A learning support device 18 and an order management server 19 are included. The modality 11, the interpretation doctor terminal 12, the clinician terminal 13, the image management server 14, the image database 15, the interpretation report server 16, the interpretation report database 17, the learning support device 18, and the order management server 19 are laid in a medical facility. Are interconnected in a communicable state via a network 21 which is a local network such as a LAN (Local Area Network). When the radiogram interpretation terminal 12 is installed in another hospital or clinic, the local networks of each hospital may be connected to each other via the Internet or a dedicated line.

This medical information system 2 is based on examination orders from clinicians using a well-known ordering system, imaging and storage of a region to be examined of a subject, interpretation of images taken by an interpreting physician, and creation of an interpretation report And a system for browsing an interpretation report and a detailed observation of an image to be interpreted by a requesting clinician.

An application program for causing each device to function as a component of the medical information system 2 is installed. The application program may be installed from a recording medium such as a CD-ROM or after being downloaded from a storage device of a server connected via a network such as the Internet.

The modality 11 shoots a region to be examined of a subject to generate an inspection image 22 representing the region, and adds incidental information defined by the DICOM (Digital Imaging and Communication Communication in Medical) standard to the image. Output device. The modality 11 includes a device that captures an image having three-dimensional organ information as the examination image 22. Specific examples include a CT (Computed Tomography) apparatus 11A, an MRI (Magnetic Resonance Imaging) apparatus 11B, a PET (Positron Emission Tomography) apparatus (not shown), an ultrasonic apparatus (not shown). Or a CR (Computed Radiography) apparatus 11C using a flat panel X-ray detector (FPD: flat panel detector). In the following, a lung is exemplified as an organ to be inspected that is imaged by the modality 11 and generates an image.

The radiology doctor terminal 12 is a computer used by radiology radiology doctors to interpret images and create interpretation reports. A CPU (Central Processing Unit), a main storage device, an auxiliary storage device, an input / output interface, a communication interface, an input A known hardware configuration such as a device, a display device, and a data bus is provided, a known operation system is installed, and the display device has one or more high-definition displays. In this interpretation doctor terminal 12, an image transmission request to the image management server 14, display of an image received from the image management server 14, automatic detection and highlighting of a portion that appears to be a lesion in the image, and creation and display of an interpretation report 23 Each process is performed by executing a software program for each process. The interpretation doctor terminal 12 transfers the generated interpretation report 23 to the interpretation report server 16 via the network 21 and requests registration of the interpretation report in the interpretation report database 17.

The clinician terminal 13 is a computer used by doctors in the clinic for detailed observation of images, reading of interpretation reports, browsing and input of electronic medical records, and the like, CPU, main storage device, auxiliary storage device, input / output It has a well-known hardware configuration such as an interface, a communication interface, an input device, a display device, and a data bus. A well-known operation system is installed, and the display device has one or more high-definition displays. Yes. In this clinician terminal 13, an image browsing request to the image management server 14, display of an image received from the image management server 14, automatic detection or highlighting of a portion that appears to be a lesion in the image, and interpretation report to the interpretation report server 16. Each process such as a browsing request and display of an interpretation report received from the interpretation report server 16 is performed by executing a software program for each process.

The image management server 14 incorporates a software program that provides a database management system (DBMS) function in a general-purpose computer. The image management server 14 includes a large-capacity storage in which the image database 15 is configured. This storage may be a large-capacity hard disk device connected to the image management server 14 through a data bus, or connected to a NAS (Network Attached Storage) or a SAN (Storage Area Network) connected to the network 21. It may be a disk device.

In the image database 15, examination images (image data) 22 obtained by photographing a plurality of patients with the modality 11 and accompanying information are registered. The incidental information includes, for example, an image ID (identification) for identifying individual images, a patient ID for identifying a subject, an examination ID for identifying examinations, and a unique ID assigned to each examination image 22 ( (UID: unique identification), examination date when the examination image 22 was generated, examination time, type of modality 11 used in the examination to obtain the examination image, patient information such as patient name, age and gender, examination Information such as a part (imaging part), an imaging condition (whether or not a contrast medium is used or a radiation dose), and a series number when a plurality of tomographic images are acquired in one examination is included.

When the image management server 14 receives a browsing request from the interpretation doctor terminal 12 via the network 21, the image management server 14 searches for the examination image registered in the image database 15 and reads the extracted examination image as the interpretation of the request source. Transmit to the medical terminal 12.

When the interpretation report server 16 receives a registration request for the interpretation report 23 from the interpretation doctor terminal 12 when a software program for providing a database management system (DataBase Management System: DBMS) function is incorporated in a general-purpose computer, the interpretation report is received. 23 is prepared in a database format and registered in the interpretation report database 17.

The interpretation report database 17 includes, for example, an image ID for identifying an interpretation target image or a representative image, an interpretation doctor ID for identifying an interpretation doctor who has performed interpretation, a lesion name, lesion position information, findings, and belief in findings. An interpretation report 23 in which information such as the degree is recorded is registered.

The order management server 19 receives the examination order issued by the clinician terminal 13 and manages the accepted examination order. The examination order includes, for example, an order ID for identifying each examination order, a terminal ID or a clinician ID of the clinician terminal 13 that has issued the examination order, and a patient to be imaged by the examination order (hereinafter, referred to as “order ID”). Patient ID of the subject), examination purpose such as follow-up observation, imaging parts such as head and chest, orientation such as lying on the back and lying down. The radiology examination engineer confirms the contents of the examination order with the order management server 19, sets the imaging condition corresponding to the confirmed examination order in the modality 11, and takes a medical image.

Next, the region of interest discrimination device 25 of the present invention will be described with reference to FIGS. The region-of-interest discriminating device 25 is incorporated in, for example, the interpretation doctor terminal 12 and discriminates the image of each area of an organ or a part that appears to be a lesion in the examination image 22 and the name of each area of the organ and the region of interest. The interpretation doctor terminal 12 performs color-coded display of each area of the organ or highlighting of the region of interest based on the determination result.

FIG. 2 is a functional block of the learning support device 18 constituting the region of interest discrimination device 25. The region-of-interest discrimination device 25 of the present invention is used together with the learning support device 18 connected to the network 21 and the interpretation report database 17 (see FIG. 1). The interpretation report database 17 functions as a storage unit of the present invention.

The learning support device 18 is composed of a general-purpose computer, and includes a main storage device such as a CPU, HDD (hard disc drive), and SSD (solid state drive), an auxiliary storage device, an input / output interface, a communication interface, an input device, and a display device. In addition, a known hardware configuration such as a data bus is provided, and a known operation system is installed. Further, data is transmitted / received to / from the image database 15 and the interpretation report database 17 connected to the network 21 via the communication interface.

In this embodiment, the learning support device 18 is provided independently of the interpretation doctor terminal 12, the clinician terminal 13, the image management server 14, and the interpretation report server 16, but is not limited thereto. You may provide in any one of these servers or a terminal.

2, the learning support device 18 includes an acquisition unit 26, a registration unit 27, a storage device 28, a learning unit 29, and a control unit 31.

The obtaining unit 26 analyzes the image interpretation report 23 to obtain the image of the region of interest included in the inspection image 22 of the image interpretation report 23 and the name of the region of interest included in the character information of the image interpretation report 23. In the present embodiment, the name of the lung area is acquired as the name of the region of interest.

As shown in FIG. 3, the lung as an anatomical organ is divided into lung lobes or lung areas. The right lung RL is divided into right upper lobe RUL (rightｒupper love), right middle lobe RML (right middle love), and right lower lobe RLL (rightＬlowerｒlove) as the lung lobe, and the left lung LL is the upper left lobe as the lung lobe It is divided into LUL (left upper love) and left lower leaf LLL (left lower love).

The right upper lobe RUL is the right pulmonary apex S1 (hereinafter abbreviated as right S1. The same applies to the following lung areas), the right rear upper lobe S2, and the right front upper lobe. It is divided into S3. The right middle lobe RML is divided into a right outer middle lobe section S4 and a right inner middle lobe section S5 as lung sections. The lower right lobe RLL is divided into upper right lobe / lower lobe section S6, right inner lung base section S7, right front lung base section S8, right outer lung base section S9, and right rear lung base section S10 as lung sections.

Further, the left upper lobe LUL is divided into a left posterior apex segment S1 + 2, a left anterior upper lobe segment S3, a left upper tongue segment S4, and a left lower tongue segment S5 as lung segments. The lower left lobe LLL is divided into a left upper lobe / lower lobe segment S6, a left anterior lung floor segment S8, a left outer lung floor segment S9, and a left rear lung floor segment S10.

The control unit 31 controls the processing flow of the acquisition unit 26, the registration unit 27, and the learning unit 29. A process in which the acquisition unit 26 acquires the image of the region of interest included in the examination image 22 of the interpretation report 23 and the name of the region of interest included in the character information of the interpretation report 23 will be described with reference to FIGS. 4 and 5. .

The interpretation report 23 includes an inspection image 22 to be interpreted, supplementary information 23A, finding information 23B, and link information 23C. The accompanying information 23A is character information attached to the examination image 22 to be interpreted, such as a patient ID, examination ID, and examination date. The finding information 23 B is obtained by editing the findings of the interpreting doctor who has interpreted the examination image 22 to be interpreted, and is character information input by the interpreting doctor terminal 12. The link information 23C is used when the interpretation report 23 is displayed on the display as will be described later, and is link information that associates the finding information 23B with the position information of the region of interest included in the examination image 22.

FIG. 5 is an example of a display screen 32 when the interpretation report 23 is displayed on the display of the interpretation doctor terminal 12 or the clinician terminal 13. In this example, in order from the top, an auxiliary information display field 32A in which the auxiliary information 23A is displayed, an observation field 32B in which the observation information 23B is displayed, and an image display in which thumbnail images of the examination image 22 to be interpreted are displayed. Column 32C.

In the example shown in FIG. 5, the finding information 23 B is displayed in the finding column 32 B, “a clear lung nodule with φ = 30 mm is recognized in the right S 1. In this case, “pulmonary nodule” indicating the lesion name of the region of interest, “right S1” which is the lung area name of the region of interest, and “φ = 30 mm” indicating the diameter dimension of the region of interest are highlighted.

Further, in the present embodiment, the interpretation report 23 includes link information 23C. This link information 23C is included in the wording of “pulmonary nodule” indicating the lesion name and the examination image 22 in the observation information 23B. Associate with the location information of the region of interest included. Specifically, the position information of the region of interest is the coordinates of the region of interest in the inspection image 22 and a range centered on the coordinates. By having the link information 23C, when the interpreting doctor terminal 12 or the clinician terminal 13 is operated and “pulmonary nodule” highlighted in the display screen 32 is selected, it is based on the position information of the related region of interest. The image 22A of the region of interest can be displayed on the display. In the example shown in FIG. 5, when “pulmonary nodule” is selected, a region of interest (a portion surrounded by a circle) having a diameter dimension of φ = 30 mm around the coordinates (X, Y, Z) in the examination image 22. An image 32D enlarged by centering on the position information of the region of interest included in the link information 23C is displayed.

In the example shown in FIGS. 4 and 5, the acquisition unit 26 analyzes the characters of the finding information 23 B of the interpretation report 23, acquires “right S 1” indicating the name of the lung region as the name of the region of interest, and interprets the interpretation report The position information including the coordinates (X, Y, Z) of the region of interest in the examination image 22 and “φ = 30 mm” indicating the diameter dimension of the region of interest is acquired from the link information 23C of 23.

The registration unit 27 registers the teacher data 33 including the region of interest image 22 A acquired by the acquisition unit 26 and the name of the region of interest in the storage device 28. In the example shown in FIG. 4, teacher data 33 including an image 22A of the region of interest, “Right S1” that is the name of the lung area, coordinates (X, Y, Z) that are position information, and a diameter φ = 30 mm are included. sign up. The storage device 28 may be a part of a storage device such as an HDD (hard disc drive) or SSD (solid state drive) provided in the learning support device 18, or may be a storage device connected via the network 21. Good.

Through the process as described above, the registration unit 27 registers a plurality of teacher data 33 and, for example, until a predetermined number of teacher data 33 is registered in the storage device 28 for machine learning described later, or an interpretation report. Until the teacher data 33 based on all the interpretation reports 23 registered in the database 17 is registered, the acquisition of the image 22A of the region of interest and the name of the region of interest from the interpretation report 23 and the registration of the teacher data 33 are repeated.

The learning unit 29 uses a plurality of teacher data 33 registered in the storage device 28 and outputs a discrimination model 34 that outputs an image 22A of the region of interest and the name of the region of interest in response to the input of the inspection image 22 of the interpretation report 23. Learning to generate. Specifically, the discrimination model 34 is generated using a machine learning method such as deep learning. For example, a plurality of teacher data 33 are input, and the machine learning algorithm is made to learn the relationship between the position information of the region of interest and the feature amount (pixel value, etc.) of each voxel. Specifically, machine learning is performed so that the error between the position information obtained when a feature amount around the region of interest among the feature amounts of each voxel is input and the position information of the region of interest in the teacher data 33 is minimized. The learning weighting coefficient used in the algorithm is updated.

As described above, the discrimination model 34 generated by the learning unit 29 is transmitted to the region of interest discrimination device 25. When the inspection image 22 such as the interpretation report 23 is input, the determination unit 35 of the region-of-interest determination device 25 outputs the image 22A of the region of interest in the inspection image 22 and the name of the region of interest using the determination model 34. .

The discrimination model 34 includes a weighting coefficient determined by using the above-described machine learning method. When the inspection image 22 is input, the discrimination model 34 is used for discriminating the region of interest image 22A and the region of interest name. It is done.

The discriminating unit 35 discriminates the image of the region of interest and the name of the region of interest in the examination image using the discrimination model 34. The discriminating unit 35 displays, for example, the discriminated region-of-interest image 36 and the region-of-interest name 37 on the display of the region-of-interest discriminating apparatus 25.

Hereinafter, the process in which the discrimination model 34 is generated from the interpretation report 23 having the lung examination image 22 and the image of the region of interest and the name of the region of interest are discriminated will be described with reference to the flowchart of FIG.

The learning support device 18 first reads the interpretation report 23 regarding the lung disease from the interpretation report database 17 (S101).

Next, the acquisition unit 26 acquires the image of the region of interest included in the examination image 22 of the interpretation report 23 and the name of the region of interest included in the character information of the interpretation report 23 (S102), and the registration unit 27 acquires The teacher data 33 including the image of the region of interest acquired by the unit 26 and the name of the region of interest is registered in the storage device 28 (S103). As described above, when the interpretation report 23 includes the observation information 23B and link information 23C that “a clear lung nodule of φ = 30 mm is recognized in the right S1”, the lung section name of the region of interest is used. A certain “right S1”, the position information of the region of interest included in the link information 23C is acquired.

Then, when the teacher data 33 for machine learning or the like is registered in the storage device 28, the learning unit 29 performs learning for generating the discrimination model 34 using a plurality of registered teacher data 33 (S104). ). The generated discrimination model 34 is transmitted to the region of interest discrimination device 25.

When a new examination image 22 is input to the region of interest discrimination device 25 (YES in S105), the discrimination unit 35 outputs the image of the region of interest and the name of the region of interest in the examination image 22 using the discrimination model 34. . For example, color-coded display of lung areas or name display of lung areas is performed (S106).

As described above, the learning support device 18 performs learning based on the teacher data including the image of the region of interest acquired from the image interpretation report 23 and the name of the region of interest, and thus has been conventionally used in medical information systems. Teacher data can be obtained from the interpretation report, and teacher data necessary for learning can be easily obtained. Furthermore, since the discrimination model 34 can be generated using the teacher data and the discrimination model 34 is generated based on the interpretation report in which correct information is recorded, the discrimination accuracy can be easily improved.

In the present embodiment, the region-of-interest discriminating device 25 capable of outputting “right S1” as the region of interest is illustrated, but the present invention is not limited to this, and a plurality of regions of interest can be discriminated simultaneously. That is, a discriminant model that can discriminate a plurality of regions of interest at the same time, such as “right S1” for a certain range of voxels in the input inspection image and “right S2” for another range of voxels. You may create it. Further, such a discrimination model may be applied to the region of interest discrimination device so that a plurality of regions of interest can be discriminated simultaneously.

In the above-described embodiment, the example in which the position information of the region of interest is acquired from the link information 23C included in the interpretation report 23 has been described. However, the present invention is not limited to this, and the interpretation report 23 has annotation information. In this case, the acquisition unit 26 may acquire the position information of the region of interest from the annotation information. The annotation information is information added as annotation to image data or the like. In the example shown in FIG. 7, an arrow 38 as annotation information is included in the inspection image 22, and the arrow 38 is around the region of interest. It is attached to. In this case, the acquisition unit 26 acquires, for example, the coordinates of the tip position of the arrow 38 in the inspection image 22 as position information.

The acquisition unit 26 may include a region of interest determination unit, and the region of interest determination unit may determine the region of interest from the inspection image 22 of the interpretation report 23. In this case, the region-of-interest discriminating unit has the same configuration as the region-of-interest discriminator 25 of the above-described embodiment. Is registered and registered as new teacher data.

As another modification, when the acquisition unit 26 analyzes the image interpretation report 23 and acquires the names of a plurality of regions of interest including a first name and a second name different from the first name. The learning unit 29 performs learning using the first teacher data including the first name as the name of the region of interest and the second teacher data including the second name as the name of the region of interest. In this case, for example, when two names “right S1” and “right S2” are acquired from the text information “Lung nodule is recognized in right S1 / right S2... The registration unit 27 registers the teacher data 33 using the name “right S1” and the teacher data 33 using the name “right S2”.

The learning unit 29 receives an error in the position information of the region of interest when the teacher data 33 using the name “right S1” is input and the region of interest when the teacher data 33 using the name “right S2” is input. The weighting coefficient for learning used in the machine learning algorithm is updated so that both of the position information errors of the machine learning algorithm are reduced. Thereby, the accuracy of learning can be improved.

In addition, when the acquisition unit 26 analyzes the interpretation report 23 and acquires names of a plurality of regions of interest, the learning unit 29 obtains region position information related to the region information of the region of interest in addition to the teacher data 33. It is good also as a structure which uses and learns. In this case, as illustrated in FIG. 8, the learning unit 29 stores in advance region position information, for example, that the right S4 exists on the side of the lung and the right S5 exists on the inside as the region position information. When the acquisition unit 26 analyzes the interpretation report 23 and acquires two names “right S4” and “right S5”, the learning unit 29 uses the region position information to acquire the lung outer surface of the region of interest. Half of the direction is learned as “right S4”, and half of the lung inner surface direction is learned as “S5”.

The acquisition unit 26 refers to the hierarchical structure database to acquire the name of the superordinate concept or the subordinate concept, and the learning unit 29 uses the teacher data 33 including the image of the region of interest and the name of the superordinate concept or the subordinate concept. Thus, the learning may be configured. The hierarchical structure database may be, for example, a part of a storage device provided in the learning support device 18 or a database connected via the network 21. The hierarchical structure database stores the names of the superordinate concepts or subordinate concepts corresponding to the superordinate concepts or subordinate concepts for the name of the region of interest. For example, for the lung, the classification of lung lobes and lung areas is stored in the hierarchical structure database, and the hierarchical structure of right lung> upper right lobe> right S1 is stored in order from the higher concept to the lower concept.

In this case, for example, when the acquisition unit 26 acquires the name “right S1”, the learning unit 29 refers to the hierarchical structure database and acquires “upper right lobe” and “right lung”, which are the names of the upper concepts. In addition to “Right S1”, the names “upper right lobe” and “right lung” are also learned as teacher data 33 in the same manner as in the above embodiment.

The acquisition unit 26 refers to the similar name database 39 to determine a representative name from a plurality of similar names, and the learning unit 29 receives the teacher data 33 including the image of the region of interest and the representative name. It is good also as a structure which uses and learns. As illustrated in FIG. 9, the similar name database 39 may be, for example, a part of the storage device provided in the learning support device 18 or a database connected via the network 21. The similar name database 39 is a database in which a plurality of similar names that are similar to each other with respect to the name of the region of interest are stored in advance. For example, the similar name database 39 includes a plurality of similar names such as area names; The name is stored. In this example, the acquisition unit 26 determines “right S3” as the representative name, and the learning unit 29 uses the teacher data 33 including the image of the region of interest and “right S3” as the name. Learning similar to the embodiment is performed.

In addition, when the interpretation report 23 is newly stored in the interpretation report database 17, the acquisition unit 26 newly acquires the image of the region of interest and the name of the region of interest, and the registration unit 27 newly uses the acquisition unit 26. New teacher data 33 including the acquired image of the region of interest and the name of the region of interest is registered, and the learning unit 29 includes the teacher data including the new teacher data 33 when the new teacher data 33 is registered. It is preferable to generate the updated discriminant model 34 by re-learning using a plurality of. Thereby, the discrimination accuracy of the discrimination model 34 can be improved.

In the above embodiment, the interpretation report 23 created by the interpretation doctor using the interpretation doctor terminal 12 or the like is taken as an example, but an image and character information similar to the interpretation report 23 of the above embodiment, such as an electronic medical record, are included. If it is.

Further, in the above embodiment, information about the region of interest is acquired from the interpretation report and registered as teacher data. However, the present invention is not limited to this, and past interpretation reports that do not include information about the region of interest are stored. If this is the case, teacher data may be registered using this past interpretation report and a newly stored interpretation report. In this case, as illustrated in FIG. 10, the acquisition unit 26 stores the previously stored interpretation report 23 and the past interpretation report 43 targeting the same patient in the interpretation report database 17. The image of the past interpretation report 43, the newly stored examination image 22 of the interpretation report 23, and the newly acquired image 22A of the region of interest are aligned by image processing or the like. An image 42B of the region of interest included in the inspection image 42 is acquired. In this case, for example, the acquisition unit 26 determines that the interpretation reports 23 and 43 are for the same patient from the patient IDs included in the

incidental information

23A and 43A.

The registration unit 27 registers the past image teacher data 44 including the region of interest image 42B acquired based on the past interpretation report 43 and the newly acquired region of interest name, and the learning unit 29 stores the past image teacher data. When 44 is registered, an updated discrimination model 34 is generated by re-learning using a plurality of teacher data 33 including the past image teacher data 44. Thereby, since the number of teacher data can be further increased, the discrimination accuracy of the discrimination model 34 can be improved.

In addition, the acquisition unit 26 uses the anatomical organ name as the name of the region of interest and the lung as the area name as an example. However, the acquisition unit 26 is not limited thereto, and as illustrated in FIG. It may be applied to leaves S1, left leaves S2, S3, S4, right leaves S5, S6, S7, etc.), or may be applied to each area of the brain. Alternatively, the acquisition unit 26 may acquire a disease name and a medical condition as the region of interest.

In the above embodiment, the learning support device 18 is provided separately from the region-of-interest discriminating device 25. However, the learning support device 18 may be integrated into the region-of-interest discriminating device 25.

In each of the above embodiments, the hardware structure of the processing unit (processing unit) that executes various processes such as the acquisition unit 26, the registration unit 27, the learning unit 29, and the determination unit 35 is, for example, as described above. The CPU is a general-purpose processor that executes programs and functions as various processing units. Various processors may be used instead of all or part of the functions realized by the CPU. More specifically, the hardware structure of these various processors is an electric circuit (circuitry) in which circuit elements such as semiconductor elements are combined. The hardware structure of the storage unit is a storage device such as a hard disk drive (HDD) or a solid state drive (SSD).

It is also possible to appropriately combine the various embodiments and various modifications described above. In addition to the program, the present invention extends to a storage medium that stores the program.

Note that the present invention can also be applied to fields other than the medical field using interpretation reports. Any specific information including character information and image information may be used. For example, the information may be applied to information using a photographic image (image data) including character information or information using SNS (social networking service) information.
[Additional Item 1]
A storage unit for storing specific information including image and character information;
By acquiring the image of the region of interest included in the image and the name of the region of interest included in the character information by analyzing the text information,
A registration unit for registering teacher data including the image of the region of interest acquired by the acquisition unit and the name of the region of interest;
A learning unit for generating a discrimination model that outputs the image of the region of interest and the name of the region of interest in response to the input of the image of the specific information using the teacher data registered in the registration unit; Learning support device.
[Additional Item 2]
The specific information includes character information included in a photographic image.
[Additional Item 3]
The specific information is SNS information.

DESCRIPTION OF SYMBOLS 2 Medical information system 11 Modality

11A CT apparatus

11B MRI apparatus

11C CR apparatus 12 Interpretation medical terminal 13 Clinician terminal 14 Image management server 15 Image database 16 Interpretation report server 17 Interpretation report database 18 Learning support apparatus 19 Order management server 21 Network 22 Inspection image 22A Image 23 Interpretation report 23A Additional information

23B Finding information

23C Link information 25 Region of interest discrimination device 26 Acquisition unit 27 Registration unit 28 Storage device 29 Learning unit 31 Control unit 32 Display screen 32A Additional information display field 32B Observation field 32C Image display Field 32D Image 33 Teacher data 34 Discrimination model 35 Discrimination unit 36 Image 37 Name 38 Arrow 39 Similar name database 42 Inspection image 42B Image 43 Interpretation report 43A Attached information 44 Past image teacher data

Claims

A storage unit storing an interpretation report including image and text information;
By acquiring the image of the region of interest included in the image and the name of the region of interest included in the character information by analyzing the interpretation report,
A registration unit for registering teacher data including the image of the region of interest acquired by the acquisition unit and the name of the region of interest;
Learning to generate a discriminant model that outputs the image of the region of interest and the name of the region of interest with respect to the input of the image of the interpretation report using a plurality of the teacher data registered in the registration unit A learning support apparatus comprising a learning unit.
The learning support device according to claim 1, wherein the acquisition unit acquires position information of the region of interest by analyzing the interpretation report.
The image interpretation report has link information that associates the finding information included in the character information with the position information of the region of interest included in the image,
The learning support device according to claim 2, wherein the acquisition unit acquires position information of the region of interest from the link information.
The interpretation report has annotation information attached around the region of interest,
The learning support device according to claim 2, wherein the acquisition unit acquires position information of the region of interest from the annotation information.
3. The learning support apparatus according to claim 2, wherein the acquisition unit acquires an image of the region of interest and position information of the region of interest by a region of interest determination unit that determines the region of interest from the image of the interpretation report.
When the learning unit acquires the names of a plurality of regions of interest including a first name and a second name different from the first name by analyzing the interpretation report, the name of the region of interest The learning is performed using the first teacher data including the first name as the first teacher data and the second teacher data including the second name as the name of the region of interest. The learning support apparatus according to item 1.
When the names of a plurality of regions of interest are acquired by analyzing the interpretation report, the learning unit performs learning using region position information related to the position information of the region of interest in addition to the teacher data. The learning support device according to claim 1, wherein the learning support device is performed.
The acquisition unit refers to a hierarchical structure database that stores a name of a superordinate concept or a subordinate concept corresponding to a superordinate concept or a subordinate concept with respect to the name of the region of interest. Get the name of the subordinate concept
The learning support device according to claim 1, wherein the learning unit performs the learning using teacher data including an image of the region of interest and a name of the superordinate concept or the subordinate concept.
The acquisition unit refers to a similar name database in which a plurality of similar names similar to each other for the name of the region of interest are stored in advance, and determines a representative name from the plurality of similar names,
The learning support device according to claim 1, wherein the learning unit performs the learning using teacher data including an image of the region of interest and the representative name.
The acquisition unit newly acquires an image of the region of interest and a name of the region of interest when the interpretation report is newly stored in the storage unit,
The registration unit registers new teacher data including an image of the region of interest newly acquired by the acquisition unit and a name of the region of interest,
The learning unit generates the updated discrimination model by re-learning using a plurality of the teacher data including the new teacher data when the new teacher data is registered. The learning support device according to any one of Items 9 to 9.
In the case where the newly stored interpretation report and the past interpretation report for the same patient are stored in the storage unit, the acquisition unit newly stores an image of the past interpretation report and the newly stored interpretation report. By aligning the image of the image interpretation report that has been newly acquired and the image of the region of interest newly acquired, the image of the region of interest included in the image of the image interpretation report in the past is acquired,
The registration unit registers past image teacher data including the image of the region of interest acquired based on the past interpretation report and the name of the newly acquired region of interest,
The said learning part produces | generates the said discrimination | determination model updated by learning again using the said teacher data including the said past image teacher data, when the said past image teacher data is registered. The learning support apparatus described.
The learning support device according to any one of claims 1 to 11, wherein the interpretation report includes an electronic medical record.
The learning support device according to any one of claims 1 to 12, wherein the acquisition unit acquires an anatomical organ name, a zone name, a disease name, and a medical condition as the name of the region of interest.
In a learning support method of a learning support device including a storage unit that stores an image interpretation report including image and character information, an acquisition unit, a registration unit, and a learning unit,
The acquisition unit acquires the image of the region of interest included in the image and the name of the region of interest included in the character information by analyzing the interpretation report;
A registration step in which the registration unit registers teacher data including the image of the region of interest acquired by the acquisition unit and the name of the region of interest;
A learning step that performs learning to generate a discrimination model that outputs the image of the region of interest and the name of the region of interest with respect to the input of the image of the interpretation report using a plurality of the teacher data Support method.
Computer
A storage unit storing an interpretation report including image and text information;
By acquiring the image of the region of interest included in the image and the name of the region of interest included in the character information by analyzing the interpretation report,
A registration unit for registering teacher data including the image of the region of interest acquired by the acquisition unit and the name of the region of interest;
Learning to generate a discriminant model that outputs the image of the region of interest and the name of the region of interest with respect to the input of the image of the interpretation report using a plurality of the teacher data registered in the registration unit A learning support program for functioning as a learning unit.
A storage unit storing an interpretation report including image and text information;
By acquiring the image of the region of interest included in the image and the name of the region of interest included in the character information by analyzing the interpretation report,
A registration unit for registering teacher data including the image of the region of interest acquired by the acquisition unit and the name of the region of interest;
Learning to generate a discriminant model that outputs the image of the region of interest and the name of the region of interest with respect to the input of the image of the interpretation report using a plurality of the teacher data registered in the registration unit The learning department,
A region-of-interest discriminating apparatus comprising: a discriminating unit that discriminates an image of the region of interest and a name of the region of interest using the discrimination model when an image of the interpretation report is input.
In a region of interest discriminating apparatus comprising a storage unit storing an image interpretation report including image and character information, an acquisition unit, a registration unit, a learning unit, and a discrimination unit,
The acquisition unit acquires the image of the region of interest included in the image and the name of the region of interest included in the character information by analyzing the interpretation report;
A registration step in which the registration unit registers teacher data including the image of the region of interest acquired by the acquisition unit and the name of the region of interest;
A learning step in which the learning unit performs learning for generating a discrimination model that outputs the image of the region of interest and the name of the region of interest with respect to the input of the image of the interpretation report using a plurality of the teacher data When,
A region-of-interest determination method comprising: a determination step of determining, when the image of the interpretation report is input, the image of the region of interest and the name of the region of interest using the determination model.
Computer
A storage unit storing an interpretation report including image and text information;
By acquiring the image of the region of interest included in the image and the name of the region of interest included in the character information by analyzing the interpretation report,
A registration unit for registering teacher data including the image of the region of interest acquired by the acquisition unit and the name of the region of interest;
Learning to generate a discriminant model that outputs the image of the region of interest and the name of the region of interest with respect to the input of the image of the interpretation report using a plurality of the teacher data registered in the registration unit The learning department,
A region-of-interest discrimination program for functioning as a discriminator for discriminating the image of the region of interest and the name of the region of interest using the discrimination model when an image of the interpretation report is input.
Computer
A storage unit storing an interpretation report including image and text information;
By acquiring the image of the region of interest included in the image and the name of the region of interest included in the character information by analyzing the interpretation report,
A registration unit for registering teacher data including the image of the region of interest acquired by the acquisition unit and the name of the region of interest;
Learning to generate a discriminant model that outputs the image of the region of interest and the name of the region of interest with respect to the input of the image of the interpretation report using a plurality of the teacher data registered in the registration unit The learning department,
A learned model for functioning as a discriminator for discriminating the image of the region of interest and the name of the region of interest using the discrimination model when the image of the interpretation report is input.