KR20190105460A - Apparatus and Method for Generating Medical Diagonosis Report - Google Patents

Abstract

The present invention relates to an apparatus for generating a medical diagnosis report comprising: an artificial intelligence learning model-based diagnosis evidence information extracting unit of receiving the same medical image diagnosed by a computer aided diagnosis (CAD) system outputting diagnosis results of determining the progress or position of a lesion of an object and predicting evaluation values by each lesion shape property which is to be the basis of decision making of the diagnosis results; and a sentence generating unit generating a medical diagnosis comment corresponding to the evaluation values by each lesion shape property. Accordingly, status information of the lesion can be notified more accurately by providing decision making evidence information of the diagnosis results of the progress or position of the lesion by the CAD system.

Description

Apparatus and Method for Generating Medical Diagonosis Report

The present invention relates to medical diagnostic technology, and more particularly, to an apparatus and method for diagnosing a disease through medical image analysis.

When performing diagnosis and treatment of diseases using medical images at medical sites, it is necessary to quantitatively measure the size, extent, and function of a lesion. Since the medical image may have different meanings according to the eyes of the user, the medical image may not recognize a lesion existing in the image by mistake. In addition, when the lesion is detected using the medical image, the lesion may not be recognized from the large-capacity medical image according to the user's skill or fatigue.

To help doctors by supplementing this, the accuracy of analysis using computer aided diagnosis (CAD) technology to help diagnose difficult cases and reduce false positives. To improve.

CAD technology analyzes medical images by intelligent high-performance image analysis system computer to determine the presence and location of lesions and provide diagnostic results. That is, a diagnosis result of whether the lesion is a malignant tumor or a benign tumor and probability information on the diagnosis result may be provided. An example of CAD technology is IBM's AI Watson in the medical field, which diagnoses diseases such as cancer. Watson for Oncology, IBM's 12-year-old, based on smart data from cancer, including colorectal cancer, increased the accuracy of breast cancer diagnosis by 91-100% in 2015.

The diagnosis result through the medical image may be result information obtained by comprehensively determining various shape attributes of the lesion appearing on the medical image. However, the CAD system only provides diagnosis results of the degree of progression such as location information and malignancy of the lesions, and does not provide evidence on why the CAD system derives such diagnosis results. Therefore, in order to hear the evidence of the diagnosis result, a separate opinion of a specialist such as a doctor is required.

In addition, although the CAD system has evolved to improve accuracy, the risk of misdiagnosis is inevitable. Therefore, as a way of verifying the reliability of the diagnosis result of the CAD system, it is required to provide reasonable judgment basis information.

Accordingly, an object of the present invention is to solve the above-described problems, the condition information of the lesion as the doctor diagnoses by providing the decision basis information of the diagnosis result of determining the progress or location of the lesion by the CAD system To make it more clear.

In addition, an object of the present invention is to improve the reliability by providing the CAD evidence information that can verify the diagnostic results.

The apparatus for generating a medical diagnosis report according to the present invention receives an identical medical image diagnosed by a CAD system that outputs a diagnosis result of determining the progress or position of an object lesion, and predicts an evaluation value for each lesion shape attribute that is a decision basis of the diagnosis result. A diagnosis basis information extracting unit based on an artificial intelligence learning model and a sentence generation unit generating a medical diagnosis description corresponding to an evaluation value of each lesion shape attribute.

The diagnostic evidence information extracting unit includes prediction images for each shape attribute trained based on deep learning or machine learning using medical images labeled with evaluation values for each lesion shape attribute, and each of the prediction models is input to the CAD system. And then the estimated value for the corresponding lesion shape property is output.

The prediction models may be a regressor that outputs one of a range of individual real values in a predetermined range as an estimated value for the lesion shape attribute.

The sentence generation unit includes a descriptive database to which individual descriptive statements for each shape attribute are mapped, a connector storage unit including connector information used to form a compound sentence by combining a plurality of short sentences, and evaluation of lesion shape attributes extracted from the descriptive database. May include a sentence assembly unit for assembling at least one individual statement corresponding to the device into a connector extracted from the connector storage unit to generate a medical diagnosis statement.

The sentence generation unit selects the lesion shape attributes among the lesion shape attributes output from the diagnostic evidence information extracting unit that are highly correlated with the diagnosis result output from the CAD system, and among the lesion shape attributes output from the diagnostic evidence information extracting unit. The apparatus may further include at least one of alignment units arranged in order of lesion shape attributes that are highly correlated with the diagnosis result output by the CAD system, and the sentence assembly unit may include individual sentences corresponding to the shape attributes selected by the selection unit, or It is possible to assemble individual statements in the order of the shape attributes arranged by the alignment unit.

Accordingly, an object of the present invention is to solve the above-mentioned problems, the condition information of the lesion as diagnosed by the physician by providing the decision basis information of the diagnosis result of determining the progress or location of the lesion by the CAD system To make it more clear.

In addition, an object of the present invention is to improve the reliability by providing the CAD evidence information that can verify the diagnostic results.

1 is a block diagram of an apparatus for generating a medical diagnosis report according to an exemplary embodiment.
2 is a detailed block diagram of a diagnosis basis information extracting unit according to an exemplary embodiment.
3A and 3B are detailed block diagrams of a diagnostic report generator according to an exemplary embodiment.
4 is a flowchart illustrating a method of generating a medical diagnosis report according to an embodiment of the present invention.
5 is a flowchart illustrating a step of extracting diagnosis basis information according to an embodiment of the present invention.
6A and 6B are flowcharts illustrating a step of generating a medical diagnosis description according to an embodiment of the present invention.

Hereinafter, a medical diagnosis report apparatus according to a preferred embodiment will be described in detail with reference to the accompanying drawings. Here, the same reference numerals are used for the same configurations, and detailed descriptions of well-known functions and configurations that may unnecessarily obscure the repeated description and the subject matter of the present invention will be omitted. Embodiments of the invention are provided to more completely describe the present invention to those skilled in the art. Accordingly, the shape and size of elements in the drawings may be exaggerated for clarity.

Combinations of each block of the block diagrams and respective steps of the flowcharts may be performed by computer program instructions (executable engines), which may be executed on a processor of a general purpose computer, special purpose computer, or other programmable data processing equipment. As such, instructions executed through a processor of a computer or other programmable data processing equipment create means for performing the functions described in each block of the block diagram or in each step of the flowchart.

These computer program instructions may also be stored in a computer usable or computer readable memory that can be directed to a computer or other programmable data processing equipment to implement functionality in a particular manner, and thus the computer usable or computer readable memory. The instructions stored therein may also produce an article of manufacture containing instruction means for performing the functions described in each block of the block diagram or in each step of the flowchart.

And computer program instructions can also be mounted on a computer or other programmable data processing equipment, such that a series of operating steps can be performed on the computer or other programmable data processing equipment to create a computer-implemented process that can be executed by the computer or other programmable data. Instructions for performing data processing equipment may also provide steps for performing the functions described in each block of the block diagram and in each step of the flowchart.

In addition, each block or step may represent a portion of a module, segment or code that includes one or more executable instructions for executing specific logical functions, and in some alternative embodiments referred to in blocks or steps It should be noted that the functions may occur out of order. For example, the two blocks or steps shown in succession may, in fact, be performed substantially concurrently, or the blocks or steps may be performed in the reverse order of the corresponding function, as required.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, embodiments of the present invention illustrated in the following may be modified in many different forms, the scope of the present invention is not limited to the embodiments described in the following. Embodiments of the present invention are provided to more fully describe the present invention to those skilled in the art.

Until now, it was pointed out that artificial intelligence only informed the results when supporting decision making and could not explain logically how it was made based on the reason. Deep learning, a representative field of artificial intelligence, enables us to extract implicit knowledge rather than explicit knowledge from large amounts of repeated data.

When applied to crime risk judgment, personnel evaluation, military operations and medical field, this method can increase efficiency, but it is hard to say that it is fair and accurate. Knowing the basis for conclusions in algorithms such as "black boxes" will help you trust the system. The recent emergence of Explainable AI (XAI) is drawing attention as a way to overcome the limitations of AI.

Explainable AI refers to artificial intelligence that presents the reasons for decision making in a way that humans can understand so that it can be used in areas where transparency and user trust are required, such as medical, legal, financial and defense. Currently, the US Defense Advanced Research Projects Agency (DARPA), which is the most active in this field, is investing about 80 billion won and doing research in the private sector such as MIT and Google. They are mainly studying the visualization process by deep learning. For example, when Deep Learning recognizes a thing as a cat, another AI tool monitors and analyzes what it was called and when it wasn't. This can be applied to cancer cells, disease diagnosis, and determining the enemy on the battlefield.

Explainable AI is another AI tool when Deep Learning recognizes something as a cat and monitors and analyzes what it was called and when it wasn't. Descriptive tools have been added to the existing deep learning process to reveal which attributes are used to judge cats. This can be applied to cancer cells, disease diagnosis, and determining the enemy on the battlefield.

The present invention applies the concept of 'Explainable AI' (XAI, Explainable AI) as described above, and attaches a separate tool called a medical diagnostic report generating device to the CAD system to provide the explanation of the evidence of the diagnosis result of the CAD system. do.

1 is a block diagram of an apparatus for generating a medical diagnosis report according to an exemplary embodiment.

Referring to FIG. 1, the CAD system 1 outputs a diagnosis result of determining the progress or position of an object lesion. Here, the diagnosis result may include the accuracy or reliability of the determination along with the determination of whether the lesion is benign or malignant.

In general, the CAD system 1 may determine whether the lesion is benign or malignant by comparing attributes extracted from the medical image with a previously stored diagnostic model. Here, the diagnostic model may be generated through machine learning using lesion shape attributes extracted from a plurality of previously collected diagnostic images, and the diagnostic model may include a combination of lesion shape attributes and a diagnosis result according to each combination. . Since the CAD system 1 is a well-known technique, detailed internal blocks will be omitted.

The medical image may be one of various medical images including a computed radiography (CR) image, a computed tomography (CT) image, an ultrasound image, and a magnetic resonance image (MRI). have. In addition, the medical image may be one frame image constituting a sequence of continuously photographed images.

In addition, although the object is shown as a lung in the illustrated medical image, this is only an example and the present invention is not limited thereto. That is, the object may be a part of the body or an animal, for example, the object may include an organ such as the liver, the heart, the uterus, the brain, the breast, the abdomen, or the blood.

According to an embodiment of the present disclosure, the apparatus may further include a medical image input unit (not shown) capable of receiving a medical image of the patient, wherein the medical image input unit is connected to the medical image photographing apparatus through an interface or a wired or wireless network. In addition, the medical image input unit may perform preprocessing by accessing the tissue image from the memory. Here, the memory may be a portable removable memory or a memory provided in the system. Here, the preprocessing process is for easily extracting lesion shape attributes from the medical image. For example, normalization, channel separation, and scaling may be performed on the medical image.

The apparatus 2 for generating a medical diagnosis report analyzes the same medical image diagnosed by the CAD system 1, and generates and outputs a report describing a decision reason for the object diagnosis result included in the ROI. The medical diagnosis report generating device 2 may be implemented in an embedded form of the CAD system 1 or may be implemented in an external form.

The medical diagnosis report generating apparatus 2 includes a diagnosis basis information extracting unit 100 and a sentence generating unit 200.

The diagnosis basis information extracting unit 100 predicts an evaluation value for each lesion shape attribute, which is a decision basis of a diagnosis result, based on an artificial intelligence learning model from the input medical image. The diagnostic evidence information extracting unit 100 will be described in detail later with reference to FIG. 2.

The sentence generation unit 200 generates a medical diagnosis description corresponding to the evaluation value for each lesion shape attribute output from the diagnosis basis information extraction unit 100. That is, it generates and outputs a diagnostic report including the description contents corresponding to the diagnosis result of the CAD system 1. A detailed description of the sentence generation unit 200 will be described later with reference to FIG. 3.

In addition to the above components, the system may include a user input unit (not shown) for generating input data for controlling the operation of the system according to the manipulation of a user (pathologist or doctor) and a display for displaying various data generated according to system operation. It may further include.

2 is a detailed block diagram of a diagnosis basis information extracting unit according to an exemplary embodiment.

The diagnosis basis information extracting unit 100 predicts and outputs an evaluation value for each of the lesion shape attributes that may be used as the lesion diagnosis basis from the medical image.

Here, the lesion shape attribute is a shape attribute of an object lesion having a meaning in a medical context, and refers to visual pattern information that helps determine whether the lesion is benign or malignant. Each lesion shape attribute is a preset classification item. Can be classified according to. Here, the evaluation value (score) may be information for predicting a score according to the findings of doctors who have visually confirmed the corresponding lesion shape property.

For the sake of understanding, referring to Table 1 below, an example of lesion shape attributes for determining benign or malignant lung tumors will be described.

Nodule properties
(Nodule characteristic) Definition  Ratings criteria  Calcification Calcification appearance in the nodule Popcorn
2. Laminated
3. Solid
4. Non-central
5. Central
6. Absent Internal structure  Expected internal composition of the nodule 1.Soft tissue
2. Fluid
3.fat
4.Air  Lobulation Whether lobular shape is apparent from margin or not 1. Marked
2.
3.
4.
5. None
Margin
How well defined the margins are 1.Poorly defined
2.
3.
4.
5. Sharp  Sphericity Dimensional shape in terms of roundness Linear
2.
3. Ovoid
4.
5. Round Spiculation Degree of exhibition of spicules 1. Marked
2.
3.
4.
5. None  Subtlety Contrast between nodule and surroundings Extremely subtle
2.Moderately subtle
3.fairly subtle
4.Moderately obvious
5. Obvious Texture  Internal density of nodule 1.Non-solid
2.
3.Part Solid
4.
5. Solid

Table 1 describes the Nodule characteristic item, Nodule characteristic item description and rating criteria for determining malignant or benign lung tumor.

Nodule characteristic is an internal structure that evaluates the appearance of calcification to assess its appearance according to the degree of calcification of the nodule, and whether fluids or fats are expected internal constituents of the nodule. Internal structure), Lobulation, which assesses whether the lobe shape is clear from the edge, Margin, which evaluates the boundary of the nodule's edge, and Sphericity, which evaluates the roundness of the shape of the nodule. ), Spiculation to assess the degree of bone fragments revealed in the nodules, Subtlety to contrast the nodules and surroundings, and Texture to evaluate the internal density of the nodules.

The diagnostic evidence information extractor 100 may output an evaluation result for each of the lesion shape attributes as described in Table 1 and output the results described in Table 2 as follows.

subtlety Internal
Structure Calcification Sphericity Margin Lobulation Spiculation Texture 4.6 1.2 5.2 1.4 2.3 4.4 2.2 1.2

Referring to <Table 1> and <Table 2>, the evaluation value (or rating value, score) of the lesion shape described in <Table 2> has a clear boundary between nodules and nodules since the sensitivity is 4.6. Since the internal structure is 1.2, it is composed of soft tissue, and the calcification is 5.2, so the calcification is Central. Since the sphericality is 1.4, the shape is almost linear. Since Margin is 2.3, the boundary is not clear, Lobulation is 2.2, so the leaflet shape is obvious, and since Spiculation is 2.2, the bone shape is clearly visible.Texture Is 1.2, so it is expected to be non-solid.

As described above, the evaluation criteria for evaluating the lesion shape attributes and the attributes are merely examples for describing the present invention, and the present invention is not limited thereto. That is, the lesion shape attributes and the evaluation criteria for each lesion shape attribute may be set by the user in consideration of characteristics according to the type of medical image (MRI, ultrasound image, etc.), the photographing site (chest, lung, thyroid, etc.).

Referring to FIG. 2, the diagnosis basis information extractor 100 uses a plurality of shape attributes predictive models 111, which are trained based on deep learning or machine learning, using medical images labeled with evaluation values of lesion shape attributes as a data set. 112, 113, ..., 11n).

In this case, the medical images used as the data set are diagnosed and may be a plurality of medical images labeled with evaluation values completed by the doctors. In addition, since evaluation values may be different for different doctors even for the same medical image, a plurality of medical image data labeled with different evaluation values may be used for the same medical image.

In addition, a learning algorithm used to generate a predictive model may include deep learning based algorithms such as deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), sparse autoencoders, deformable part models (DPMs), and SVMs (SVMs). Support Vector Machine) and Histogram of Oriented Gradients (HOG) and the like.

Each of the attribute prediction models 111, 112, 113,..., And 11n receives a medical image input to the CAD system and outputs an estimated value predicted for a corresponding lesion shape attribute. That is, mapped to one of the shape properties, for example, the first shape property prediction model 111 predicts a sensitivity evaluation, and the second shape property prediction model 112 predicts an internal structure evaluation value. The third shape attribute prediction model 113 may predict sphericality evaluation values, and predict and output the nth shape attribute prediction model 11n bone fragmentation evaluation values.

According to one aspect, the prediction models 111, 112, 113, ..., 11n are regressors that output as one of a range of individual real values in a predetermined range as an estimated value for the lesion shape attribute. Can be. For example, the evaluation value as described in <Table 1> may be a real value such as 4.6 or 1.2, as described in <Table 2>, in the range of 1 to 6.

According to another aspect, the prediction models 111, 112, 113, ..., 11n may be a classifier that outputs one of the values of '0' and '1' as an estimated value for predicting the lesion shape attribute. have. For example, when the evaluation criteria are set as shown in Table 1, as shown in Table 2, the information classified once more as 4.6 with a real value of '4.6' and with a real value of 1.2 as '0' It can be output as an estimate.

Alternatively, when the prediction models 111, 112, 113,..., And 11n are classifiers, the evaluation values predicted for the lesion shape attribute are the rating values disclosed in Table 1 or Table 2 above. In (Scores) it can be output as the value of one of the specific scores.

The vector generator 100 converts the values of the lesion shape attributes into a vector and outputs the converted values. For example, referring to Table 2, when X ₁ is Subtlety, X ₂ is Internal Structure, ....., X ₈ is Texture, the vector of (4.6, 1.2, ..., 1.2) is Can be generated. At this time, since the range of the evaluation value may be different for each of the lesion shape attributes, the evaluation value may be normalized for each of the lesion shape attributes.

3A and 3B are detailed block diagrams of a diagnostic report generator according to an exemplary embodiment.

Referring to FIG. 3A, according to an aspect, the sentence generation unit 200 includes a description database 210, a connector storage unit 220, and a sentence assembly unit 230. According to another aspect, the sentence generator 200 may further include at least one of the selector 240 and the alignment unit 250.

The comment database 210 stores individual comments mapped to shape attributes. For example, the individual descriptions may have a separate description of 'the edge of the edge is unclear' that is mapped when the lesion shape property is Margin and the evaluation score is '1', and the lesion shape property is a spherical map ( Sphericity), and if the evaluation score is '5', there may be stored a separate description of 'nodule is round'.

The connector storage unit 220 stores the connector information used to form a compound sentence by combining a plurality of short sentences. For example, it may include 'because', 'because of', and the like.

The sentence assembly unit 230 generates a medical diagnosis statement by assembling at least one individual comment corresponding to the evaluation value for each lesion shape attribute extracted from the comment database 210 into the connector extracted from the connector storage unit 220. . For example, as shown in Figure 3, the 'tumor is irregular border,' 'about 30mm in size' and 'the calcification hardly progressed', etc. 'to', '~' and '~ And then assembled to connect to a back light to generate a medical diagnostic description. As shown in FIG. 3, the at least one individual statements corresponding to the previous evaluation values of the lesion shape attributes are connected, and the diagnosis result arranged as the final sentence is connected to 'because of', and thus the basis for determining the diagnosis result is You can construct a sentence to clarify what it is.

Referring to FIG. 3B, according to another aspect, the selection unit 240 may have a lesion shape attribute that is highly correlated with a diagnosis result output by the CAD system 1 among the lesion shape attributes output from the diagnosis evidence information extractor 100. Select them. That is, not all of the evaluation values of the lesion shape attributes described in Table 2 may have contributed significantly to the diagnosis result as a decision basis. Therefore, since it is unnecessary to describe all the evaluation values for each lesion shape attribute as the medical diagnosis statement, only the lesion shape attributes that contributed decisively to the diagnosis result may be selected so that the medical diagnosis statement is generated by the sentence assembly unit 230. have.

The alignment unit 250 sorts the lesion shape attributes among the lesion shape attributes output from the diagnosis evidence information extractor 100 in order of the lesion shape attributes having a high correlation with the diagnosis result output by the CAD system 1. That is, the evaluation values for the lesion shape attributes listed in Table 2 will be in order of contributing to the diagnosis result as a decision basis. Accordingly, the medical diagnosis statement may be generated by the sentence assembly unit 230 in the order in which the evaluation values for the lesion shape attributes are contributed.

To this end, the selection unit 240 and the alignment unit 250 are implemented as a learning model that can classify shape attributes used as evidence for determination of malignant or benign tumors, or refer to a database having pre-classified information. can do. In general, malignant tumors have marginally defined margins, while in benign tumors, the sphericality of the nodules is round and the edges are sharp. Can be. At this time, the shape, corners, etc. of the nodule may be a criterion for determining the benign or malignant tumor. Therefore, each lesion shape property may be classified by setting properties for determining benign tumors and malignant tumors as classification items, such as shapes and margins of nodules.

Then, the sentence assembly unit 230 assembles individual sentences corresponding to the shape attributes selected by the selector 240 or individual descriptions in the order of the shape attributes arranged by the alignment unit. For example, as shown in FIG. 3, the border, size, and calcification of the tumor may be selected and generated based on the evidence associated with 40% malignant tumor as a result of the CAD system 10 diagnosis.

Here, although the sorting unit 240 and the alignment unit 250 are arranged in this order in FIG. 3B, the present invention is not limited thereto. That is, only one of the selection unit 240 and the alignment unit 250 may be used, or the arrangement order between the two components may be changed.

Here, as shown in FIG. 3, the medical diagnosis statement may include a lesion shape attribute that is a decision result of a diagnosis result of the CAD system 1 including a progress level or probability of an object lesion, a region of interest of a medical image, and a diagnosis result. It may contain a descriptive text indicating a star rating.

According to another embodiment, the diagnostic report generator 200 may be implemented in the form of a sentence generation learning model generated using a learning algorithm of a recurrent neural network (RNN) or a long-short term memory (LSTM). That is, a data set labeled with an attribute report of lesion shape attributes may be trained using a learning algorithm of Recurrent Neural Network (RNN) or Long-Short Term Memory (LSTM).

Recurrent Neural Network (RNN) or Long-Short Term Memory (LSTM) is a learning algorithm that learns time-series data that changes over time and predicts artificial intelligence. RNN is the deepest network structure in deep learning. Examples of time series data include stock prices, human movements, climate, the number of Internet access users, and search terms. LSTM is a long-term term memory algorithm that solves the problem of vanishing gradient problem because the neural network is so deep that the gate unit is placed at each node.

When the diagnostic report generator 200 according to the present invention is a model trained by a learning algorithm of a Recurrent Neural Network (RNN) or a Long-Short Term Memory (LSTM), the neighbor report may be arranged according to the association between the lesion shape attributes. You can also adjust the input sequence. Alternatively, the medical image may be effectively used when the input medical image is an image frame sequentially arranged over time.

4 is a flowchart illustrating a method of generating a medical diagnosis report according to an embodiment of the present invention. The method for generating a medical diagnosis report to be described later may be understood to have the medical diagnosis report generating device 2 and its components shown in FIGS.

Referring to FIG. 4, the apparatus 2 for generating a medical diagnosis report receives the same medical image diagnosed by the CAD system 1 which outputs a diagnosis result of determining the progress or position of an object lesion, and thus the decision result of the diagnosis result is determined. The estimated value of each lesion shape attribute is predicted based on an artificial intelligence learning model (S410). A detailed description of S410 will be described later with reference to FIG. 5.

The medical diagnosis report generating device 2 generates a medical diagnosis description corresponding to the evaluation value for each lesion shape attribute (S420). That is, it generates and outputs a diagnostic report including the description contents corresponding to the diagnosis result of the CAD system 1. A detailed description of S420 will be described later with reference to FIGS. 6A and 6B.

5 is a flowchart illustrating a step of extracting diagnosis basis information according to an embodiment of the present invention.

Referring to FIG. 5, the diagnosis basis information extracting unit 100 is an evaluation value for each of the lesion shape attributes that may be used as the diagnosis basis as the same image input to the CAD system 1 is input (S510). (S520).

Here, the lesion shape attribute is a shape attribute of an object lesion having a meaning in a medical context, and refers to visual pattern information that helps determine whether the lesion is benign or malignant. Each lesion shape attribute is a preset classification item. Can be classified according to. Here, the evaluation value may be information for predicting a score according to the findings of doctors who visually confirmed the lesion shape property.

S520 is performed by each of a plurality of shape attribute prediction models 111, 112, 113,..., And 11 n trained based on deep learning or machine learning using medical images labeled with evaluation values of lesion shape attributes as data sets. Is performed.

According to an aspect, the estimated value for the lesion shape attribute may be output as one of a range of individual real values. For example, the evaluation value as described in <Table 1> may be a real value such as 4.6 or 1.2, as described in <Table 2>, in the range of 1 to 6.

According to another aspect, it may be output as one of the values of '0' and '1' as the estimated value for the lesion shape attribute. For example, when the evaluation criteria are set as shown in Table 1, as shown in Table 2, the information classified once more as 4.6 with a real value of '4.6' and with a real value of 1.2 as '0' It can be output as an estimate.

Alternatively, the evaluation value predicted for the lesion shape attribute may be output as one of a specific score from the rating values (scores) disclosed in the above-described <Table 1> or <Table 2>.

The diagnosis basis information extracting unit 100 converts the vector into a vector consisting of evaluation values for each of the lesion shape attributes and outputs the converted vector (S530). For example, referring to Table 2, when X ₁ is Subtlety, X ₂ is Internal Structure, ....., X ₈ is Texture, the vector of (4.6, 1.2, ..., 1.2) is Can be generated. At this time, since the range of the evaluation value may be different for each of the lesion shape attributes, the evaluation value may be normalized for each of the lesion shape attributes.

6A and 6B are flowcharts illustrating a step of generating a medical diagnosis description according to an embodiment of the present invention.

Referring to FIG. 6A, according to an aspect, the sentence generation unit 200 may input at least one individual description corresponding to each of the evaluation values for the lesion shape attributes as a vector including the evaluation values for the lesion shape attributes is input (S610). Extract (S620). Then, the sentence generation unit 200 generates a medical diagnosis statement by assembling the connector corresponding to the extracted individual description (S630). For example, as shown in Figure 3, the 'tumor is irregular border,' 'about 30mm in size' and 'the calcification hardly progressed', etc. 'to', '~' and '~ And then assembled to connect to a back light to generate a medical diagnostic description. As shown in FIG. 3, the at least one individual statements corresponding to the previous evaluation values of the lesion shape attributes are connected, and the diagnosis result arranged as the final sentence is connected to 'because of', and thus the basis for determining the diagnosis result is You can construct a sentence to clarify what it is.

Referring to FIG. 6B, according to another aspect, the method may further include steps S613 and S616 as compared to FIG. 6A. The sentence generation unit 200 selects lesion shape attributes having a high correlation with the diagnosis result output by the CAD system 1 among the lesion shape attributes as a vector including evaluation values for the lesion shape attributes is input (S610) (S613). ). That is, not all of the evaluation values of the lesion shape attributes described in Table 2 may have contributed significantly to the diagnosis result as a decision basis. Therefore, since it is unnecessary to describe all the evaluation values for the lesion shape attributes as the medical diagnosis statement, only the lesion shape attributes contributing decisively to the diagnosis result may be selected so that the medical diagnosis statement is generated.

In addition, the sentence generation unit 200 arranges the lesion shape attributes in order of the lesion shape attributes having a high correlation with the diagnosis result output by the CAD system 1 (S616). That is, the evaluation values for the lesion shape attributes listed in Table 2 will be in order of contributing to the diagnosis result as a decision basis. Accordingly, the medical diagnosis statement may be generated by the sentence assembly unit 230 in the order in which the evaluation values for the lesion shape attributes are contributed.

Here, although FIG. 6B is performed in the order of S613 and S616, this is merely an example and the present invention is not limited thereto. That is, only one of S613 and S616 may be used, or the order between S613 and S616 may be reversed.

Then, the sentence generator 200 extracts individual sentences corresponding to the selected shape attributes or individual descriptions in the order of the shape attribute (S620), and assembles the individual descriptions into the extracted connector to generate a medical diagnosis statement. (S630). For example, as shown in FIG. 3, the border, size, and calcification of the tumor may be selected and generated based on the evidence associated with 40% malignant tumor as a result of the CAD system 10 diagnosis.

Here, as shown in FIG. 3, the medical diagnosis statement may include a lesion shape attribute that is a decision result of a diagnosis result of the CAD system 1 including a progress level or probability of an object lesion, a region of interest of a medical image, and a diagnosis result. It may contain a descriptive text indicating a star rating.

The present invention can be used industrially in the field of medical diagnostic technology.

1: CAD system
2: medical diagnostic report generating device
100: diagnostic evidence information extraction unit
111 ~ 11n: Shape Property Prediction Model
120: vector generator
200: sentence generation unit
210: comment database
220: connector storage unit
230: sentence assembly
240: sorting unit
250: alignment unit

Claims (5)

Computer Aided Diagnosis (hereinafter referred to as 'CAD') that outputs the diagnosis result that determines the progress or location of the object lesion. The lesion shape that is the basis for decision-making of the diagnosis result by receiving the same medical image diagnosed by the system. Diagnostic evidence information extraction unit based on artificial intelligence learning model for predicting the evaluation value of each property;
A sentence generation unit generating a medical diagnosis description corresponding to the evaluation value for each lesion shape attribute;
Medical diagnostic report generation device comprising a.
The method of claim 1, wherein the diagnostic evidence information extraction unit,
Includes predictive models by shape attribute trained on the basis of deep learning or machine learning using medical images labeled with lesion shape attribute evaluation values as a data set,
A medical diagnostic report generating device, each of which predictive models receives a medical image input to a CAD system and outputs an estimated value for a corresponding lesion shape attribute. The method of claim 2, wherein the prediction models are
A regressor for outputting the estimated value of the lesion shape attribute as one of a predetermined range of real numbers;
or
An apparatus for generating a medical diagnosis report, which is one of a classifier that outputs a predicted evaluation value of a lesion shape attribute as a value of a specific score.
The method of claim 1, wherein the sentence generation unit,
A comment database in which individual comments having a predetermined format are mapped to shape attributes;
A connector storage unit including connector information used to form a compound sentence by combining a plurality of short sentences;
A sentence assembly unit for generating a medical diagnosis statement by assembling at least one individual comment corresponding to the evaluation value of the lesion shape attribute extracted from the description database into a connector extracted from the connector storage unit;
Medical diagnostic report generation device comprising a.
The method of claim 4, wherein the sentence generation unit,
A selection unit for selecting lesion shape attributes that are highly correlated with the diagnosis result output by the CAD system among the lesion shape attributes output from the diagnosis evidence information extracting unit;
An alignment unit for sorting the lesion shape attributes among the lesion shape attributes output from the diagnosis evidence information extracting unit in order of the lesion shape attributes having a high correlation with the diagnosis result output by the CAD system; At least one of,
The sentence assembly part,
A medical diagnostic report generating device for assembling individual sentences corresponding to the shape attributes selected by the selection unit or individual descriptions in the order of the shape attributes arranged by the alignment unit.

Images