CN116258136A - Error detection model training method, medical image report detection method, system and equipment - Google Patents
Error detection model training method, medical image report detection method, system and equipment Download PDFInfo
- Publication number
- CN116258136A CN116258136A CN202310144833.4A CN202310144833A CN116258136A CN 116258136 A CN116258136 A CN 116258136A CN 202310144833 A CN202310144833 A CN 202310144833A CN 116258136 A CN116258136 A CN 116258136A
- Authority
- CN
- China
- Prior art keywords
- error
- model
- sample
- error detection
- medical image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 247
- 238000012549 training Methods 0.000 title claims abstract description 188
- 238000000034 method Methods 0.000 title claims abstract description 88
- 230000008569 process Effects 0.000 claims abstract description 15
- 238000012360 testing method Methods 0.000 claims description 60
- 238000004590 computer program Methods 0.000 claims description 14
- 230000002452 interceptive effect Effects 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 6
- 230000001960 triggered effect Effects 0.000 claims description 6
- 238000010586 diagram Methods 0.000 description 12
- 238000003860 storage Methods 0.000 description 9
- 230000006870 function Effects 0.000 description 7
- 238000012805 post-processing Methods 0.000 description 6
- 238000005070 sampling Methods 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 4
- 238000002372 labelling Methods 0.000 description 4
- 239000013589 supplement Substances 0.000 description 4
- 238000002059 diagnostic imaging Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000012544 monitoring process Methods 0.000 description 3
- 238000003058 natural language processing Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 2
- 238000002591 computed tomography Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 201000010099 disease Diseases 0.000 description 2
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 2
- 238000009826 distribution Methods 0.000 description 2
- 238000002595 magnetic resonance imaging Methods 0.000 description 2
- 238000003062 neural network model Methods 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 230000002829 reductive effect Effects 0.000 description 2
- 230000001502 supplementing effect Effects 0.000 description 2
- 241000613118 Gryllus integer Species 0.000 description 1
- 206010056342 Pulmonary mass Diseases 0.000 description 1
- 210000001015 abdomen Anatomy 0.000 description 1
- 230000005856 abnormality Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000001746 atrial effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000002405 diagnostic procedure Methods 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000000670 limiting effect Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000004091 panning Methods 0.000 description 1
- 230000036961 partial effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/232—Orthographic correction, e.g. spell checking or vowelisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/205—Parsing
- G06F40/226—Validation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H15/00—ICT specially adapted for medical reports, e.g. generation or transmission thereof
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Software Systems (AREA)
- Epidemiology (AREA)
- Medical Informatics (AREA)
- Primary Health Care (AREA)
- Public Health (AREA)
- Medical Treatment And Welfare Office Work (AREA)
Abstract
The application provides an error detection model training method, a medical image report detection system and medical image report detection equipment. The training method comprises the following steps: determining a first recognition model trained based on a plurality of first samples of the medical image report; acquiring a plurality of second sample texts of the medical image report, wherein at least one sample sentence with a first label in the second sample text is an error sample sentence containing an error word; training the first recognition model based on the at least one sample sentence to obtain a second recognition model; in the process of training the first recognition model, if the first recognition model is monitored that the first recognition model cannot recognize a first error sample sentence in at least one sample sentence, the first error sample sentence is sent to a user, and the user determines a first error word in the first error sample sentence and adds the first error word to an error detection rule set; and determining an error detection model according to the second identification model and the error detection rule set. The scheme can train with low training cost to obtain the error detection model with better performance.
Description
Cross reference
The present application is incorporated by reference in its entirety into chinese patent application No. 202310036606.X entitled "error detection model training method, medical image report detection method, system and apparatus" filed on 10 month 2023, 01.
Technical Field
The application relates to the technical field of intelligent medical treatment, in particular to an error detection model training method, a medical image report detection system and medical image report detection equipment.
Background
With the development of modern medicine, medical images have become an indispensable important part in assisting doctors in diagnostic procedures, and doctors often write an electronic medical image report for each medical image, wherein the medical image report mainly comprises quantitative description about whether abnormality exists in the medical image, diagnostic comments obtained based on medical image performance and clinical data analysis, and the like.
Because the medical image report is manually written, some errors, such as spelling errors, vocabulary errors and the like, are unavoidable, and at present, a rule-based method is mainly adopted in the aspect of error detection of the medical image report, but the rule-based method needs to manually formulate a word stock, has high cost and cannot identify error types outside the word stock.
Disclosure of Invention
Aiming at the problem of error detection of the existing medical image report, the embodiment of the application provides an error detection model training method, a medical image report detection method, a system and equipment.
In one embodiment of the present application, an error detection model training method is provided, wherein the detection model is used for error detection of a medical image report. The method comprises the following steps:
determining a first recognition model, the first recognition model being trained based on a plurality of first text samples of the medical image report;
acquiring a plurality of second sample texts of the medical image report; the second sample text comprises at least one sample sentence, wherein the sample sentence with a first label in the at least one sample sentence is an error sample sentence containing an error word, and the sample sentence without the first label is a correct sample sentence without the error word;
training the first recognition model based on at least one sample sentence included in the plurality of first samples to obtain a second recognition model;
in the training process of the first recognition model, if the first recognition model is monitored that the first recognition model cannot recognize a first error sample sentence in the at least one sample sentence, the first error sample sentence is sent to a user, so that the user determines a first error word in the first error sample sentence and adds the first error word to an error detection rule set;
And determining the error detection model according to the second identification model and the error detection rule set.
In another embodiment of the present application, there is also provided a medical report detection method, the method comprising:
determining a target medical image report to be detected;
obtaining an error detection model, wherein the error detection model is obtained through training by the error detection model training method provided by the embodiment of the application;
and carrying out error detection on the target medical image report by utilizing the error detection model.
In yet another embodiment of the present application, there is also provided a medical image report detection system, the system including:
the server is used for determining a target medical image report to be detected; obtaining an error detection model, wherein the error detection model is obtained through training by the error detection model training method provided by the embodiment of the application; performing error detection on the target medical image report by using the error detection model;
and the client is used for providing an interactive interface, and displaying the error detection result of the target medical image report on the interactive interface.
In yet another embodiment of the present application, an electronic device is also provided. The electronic device includes: a memory and a processor, wherein the memory is used for storing a computer program; the processor is coupled to the memory, and is configured to execute the computer program stored in the memory, to implement steps in an error detection model training method provided in one embodiment of the present application, or to implement steps in a medical image report detection method provided in another embodiment of the present application.
According to the technical scheme provided by the embodiments of the application, when an error detection model for carrying out error detection on a medical image report is trained, a first recognition model is determined first, and the first recognition model is obtained by training a plurality of first text samples based on the medical image report; and then, acquiring a plurality of second sample texts comprising at least one sample sentence of the medical image report, wherein the sample sentence with the first label in the at least one sample sentence is an error sample sentence containing an error word, and the sample sentence without the first label is a correct sample sentence without the error word, and training the first recognition model by utilizing the at least one sample sentence comprising the acquired plurality of first sample texts to obtain a second recognition model. Through training by utilizing at least one sample sentence included in the first sample text and the second sample text in the two training stages, the second recognition model obtained through training has higher understanding ability on text semantics, and the accuracy of error recognition can be effectively improved. Further, in the training process of the first recognition model, if it is monitored that the first recognition model cannot recognize the first error sample sentence in the at least one sample sentence, the first error sample sentence is sent to the user, so that the first error word in the first error sample sentence is determined and added to the error detection rule set; finally, the error detection model is determined according to the second recognition model and the error detection rule set, and the error detection rule set is combined to supplement the difficult errors which cannot be recognized by the second recognition model obtained through training in the two training stages, so that the error recognition precision of the error detection model can be further improved, and the problems of large burden and high cost of manually constructing a word stock (such as the error detection rule set) in the process of directly constructing the error detection model based on a rule method can be avoided. In summary, the scheme can train with low training cost to obtain the required error detection model with better performance. And then, performing error detection on the target medical report to be detected by using the trained error detection model, so that higher error detection accuracy can be ensured.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief description will be given below of the drawings that are needed to be utilized in the embodiments or the prior art descriptions, and it is obvious that the drawings in the following description are some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of an error detection model training method according to an embodiment of the present application;
FIG. 2a is a diagram illustrating an example of medical image reporting according to an embodiment of the present application;
FIG. 2b is a schematic diagram of training a pre-training language model according to an embodiment of the present application;
FIG. 3 is a simplified schematic diagram of training of an error detection model according to one embodiment of the present application;
FIG. 4 is a flowchart illustrating a medical image report detection method according to an embodiment of the present disclosure;
FIG. 5 is a schematic structural diagram of a medical image report detection system according to an embodiment of the present disclosure;
FIG. 6 is a schematic structural diagram of an error detection model training device according to an embodiment of the present disclosure;
FIG. 7 is a schematic structural diagram of a medical image report detection device according to an embodiment of the present disclosure;
Fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present application;
fig. 9 is a schematic structural diagram of a computer program product according to an embodiment of the present application.
Detailed Description
Along with the popularization and promotion of medical imaging equipment, medical imaging reports have become increasingly important bases for assisting clinicians in diagnosing diseases. The present medical image report is often an electronic report written by radiologists for medical images, and various errors such as spelling errors, vocabulary errors, grammar errors and the like are unavoidable in the content of the report. Since the medical image report is a document which is communicated with affine medical doctors and clinicians and has a certain medical legal responsibility, if there is an error in the medical image report, on one hand, the medical image report may cause trouble to the clinical doctors when the clinical doctors diagnose diseases according to the medical image report, on the other hand, the medical image report is provided to the patients, and the medical skill level and the quality of the medical doctors may be questioned by the patients due to the error in the medical image report, and even the doctor-patient contradiction may be caused. Therefore, error detection of medical image reports to control report quality is a very important issue.
Currently, error detection of medical image reports is mainly implemented by artificial intelligence technology means of Natural language processing (Natural LanguageProcessing, NPL), and more specifically, by using a rule-based modeling method. The modeling method based on rules needs to manually make word stock, a large amount of various errors need to be collected in the process of making the word stock, the problems of long time, high cost and difficulty in exhausting all possible error types exist, and error types outside the word stock cannot be identified. In addition, as most medical image reports are unstructured texts, the medical image reports of different categories also have larger differences in corpus range and semantic logic, if the medical image reports of different categories are independently modeled, the accuracy is high, but the application range is narrow, and the cost is higher; if the mixed modeling is performed for the medical image report major class, the applicability in different classes of medical image reports can be better considered, but the accuracy has limitation and lower accuracy.
In order to solve the above problems, the embodiments of the present application provide a technical solution for medical image report error detection. In order to enable those skilled in the art to better understand the present application, the following description will make clear and complete descriptions of the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application.
In some of the flows described in the specification, claims, and drawings described above, a plurality of operations occurring in a particular order are included, and the operations may be performed out of order or concurrently with respect to the order in which they occur. The sequence numbers of operations such as 101, 102, etc. are merely used to distinguish between the various operations, and the sequence numbers themselves do not represent any order of execution. In addition, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel. It should be noted that, the descriptions of "first" and "second" herein are used to distinguish different messages, devices, modules, etc., and do not represent a sequence, and are not limited to the "first" and the "second" being different types. Furthermore, the embodiments described below are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.
The method and the device are beneficial to being realized by a trained error detection model when error detection is carried out on the medical image report. Fig. 1 shows a flow chart of an error detection model training method provided in an embodiment of the present application, where an execution body of the method may be an electronic device with logic processing capability, and the electronic device may be a client or a server. The client may be hardware integrated on the terminal and provided with an embedded program, or may be an application software installed in the terminal, or may be a tool software embedded in an operating system of the terminal, which is not limited in this embodiment. The terminal may be any device with certain computing capabilities, such as a smart phone, a notebook computer, a smart wearable device, a PDA (Personal Digital Assistant ), a desktop computer, etc. The server may be a single server, a server cluster formed by a plurality of servers, a cloud or virtual server, and the embodiment is not limited in detail. In the above description, the error detection model is used for performing error detection on the medical image report. As shown in fig. 1, the error detection model training method provided in this embodiment includes the following steps:
101. Determining a first recognition model, the first recognition model being trained based on a plurality of first text samples of the medical image report;
102. acquiring a plurality of second sample texts of the medical image report; the second sample text comprises at least one sample sentence, wherein the sample sentence with a first label in the at least one sample sentence is an error sample sentence containing an error word, and the sample sentence without the first label is a correct sample sentence without the error word;
103. training the first recognition model based on at least one sample sentence included in the second samples to obtain a second recognition model;
104. in the training process of the first recognition model, if the first recognition model is monitored that the first recognition model cannot recognize a first error sample sentence in the at least one sample sentence, the first error sample sentence is sent to a user, so that the user determines a first error word in the first error sample sentence and adds the first error word to an error detection rule set;
105. and determining the error detection model according to the second identification model and the error detection rule set.
In the above 101, the first sample text is a plurality of sample texts, which are generated according to the acquired plurality of historical medical image report information, and description of the historical medical image report information is referred to below. In order to reduce training cost, the embodiment uses a first sample text of a medical image report to train to obtain a first recognition model based on a model fine tuning method. The method based on model fine tuning refers to: training on sample data of a new task by using the existing pre-training model resources for solving the problem similar to the new task, so as to finely tune parameters of the pre-training model, and enabling the finely tuned pre-training model to be adapted to the new task, thereby rapidly obtaining a model adapted to the new task; the pre-training model is a model obtained by training on a large standard data set. The model adaptive to the new task is obtained by training based on the model fine tuning method, so that the model required by training the new task from scratch can be avoided, a better model training effect can be obtained by using less training sample data, and the training cost can be greatly reduced. Based on the foregoing, in one implementation technical solution, the "determining the first recognition model" in the foregoing 101 may specifically include:
1011. Acquiring a plurality of historical medical image report information;
1012. generating the first sample texts according to the historical medical image report information;
1013. training a pre-training language model to be trained by using the plurality of first text samples to obtain the first recognition model.
In the 1011, the plurality of history medical image report information includes a plurality of history medical image reports and a second tag corresponding to each history medical image report. The second tag is used to identify whether the corresponding historical medical image report is a correct medical image report. Proper medical image reporting refers to: the report text content in the medical image report is completely correct, and no error content (such as error vocabulary, error words, etc.) exists. For example, if the first label is "1", it indicates that the historical medical image report corresponding to the first label is a correct medical image report; if the first label is "0", it indicates that the historical medical image report corresponding to the first label is an erroneous medical image report.
In order to adapt the trained model to various types of medical image reports, the historical medical image reports may be extracted from the medical image report sets by sampling. The sampled historical medical image report can be sent to a client used by a user (such as an affine physician) to be displayed on an interactive interface provided by the client so as to mark the historical medical image report by the user, thereby obtaining a second label of the historical medical image report. That is, a specific implementation of the above 1011 "obtaining a plurality of historical medical report information" may include the following steps:
10111. Acquiring a plurality of historical medical image report sets; wherein a plurality of historical medical image reports in a set of historical medical image reports belong to the same category;
10112. sampling a plurality of historical image report sets to extract a set number of historical image reports from each historical image report set;
10113. extracting a set number of historical image reports from each historical image report set respectively, and sending the historical image reports to a user so that the user marks a corresponding second label for the received historical image reports;
10114. and determining a plurality of pieces of historical medical report information according to the set number of historical image reports and the corresponding second labels respectively extracted from each historical image report set.
In particular, the set of historical medical image reports may be constructed by a user (e.g., radiologist) collecting a number (e.g., 2 thousands, 5 thousands, 1 thousands, etc.) of historical medical image reports, which may be derived from, but not limited to, a corresponding medical image report database. The types of the historical medical image reports are divided according to human body parts, and can comprise image reports of parts such as head, neck, chest, abdomen, basin, limbs and the like; divided by imaging modality of medical images, image reports that may include X-ray, CT (Computed Tomography, electronic computed tomography), MRI (Magnetic Resonance Imaging ), etc.; the modal partitioning may include panning or enhancing, etc. of the image report. When sampling is carried out on a plurality of historical image report sets, sampling can be carried out according to the same proportion, and sampling balance among various historical image reports is ensured. After the extracted historical image report is sent to the user, the user can use a text labeling tool (such as a Brat tool) to label the received historical image report.
It should be noted that, in the case that each of the historical medical image reports in the historical medical image report set has been labeled with a corresponding second label in advance, the step 10113 may not be executed any more, and the acquisition of the plurality of historical medical report information may be completed by directly sampling only the plurality of historical medical image report sets.
In 1012 above, as with the example of medical image reporting shown in fig. 2a, a medical image report generally includes, but is not limited to: patient information (e.g., patient name, age, sex, etc.), medical images (e.g., image map), descriptive text associated with medical images (e.g., image description (also referred to as diagnostic view, content obtained for a doctor from machine-acquired images, e.g., atrial positive #,) diagnostic comments), date of examination, physician information (e.g., physician name), etc. Since the description text related to the medical image in the medical image report is closely related to the condition of the patient, the description text is important text content affecting the quality of the medical image report. For this reason, in the training of the model, the description text related to the medical image in the medical image report is mainly used as a training sample. Based on this, 1012 "generate the plurality of first samples according to the plurality of historical medical image reports" may be implemented by the following specific steps:
10121. Extracting descriptive text related to medical images in the plurality of historical medical image reports;
10122. processing the description text to obtain a processed description text conforming to the preset text length;
10123. determining a third label of the processed description text corresponding to the plurality of historical medical image reports according to the second labels corresponding to the historical medical image reports;
1024. and determining positive sample text and negative sample text from the processed description text corresponding to the plurality of historical medical image reports according to the third label.
In particular, a related text extraction algorithm may be used to extract descriptive text related to medical relevance in the historical medical image report. Since the length of the text required to be input by the pre-training language model is always certain when the corresponding pre-training language model is used for training in the follow-up, the extracted descriptive text needs to be aligned, so that the length of the text of the descriptive text after processing can meet the length of the input text required by the pre-training language model, namely the preset text length is the length of the input text required by the pre-training language model. Specifically, if the text length of the description text is greater than the preset text length, the description text may be intercepted, for example, the header of the description text may be intercepted as the starting position; if the text length of the description text is smaller than the preset text length, the description text can be filled, for example, characters (such as punctuation marks, numbers, letters, spaces and the like) can be filled at the tail part of the description text; if the text length of the descriptive text is equal to the preset text length, the descriptive text is not processed.
And then, directly using the second label of the historical medical image report as a third label of the processed description text corresponding to the historical medical image report, or automatically labeling a new third label for the processed description text corresponding to the historical medical image report based on the second label of the historical medical image report, wherein the third label is used for identifying whether the corresponding processed description text is a correct description text. For example, if the third tag identifies that the corresponding post-processing descriptive text is correct text without error content, the post-processing descriptive text is positive sample text; if the third tag identifies that the corresponding post-processing descriptive text is an error text containing error content, the post-processing descriptive text is a negative sample text.
It should be noted that step 10121 is an optional step. For example, if the user is to collect the historical medical image report to construct the set of historical medical image reports, if the historical medical image report is obtained by the user from the corresponding medical image report database and the derived report item selected at the time of the derivation is only descriptive text related to the medical image, that is, the historical medical image report includes text content only descriptive text related to the medical image, the step 10121 may not be executed. In addition, since the plurality of historical medical image reports are sampled from the historical medical image report set, the historical medical image reports in the historical medical image report set may have problems of repetition, incompleteness and the like, and based on the historical medical image reports, preprocessing may be performed before the corresponding sample text is generated by using the plurality of historical medical image reports to ensure the quality of the finally generated sample text. That is, before the step 10121, the method provided in this embodiment may further include the following steps:
10120. Preprocessing a plurality of historical medical image reports; wherein the pretreatment comprises at least one of the following: cleaning reports (e.g., removing duplicate reports), deleting invalid reports (e.g., deleting incomplete reports, deleting non-medical image reports, etc.).
In 1013 above, the pre-trained language model may be, but is not limited to: the language-dependent machine learning network model (such as a neural network model, a deep neural network model, etc.) may be, for example, a BERT (Bidirectional Encoder Representation from Transformers, transform-based bi-directional coding language characterization) model, a GPT (generating Pre-Training) model, a ERNIE (Enhanced Language Representation with Informative Entities) model, or other natural language model. Training the pre-training language model is stopped after a preset training stopping condition is reached, and the trained pre-training language model is determined to be a first recognition model; the training stop condition may be, but is not limited to, a recognition error rate of the model, the number of training times, or the like. Based on this, in a specific implementation technical solution, the foregoing 1013 "training the pre-training language model to be trained to obtain the first recognition model by using the plurality of first samples" may be implemented by the following specific steps:
10131. Inputting the first sample texts into the pre-training language model, and executing the pre-training language model to obtain first prediction labels of the first sample texts;
10132. determining a first recognition error rate according to the first prediction tag and the third tags corresponding to the plurality of first sample texts;
10133. if the first recognition error rate is greater than or equal to a first threshold, fine tuning parameters of the pre-training language model, and returning to execute the step 10131;
10134. and if the first recognition error rate is smaller than the first threshold value, stopping training the pre-training language model, and determining the trained pre-training language model as the first recognition model.
In specific implementation, the first sample text, that is, the post-processing description text referred to above, and correspondingly, the third label corresponding to the first sample text, that is, the third label referred to above, refers to the post-processing description text. When determining the first recognition error rate, different indexes used for representing the performance of the model, such as common indexes of loss, accuray (accuracy) and the like, can be adopted to determine according to a first prediction label and a third label corresponding to a plurality of first samples; the calculation of the index may be performed on a training data set (i.e. the first plurality of samples described above), but is not limited to, so as to understand the "learning" condition of the model.
For example, after each iteration training of the pre-training language model is finished, a corresponding loss value can be calculated as a first recognition error rate by using the loss function according to a first prediction tag and a third tag corresponding to a plurality of first sample texts. The loss function may be, but is not limited to, a cross entropy loss function (also called log loss function), whose corresponding expression is as follows:
wherein N is the total number of samples of the first sample text; y is i A third label representing the ith first sample text, the third label being capable of representing the corresponding first sample text as either positive sample text or negative sample text, such as: if the third label is 1, the corresponding first sample text is represented as positive sample text; if the third label is 0, the corresponding first sample text is represented as negative sample text. q represents the probability distribution of the sample tags of the N first texts (i.e. the third tag described above); p represents the probability distribution of predictive labels of N first sample texts (i.e. the first predictive labels described above), in particular P (y i ) Representing the ith first sample y i Is the predictive probability (i.e., predictive label) of the positive sample text. H p (q) is the total Loss value. Further, assuming that there are L total network layers in the pre-trained language model, the Loss value loss_L of the jth network layer j The method comprises the following steps:
Loss_L j ＝-W j [Y j *logX j +(1-Y j )*log(1-X j )]
wherein X is j For the input of the jth network layer, Y j For the j-th network layerOutput predictive label, W j Is a parameter (weight) of the j-th network layer.
If H is calculated as above p When the pre-training language model is greater than or equal to a preset loss threshold (i.e., a first threshold), it indicates that the performance of the pre-training language model does not reach the requirement, and at this time, the parameters of the pre-training language model are fine-tuned, specifically, according to the actual training situation, the parameters (weights) of all network layers of the pre-training language model may be adjusted, or the parameters of a part of network layers of the pre-training language model may also be adjusted, which is not limited herein; wherein, in the process of fine tuning parameters of the pre-training language model, a gradient descent method can be used, but is not limited to. Thereafter, based on the fine-tuned pre-trained language model, the above step 10131 is performed back. If H p And if the performance of the pre-training language model reaches the requirement, training of the pre-training language model is stopped, and the pre-training language model is determined to be a first recognition model.
For another example, the prediction labels of the first sample texts can be compared with the corresponding third labels to determine the number n of correctly identified sample texts in the plurality of first sample texts; then, the ratio of the number N of correctly recognized sample texts to the total number N of samples of the plurality of first sample texts is calculated, so as to obtain the recognition accuracy (i.e., N/N). The first recognition error rate is a difference between a set value (e.g. integer 1) and the recognition accuracy, so that it can be determined whether to stop training the pre-training language model according to the first recognition error rate.
FIG. 2b illustrates an example of training a pre-trained language model using a plurality of first samples.
In 102, the plurality of second sample texts may be generated according to a certain number (e.g., 500) of historical medical image reports collected by the user (e.g., radiologist) and including various errors, where the user may only mark the erroneous sentences in which there are errors when marking the historical medical image reports with errors, and does not need to mark specific error contents, because the training manner adopted in the subsequent training of the first recognition model by using the second sample texts is weak supervision. The weak supervision is a training method for giving a small quantity of labels or incomplete labels to the model, so that the model can learn autonomously according to the small quantity of label information, and the characteristics of the overall sample can be inferred. In this way, the labeling method is adopted, so that the generated sample sentences with labels (i.e. the first labels) in the second sample texts are error sample sentences containing error words, and the unlabeled sample sentences are correct sample sentences without error words.
For specific implementation of the second sample text generation, reference may be made to the specific implementation of the first sample text generation described above.
After the plurality of second sample texts are obtained, sample sentences contained in the plurality of second sample texts can be used for training the first recognition model, and training can be achieved through the training process of the pre-training language model. Based on this, in one implementation solution, 103 "training the first recognition model based on at least one sample sentence included in the plurality of second sample texts to obtain a second recognition model" may specifically include:
1031. inputting the at least one sample sentence into the first recognition model, and executing the first recognition model to obtain a second prediction tag of the at least one sample sentence;
1032. determining a second recognition error rate according to the second prediction tag and the first tag information of the at least one sample sentence;
1033. if the second recognition error rate is greater than or equal to a second threshold value, performing fine adjustment on parameters of the first recognition model, and returning to the step of executing the at least one sample sentence input to the first recognition model, and executing the first recognition model to obtain a prediction label of the at least one sample sentence;
1034. And if the second recognition error rate is smaller than a second threshold value, stopping training the first recognition model, and determining the trained first recognition model as the second recognition model.
In implementation, according to the second prediction tag and the first tag information corresponding to at least one sample statement, different indexes for representing the performance of the model are adopted to determine the second recognition error rate, such as common indexes of loss, accuray and the like. Regarding the calculation of the second recognition error rate, reference may be made to the example of the calculation of the first recognition error rate given above. After that, for the description of performing the above steps 1033 to 1034 according to the second recognition error rate, reference may also be made to the examples given above for the first recognition error rate.
Further, in the training process, the error sentence which is not correctly identified in at least one sample sentence can be sent to the user, so that the error word in the error sample sentence which is not successfully identified by the model is manually extracted, a corresponding word stock (namely an error detection rule set described below) is established, and errors which are still not identified after the model is trained later can be conveniently supplemented by adopting a rule-based method. That is, the step 103 may further include the following specific steps:
1035. Comparing the first prediction tag corresponding to the first error sample sentence in the at least one sample sentence with the first tag;
1036. if the second prediction label corresponding to the first error sample sentence is consistent with the first label, the first error sample sentence is identified;
1037. if the second prediction label corresponding to the first error sample sentence is inconsistent with the first label, the first error sample sentence is not recognized, and the operation of "sending the first error sample sentence to the user" in 104 is triggered.
In 104-105, the error detection rule set may be added to the second recognition model to obtain an error detection model, so as to improve performance of the error detection model.
In the technical scheme provided by the embodiment, when an error detection model for carrying out error detection on a medical image report is trained, a first recognition model is determined first, and the first recognition model is obtained by training a plurality of first text samples based on the medical image report; and then, acquiring a plurality of second sample texts comprising at least one sample sentence of the medical image report, wherein the sample sentence with the first label in the at least one sample sentence is an error sample sentence containing an error word, and the sample sentence without the first label is a correct sample sentence without the error word, and training the first recognition model by utilizing the at least one sample sentence comprising the acquired plurality of first sample texts to obtain a second recognition model. Through training by utilizing at least one sample sentence included in the first sample text and the second sample text in the two training stages, the second recognition model obtained through training has higher understanding ability on text semantics, and the accuracy of error recognition can be effectively improved. Further, in the training process of the first recognition model, if it is monitored that the first recognition model cannot recognize the first error sample sentence in the at least one sample sentence, the first error sample sentence is sent to the user, so that the first error word in the first error sample sentence is determined and added to the error detection rule set; finally, the error detection model is determined according to the second recognition model and the error detection rule set, and the error detection rule set is combined to supplement the difficult errors which cannot be recognized by the second recognition model obtained through training in the two training stages, so that the error recognition precision of the error detection model can be further improved, and the problems of large burden and high cost of manually constructing a word stock (such as the error detection rule set) in the process of directly constructing the error detection model based on a rule method can be avoided. In summary, the scheme can train with low training cost to obtain the required error detection model with better performance.
Furthermore, in order to verify the performance of the error detection model, the error detection model can be tested by a test sample text set corresponding to the medical image report, and the error detection model is optimized when the performance of the error detection model is determined to not meet the requirement according to the test result. That is, the method provided in this embodiment may further include the following steps:
106. acquiring a test sample text set of a medical image report;
107. based on the test sample text set, the detection model is tested to optimize the error detection model.
In a specific implementation, the test sample text set includes a plurality of third sample texts and fourth tag information of each third sample text, the plurality of test sample texts are generated according to a plurality of test sample medical image reports, and for specific implementation of test sample generation, reference may be made to related descriptions generated by the first sample text or the second sample text. The fourth tag information of one third sample text may include at least one fourth tag, one fourth tag identifying any one of the following: the third sample text is a positive sample text or a negative sample text, an error sentence, an error word, and the like. When testing the error detection model by using the test sample text set, whether the model performance needs to be optimized can be determined according to the error detection error rate of the error detection model. Based on this, in one implementation solution, 107 "test the detection model to optimize the error detection model based on the test sample text set" may specifically include:
1071. Inputting the plurality of third sample texts into the error detection model, and executing the test result of the tag information to the error detection model to obtain a test result containing third prediction tag information corresponding to each third sample text;
1072. determining the error detection error rate of the error detection model by the test result and the fourth label information;
1073. if the error detection error rate is greater than or equal to a third threshold value, optimizing the error detection model based on the test result and the fourth tag information, and returning to the step of executing the test sample text set based on the test sample text set to test the error detection model to optimize the error detection model;
1074. and if the error detection error rate is smaller than the third threshold value, stopping testing the error detection model.
In the specific implementation, the values of indexes such as loss and Accuray can be calculated as the error detection error rate of the error detection model according to the third prediction tag information corresponding to each third sample text and the fourth tag information corresponding to each third sample text, which are included in the test result. For a specific implementation of the calculation of the values of the indicators loss, accuray, etc., reference is made to the examples given above for the first recognition error rate. When the error detection error rate is greater than or equal to a third threshold value, the error detection model can be optimized by supplementing error words which cannot be identified by the error detection model in the third sample text to the error detection rule set. That is, in a specific implementation manner, "optimize the error detection model based on the test result and the third tag information" in 1073 may be implemented by the following specific steps:
10731. Determining second error words which cannot be identified by the error detection model in the plurality of third sample texts based on the test result and the fourth label information, and adding the error detection rule set;
10732. based on the added error detection rule set, the above 105 is triggered.
In particular, the determination of the second error word and the addition of the second error word to the error detection rule set may be performed by the user. Based on this, 10731 "determine, based on the test result and the third tag information, a second error word that cannot be identified by the error detection model in the plurality of third sample texts and add the error detection rule set" may be implemented by the following specific steps:
107311, determining a target third sample text in which third predicted tag information in the plurality of third sample texts is different from corresponding fourth tag information based on the test result and the fourth tag information;
107312, sending the target third sample text and the difference tag information between the fourth prediction tag information and the fourth tag information corresponding to the target third sample text to a user, determining a second error word which cannot be identified by the error detection model from the target third sample text according to the difference tag information by the user, and adding the second error word to the error detection rule set.
For example, if there is a difference between the third predictive label information corresponding to the third sample text and the fourth label information corresponding to the third sample text, the third predictive label information corresponding to the third sample text and the pair thereof can be determinedDifferential tag information between the fourth tag information to be used, such as: if the third predictive label a in the third predictive label information 11 And a fourth tag a in the fourth tag information 21 The difference label information includes the third prediction label a when the difference label information is the label of the sample sentence a in the target third sample text A and the label is inconsistent (i.e. there is a difference between the two) 11 And a fourth label a 21 . And then, the difference label information can be carried in a corresponding target third sample text A and sent to a client used by a user, and the difference label information is displayed on an interactive interface provided by the client, so that the user can quickly position the position, which cannot be identified by an error detection model, in the target third sample text according to the displayed difference label information, further determine a second error word, which cannot be identified by the error detection model, and add the second error word to an error detection rule set. Responding to the addition completion operation triggered by the user, namely triggering the error detection rule set based on the addition, and returning to execute the step 105 to obtain an optimized error detection model; further, the above step 1071 is executed again based on the optimized error detection model, so as to continue testing and optimizing the error detection model until the test stopping condition is met.
It should be noted that, the above threshold values, such as the first threshold value, the second threshold value, the third threshold value, and the like, are flexibly set according to practical situations. For example, the first threshold may be 50%, that is, training the pre-trained language model until the recognition error rate is less than 50% and stopping training, thereby obtaining a first recognition model; the second threshold may be 30%, that is, training the first recognition model until the recognition error rate is less than 30%, and stopping training, thereby obtaining a second recognition model; the third threshold may be 10%, that is, the error detection model is tested to have an identification error rate less than 10%, and then the test and optimization are stopped, so as to obtain the final error detection model.
In summary, the error detection model training process provided by the above-described embodiments of the application can be summarized simply as three phases as shown in FIG. 3:
the first stage: fine tuning training phase based on pre-training language model
Because human language is very complex, it is very difficult to train an identification model for identifying text from scratch, and currently, the dominant technology is to train the preliminary model continuously in the sub-domain through a pre-trained preliminary model (i.e., a pre-training language model) published in public, so as to obtain a model capable of meeting new tasks. Based on the method, because the medical image report is a part of Chinese big data, and has general Chinese characteristics and special medical words and expression methods, in order to reduce training cost, the method adopts a fine-tuning training method by means of a pre-training language model in a first training stage, and trains the pre-training language model by utilizing a plurality of first texts generated by a large number of medical image reports, so that a first recognition model obtained by training can be used for primarily understanding the combination rule of common words and words in medical imaging.
The first stage can be understood as training the model from the whole medical image report, so that the first recognition model obtained by training can simply recognize whether the whole medical image report has errors or not, and the first recognition model is not required to specifically recognize specific error contents in the medical image report.
And a second stage: weak supervision learning training phase based on incomplete error labeling
The weak supervision learning is a training method for giving a small amount of labels or incomplete labels to a model, so that the model can learn autonomously according to the small amount of label information, thereby reasoning the characteristics of a general sample. In this stage, the first recognition model is trained by using weak supervised learning to obtain the second recognition model, so that when a doctor collects a sample, only the whole sentence or paragraph containing an error in the medical image report of the sample needs to be marked, the specific error keyword or word in the sentence does not need to be standardized, and the model can learn the semantic features of the error sentence or paragraph by itself in the training process.
The second stage can be understood as training the model from the sentences in the medical image report, so that the trained second recognition model can specifically recognize the specific error content in the medical image report. For example, during training, a sentence such as "left lung nodule, discomfort follow-up treatment" is input to the first recognition model, and a partial word such as "discomfort follow-up treatment" in the sentence is not input.
And a third stage: and carrying out rule supplement on the second recognition model based on the recognition method of the rule.
Rule-based recognition methods (i.e., the rule-based modeling methods referred to above) refer to specifying a number of particular keywords or words that, once they appear in a sentence, are determined to be erroneous. The rule-based identification method is limited in that: only fixed combinations of erroneous words can be identified, and the number and type of erroneous words is not exhaustive or quantitative. In the present application, for errors that cannot be identified after training in the two model training phases, a rule-based method is adopted to supplement the second identification model, so as to finally obtain an error detection model that meets the requirements.
In summary, the scheme of the application is based on natural language processing technology, and the required error detection model is obtained through training in three stages. Specifically, in the first two stages, the models are trained through a fine tuning training method and a weak supervision learning training method, so that the improvement of the text semantic understanding capability of the models is realized, and the trained models (second recognition models) can recognize the characteristics of the wrong words and the overall semantic and word sequence characteristics of sentences or paragraphs and the like containing the wrong words. And then, on the basis of the second recognition model obtained through the training of the two stages, the recognition method based on the rules is used for supplementing the difficult errors which cannot be recognized by the model, so that an error detection model is constructed, the problem that a doctor needs to carry out large-scale rule marking when the required error detection model is constructed by directly using the recognition method of the rules can be avoided, and the task amount of manually constructing a word stock can be reduced.
Fig. 4 is a schematic flow chart of a medical image report detection method according to another embodiment of the present application, where an execution subject of the method may be an electronic device with logic processing capability, and the electronic device may be a client or a server. For specific description of the client or the server, reference may be made to the relevant content in the other embodiments above. As shown in fig. 4, the method provided in this embodiment includes the following steps:
201. determining a target medical image report to be detected;
202. obtaining an error detection model, wherein the monitoring model is obtained by training a training method of the error detection model, which is provided by the embodiment of the application and shown in fig. 1;
203. and carrying out error detection on the target medical image report by utilizing the error detection model.
In particular, the target medical image report may be an electronic image report that has just been written by a user (e.g., radiologist). After the user composes the corresponding target medical image report, a determination operation may be clicked. The execution main body responds to the operation triggered by the user, acquires an error detection model and utilizes the error detection model to perform error detection on the target medical image report; in addition, the error detection result can be sent to the user to be displayed on the client used by the user. If the displayed error detection result prompts that the target medical image report has errors, the user can also quickly modify the error content in the target medical image report according to the displayed error detection result. Based on this, the method provided in this embodiment may further include:
204. Displaying the error detection result to a user;
205. and updating the target medical image report in response to a user modifying operation of the target medical report based on the error detection result.
Fig. 5 shows a medical image report detection system provided in a further embodiment of the present application, the medical image report detection system comprising: a server side 32 and a client side 31; wherein,,
the server 32 determines a target medical image report to be detected; obtaining an error detection model, wherein the error detection model is obtained through training by the error detection model training method provided by the embodiment of the application; and carrying out error detection on the target medical image report by utilizing the error detection model.
The client 31 is configured to provide an interactive interface, and display the error detection result of the target medical image report on the interactive interface.
For details of the server and the client, reference may be made to the related content in the other embodiments, and details are not repeated here. In addition, the functions executed by the server and the client are not described in detail, and reference may be made to the related contents in other embodiments. In addition to the above steps, the medical image report detection system provided in the embodiment of the present application may further include other part or all of the steps in the above embodiments, and specifically, reference may be made to the corresponding content of the above embodiments, which is not repeated herein.
FIG. 6 is a schematic structural diagram of an error detection model training apparatus according to an embodiment of the present application, where the error detection model is used for error detection of a medical report. As shown in fig. 6, the error detection model training apparatus includes: a determining module 41, an acquiring module 42, a training module 43 and a monitoring and transmitting module 44; wherein,,
a determining module 41 for determining a first recognition model, the first recognition model being trained based on a plurality of first samples of the medical image report;
an acquisition module 42 for acquiring a plurality of second sample texts of the medical image report; the second sample text comprises at least one sample sentence, wherein the sample sentence with a first label in the at least one sample sentence is an error sample sentence containing an error word, and the sample sentence without the first label is a correct sample sentence without the error word;
a training module 43, configured to train the first recognition model based on at least one sample sentence included in the plurality of first samples, so as to obtain a second recognition model;
a monitoring and sending module 44, configured to send, in a training process for the first recognition model, if it is monitored that the first recognition model cannot recognize a first error sample sentence in the at least one sample sentence, the first error sample sentence to a user, so that the user determines a first error word in the first error sample sentence and adds the first error word to an error detection rule set;
The determining module 41 is further configured to determine the error detection model according to the second recognition model and the error detection rule set.
Further, the determining module 41, when used for determining the first recognition model, is specifically configured to: acquiring a plurality of historical medical image report information; generating the first sample texts according to the historical medical image report information; training a pre-training language model to be trained by using the plurality of first text samples to obtain the first recognition model.
Further, the plurality of historical medical image report information includes: a plurality of historical medical image reports and second labels corresponding to the historical medical image reports; the second label is used for identifying whether the corresponding historical medical image report is a correct medical image report or not; and
the determining module 41, when configured to generate a plurality of first sample texts according to the plurality of historical medical image report information, is specifically configured to: extracting descriptive text related to the medical images in the plurality of medical image reports; processing the description text to obtain a processed description text conforming to the preset text length; determining a third label of the processed description text corresponding to the plurality of historical medical image reports according to the second labels corresponding to the historical medical image reports; and determining positive sample text and negative sample text from the processed description text corresponding to the plurality of historical medical image reports according to the third label.
Further, the determining module 41, when configured to train the pre-training language model to be trained by using the plurality of first text samples to obtain the first recognition model, is specifically configured to: inputting the first sample texts into the pre-training language model, and executing the pre-training language model to obtain first prediction labels of the first sample texts; determining a first recognition error rate according to the first prediction tag and the third tags corresponding to the plurality of first sample texts; if the first recognition error rate is greater than or equal to a first threshold value, performing fine adjustment on parameters of the pre-training language model, and returning to the step of inputting the plurality of first sample texts into the pre-training language model, and executing the pre-training language model to obtain predictive labels of the plurality of first sample texts; and if the first recognition error rate is smaller than the first threshold value, stopping training the pre-training language model, and determining the trained pre-training language model as the first recognition model.
Further, the training module 43 is specifically configured to, when training the first recognition model based on at least one sample sentence included in the plurality of first texts to obtain a second recognition model: inputting the at least one sample sentence into the first recognition model, and executing the first recognition model to obtain a second prediction tag of the at least one sample sentence; determining a second recognition error rate according to the second prediction tag and the first tag information of at least one sample sentence; if the second recognition error rate is greater than or equal to a second threshold value, performing fine adjustment on parameters of the first recognition model, and returning to the step of executing the at least one sample sentence input to the first recognition model, and executing the first recognition model to obtain a prediction label of the at least one sample sentence; and if the second recognition error rate is smaller than a second threshold value, stopping training the first recognition model, and determining the trained first recognition model as the second recognition model.
Further, the error detection model training device provided in this embodiment further includes:
the comparison module is used for comparing the second prediction tag corresponding to the first error sample statement in the at least one sample statement with the first tag to obtain a comparison result;
the determining module 41 is further configured to determine, according to a comparison result, whether the second prediction tag corresponding to the first error sample sentence is consistent with the first tag; if the second prediction label corresponding to the first error sample sentence is consistent with the first label, the first error sample sentence is identified; if the second prediction label corresponding to the first error sample sentence is inconsistent with the first label, the first error sample sentence is not recognized, and the operation of sending the first error sample sentence to a user is triggered.
Further, the acquiring module 42 is further configured to acquire a test sample text set of the medical image report; and, the error detection training device provided in this embodiment further includes: and the test module is used for testing the error detection model based on the test sample text set so as to optimize the error detection model.
Further, the test sample set includes: a plurality of third sample texts and fourth tag information of each third sample text; and the test module is specifically configured to, when configured to test the error detection model based on the test sample text set to optimize the error detection model: inputting the plurality of third sample texts into the error detection model, and executing the error detection model to obtain a test result containing third prediction tag information corresponding to each third sample text; determining an error detection error rate of the error detection model based on the test result and the fourth tag information; if the error detection error rate is greater than or equal to a third threshold value, optimizing the error detection model based on the test result and the third tag information, and returning to the step of executing the test sample text set based on the test sample text set to test the error detection model to optimize the error detection model; and if the error detection error rate is smaller than the third threshold value, stopping testing the error detection model.
Further, the test module is specifically configured to, when configured to optimize the error detection model based on the test result and the fourth tag information: determining second error words which cannot be identified by the error detection model in the plurality of third sample texts based on the test result and the fourth label information, and adding the second error words to the error detection rule set; and triggering the operation of determining the error detection model according to the second identification model and the error detection rule set based on the added error detection rule set.
What needs to be explained here is: the error detection model training device provided in this embodiment may implement the technical solution described in the embodiment of the detection model training method shown in fig. 1, and the specific implementation principle of each module or unit may refer to the corresponding content in the embodiment of the detection model training method shown in fig. 1, which is not described herein.
Fig. 7 is a schematic structural diagram of a medical image report detection device according to another embodiment of the present application. As shown in fig. 7, the medical image report detection apparatus includes: a determining module 51, an acquiring module 52 and a detecting module 53; wherein,,
a determining module 51, configured to determine a target medical image report to be detected;
an obtaining module 52, configured to obtain an error detection model, where the error detection model is obtained by training an error detection model training method according to an embodiment of the present application as shown in fig. 1;
and the detection module 53 is configured to perform error detection on the target medical image report by using the error detection model.
What needs to be explained here is: the medical image report detection device provided in this embodiment may implement the technical solution described in the embodiment of the medical image report detection method shown in fig. 4, and the specific implementation principle of each module or unit may refer to the corresponding content in the embodiment of the medical image report method shown in fig. 4, which is not described herein.
Fig. 8 shows a schematic structural diagram of an electronic device according to an embodiment of the present application. As shown in fig. 8, the electronic device includes: a memory 61 and a processor 62. Wherein the memory 61 is configured to store one or more computer instructions; the processor 62 is coupled to the memory 61, and is configured to execute one or more computer instructions (e.g., those implementing data storage logic) for implementing the error detection model training method provided by the embodiments of the present application or for implementing the medical image report detection method provided by the embodiments of the present application.
The memory 61 may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.
Further, as shown in fig. 8, the electronic device may further include: communication component 63, power component 64, audio component 65, display 66, and other components. Only some of the components are schematically shown in fig. 8, which does not mean that the electronic device only comprises the components shown in fig. 8.
Accordingly, the embodiments of the present application further provide a computer readable storage medium storing a computer program, where the computer program when executed by a computer can implement steps or functions in the error detection model training method or the medical image report detection method provided in the foregoing embodiments.
The methods in this application may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. Fig. 9 schematically shows a block diagram of a computer program product provided by the present application. The computer program product comprises computer program/instructions 71 which, when the computer program/instructions 71 are executed by a processor, such as the processor 62 shown in fig. 7, may fully or partially perform the flow or functions of the error detection model training method or the medical image report detection method described herein. The computer may be a general purpose computer, a special purpose computer, a computer network, a network device, a user device, a core network device, an OAM, or other programmable apparatus.
The computer program or instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer program or instructions may be transmitted from one website site, computer, server, or data center to another website site, computer, server, or data center by wired or wireless means. The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that integrates one or more available media. The usable medium may be a magnetic medium, e.g., floppy disk, hard disk, tape; but also optical media such as digital video discs; but also semiconductor media such as solid state disks. The computer readable storage medium may be volatile or nonvolatile storage medium, or may include both volatile and nonvolatile types of storage medium.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and are not limiting thereof; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the corresponding technical solutions.
Claims (12)
1. The error detection model training method is characterized in that the error detection model is used for carrying out error detection on a medical image report; the method comprises the following steps:
determining a first recognition model, the first recognition model being trained based on a plurality of first text samples of the medical image report;
acquiring a plurality of second sample texts of the medical image report; the second sample text comprises at least one sample sentence, wherein the sample sentence with a first label in the at least one sample sentence is an error sample sentence containing an error word, and the sample sentence without the first label is a correct sample sentence without the error word;
training the first recognition model based on at least one sample sentence included in the plurality of second sample texts to obtain a second recognition model;
in the training process of the first recognition model, if the first recognition model is monitored that the first recognition model cannot recognize a first error sample sentence in the at least one sample sentence, the first error sample sentence is sent to a user, so that the user determines a first error word in the first error sample sentence and adds the first error word to an error detection rule set;
and determining the error detection model according to the second identification model and the error detection rule set.
2. The method of claim 1, wherein determining the first recognition model comprises:
acquiring a plurality of historical medical image report information;
generating the first sample texts according to the historical medical image report information;
training a pre-training language model to be trained by using the plurality of first text samples to obtain the first recognition model.
3. The method of claim 2, wherein the plurality of historical medical image reporting information comprises: a plurality of historical medical image reports and second labels corresponding to the historical medical image reports; the second label is used for identifying whether the corresponding historical medical image report is a correct medical image report or not; and
generating a plurality of first sample texts according to the plurality of historical medical image report information, wherein the first sample texts comprise:
extracting descriptive text related to the medical images in the plurality of medical image reports;
processing the description text to obtain a processed description text conforming to the preset text length;
determining a third label of the processed description text corresponding to the plurality of historical medical image reports according to the second labels corresponding to the historical medical image reports;
And determining positive sample text and negative sample text from the processed description text corresponding to the plurality of historical medical image reports according to the third label.
4. A method according to claim 3, wherein training a pre-training language model to be trained using the plurality of first text samples to obtain the first recognition model comprises:
inputting the first sample texts into the pre-training language model, and executing the pre-training language model to obtain first prediction labels of the first sample texts;
determining a first recognition error rate according to the first prediction tag and the third tags corresponding to the plurality of first sample texts;
if the first recognition error rate is greater than or equal to a first threshold value, performing fine adjustment on parameters of the pre-training language model, and returning to the step of inputting the plurality of first sample texts into the pre-training language model, and executing the pre-training language model to obtain predictive labels of the plurality of first sample texts;
and if the first recognition error rate is smaller than the first threshold value, stopping training the pre-training language model, and determining the trained pre-training language model as the first recognition model.
5. The method of any of claims 1-4, wherein training the first recognition model to obtain a second recognition model based on at least one sample sentence included in the plurality of second sample texts comprises:
inputting the at least one sample sentence into the first recognition model, and executing the first recognition model to obtain a second prediction tag of the at least one sample sentence;
determining a second recognition error rate according to the second prediction tag and the first tag information of at least one sample sentence;
if the second recognition error rate is greater than or equal to a second threshold value, performing fine adjustment on parameters of the first recognition model, and returning to the step of executing the at least one sample sentence input to the first recognition model, and executing the first recognition model to obtain a prediction label of the at least one sample sentence;
and if the second recognition error rate is smaller than a second threshold value, stopping training the first recognition model, and determining the trained first recognition model as the second recognition model.
6. The method as recited in claim 5, further comprising:
Comparing the second prediction tag corresponding to the first error sample sentence in the at least one sample sentence with the first tag;
if the second prediction label corresponding to the first error sample sentence is consistent with the first label, the first error sample sentence is identified;
if the second prediction label corresponding to the first error sample sentence is inconsistent with the first label, the first error sample sentence is not recognized, and the operation of sending the first error sample sentence to a user is triggered.
7. The method according to any one of claims 1 to 4, further comprising:
acquiring a test sample text set of a medical image report;
based on the test sample text set, the error detection model is tested to optimize the error detection model.
8. The method of claim 7, wherein the test sample set comprises: a plurality of third sample texts and fourth tag information of each third sample text; and
testing the error detection model to optimize the error detection model based on the test sample text set, comprising:
inputting the plurality of third sample texts into the error detection model, and executing the error detection model to obtain a test result containing third prediction tag information corresponding to each third sample text;
Determining an error detection error rate of the error detection model based on the test result and the fourth tag information;
if the error detection error rate is greater than or equal to a third threshold value, optimizing the error detection model based on the test result and the fourth tag information, and returning to the step of executing the test sample text set based on the test sample text set to test the error detection model to optimize the error detection model;
and if the error detection error rate is smaller than the third threshold value, stopping testing the error detection model.
9. The method of claim 8, wherein optimizing the error detection model based on the test result and the fourth tag information comprises:
determining second error words which cannot be identified by the error detection model in the plurality of third sample texts based on the test result and the fourth label information, and adding the second error words to the error detection rule set;
and triggering the operation of determining the error detection model according to the second identification model and the error detection rule set based on the added error detection rule set.
10. A medical image report detection method, comprising:
determining a target medical image report to be detected;
Obtaining an error detection model, wherein the error detection model is trained by the error detection model training method according to any one of claims 1 to 9;
and carrying out error detection on the target medical image report by utilizing the error detection model.
11. A medical image report detection system, comprising:
the server is used for determining a target medical image report to be detected; obtaining an error detection model, wherein the error detection model is trained by the error detection model training method according to any one of claims 1 to 9; performing error detection on the target medical image report by using the error detection model;
the client is used for providing an interactive interface; and displaying the error detection result reported by the target medical image on the interactive interface.
12. An electronic device, comprising: a memory and a processor, wherein,
the memory is used for storing a computer program;
the processor is coupled to the memory for executing the computer program stored in the memory for implementing the detection model training method according to any one of claims 1 to 9 or for implementing the medical image report detection method according to claim 10.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310036606 | 2023-01-10 | ||
| CN202310036606X | 2023-01-10 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116258136A true CN116258136A (en) | 2023-06-13 |
Family
ID=86680529
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310144833.4A Pending CN116258136A (en) | 2023-01-10 | 2023-02-14 | Error detection model training method, medical image report detection method, system and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116258136A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118093527A (en) * | 2024-04-24 | 2024-05-28 | 脉得智能科技(无锡)有限公司 | Report quality inspection method and device and electronic equipment |
-
2023
- 2023-02-14 CN CN202310144833.4A patent/CN116258136A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118093527A (en) * | 2024-04-24 | 2024-05-28 | 脉得智能科技(无锡)有限公司 | Report quality inspection method and device and electronic equipment |
| CN118093527B (en) * | 2024-04-24 | 2024-08-16 | 脉得智能科技(无锡)有限公司 | Report quality inspection method and device and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10929420B2 (en) | Structured report data from a medical text report | |
| US11282196B2 (en) | Automated patient complexity classification for artificial intelligence tools | |
| US20220044812A1 (en) | Automated generation of structured patient data record | |
| CN111540468B (en) | ICD automatic coding method and system for visualizing diagnostic reasons | |
| US8612261B1 (en) | Automated learning for medical data processing system | |
| US11810654B2 (en) | Method and system for automatically generating a section in a radiology report | |
| CN110705293A (en) | Electronic medical record text named entity recognition method based on pre-training language model | |
| EP3956901A1 (en) | Computer-implemented natural language understanding of medical reports | |
| WO2020123723A1 (en) | System and method for providing health information | |
| CN112541066B (en) | Text-structured-based medical and technical report detection method and related equipment | |
| US10984024B2 (en) | Automatic processing of ambiguously labeled data | |
| WO2018188981A1 (en) | Drawing conclusions from free form texts with deep reinforcement learning | |
| CN111477320B (en) | Treatment effect prediction model construction system, treatment effect prediction system and terminal | |
| US11295080B2 (en) | Automatic detection of context switch triggers | |
| US11763081B2 (en) | Extracting fine grain labels from medical imaging reports | |
| CN113159134A (en) | Intelligent diagnosis evaluation method based on mammary gland structural report | |
| CN115910263A (en) | PET/CT image report conclusion auxiliary generation method and device based on knowledge graph | |
| Hartsock et al. | Vision-language models for medical report generation and visual question answering: A review | |
| CN116258136A (en) | Error detection model training method, medical image report detection method, system and equipment | |
| CN117216322A (en) | Method, device, equipment and storage medium for generating electronic medical record | |
| WO2023219836A1 (en) | Method for automating radiology workflow | |
| CN110060749B (en) | Intelligent electronic medical record diagnosis method based on SEV-SDG-CNN | |
| EP4300505A1 (en) | Medical structured reporting workflow assisted by natural language processing techniques | |
| JP2022128583A (en) | Medical information processing device, medical information processing method and medical information processing program | |
| CN113223698B (en) | Emergency hierarchical processing method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |