CN110362829B - Quality evaluation method, device and equipment for structured medical record data - Google Patents

Quality evaluation method, device and equipment for structured medical record data Download PDF

Info

Publication number
CN110362829B
CN110362829B CN201910639668.3A CN201910639668A CN110362829B CN 110362829 B CN110362829 B CN 110362829B CN 201910639668 A CN201910639668 A CN 201910639668A CN 110362829 B CN110362829 B CN 110362829B
Authority
CN
China
Prior art keywords
keyword
record data
medical record
type
structured
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910639668.3A
Other languages
Chinese (zh)
Other versions
CN110362829A (en
Inventor
张志立
黄海峰
李坤
肖珺
石晓坤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Baidu Netcom Science and Technology Co Ltd
Original Assignee
Beijing Baidu Netcom Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Baidu Netcom Science and Technology Co Ltd filed Critical Beijing Baidu Netcom Science and Technology Co Ltd
Priority to CN201910639668.3A priority Critical patent/CN110362829B/en
Publication of CN110362829A publication Critical patent/CN110362829A/en
Application granted granted Critical
Publication of CN110362829B publication Critical patent/CN110362829B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25—Integrating or interfacing systems involving database management systems
    • G06F16/258—Data format conversion from or to a database
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/205—Parsing
    • G06F40/216—Parsing using statistical methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H10/00—ICT specially adapted for the handling or processing of patient-related medical or healthcare data
    • G16H10/60—ICT specially adapted for the handling or processing of patient-related medical or healthcare data for patient-specific data, e.g. for electronic patient records
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
    • Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Abstract

The embodiment of the invention provides a method, a device and equipment for evaluating the quality of structured medical record data, wherein the method comprises the following steps: according to a medical type, acquiring a plurality of structured medical record data to be evaluated corresponding to the medical type from a medical database, wherein each structured medical record data comprises at least one key value pair data, and each key value pair data comprises a keyword and a value corresponding to the keyword; acquiring a labeling result corresponding to each piece of structured medical record data, wherein the labeling result comprises each keyword and a comparison result corresponding to each keyword; and evaluating the quality of the structured medical record data corresponding to the medical type according to the labeling result corresponding to each structured medical record data. Through the process, the quality evaluation process of the structured medical record data is realized; in addition, the quality of the structured medical record data of different medical types can be determined, and the accuracy of quality evaluation is improved.

Description

Quality evaluation method, device and equipment for structured medical record data
Technical Field
The embodiment of the invention relates to the technical field of computers, in particular to a method, a device and equipment for evaluating the quality of structured medical record data.
Background
At present, medical records are widely applied in the fields of clinical scientific research, public health big data, hospital data intelligent management, intelligent follow-up visit and the like. The original medical record is typically in an unstructured form. In order to improve the convenience of the medical record in the above fields, it is necessary to convert the original medical record into a structured form.
Specifically, the unstructured data in the original medical record is cleaned by adopting a data structuring processing technology, so that the unstructured data is converted into structured data. Illustratively, the data in the original medical record is processed into a structured form with uniform field naming in units of chapters, or features such as duration of symptoms, positivity and negativity, diagnosis certainty of diseases and the like are extracted from the original medical record. Through the data structuring processing process, the original medical records which can only be understood by doctors usually are converted into the structured medical records which can be understood by a computer.
After converting the original medical record data into the structured medical record data, the quality of the structured medical record data needs to be evaluated to determine the reliability of the structured processing algorithm.
Disclosure of Invention
The embodiment of the invention provides a method, a device and equipment for evaluating the quality of structured medical record data, which are used for evaluating the quality of the structured medical record data.
In a first aspect, an embodiment of the present invention provides a method for evaluating quality of structured medical record data, including:
according to a medical type, acquiring a plurality of structured medical record data to be evaluated corresponding to the medical type from a medical database, wherein the medical database is used for storing original medical record data of different medical types and structured medical record data obtained by carrying out structured processing on the original medical record data, each structured medical record data comprises at least one key value pair data, and each key value pair data comprises a keyword and a value corresponding to the keyword;
acquiring a labeling result corresponding to each structured medical record data, wherein the labeling result comprises each keyword and a comparison result corresponding to each keyword; the labeling result is obtained by comparing the structured medical record data with the original medical record data;
and evaluating the quality of the structured medical record data corresponding to the medical type according to the labeling result corresponding to each structured medical record data.
Alternatively to this, the first and second parts may,
the type of the comparison result corresponding to each keyword is any one of the following types: correct conversion, incorrect keyword recall, and unrerecalled keyword; the evaluating the quality of the structured medical record data corresponding to the medical type according to the labeling result corresponding to each structured medical record data comprises:
for each first keyword, obtaining a comparison result corresponding to the first keyword from the labeling results corresponding to the plurality of structured medical record data, wherein the first keyword is any one of the keywords;
obtaining first result analysis information according to a comparison result corresponding to the first keyword, wherein the first result analysis information comprises: the type of the comparison result is the number of the first keywords with correct conversion, the type of the comparison result is the number of the first keywords with wrong keyword recalls, and the type of the comparison result is the number of the first keywords with unrecalled keywords;
according to the first result analysis information, acquiring the accuracy and the recall rate corresponding to the first keyword;
and evaluating the quality of the structured medical record data corresponding to the medical type according to the accuracy and the recall rate corresponding to each keyword.
In the alternative,
the obtaining of the accuracy and the recall rate corresponding to the first keyword according to the first result analysis information includes:
acquiring the recalled number of the first keywords according to the comparison result type, the number of the first keywords with correct conversion, the number of the first keywords with wrong conversion and the number of the first keywords with wrong keyword recall;
acquiring the accuracy corresponding to the first keyword according to the number of the first keyword which is correctly converted and the number of recalled first keywords as the type of the comparison result;
and acquiring the recall rate corresponding to the first keyword according to the recalled quantity of the first keyword, the quantity of the first keyword with the type of the comparison result as the keyword error recall and the quantity of the first keyword with the type of the comparison result as the keyword unrerecalling.
Alternatively to this, the first and second parts may,
the type of the comparison result corresponding to each keyword is any one of the following types: correct conversion, incorrect keyword recall, and unrerecalled keyword; the evaluating the quality of the structured medical record data corresponding to the medical type according to the labeling result corresponding to each structured medical record data comprises:
according to the labeling result corresponding to each structured medical record data, second result analysis information is obtained, and the second result analysis information comprises: the type of the comparison result is the number of each keyword which is correctly converted, the type of the comparison result is the number of each keyword which is wrongly recalled by the keyword, and the type of the comparison result is the number of each keyword which is not recalled by the keyword;
and according to the second result analysis information, acquiring the accuracy and the recall rate corresponding to the structured medical record data corresponding to the medical type.
In the alternative,
the labeling result further comprises a correction result, and the correction result comprises: the type of the comparison result is a correction value corresponding to each keyword with conversion error, and the type of the comparison result is each keyword which is not recalled by the keyword and a value corresponding to each keyword;
after the quality of the structured medical record data corresponding to the medical type is evaluated according to the labeling result corresponding to each structured medical record data, the method further comprises the following steps:
displaying abnormal data in the plurality of structured medical record data, and displaying the correction result; and the abnormal data is key value pair data with the type of the comparison result being a conversion error, a keyword error recall or a keyword non-recall.
Alternatively to this, the first and second parts may,
the acquiring, according to the medical type, a plurality of structured medical record data to be evaluated corresponding to the medical type from a medical database includes:
acquiring the medical type to be evaluated and the maximum evaluation number according to the evaluation task;
acquiring a plurality of structured medical record data to be evaluated corresponding to the medical type from a medical database according to the medical type and the maximum evaluation quantity; wherein the number of structured medical record data is less than or equal to the maximum number of evaluations.
Alternatively to this, the first and second parts may,
the acquiring, from a medical database, a plurality of pieces of structured medical record data to be evaluated corresponding to the medical type according to the medical type and the maximum evaluation quantity includes:
acquiring a target set from the medical database, wherein the target set comprises structured medical record data corresponding to the medical type;
if the quantity of the structured medical record data in the target set is larger than the maximum evaluation quantity, randomly selecting the structured medical record data with the maximum evaluation quantity from the target set as the structured medical record data to be evaluated;
and if the quantity of the structured medical record data in the target set is less than or equal to the maximum evaluation quantity, taking all the structured medical record data in the target set as the structured medical record data to be evaluated.
In a second aspect, an embodiment of the present invention provides an apparatus for evaluating quality of structured medical record data, including:
the medical record management system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring a plurality of structured medical record data to be evaluated, which correspond to medical types, from a medical database according to the medical types, the medical database is used for storing original medical record data of different medical types and carrying out structured processing on the original medical record data to obtain structured medical record data, each piece of structured medical record data comprises at least one piece of key-value pair data, and each piece of key-value pair data comprises a keyword and a value corresponding to the keyword;
a second obtaining module, configured to obtain a labeling result corresponding to each piece of structured medical record data, where the labeling result includes each keyword and a comparison result corresponding to each keyword; the labeling result is obtained by comparing the structured medical record data with the original medical record data;
and the quality evaluation module is used for evaluating the quality of the structured medical record data corresponding to the medical type according to the labeling result corresponding to each structured medical record data.
Optionally, the type of the comparison result corresponding to each keyword is any one of the following: correct conversion, incorrect keyword recall, and unrerecalled keyword; the quality assessment module is specifically configured to:
for each first keyword, obtaining a comparison result corresponding to the first keyword from the labeling results corresponding to the plurality of structured medical record data, wherein the first keyword is any one of the keywords;
obtaining first result analysis information according to a comparison result corresponding to the first keyword, wherein the first result analysis information comprises: the type of the comparison result is the number of the first keywords which are correctly converted, the type of the comparison result is the number of the first keywords which are wrongly recalled by the keywords, and the type of the comparison result is the number of the first keywords which are not recalled by the keywords;
acquiring the accuracy and recall rate corresponding to the first keyword according to the first result analysis information;
and evaluating the quality of the structured medical record data corresponding to the medical type according to the accuracy and the recall rate corresponding to each keyword.
Optionally, the quality evaluation module is specifically configured to:
acquiring the number of recalled first keywords according to the comparison result type, which is the number of the first keywords with correct conversion, the comparison result type, which is the number of the first keywords with wrong conversion, and the comparison result type, which is the number of the first keywords with wrong keyword recall;
acquiring the accuracy corresponding to the first keyword according to the number of the first keyword which is correctly converted and the number of recalled first keywords as the type of the comparison result;
and acquiring the recall rate corresponding to the first keyword according to the recalled quantity of the first keyword, the quantity of the first keyword with the type of the comparison result as the keyword error recall and the quantity of the first keyword with the type of the comparison result as the keyword unrerecalling.
Optionally, the type of the comparison result corresponding to each keyword is any one of the following: correct conversion, incorrect keyword recall, and unrerecalled keyword; the quality assessment module is specifically configured to:
according to the labeling result corresponding to each structured medical record data, second result analysis information is obtained, and the second result analysis information comprises: the type of the comparison result is the number of each keyword which is correctly converted, the type of the comparison result is the number of each keyword which is wrongly recalled by the keyword, and the type of the comparison result is the number of each keyword which is not recalled by the keyword;
and according to the second result analysis information, acquiring the accuracy and the recall rate corresponding to the structured medical record data corresponding to the medical type.
Optionally, the annotation result further includes a correction result, and the correction result includes: the type of the comparison result is a correction value corresponding to each keyword with conversion error, and the type of the comparison result is each keyword which is not recalled by the keywords and a value corresponding to each keyword; the device further comprises:
the display module is used for displaying abnormal data in the plurality of structured medical record data and displaying the correction result; and the abnormal data is key value pair data with the type of the comparison result being conversion error, keyword error recall or keyword unrerecated.
Optionally, the first obtaining module is specifically configured to:
acquiring the medical type to be evaluated and the maximum evaluation number according to the evaluation task;
acquiring a plurality of structured medical record data to be evaluated corresponding to the medical type from a medical database according to the medical type and the maximum evaluation quantity; wherein the number of the structured medical record data is less than or equal to the maximum number of evaluations.
Optionally, the first obtaining module is specifically configured to:
acquiring a target set from the medical database, wherein the target set comprises structured medical record data corresponding to the medical type;
if the quantity of the structured medical record data in the target set is larger than the maximum evaluation quantity, randomly selecting the structured medical record data with the maximum evaluation quantity from the target set as the structured medical record data to be evaluated;
and if the quantity of the structured medical record data in the target set is less than or equal to the maximum evaluation quantity, taking all the structured medical record data in the target set as the structured medical record data to be evaluated.
In a third aspect, an embodiment of the present invention provides a quality evaluation apparatus for structured medical record data, including: memory, a processor and a computer program, the computer program being stored in the memory, the processor running the computer program to perform the method according to any of the first aspects.
In a fourth aspect, the present invention provides a computer-readable storage medium including a computer program, which when executed by a processor implements the method according to any one of the first aspect.
The embodiment of the invention provides a method, a device and equipment for evaluating the quality of structured medical record data, wherein the method comprises the following steps: according to a medical type, acquiring a plurality of structured medical record data to be evaluated corresponding to the medical type from a medical database, wherein the medical database is used for storing original medical record data of different medical types and structured medical record data obtained by carrying out structured processing on the original medical record data, each piece of structured medical record data comprises at least one piece of key value pair data, and each piece of key value pair data comprises a keyword and a value corresponding to the keyword; acquiring a labeling result corresponding to each piece of structured medical record data, wherein the labeling result comprises each keyword and a comparison result corresponding to each keyword; the labeling result is obtained by comparing the structured medical record data with the original medical record data; and evaluating the quality of the structured medical record data corresponding to the medical type according to the labeling result corresponding to each structured medical record data. Through the process, the quality evaluation process of the structured medical record data is realized; in addition, when the structured medical record data is evaluated, the quality of the structured medical record data of different medical types can be determined by evaluating the structured medical record data of different medical types, and the accuracy of quality evaluation is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic diagram of a quality evaluation process of structured medical record data according to an embodiment of the present invention;
fig. 2 is a schematic flowchart of a method for evaluating quality of structured medical record data according to an embodiment of the present invention;
fig. 3 is a schematic flow chart of quality evaluation using keyword dimensions according to an embodiment of the present invention;
FIG. 4 is a schematic flow chart of quality assessment using overall dimension according to an embodiment of the present invention;
FIG. 5 is a flowchart illustrating a method for quality assessment of structured medical record data according to another embodiment of the present invention;
fig. 6 is a schematic structural diagram of an apparatus for quality assessment of structured medical record data according to an embodiment of the present invention;
FIG. 7 is a schematic structural diagram of an apparatus for quality assessment of structured medical record data according to another embodiment of the present invention;
fig. 8 is a schematic structural diagram of a quality evaluation apparatus for structured medical record data according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without making any creative effort based on the embodiments in the present invention, belong to the protection scope of the present invention.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are, for example, capable of operation in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Data in a computer informatization system is divided into structured data and unstructured data. For the convenience of the following description, the meaning and form of the unstructured data and the structured data are described first.
Unstructured data refers to data that is irregular or incomplete in data structure, has no predefined data model, and is inconvenient to present in a database two-dimensional logical table. The format and standards of unstructured data are diverse and typically take the form: text documents, email messages, chat logs, pictures, audio and video, and the like.
The structured data is also called row data, is data logically expressed and realized by a two-dimensional table structure, strictly follows the data format and length specification, and is mainly stored and managed by a relational database. Structured data is typically represented in the form of key-value pairs, i.e. in the form of keys (key) -values of keys (value). Of course, in some scenarios, the key may also be referred to as a field, and the value of the key may also correspond to an attribute referred to as a field.
Due to the diversity of formats and standards of unstructured data, unstructured data is difficult to understand and process by computers, and a great deal of inconvenience is brought when a large amount of unstructured data is used in the fields of storage, retrieval, intelligent analysis and the like. Therefore, in many scenarios, especially scenarios requiring analysis of large amounts of unstructured data, structured processing techniques are required to convert unstructured data into structured data. Further, in order to ensure the accuracy and validity of the converted structured data, the quality of the above-mentioned structured processing needs to be evaluated.
Fig. 1 is a schematic diagram of a quality evaluation process of structured medical record data according to an embodiment of the present invention. As shown in fig. 1, the original medical record data is unstructured, and structured medical record data is obtained by performing structured processing on the original medical record data. And then, comparing and labeling the original medical record data and the corresponding structured medical record data to obtain a labeling result. Wherein, the labeling result may indicate which key-value pairs are correctly converted, which key-value pairs are incorrectly converted, and the like. And finally, evaluating the quality of the structuring treatment according to the labeling result.
In this embodiment, the "quality of structured medical record data" may also be referred to as "quality of structured processing", and the meaning of the characterization is: and the structured medical record data obtained by the original medical record data through the structured processing and the primary medical record data are matched. Illustratively, if the degree of match between the structured medical record data and the corresponding original medical record data is high, it indicates that the quality of the structured processing is high, that is, the quality of the structured medical record data is high. If the goodness of fit between the structured medical record data and the corresponding original medical record data is low, it indicates that the quality of the structured processing is low, that is, the quality of the structured medical record data is low. For example, if the structured medical record data obtained by the original medical record data through the structured processing includes 10 key value pairs, and if the accuracy threshold of the quality evaluation is 80%, when more than or equal to 8 of the 10 key value pairs are converted correctly, the quality of the structured processing is considered to be high, and when less than 8 of the 10 key value pairs are converted correctly, the quality of the structured processing is considered to be low.
Referring to fig. 1, in a possible scenario, a structured processing device is used to implement a process of converting original medical record data into structured medical record data, a contrast labeling device is used to implement a process of performing contrast labeling on the structured medical record data and the original medical record data, and a quality evaluation device is used to analyze a labeling result. The above-mentioned structural processing device, the comparison and labeling device and the quality evaluation device may be respectively disposed in different electronic devices, or two or three of them may be integrated into one electronic device. This embodiment is not particularly limited thereto.
The technical means of the present invention will be described in detail with reference to specific examples. These several specific embodiments may be combined with each other below, and details of the same or similar concepts or processes may not be repeated in some embodiments.
Fig. 2 is a flowchart illustrating a method for evaluating quality of structured medical record data according to an embodiment of the present invention, where the method of this embodiment may be executed by a quality evaluation apparatus, and the apparatus may be in the form of software and/or hardware. As shown in fig. 2, the method of the present embodiment includes:
s201: according to a medical type, acquiring a plurality of structured medical record data to be evaluated corresponding to the medical type from a medical database, wherein the medical database is used for storing original medical record data of different medical types and structured medical record data obtained by carrying out structured processing on the original medical record data, each structured medical record data comprises at least one key value pair data, and each key value pair data comprises a keyword and a value corresponding to the keyword.
The method of the embodiment can be used for evaluating the quality of the structured medical record data corresponding to a certain medical treatment type. Wherein the medical type may be divided by different dimensions. The classification can be performed according to the medical record types, such as: admission medical records, outpatient medical records, emergency medical records, discharge medical records, and the like; it can also be divided according to the type of hospital, for example: children medical treatment, gynecological medical treatment, comprehensive hospitals and the like; division can also be made according to the type of department, for example: pediatric, obstetrical, surgical, medical, etc. And storing the original medical record data and the structured medical record data corresponding to different medical types in a medical database, and acquiring the structured medical record data corresponding to the medical type from the medical database according to the medical type to be evaluated during evaluation.
It is to be understood that the original medical record data is usually in an unstructured form, and the original medical record data can be converted into structured medical record data by using a structured processing technology. Illustratively, the data in the original medical record is processed into a structured form with uniform field naming in units of chapters, or features such as duration of symptoms, positivity and negativity, diagnosis certainty of diseases and the like are extracted from the original medical record. Through the structured processing process, the original medical record which can be generally understood only by a doctor is converted into the structured medical record which can be understood by a computer. Note that the present embodiment does not specifically limit the structuring process technique. The process of structured processing may be performed using structured processing means.
In this embodiment, the structured medical record data is stored in a key-value pair manner. The data in each original medical record is called original medical record data, and the data obtained by converting the data in one original medical record is called structured medical record data. As can be appreciated, each structured medical record data includes one or more key value pair data. Each key-value pair data includes a key (key) and a value (value) corresponding to the key.
S202: acquiring a labeling result corresponding to each structured medical record data, wherein the labeling result comprises each keyword and a comparison result corresponding to each keyword; and the labeling result is obtained by comparing the structured medical record data with the original medical record data.
The labeling result indicates the difference obtained by comparing the structured medical record data with the original medical record data. For example, the comparison result corresponding to each keyword can be used to indicate whether the keyword is correctly converted.
Optionally, the type of the comparison result corresponding to each keyword is any one of the following: correct translation, wrong translation, missed keyword recall, and unrerecalled keyword. For example, for a certain keyword, if the keyword and the value obtained by structuring are both converted correctly, the type of the comparison result corresponding to the keyword is "converted correctly"; if the value of the key word obtained by structuring is wrong in conversion, the type of the comparison result corresponding to the key word is wrong in conversion; if the keyword and the value are not required to be identified, but the keyword is identified in the structuring processing process, the type of the comparison result corresponding to the keyword is 'keyword false recall'; if a certain keyword and value need to be identified, but the structuring processing process is not identified, the type of the comparison result corresponding to the keyword is "keyword unrerecalling".
In a possible embodiment, the process of comparing the structured medical record data with the original medical record data to obtain the annotation result can be executed by the comparison and annotation device. Illustratively, the comparison and labeling device extracts keyword information in the original medical record data and values corresponding to the keyword information, compares the extracted information with key value pairs in the structured medical record data to obtain comparison results, and labels respective comparison results for each keyword.
In another possible implementation, a professional annotator can also compare the structured medical record data with the original medical record data to obtain a comparison result corresponding to each keyword in the structured medical record data.
Optionally, the annotation result may further include a correction result, and the correction result includes: the type of the comparison result is the corrected value corresponding to each keyword with conversion error, and the type of the comparison result is each keyword which is not recalled by the keyword and the value corresponding to each keyword.
In the above comparison and annotation process, for the keyword with the type of the comparison result being "conversion error", a correction value corresponding to the keyword may also be obtained according to the original medical record data, and the correction value is also added to the annotation result.
For example, in the contrast labeling process, for a keyword of which the type of the comparison result is "keyword unrerecalling", the keyword and the value of the keyword may be added to the labeling result according to the original medical record data.
In this embodiment, the annotation result corresponding to each piece of structured medical record data is stored in the result file. Illustratively, the result file may be a JSON file.
Illustratively, the data in the result file is stored in the form of values corresponding to keys.
{ "key1": { "value": "+", "type": "conversion correct/conversion error/keyword missed recall/keyword not recalled" },
"key2": { "value": "x", "type": "conversion correct/conversion error/keyword missed recall/keyword not recalled" }
S203: and evaluating the quality of the structured medical record data corresponding to the medical type according to the labeling result corresponding to each structured medical record data.
In this embodiment, the quality of the structured medical record data corresponding to the medical type is evaluated according to the labeling result. For example, thresholds may be set for the types of comparison results, respectively, and assuming that the threshold for correct conversion is set to 80%, then: if the percentage of each keyword labeled as the correct type of conversion in the labeling result is greater than or equal to 80%, it is determined that the quality of the structured processing is high, that is, the accuracy of the structured medical record data is high. If the percentage of each keyword labeled as the correct type of conversion is less than 80% in the labeling result, it is determined that the quality of the structured processing is low, that is, the accuracy of the structured medical record data is low.
It should be noted that, in practical applications, different thresholds may be set according to types of various comparison results, so as to perform multi-dimensional quality evaluation on the structured processing, thereby improving comprehensiveness of the quality evaluation.
It can be understood that, in the process of structuring the original medical record data, the conversion quality of the same structuring algorithm for data of different medical treatment types may be different. For example, a structure processing algorithm may have a higher quality of conversion for hospital admission records and a lower quality of conversion for hospital discharge records. Therefore, when the structured medical record data is evaluated, the quality of the structured medical record data of different medical types can be determined by evaluating the structured medical record data of different medical types, and the accuracy of quality evaluation is improved.
In the following, a specific quality evaluation process is described using a keyword dimension and an overall dimension in combination with two possible embodiments.
In a first possible implementation, the keyword dimension may be used for quality assessment. Fig. 3 is a schematic flowchart of a quality evaluation process using keyword dimensions according to an embodiment of the present invention. As shown in fig. 3, the method includes:
s301: and for each first keyword, obtaining a comparison result corresponding to the first keyword from the labeling results corresponding to the plurality of structured medical record data, wherein the first keyword is any one of the keywords.
S302: according to a comparison result corresponding to the first keyword, obtaining first result analysis information, wherein the first result analysis information comprises: the type of the comparison result is the number of the first keywords with correct conversion, the type of the comparison result is the number of the first keywords with wrong keyword recalls, and the type of the comparison result is the number of the first keywords with unrecalled keywords.
S303: and according to the first result analysis information, acquiring the accuracy and the recall rate corresponding to the first keyword.
For example, it is assumed that each of 1000 pieces of structured medical record data to be evaluated includes 10 keywords, which are key1 to key10. The labeling result corresponding to each structured medical record data comprises the comparison result corresponding to each of the 10 keywords. For example: the labeling result corresponding to one of the structured medical record data is as follows:
{ "key1": { "value": "+", "type": the "switch is correct" },
"key2": { "value": "x", "type": the "switch is correct" },
"key3": { "value": "+", "type": "a switch error" },
"key4": { "value": "+", "type": the "conversion error" is corrected by the "conversion error",
"key5": { "value": "x", "type": "switch over is correct" },
"key6": { "value": "x", "type": "switch over is correct" },
"key7": { "value": "x", "type": "keyword missed recall" },
"key8": { "value": "+", "type": the "switch is correct" },
"key9": { "value": "x", "type": "keyword not recalled" },
"key10": { "value": "x", "type": "correct conversion" }
In this embodiment, the quality evaluation may be performed from the dimension of each keyword in the above 10 keywords. Specifically, the accuracy and recall rate corresponding to each keyword are used for evaluation.
For example, S301 to S303 in the present embodiment are respectively executed for each of the above 10 keywords. Taking the keyword key1 as an example, the comparison result corresponding to the keyword key1 is obtained from the labeling results corresponding to the 1000 structured medical record data.
Further, obtaining first result analysis information from the comparison result corresponding to the keyword key1 includes: the type of the comparison result is the number of the keywords key1 with correct conversion, the type of the comparison result is the number of the keywords key1 with wrong keyword recalling, and the type of the comparison result is the number of the keywords key1 with unrecalled keywords.
Further, according to the first result analysis information, the accuracy rate and the recall rate corresponding to the keyword key1 are obtained. Wherein the accuracy rate indicates what proportion of the keywords identified in the structuring process are correctly converted by the conversion. Recall indicates how many proportion of the keywords that should be identified in the structuring process are identified.
In a possible implementation manner, the number of recalled keywords key1 is obtained according to the number of the keywords key1 with the correct conversion type as the type of the comparison result, the number of the keywords key1 with the wrong conversion type as the type of the comparison result, and the number of the keywords key1 with the wrong keyword recall type as the type of the comparison result. And acquiring the accuracy corresponding to the keyword key1 according to the number of the keyword keys 1 with correct conversion and the number of the recalled keyword keys 1 as the type of the comparison result. And acquiring the recall rate corresponding to the keyword key1 according to the number of recalled keywords key1, the number of the keywords key1 of which the type of the comparison result is the keyword error recall, and the number of the keywords key1 of which the type of the comparison result is the keyword non-recall.
Illustratively, the number of the keywords key1 recalled = the number of the keywords key1 of which the type of the comparison result is correct for conversion + the number of the keywords key1 of which the type of the comparison result is wrong for keyword recall.
The accuracy rate of the key1 = that the type of the comparison result is the number of the correctly converted key 1/the number of the recalled key1
The recall rate corresponding to the keyword key1 = (the number of recalled keyword keys 1-the type of the comparison result is the number of the keyword keys 1 recalled by mistake)/(the number of recalled keyword keys 1-the type of the comparison result is the number of the keyword keys 1 recalled by mistake + the type of the comparison result is the number of the keyword keys 1 not recalled by mistake)
S304: and evaluating the quality of the structured medical record data corresponding to the medical type according to the accuracy and the recall rate corresponding to each keyword.
And respectively executing S301 to S303 to the 10 keywords to obtain the accuracy and the recall rate corresponding to the 10 keywords. In step S304, the quality of the structured processing can be visually obtained by the accuracy and recall ratio corresponding to each keyword. Illustratively, if the accuracy rate or the recall rate of a certain keyword is low, the structured processing algorithm may be optimized according to the original medical record data corresponding to the keyword.
Of course, under the condition of a large number of keywords, the accuracy and the recall rate corresponding to each keyword may be further subjected to weighting operation to obtain an overall quality evaluation result corresponding to a plurality of structured medical record data.
In a second possible implementation, the overall dimension may be used for quality assessment. Fig. 4 is a schematic flowchart of quality evaluation by using overall dimensions according to an embodiment of the present invention. As shown in fig. 4, the method includes:
s401: according to the labeling result corresponding to each structured medical record data, second result analysis information is obtained, and the second result analysis information comprises: the type of the comparison result is the number of each keyword with correct conversion, the type of the comparison result is the number of each keyword with wrong keyword recall, and the type of the comparison result is the number of each keyword without keyword recall.
S402: and acquiring the accuracy and recall rate corresponding to the structured medical record data corresponding to the medical type according to the second result analysis information.
The embodiment of this embodiment is similar to the embodiment shown in fig. 3, except that the keywords are not distinguished, and all keywords are considered together. That is, included in the second result analysis information is: in the labeling results corresponding to the 1000 structured medical record data, the type of the comparison result is the number of each keyword which is correctly converted, the type of the comparison result is the number of each keyword which is wrongly recalled by the keywords, and the type of the comparison result is the number of each keyword which is not recalled by the keywords.
Further, the accuracy and recall corresponding to the structuring processing in S402 are similar to those in the embodiment shown in fig. 3, and are not described herein again.
The accuracy and recall in this embodiment may also be presented in the form of a chart, for example: line graphs, bar graphs, etc. Furthermore, evaluation results of different batches or different medical types can be displayed in a chart so as to visually reflect the quality change condition in the structural processing iteration process.
The quality evaluation method for the structured medical record data provided by the embodiment comprises the following steps: according to a medical type, acquiring a plurality of structured medical record data to be evaluated corresponding to the medical type from a medical database, wherein the medical database is used for storing original medical record data of different medical types and structured medical record data obtained by carrying out structured processing on the original medical record data, each structured medical record data comprises at least one key value pair data, and each key value pair data comprises a keyword and a value corresponding to the keyword; acquiring a labeling result corresponding to each piece of structured medical record data, wherein the labeling result comprises each keyword and a comparison result corresponding to each keyword; the labeling result is obtained by comparing the structured medical record data with the original medical record data; and evaluating the quality of the structured medical record data corresponding to the medical type according to the labeling result corresponding to each structured medical record data. Through the process, the quality evaluation process of the structured medical record data is realized; in addition, when the structured medical record data is evaluated, the quality of the structured medical record data of different medical types can be determined by evaluating the structured medical record data of different medical types, and the accuracy of quality evaluation is improved.
Fig. 5 is a flowchart illustrating a method for evaluating quality of structured medical record data according to another embodiment of the present invention. As shown in fig. 5, the method of the present embodiment includes:
s501: and acquiring the medical type to be evaluated and the maximum evaluation number according to the evaluation task.
S502: and acquiring a plurality of structured medical record data to be evaluated corresponding to the medical type from a medical database according to the medical type and the maximum evaluation number.
Wherein the number of the structured medical record data is less than or equal to the maximum number of evaluations.
In one possible implementation, the user creates an evaluation task in the quality evaluation device, and formulates the medical type and the maximum evaluation number corresponding to the evaluation task. And after the task is successfully established, the quality evaluation device screens out the structured medical record data to be evaluated from the medical database.
Optionally, a target set is obtained from the medical database, where the target set includes structured medical record data corresponding to the medical type; if the quantity of the structured medical record data in the target set is larger than the maximum evaluation quantity, randomly selecting the structured medical record data with the maximum evaluation quantity from the target set as the structured medical record data to be evaluated; and if the quantity of the structured medical record data in the target set is less than or equal to the maximum evaluation quantity, taking the structured medical record data in the target set as the structured medical record data to be evaluated.
The above-mentioned random selection process can be implemented by using a random selection algorithm. Illustratively, a unique identifier is allocated to each structured medical record data in the target set, and a random scrambling algorithm is adopted to scramble the data in the target set. Assuming that the maximum number of evaluations is N, starting from 0, structured medical record data identified as 0-N-1 are selected as the data to be evaluated.
S503: acquiring a labeling result corresponding to each piece of structured medical record data, wherein the labeling result comprises each keyword and a comparison result corresponding to each keyword, and the type of the comparison result is any one of the following types: correct translation, incorrect translation, missed keyword recall, and unrerecalled keyword.
S504: and evaluating the quality of the structured medical record data corresponding to the medical type according to the labeling result corresponding to each structured medical record data.
In this embodiment, the specific implementation processes of S503 and S504 are similar to those of the above embodiments, and are not described herein again.
S505: and displaying abnormal data in the plurality of structured medical record data, and displaying the correction result.
And the abnormal data is key value pair data with the type of the comparison result being a conversion error, a keyword error recall or a keyword non-recall. The correction result includes: the type of the comparison result is the corrected value corresponding to each keyword with conversion error, and the type of the comparison result is each keyword which is not recalled by the keyword and the value corresponding to each keyword.
Illustratively, the abnormal data and the correction result are derived from a medical database and displayed. The presented data can be used as the analysis basis for the structured quality improvement of medical record data. Furthermore, the structured medical record data in the medical database can be updated according to the corrected result after evaluation, so that the accuracy of the structured medical record data is improved.
Fig. 6 is a schematic structural diagram of a quality evaluation apparatus for structured medical record data according to an embodiment of the present invention. As shown in fig. 6, the apparatus 600 for evaluating quality of structured medical record data according to this embodiment includes: a first acquisition module 601, a second acquisition module 602, and a quality assessment module 603.
The first obtaining module 601 is configured to obtain, according to a medical type, multiple pieces of structured medical record data to be evaluated, which correspond to the medical type, from a medical database, where the medical database is configured to store original medical record data of different medical types, and structured medical record data obtained by performing structured processing on each piece of the original medical record data, where each piece of the structured medical record data includes at least one piece of key-value pair data, and each piece of the key-value pair data includes a keyword and a value corresponding to the keyword;
a second obtaining module 602, configured to obtain a labeling result corresponding to each structured medical record data, where the labeling result includes each keyword and a comparison result corresponding to each keyword; the labeling result is obtained by comparing the structured medical record data with the original medical record data;
the quality evaluation module 603 is configured to evaluate the quality of the structured medical record data corresponding to the medical type according to the labeling result corresponding to each structured medical record data.
The apparatus provided in this embodiment may be used to implement the method embodiment shown in fig. 2, and the implementation principle and technical effects are similar, which are not described herein again.
Fig. 7 is a schematic structural diagram of an apparatus for quality assessment of structured medical record data according to another embodiment of the present invention. As shown in fig. 7, based on the embodiment shown in fig. 6, the apparatus 600 for evaluating quality of structured medical record data provided in this embodiment may further include: a module 604 is shown.
Optionally, the type of the comparison result corresponding to each keyword is any one of the following: correct conversion, incorrect keyword recall, and unrerecalled keyword; the quality evaluation module 603 is specifically configured to:
for each first keyword, obtaining a comparison result corresponding to the first keyword from the labeling results corresponding to the plurality of structured medical record data, wherein the first keyword is any one of the keywords;
obtaining first result analysis information according to a comparison result corresponding to the first keyword, wherein the first result analysis information comprises: the type of the comparison result is the number of the first keywords which are correctly converted, the type of the comparison result is the number of the first keywords which are wrongly recalled by the keywords, and the type of the comparison result is the number of the first keywords which are not recalled by the keywords;
acquiring the accuracy and recall rate corresponding to the first keyword according to the first result analysis information;
and evaluating the quality of the structured medical record data corresponding to the medical type according to the accuracy and the recall rate corresponding to each keyword.
Optionally, the quality evaluation module 603 is specifically configured to:
acquiring the recalled number of the first keywords according to the comparison result type, the number of the first keywords with correct conversion, the number of the first keywords with wrong conversion and the number of the first keywords with wrong keyword recall;
acquiring the accuracy corresponding to the first keyword according to the type of the comparison result, namely the number of the first keyword which is correctly converted and the number of the first keyword which is recalled;
and acquiring the recall rate corresponding to the first keyword according to the recalled quantity of the first keyword, the quantity of the first keyword with the type of the comparison result as the keyword error recall and the quantity of the first keyword with the type of the comparison result as the keyword unrerecalling.
Optionally, the type of the comparison result corresponding to each keyword is any one of the following: correct conversion, incorrect keyword recall, and unrerecalled keyword; the quality evaluation module 603 is specifically configured to:
according to the labeling result corresponding to each structured medical record data, second result analysis information is obtained, and the second result analysis information comprises: the type of the comparison result is the number of each keyword with correct conversion, the type of the comparison result is the number of each keyword with wrong keyword recall, and the type of the comparison result is the number of each keyword without keyword recall;
and according to the second result analysis information, acquiring the accuracy and the recall rate corresponding to the structured medical record data corresponding to the medical type.
Optionally, the annotation result further includes a correction result, and the correction result includes: the type of the comparison result is a correction value corresponding to each keyword with conversion error, and the type of the comparison result is each keyword which is not recalled by the keyword and a value corresponding to each keyword;
the display module 604 is configured to display abnormal data in the plurality of structured medical record data and display the correction result; and the abnormal data is key value pair data with the type of the comparison result being a conversion error, a keyword error recall or a keyword non-recall.
Optionally, the first obtaining module 601 is specifically configured to:
acquiring the medical type to be evaluated and the maximum evaluation number according to the evaluation task;
acquiring a plurality of structured medical record data to be evaluated corresponding to the medical type from a medical database according to the medical type and the maximum evaluation quantity; wherein the number of structured medical record data is less than or equal to the maximum number of evaluations.
Optionally, the first obtaining module 601 is specifically configured to:
acquiring a target set from the medical database, wherein the target set comprises structured medical record data corresponding to the medical type;
if the quantity of the structured medical record data in the target set is larger than the maximum evaluation quantity, randomly selecting the structured medical record data with the maximum evaluation quantity from the target set as the structured medical record data to be evaluated;
and if the quantity of the structured medical record data in the target set is less than or equal to the maximum evaluation quantity, taking all the structured medical record data in the target set as the structured medical record data to be evaluated.
The apparatus provided in this embodiment may be configured to implement the technical solution of any of the above method embodiments, and the implementation principle and the technical effect are similar, which are not described herein again.
Fig. 8 is a schematic structural diagram of a quality evaluation apparatus for structured medical record data according to an embodiment of the present invention. As shown in fig. 8, the quality evaluation apparatus 800 for structured medical record data according to this embodiment includes: a processor 801 and a memory 802; a memory 802 for storing a computer program; a processor 801 for executing the computer program stored in the memory to implement the method in the above embodiments. Reference may be made in particular to the description relating to the method embodiments described above.
Alternatively, the memory 802 may be separate or integrated with the processor 801.
When the memory 802 is a device separate from the processor 801, the quality assessment apparatus 800 for structured medical record data can further include: a bus 803 for connecting the memory 802 and the processor 801.
The quality evaluation device for structured medical record data provided by this embodiment can be used to implement the technical scheme in any of the above method embodiments, and the implementation principle and technical effect are similar, which is not described herein again.
An embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium includes a computer program, and the computer program is used to implement the technical solutions in any of the above method embodiments.
An embodiment of the present invention further provides a chip, including: the system comprises a memory, a processor and a computer program, wherein the computer program is stored in the memory, and the processor runs the computer program to execute the technical scheme of any one of the method embodiments.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is only one logical functional division, and other divisions may be realized in practice, for example, a plurality of modules may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or modules, and may be in an electrical, mechanical or other form.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
In addition, functional modules in the embodiments of the present invention may be integrated into one processing unit, or each module may exist alone physically, or two or more modules are integrated into one unit. The unit formed by the modules can be realized in a hardware mode, and can also be realized in a mode of hardware and a software functional unit.
The integrated module implemented in the form of a software functional module may be stored in a computer-readable storage medium. The software functional module is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device) or a processor (processor) to execute some steps of the methods according to the embodiments of the present invention.
It should be understood that the Processor may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the methods disclosed in the incorporated application may be directly implemented by a hardware processor, or may be implemented by a combination of hardware and software modules in the processor.
The memory may comprise a high-speed RAM memory, and may further comprise a non-volatile storage NVM, such as at least one disk memory, and may also be a usb disk, a removable hard disk, a read-only memory, a magnetic or optical disk, etc.
The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended ISA (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, the buses in the figures of the present invention are not limited to only one bus or one type of bus.
The storage medium may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.
An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. Of course, the storage medium may also be integral to the processor. The processor and the storage medium may reside in an Application Specific Integrated Circuits (ASIC). Of course, the processor and the storage medium may reside as discrete components in an electronic device or host device.
Those of ordinary skill in the art will understand that: all or a portion of the steps of implementing the above-described method embodiments may be performed by hardware associated with program instructions. The program may be stored in a computer-readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and these modifications or substitutions do not depart from the spirit of the corresponding technical solutions of the embodiments of the present invention.

Claims (16)

1. A method for evaluating the quality of structured medical record data is characterized by comprising the following steps:
according to a medical type, acquiring a plurality of structured medical record data to be evaluated corresponding to the medical type from a medical database, wherein the medical database is used for storing original medical record data of different medical types and structured medical record data obtained by carrying out structured processing on the original medical record data, each piece of structured medical record data comprises at least one piece of key value pair data, and each piece of key value pair data comprises a keyword and a value corresponding to the keyword;
acquiring a labeling result corresponding to each piece of structured medical record data, wherein the labeling result comprises each keyword and a comparison result corresponding to each keyword; the labeling result is obtained by comparing the structured medical record data with the original medical record data;
and evaluating the quality of the structured medical record data corresponding to the medical type according to the labeling result corresponding to each structured medical record data.
2. The method according to claim 1, wherein the type of the comparison result corresponding to each keyword is any one of the following types: correct conversion, wrong recall of keywords and no recall of keywords; the evaluating the quality of the structured medical record data corresponding to the medical type according to the labeling result corresponding to each structured medical record data comprises:
for each first keyword, obtaining a comparison result corresponding to the first keyword from labeling results corresponding to the multiple structured medical record data, wherein the first keyword is any one of the keywords;
obtaining first result analysis information according to a comparison result corresponding to the first keyword, wherein the first result analysis information comprises: the type of the comparison result is the number of the first keywords with correct conversion, the type of the comparison result is the number of the first keywords with wrong keyword recalls, and the type of the comparison result is the number of the first keywords with unrecalled keywords;
according to the first result analysis information, acquiring the accuracy and the recall rate corresponding to the first keyword;
and evaluating the quality of the structured medical record data corresponding to the medical type according to the accuracy and the recall rate corresponding to each keyword.
3. The method according to claim 2, wherein the obtaining the accuracy and the recall ratio corresponding to the first keyword according to the first result analysis information comprises:
acquiring the number of recalled first keywords according to the comparison result type, which is the number of the first keywords with correct conversion, the comparison result type, which is the number of the first keywords with wrong conversion, and the comparison result type, which is the number of the first keywords with wrong keyword recall;
acquiring the accuracy corresponding to the first keyword according to the number of the first keyword which is correctly converted and the number of recalled first keywords as the type of the comparison result;
and acquiring the recall rate corresponding to the first keyword according to the recalled quantity of the first keyword, the quantity of the first keyword with the type of the comparison result as the keyword error recall and the quantity of the first keyword with the type of the comparison result as the keyword unrerecalling.
4. The method according to claim 1, wherein the type of the comparison result corresponding to each keyword is any one of the following types: correct conversion, incorrect keyword recall, and unrerecalled keyword; the evaluating the quality of the structured medical record data corresponding to the medical type according to the labeling result corresponding to each structured medical record data comprises:
according to the labeling result corresponding to each structured medical record data, second result analysis information is obtained, and the second result analysis information comprises: the type of the comparison result is the number of each keyword which is correctly converted, the type of the comparison result is the number of each keyword which is wrongly recalled by the keyword, and the type of the comparison result is the number of each keyword which is not recalled by the keyword;
and according to the second result analysis information, acquiring the accuracy and the recall rate corresponding to the structured medical record data corresponding to the medical type.
5. The method according to any one of claims 1 to 4, wherein the annotation result further comprises a modification result, and the modification result comprises: the type of the comparison result is a correction value corresponding to each keyword with conversion error, and the type of the comparison result is each keyword which is not recalled by the keyword and a value corresponding to each keyword;
after the quality of the structured medical record data corresponding to the medical type is evaluated according to the labeling result corresponding to each structured medical record data, the method further comprises the following steps:
displaying abnormal data in the plurality of structured medical record data, and displaying the correction result; and the abnormal data is key value pair data with the type of the comparison result being a conversion error, a keyword error recall or a keyword non-recall.
6. The method according to any one of claims 1 to 4, wherein the obtaining a plurality of structured medical record data to be evaluated corresponding to the medical type from a medical database according to the medical type comprises:
acquiring the medical type to be evaluated and the maximum evaluation number according to the evaluation task;
acquiring a plurality of structured medical record data to be evaluated corresponding to the medical type from a medical database according to the medical type and the maximum evaluation quantity; wherein the number of structured medical record data is less than or equal to the maximum number of evaluations.
7. The method according to claim 6, wherein the obtaining a plurality of structured medical record data to be evaluated corresponding to the medical type from a medical database according to the medical type and the maximum evaluation number comprises:
acquiring a target set from the medical database, wherein the target set comprises structured medical record data corresponding to the medical type;
if the quantity of the structured medical record data in the target set is larger than the maximum evaluation quantity, randomly selecting the structured medical record data with the maximum evaluation quantity from the target set as the structured medical record data to be evaluated;
and if the quantity of the structured medical record data in the target set is less than or equal to the maximum evaluation quantity, taking all the structured medical record data in the target set as the structured medical record data to be evaluated.
8. An apparatus for quality assessment of structured medical record data, comprising:
the medical record management system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring a plurality of structured medical record data to be evaluated, which correspond to medical types, from a medical database according to the medical types, the medical database is used for storing original medical record data of different medical types and carrying out structured processing on the original medical record data to obtain structured medical record data, each piece of structured medical record data comprises at least one piece of key-value pair data, and each piece of key-value pair data comprises a keyword and a value corresponding to the keyword;
a second obtaining module, configured to obtain a labeling result corresponding to each structured medical record data, where the labeling result includes each keyword and a comparison result corresponding to each keyword; the labeling result is obtained by comparing the structured medical record data with the original medical record data;
and the quality evaluation module is used for evaluating the quality of the structured medical record data corresponding to the medical type according to the labeling result corresponding to each structured medical record data.
9. The apparatus according to claim 8, wherein the type of the comparison result corresponding to each keyword is any one of the following: correct conversion, incorrect keyword recall, and unrerecalled keyword; the quality assessment module is specifically configured to:
for each first keyword, obtaining a comparison result corresponding to the first keyword from the labeling results corresponding to the plurality of structured medical record data, wherein the first keyword is any one of the keywords;
according to a comparison result corresponding to the first keyword, obtaining first result analysis information, wherein the first result analysis information comprises: the type of the comparison result is the number of the first keywords which are correctly converted, the type of the comparison result is the number of the first keywords which are wrongly recalled by the keywords, and the type of the comparison result is the number of the first keywords which are not recalled by the keywords;
according to the first result analysis information, acquiring the accuracy and the recall rate corresponding to the first keyword;
and evaluating the quality of the structured medical record data corresponding to the medical type according to the accuracy and recall rate corresponding to each keyword.
10. The apparatus of claim 9, wherein the quality assessment module is specifically configured to:
acquiring the recalled number of the first keywords according to the comparison result type, the number of the first keywords with correct conversion, the number of the first keywords with wrong conversion and the number of the first keywords with wrong keyword recall;
acquiring the accuracy corresponding to the first keyword according to the type of the comparison result, namely the number of the first keyword which is correctly converted and the number of the first keyword which is recalled;
and acquiring the recall rate corresponding to the first keyword according to the recalled quantity of the first keyword, the quantity of the first keyword with the type of the comparison result as the keyword error recall and the quantity of the first keyword with the type of the comparison result as the keyword unrerecalling.
11. The apparatus of claim 8, wherein the type of the comparison result corresponding to each keyword is any one of the following: correct conversion, wrong recall of keywords and no recall of keywords; the quality assessment module is specifically configured to:
according to the labeling result corresponding to each structured medical record data, second result analysis information is obtained, and the second result analysis information comprises: the type of the comparison result is the number of each keyword with correct conversion, the type of the comparison result is the number of each keyword with wrong keyword recall, and the type of the comparison result is the number of each keyword without keyword recall;
and acquiring the accuracy and recall rate corresponding to the structured medical record data corresponding to the medical type according to the second result analysis information.
12. The apparatus according to any one of claims 8 to 11, wherein the labeling result further comprises a modification result, and the modification result comprises: the type of the comparison result is a correction value corresponding to each keyword with conversion error, and the type of the comparison result is each keyword which is not recalled by the keyword and a value corresponding to each keyword; the device further comprises:
the display module is used for displaying abnormal data in the plurality of structured medical record data and displaying the correction result; and the abnormal data is key value pair data with the type of the comparison result being a conversion error, a keyword error recall or a keyword non-recall.
13. The apparatus according to any one of claims 8 to 11, wherein the first obtaining module is specifically configured to:
acquiring the medical type to be evaluated and the maximum evaluation number according to the evaluation task;
acquiring a plurality of structured medical record data to be evaluated corresponding to the medical type from a medical database according to the medical type and the maximum evaluation quantity; wherein the number of structured medical record data is less than or equal to the maximum number of evaluations.
14. The apparatus of claim 13, wherein the first obtaining module is specifically configured to:
acquiring a target set from the medical database, wherein the target set comprises structured medical record data corresponding to the medical type;
if the quantity of the structured medical record data in the target set is larger than the maximum evaluation quantity, randomly selecting the structured medical record data with the maximum evaluation quantity from the target set as the structured medical record data to be evaluated;
and if the quantity of the structured medical record data in the target set is less than or equal to the maximum evaluation quantity, taking all the structured medical record data in the target set as the structured medical record data to be evaluated.
15. A quality evaluation apparatus for structured medical record data, comprising: memory, a processor and a computer program, the computer program being stored in the memory, the processor running the computer program to perform the method of any of claims 1 to 7.
16. A computer-readable storage medium, characterized in that the computer-readable storage medium comprises a computer program which, when executed by a processor, carries out the method of any one of claims 1 to 7.
CN201910639668.3A 2019-07-16 2019-07-16 Quality evaluation method, device and equipment for structured medical record data Active CN110362829B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910639668.3A CN110362829B (en) 2019-07-16 2019-07-16 Quality evaluation method, device and equipment for structured medical record data

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910639668.3A CN110362829B (en) 2019-07-16 2019-07-16 Quality evaluation method, device and equipment for structured medical record data

Publications (2)

Publication Number Publication Date
CN110362829A CN110362829A (en) 2019-10-22
CN110362829B true CN110362829B (en) 2023-01-03

Family

ID=68219379

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910639668.3A Active CN110362829B (en) 2019-07-16 2019-07-16 Quality evaluation method, device and equipment for structured medical record data

Country Status (1)

Country Link
CN (1) CN110362829B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110866557B (en) * 2019-11-12 2022-12-13 贵州医渡云技术有限公司 Data evaluation method and device, storage medium and electronic device
CN113052410A (en) * 2019-12-26 2021-06-29 医渡云（北京）技术有限公司 Quality control method and device for electronic medical record data
CN111986750B (en) * 2020-07-27 2023-12-26 北京天健源达科技股份有限公司 Structural detection method for electronic medical record template
CN112184084B (en) * 2020-11-05 2023-08-08 北京嘉和海森健康科技有限公司 Medical record learning quality assessment method and device
CN113380363B (en) * 2021-06-24 2023-04-07 湖南创星科技股份有限公司 Medical data quality evaluation method and system based on artificial intelligence

Citations (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104182844A (en) * 2014-08-27 2014-12-03 浪潮软件股份有限公司 Hospitalization service performance evaluation method of medical institutions
CN104408667A (en) * 2014-11-20 2015-03-11 深圳供电局有限公司 Comprehensive assessment method and system of power quality
CN105808951A (en) * 2016-03-10 2016-07-27 宁波金唐软件有限公司 Quality control method and apparatus for electronic medical record
CN106383853A (en) * 2016-08-30 2017-02-08 刘勇 Realization method and system for electronic medical record post-structuring and auxiliary diagnosis
CN106445901A (en) * 2016-09-28 2017-02-22 医渡云（北京）技术有限公司 Method and device for designing case report form
CN106951705A (en) * 2017-03-16 2017-07-14 天津艾登科技有限公司 A kind of data quality accessment system serviced for inpatient medical
CN107683477A (en) * 2015-06-05 2018-02-09 利姆博思医学科技有限责任公司 Data quality management system and method
CN107767929A (en) * 2017-11-13 2018-03-06 医渡云（北京）技术有限公司 CRF filling method, device, electronic equipment and storage medium
CN107833595A (en) * 2017-10-12 2018-03-23 山东大学 Medical big data multicenter integration platform and method
CN107908768A (en) * 2017-09-30 2018-04-13 北京颐圣智能科技有限公司 Method, apparatus, computer equipment and the storage medium of electronic health record processing
CN108038131A (en) * 2017-11-17 2018-05-15 上海数据交易中心有限公司 Data Quality Analysis preprocess method and device, storage medium, terminal
CN108170468A (en) * 2017-12-28 2018-06-15 中山大学 The method and its system of a kind of automatic detection annotation and code consistency
CN108897682A (en) * 2018-06-25 2018-11-27 郑州云海信息技术有限公司 A kind of iteration tests outcome evaluation method based on Python
CN109256212A (en) * 2018-08-17 2019-01-22 上海米因医疗器械科技有限公司 Bone health assessment models construction method, device, equipment, medium and appraisal procedure
CN109445948A (en) * 2018-11-15 2019-03-08 济南浪潮高新科技投资发展有限公司 A kind of data mark crowdsourcing plateform system and crowdsourcing data mask method based on intelligent contract
CN109473149A (en) * 2018-11-09 2019-03-15 天津开心生活科技有限公司 Data Quality Assessment Methodology, device, electronic equipment and computer-readable medium
CN109522302A (en) * 2018-11-09 2019-03-26 南京医渡云医学技术有限公司 Medical data processing method, device, electronic equipment and computer-readable medium
CN109582661A (en) * 2018-11-23 2019-04-05 金色熊猫有限公司 Data structured appraisal procedure, device, storage medium and electronic equipment
CN109710670A (en) * 2018-12-11 2019-05-03 河南通域医疗科技有限公司 A method of case history text is converted into structural metadata from natural language

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6931327B2 (en) * 2003-08-01 2005-08-16 Dexcom, Inc. System and methods for processing analyte sensor data
US20120245952A1 (en) * 2011-03-23 2012-09-27 University Of Rochester Crowdsourcing medical expertise
US20140324553A1 (en) * 2012-08-01 2014-10-30 Michael Joseph Rosenberg Computer-Assisted Method for Adaptive, Risk-Based Monitoring of Clinical Studies
US9626757B2 (en) * 2013-08-08 2017-04-18 Washington University System and method for the validation and quality assurance of computerized contours of human anatomy
US10813572B2 (en) * 2015-12-11 2020-10-27 Electronic Caregiver, Inc. Intelligent system for multi-function electronic caregiving to facilitate advanced health diagnosis, health monitoring, fall and injury prediction, health maintenance and support, and emergency response
US10303425B2 (en) * 2016-03-21 2019-05-28 Patient Prism LLC Interactive keyword cloud
US20190156923A1 (en) * 2017-11-17 2019-05-23 LunaPBC Personal, omic, and phenotype data community aggregation platform

Patent Citations (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104182844A (en) * 2014-08-27 2014-12-03 浪潮软件股份有限公司 Hospitalization service performance evaluation method of medical institutions
CN104408667A (en) * 2014-11-20 2015-03-11 深圳供电局有限公司 Comprehensive assessment method and system of power quality
CN107683477A (en) * 2015-06-05 2018-02-09 利姆博思医学科技有限责任公司 Data quality management system and method
CN105808951A (en) * 2016-03-10 2016-07-27 宁波金唐软件有限公司 Quality control method and apparatus for electronic medical record
CN106383853A (en) * 2016-08-30 2017-02-08 刘勇 Realization method and system for electronic medical record post-structuring and auxiliary diagnosis
CN106445901A (en) * 2016-09-28 2017-02-22 医渡云（北京）技术有限公司 Method and device for designing case report form
CN106951705A (en) * 2017-03-16 2017-07-14 天津艾登科技有限公司 A kind of data quality accessment system serviced for inpatient medical
CN107908768A (en) * 2017-09-30 2018-04-13 北京颐圣智能科技有限公司 Method, apparatus, computer equipment and the storage medium of electronic health record processing
CN107833595A (en) * 2017-10-12 2018-03-23 山东大学 Medical big data multicenter integration platform and method
CN107767929A (en) * 2017-11-13 2018-03-06 医渡云（北京）技术有限公司 CRF filling method, device, electronic equipment and storage medium
CN108038131A (en) * 2017-11-17 2018-05-15 上海数据交易中心有限公司 Data Quality Analysis preprocess method and device, storage medium, terminal
CN108170468A (en) * 2017-12-28 2018-06-15 中山大学 The method and its system of a kind of automatic detection annotation and code consistency
CN108897682A (en) * 2018-06-25 2018-11-27 郑州云海信息技术有限公司 A kind of iteration tests outcome evaluation method based on Python
CN109256212A (en) * 2018-08-17 2019-01-22 上海米因医疗器械科技有限公司 Bone health assessment models construction method, device, equipment, medium and appraisal procedure
CN109473149A (en) * 2018-11-09 2019-03-15 天津开心生活科技有限公司 Data Quality Assessment Methodology, device, electronic equipment and computer-readable medium
CN109522302A (en) * 2018-11-09 2019-03-26 南京医渡云医学技术有限公司 Medical data processing method, device, electronic equipment and computer-readable medium
CN109445948A (en) * 2018-11-15 2019-03-08 济南浪潮高新科技投资发展有限公司 A kind of data mark crowdsourcing plateform system and crowdsourcing data mask method based on intelligent contract
CN109582661A (en) * 2018-11-23 2019-04-05 金色熊猫有限公司 Data structured appraisal procedure, device, storage medium and electronic equipment
CN109710670A (en) * 2018-12-11 2019-05-03 河南通域医疗科技有限公司 A method of case history text is converted into structural metadata from natural language

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
"基于电子病历的临床医疗大数据挖掘流程与方法";阮彤 等;《大数据》;20170920;第83-98页 *

Also Published As

Publication number Publication date
CN110362829A (en) 2019-10-22

Similar Documents

Publication Publication Date Title
CN110362829B (en) Quality evaluation method, device and equipment for structured medical record data
US11416344B2 (en) Partial database restoration
US9690935B2 (en) Identification of obfuscated computer items using visual algorithms
US20150286783A1 (en) Peer group discovery for anomaly detection
US10380252B2 (en) Systems and methods for analyzing document coverage
US7809747B2 (en) Fuzzy database matching
CN106909811B (en) Method and device for processing user identification
CN109074858B (en) Hospital matching of de-identified healthcare databases without distinct quasi-identifiers
US11176110B2 (en) Data updating method and device for a distributed database system
CN112860997A (en) Medical resource recommendation method, device, equipment and storage medium
US11531689B2 (en) Information processing apparatus, information processing method, and non-transitory computer readable medium
CN105117489B (en) Database management method and device and electronic equipment
WO2019196211A1 (en) Method for filtering medical records, terminal device, and computer readable storage medium
CN112420150B (en) Medical image report processing method and device, storage medium and electronic equipment
JP5952441B2 (en) Method for identifying secret data, electronic apparatus and computer-readable recording medium
CN111415760B (en) Doctor recommendation method, doctor recommendation system, computer equipment and storage medium
CN111507405A (en) Picture labeling method and device, electronic equipment and computer readable storage medium
CN109241381B (en) Information matching method and device
CN115759040A (en) Electronic medical record analysis method, device, equipment and storage medium
CN113658711A (en) Medical data localization method and device, computer equipment and storage medium
JP6123372B2 (en) Information processing system, name identification method and program
US11321372B2 (en) Method and system for a natural language processing using data streaming
CN112309586A (en) Method and system for improving adaptation degree of pushing information of medical robot and user diseases
CN117493466B (en) Financial data synchronization method and system
TWI776020B (en) Apparatus, method, and computer program product thereof for locating user interests

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant