CN111339748A - Analytical model evaluation method, analytical model evaluation device, analytical model evaluation equipment and analytical model evaluation medium - Google Patents
Analytical model evaluation method, analytical model evaluation device, analytical model evaluation equipment and analytical model evaluation medium Download PDFInfo
- Publication number
- CN111339748A CN111339748A CN202010095964.4A CN202010095964A CN111339748A CN 111339748 A CN111339748 A CN 111339748A CN 202010095964 A CN202010095964 A CN 202010095964A CN 111339748 A CN111339748 A CN 111339748A
- Authority
- CN
- China
- Prior art keywords
- analysis
- model
- statement
- determining
- analytical
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000011156 evaluation Methods 0.000 title claims abstract description 49
- 238000004458 analytical method Methods 0.000 claims abstract description 311
- 238000000034 method Methods 0.000 claims abstract description 24
- 238000012545 processing Methods 0.000 claims abstract description 14
- 230000004304 visual acuity Effects 0.000 claims abstract description 5
- 238000012216 screening Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 11
- 238000010586 diagram Methods 0.000 description 12
- 230000002159 abnormal effect Effects 0.000 description 8
- 238000012986 modification Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 238000012163 sequencing technique Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 238000003058 natural language processing Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000004075 alteration Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
Images
Landscapes
- Machine Translation (AREA)
Abstract
The application discloses an assessment method, a device, equipment and a medium of an analytic model, belonging to the technical field of data processing, wherein the method comprises the steps of respectively analyzing each statement to be analyzed through the analytic model contained in an analytic set; respectively determining an analytic model for successfully analyzing each statement, wherein each statement corresponds to one type of analytic model; respectively determining the analysis number of the sentences corresponding to each type of analysis model; and obtaining the evaluation result of the resolving power of any type of resolving model according to the resolving quantity of each type of resolving model. Therefore, when the sentences are analyzed through the plurality of analysis models, the analysis capability of any type of analysis model is evaluated through the analysis number of the sentences successfully analyzed by various types of analysis models, and the accuracy of the evaluation of the analysis capability of the analysis models is improved.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a method, an apparatus, a device, and a medium for evaluating an analytic model.
Background
With the development of Language technology, sentences are generally analyzed by using an analysis model generated by Natural Language Processing (NLP) technology to obtain an analysis result, so that corresponding operation Processing can be performed according to the analysis result.
In the prior art, since the resolution capability of one resolution model generally cannot achieve one hundred percent of successful resolution, a plurality of resolution models are generally adopted to resolve a sentence to be resolved.
However, when a plurality of analysis models are used for analysis, it is difficult to evaluate the analysis capability of a specific analysis model in the analysis process, and the obtained evaluation accuracy is poor.
Therefore, how to evaluate the analysis capability of each analysis model is an urgent problem to be solved.
Disclosure of Invention
The embodiment of the application provides an analytical model evaluation method, device, equipment and medium, which are used for improving evaluation accuracy when evaluating analytical capabilities of various analytical models.
In one aspect, an evaluation method for an analytic model is provided, including:
acquiring a plurality of sentences to be analyzed;
analyzing each statement respectively through an analysis model contained in an analysis set, wherein the analysis set comprises a first type of analysis model and a second type of analysis model;
respectively determining an analytic model for successfully analyzing each statement, wherein each statement corresponds to one type of analytic model;
respectively determining the analysis number of the sentences corresponding to each type of analysis model;
and obtaining the evaluation result of the resolving power of any type of resolving model according to the resolving quantity of each type of resolving model.
Preferably, the parsing each statement through the parsing model included in the parsing set includes:
respectively aiming at each statement, the following steps are executed: and analyzing the sentences sequentially through each analysis model contained in the analysis set according to the set model sequence of each analysis model until the analysis is determined to be successful or the analysis is determined to be failed.
Preferably, the set model sequence is determined according to the analysis duration and the model weight of each acquired analysis model.
Preferably, each statement is parsed by a parsing model included in the parsing set, and the parsing further includes:
when the parsing is determined to be successful, recording a parsing model identifier corresponding to a parsing model for successfully parsing the sentence;
when the analysis of each analysis model is determined to be failed, determining the set analysis model identifier as the identifier corresponding to the sentence, and recording the set analysis model identifier;
respectively determining an analysis model for successfully analyzing each statement, comprising the following steps:
and determining the corresponding analytical model according to the analytical model identifier corresponding to each recorded statement.
Preferably, after determining the parsing model successfully parsed for each statement respectively, the method further includes:
obtaining the analysis result of each statement, wherein the analysis result is obtained by analyzing the statement through an analysis model;
matching the analysis result of each statement with the acquired corresponding expected result to obtain corresponding matching degree;
screening out sentences the matching degree of which does not accord with preset matching conditions;
and determining the analytical model corresponding to the screened statement as the model to be optimized.
Preferably, further comprising:
respectively aiming at each analytical model, the following steps are executed:
screening out sentences of which the matching degree does not meet preset matching conditions from the sentences corresponding to the analytic model;
determining a first statement quantity of each statement corresponding to the analysis model and a second statement quantity of the screened statements;
and determining the accuracy of the analytical model according to the first statement quantity and the second statement quantity.
Preferably, before parsing each sentence through the parsing model included in the parsing set, the method further includes:
acquiring a plurality of analysis models for analyzing sentences;
and classifying the acquired multiple analytical models according to a set classification rule to obtain various analytical models.
In one aspect, an apparatus for evaluating an analytical model is provided, including:
the system comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring a plurality of sentences to be analyzed;
the analysis unit is used for analyzing each statement respectively through analysis models contained in an analysis set, and the analysis set comprises a first type of analysis model and a second type of analysis model;
the first determining unit is used for respectively determining an analytic model which is successfully analyzed for each statement, and each statement corresponds to one type of analytic model;
the second determining unit is used for respectively determining the analysis quantity of the sentences corresponding to each type of analysis model;
and the obtaining unit is used for obtaining the evaluation result of the analysis capability of any type of analysis model according to the analysis quantity of each type of analysis model.
Preferably, the parsing unit is configured to:
respectively aiming at each statement, the following steps are executed: and analyzing the sentences sequentially through each analysis model contained in the analysis set according to the set model sequence of each analysis model until the analysis is determined to be successful or the analysis is determined to be failed.
Preferably, the set model sequence is determined according to the analysis duration and the model weight of each acquired analysis model.
Preferably, the parsing unit is further configured to:
when the parsing is determined to be successful, recording a parsing model identifier corresponding to a parsing model for successfully parsing the sentence;
when the analysis of each analysis model is determined to be failed, determining the set analysis model identifier as the identifier corresponding to the sentence, and recording the set analysis model identifier;
respectively determining an analysis model for successfully analyzing each statement, comprising the following steps:
and determining the corresponding analytical model according to the analytical model identifier corresponding to each recorded statement.
Preferably, the first determination unit is further configured to:
obtaining the analysis result of each statement, wherein the analysis result is obtained by analyzing the statement through an analysis model;
matching the analysis result of each statement with the acquired corresponding expected result to obtain corresponding matching degree;
screening out sentences the matching degree of which does not accord with preset matching conditions;
and determining the analytical model corresponding to the screened statement as the model to be optimized.
Preferably, the first determination unit is further configured to:
respectively aiming at each analytical model, the following steps are executed:
screening out sentences of which the matching degree does not meet preset matching conditions from the sentences corresponding to the analytic model;
determining a first statement quantity of each statement corresponding to the analysis model and a second statement quantity of the screened statements;
and determining the accuracy of the analytical model according to the first statement quantity and the second statement quantity.
Preferably, the parsing unit further:
acquiring a plurality of analysis models for analyzing sentences;
and classifying the acquired multiple analytical models according to a set classification rule to obtain various analytical models.
In one aspect, there is provided a control apparatus comprising:
at least one memory for storing program instructions;
and the at least one processor is used for calling the program instructions stored in the memory and executing the steps of the evaluation method of any analysis model according to the obtained program instructions.
In one aspect, a computer-readable storage medium is provided, on which a computer program is stored, which computer program, when being executed by a processor, carries out the steps of any one of the above described analytical model evaluation methods.
In the evaluation method, the evaluation device, the evaluation equipment and the evaluation medium for the analytic models, each statement to be analyzed is analyzed through the analytic models contained in the analytic set; respectively determining an analytic model for successfully analyzing each statement, wherein each statement corresponds to one type of analytic model; respectively determining the analysis number of the sentences corresponding to each type of analysis model; and obtaining the evaluation result of the resolving power of any type of resolving model according to the resolving quantity of each type of resolving model. Therefore, when the sentences are analyzed through the plurality of analysis models, the analysis capability of any type of analysis model is evaluated through the analysis number of the sentences successfully analyzed by various types of analysis models, and the accuracy of the evaluation of the analysis capability of the analysis models is improved.
Additional features and advantages of the application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the application. The objectives and other advantages of the application may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
fig. 1 is a flowchart of an embodiment of an evaluation method for an analytic model in an embodiment of the present application;
FIG. 2 is a flow chart of an analysis implementation in an embodiment of the present application;
FIG. 3 is a flow chart of an implementation of model determination to be optimized in an embodiment of the present application;
FIG. 4 is a flow chart illustrating an implementation of accuracy determination in an embodiment of the present application;
FIG. 5 is a schematic structural diagram of an apparatus for evaluating an analytical model according to an embodiment of the present disclosure;
fig. 6 is a schematic structural diagram of a control device in an embodiment of the present application.
Detailed Description
In order to make the purpose, technical solution and beneficial effects of the present application more clear and more obvious, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
In order to improve the evaluation accuracy when evaluating the analytical capabilities of various analytical models, embodiments of the present application provide an analytical model evaluation method, apparatus, device, and medium.
Referring to fig. 1, a flowchart of an embodiment of an analytical model evaluation method according to the present invention is shown.
The specific implementation flow of the method is as follows:
step 100: the control device obtains a plurality of statements to be parsed.
Specifically, the control device may be a terminal device or a server having a data processing capability.
Wherein, the terminal equipment: may be a mobile terminal, a fixed terminal, or a portable terminal such as a mobile handset, station, unit, device, multimedia computer, multimedia tablet, internet node, communicator, desktop computer, laptop computer, notebook computer, netbook computer, tablet computer, Personal Communication Systems (PCS) device, personal navigation device, Personal Digital Assistant (PDA), audio/video player, or any combination thereof, including accessories and peripherals of such devices, or any combination thereof. It is also contemplated that the terminal device can support any type of interface to the user (e.g., wearable device), and the like.
A server: servers, mainframe computing devices, etc. provided by various services. The server may be one or more servers. The server may also be a physical server or a virtual server, etc.
Step 101: and the control equipment analyzes each statement respectively through the analysis model contained in the analysis set.
Specifically, before step 101 is executed, the control device obtains a plurality of analysis models for analyzing a sentence, classifies the obtained plurality of analysis models according to a set classification rule, obtains various types of analysis models, and obtains an analysis set composed of various types of analysis models.
The analysis set includes a first type of analysis model and a second type of analysis model, and may also include other types of analysis models, and each type of analysis model may include one analysis model or may include multiple analysis models, which is not limited herein.
When the control device classifies the acquired multiple analysis models according to the set classification rule, the following modes can be adopted:
the first mode is as follows: according to the analytical algorithm adopted by the analytical model, the analytical models adopting the same analytical principle are divided into the same class.
For example, analytical models using neural networks are classified into the same class of analytical models.
The second way is: and setting the grade of the analytical model according to the actual application requirements, and dividing the analytical models in the same grade into the same class.
For example, the analysis models included in the analysis set are analysis model a, analysis model B, analysis model C, and analysis model D, respectively. The analysis model A is a main model, the analysis model B, the analysis model C and the analysis model D are auxiliary models, the level of the analysis model is set to be a first level, and the level of the analysis model B, the analysis model C and the analysis model D is set to be a second level. The analytical model A is divided into a first type of analytical model, and the analytical model B, the analytical model C and the analytical model D are divided into a second type of analytical model.
In practical applications, the analytical model may be classified in other manners, which is not limited herein.
In step 101, refer to an analysis implementation flowchart shown in fig. 2. In the step 101, the control device may perform the following steps for each sentence:
s1011: and the control equipment analyzes the sentences sequentially through each analysis model contained in the analysis set according to the set model sequence of each analysis model until the analysis is determined to be successful or the analysis is determined to be failed.
The analytic model is used for carrying out natural language processing on the sentences. Different analysis models have different characteristics and different analysis time lengths.
Optionally, the set model sequence may be determined according to the analysis duration and the model weight of each acquired analysis model.
In one embodiment, the parsing time of an parsing model is an average of the parsing time of each sentence.
In practical application, the model weight may be set according to a practical application scenario, and is not limited herein.
When determining the setting model sequence, the control device may adopt any one of the following modes:
the first mode is as follows: and sequencing according to the preset model weights of the analytical models from high to low, and sequencing the analytical models with the same model weights according to the sequence of the analytical durations from short to long.
The second way is: and sequencing the top N analytical models with the highest weight in each analytical model according to the sequence of the model weights from high to low, and sequencing the rest analytical models according to the sequence of the analytical durations from short to long. Where N is a positive integer, e.g., N ═ 1.
For example, the control device puts the analytic model with the highest weight in each analytic model at the head, and sorts the remaining multiple analytic models in the order of the analytic durations from short to long.
Further, the model setting sequence may be set according to an actual application scenario, for example, the model setting sequence may be set randomly, or may be set according to the analysis model type, which is not limited herein.
The reason is that the parsing capability of one or one type of parsing model usually cannot achieve one hundred percent of successful parsing, in order to improve the parsing success rate, the sentence is parsed by multiple types of parsing models, that is, when one or one type of parsing model fails to parse the sentence, other parsing models can be used to parse the sentence, and when it is determined that the parsing model succeeds in parsing the sentence, the parsing of the sentence can be stopped.
S1012: and when the analysis is determined to be successful, the control equipment records the analysis model identification corresponding to the analysis model which is successfully analyzed by the statement.
The analysis model identifier is used to represent the analysis model, and may be represented in a form of code or text, which is not limited herein.
Since each parsing model usually processes a statement in the form of a code, in the subsequent step, the parsing model that is successfully parsed can be determined by the parsing model identifier recorded in the log.
S1013: and when the analysis of each analysis model fails, the control equipment determines the set analysis model as the identifier corresponding to the sentence, and records the set analysis model identifier.
And setting the analysis model identification as the identification selected from the analysis model identifications corresponding to the analysis models.
In one embodiment, the analysis model identifier corresponding to the first analysis model is determined as the set analysis model identifier.
This is because, in practical applications, generally, one specific analysis model is used as a main analysis model, that is, the specific analysis model and a plurality of other analysis models form an analysis set, the specific analysis model is used as a first model in a set model sequence, and the other analysis models are used as backup models to analyze a sentence for which the specific analysis model has not been successfully analyzed. And when the analysis of each analysis model fails, the default is that the specific analysis model is successfully analyzed, so that the analysis capability of each analysis model is evaluated according to the analysis success quantity of each analysis model in the subsequent steps, and the specific analysis model is optimized.
In one embodiment, when a sentence is analyzed by each analysis model, an analysis model identifier corresponding to an analysis model for successfully analyzing the sentence is recorded in the form of a program log during the analysis. And when the analysis of each analysis model fails, setting an analysis model identifier as an analysis model identifier corresponding to the analysis model which successfully analyzes the sentence.
For example, assume that each analytical model is model a, model B, model C, and model D. The set model sequence of each analytical model is model A, model B, model C and model D, and a corresponding analytical model identifier is set for each analytical model. The statement is statement E.
And the control equipment determines that the statement E is failed to be analyzed by the model A, and analyzes the statement E through the model B. And when the control equipment determines that the model B successfully analyzes the statement E, the control equipment determines that the model B is an analysis model corresponding to the statement E, and writes an analysis model identifier of the model B when a log (log) is generated. Then it can be determined that the statement is model B resolved by the resolved model identification of model B recorded in log.
Furthermore, the control device analyzes each statement through the analysis model contained in the analysis set, and can also obtain the analysis result of each analysis statement. The control device may further position the analytic model corresponding to the abnormal analytic result according to the analytic result, so as to further optimize the analytic model.
Step 102: the control device determines the analytic models successfully analyzed for each sentence respectively.
Specifically, the control device determines a corresponding analysis model according to the recorded analysis model identifier corresponding to each statement.
Further, the control device may determine the analysis model type corresponding to each sentence according to the analysis model type corresponding to the analysis model.
That is, each statement corresponds to a class of parsing models.
Step 103: and the control equipment respectively determines the analysis number of the sentences corresponding to each type of analysis model.
Specifically, the control device determines the analysis number of the sentences corresponding to each analysis model respectively, and determines the analysis number of each type of analysis model respectively according to the analysis model type and the analysis number corresponding to each analysis model.
Therefore, the analysis number of the sentences successfully analyzed by each type of analysis model can be counted.
Step 104: and the control equipment obtains the evaluation result of the analysis capability of any type of analysis model according to the analysis quantity of each type of analysis model.
Specifically, the control device determines the sum of the analytic numbers of each type of analytic model, and determines the analytic ratio of the type of analytic model according to the ratio of the analytic number of each type of analytic model to the sum, so as to obtain an evaluation result containing the analytic ratio corresponding to each type of analytic model.
The evaluation result may further include the analysis accuracy of each analysis model and a corresponding model to be optimized when the analysis result is abnormal.
Referring to an implementation flowchart for determining a model to be optimized shown in fig. 3, when locating and analyzing an abnormal model to be optimized, the control device may adopt the following steps:
step 301: and acquiring the expected result correspondingly set by each stored statement.
Specifically, the control device sets a corresponding expected result for each parsing statement in advance.
Step 302: and respectively matching the analysis result of each statement with the acquired corresponding expected result to obtain the corresponding matching degree.
Step 303: and screening out sentences the matching degree of which does not accord with the preset matching condition.
Specifically, the preset matching condition may be higher than a preset matching value. The control device screens out statements whose matching degree is not higher than a preset matching value.
In practical application, the preset matching value may be set according to a practical application scenario, for example, the preset matching value may be 0.8, which is not described herein again.
Step 304: and determining the analytical model corresponding to the screened statement as the model to be optimized.
Specifically, the control device determines a corresponding analysis model according to an analysis model identifier corresponding to the analysis statement, and determines the analysis model as a model to be optimized.
Therefore, the analytic model identification corresponding to each analytic statement can be recorded, the analytic model with abnormal analytic statement can be quickly positioned through the recorded analytic model identification, the positioning time is saved, the processing efficiency is improved, and the analytic model with abnormal analytic statement can be further optimized.
Further, the accuracy of the analytical model analysis may also be determined. Referring to an implementation flow chart of accuracy determination shown in fig. 4, the control device may further perform the following steps for each analytical model respectively:
step 401: and screening out sentences of which the matching degree does not meet the preset matching condition from the sentences corresponding to the analysis model.
Step 402: and determining the first statement quantity of each statement corresponding to the analysis model and the second statement quantity of the screened statements.
Step 403: and determining the accuracy of the analytical model according to the first statement quantity and the second statement quantity.
In one embodiment, the ratio between the second number of sentences and the first number of sentences is determined as the accuracy of the analytical model.
That is, the accuracy of the parsing is determined according to the first statement quantity of the parsing success of one parsing model and the second statement quantity of the parsing result abnormity.
Therefore, the analysis model with abnormal analysis can be rapidly positioned through the analysis model identification so as to optimize the analysis model and determine the accuracy of the analysis model.
In the embodiment of the application, when the sentences are analyzed through the plurality of analysis models, the analysis accuracy of each analysis model, the model to be optimized with abnormal analysis results and the analysis ratio of any type of analysis model can be determined, the efficiency of abnormal positioning and the accuracy of evaluation are improved, the positioning time is saved, and the evaluation results are quantized. Further, when the specified analysis model is further optimized according to the determined accuracy, the model to be optimized and the analysis ratio, each suboptimal effect can be intuitively known through an evaluation result.
Based on the same inventive concept, the embodiment of the present application further provides an evaluation apparatus for an analytic model, and because the principle of the apparatus and the device for solving the problem is similar to that of an evaluation method for an analytic model, the implementation of the apparatus can refer to the implementation of the method, and repeated details are not repeated.
As shown in fig. 5, which is a schematic structural diagram of an evaluation apparatus for an analytical model according to an embodiment of the present application, the evaluation apparatus includes:
an obtaining unit 501, configured to obtain multiple statements to be parsed;
an analysis unit 502, configured to analyze each statement respectively through an analysis model included in an analysis set, where the analysis set includes a first type of analysis model and a second type of analysis model;
a first determining unit 503, configured to determine an analysis model that is successfully analyzed for each statement, where each statement corresponds to one type of analysis model;
a second determining unit 504, configured to determine the number of parsing statements corresponding to each type of parsing model;
an obtaining unit 505, configured to obtain an evaluation result of the analytic capability of any type of analytic model according to the analytic number of each type of analytic model.
Preferably, the parsing unit 502 is configured to:
respectively aiming at each statement, the following steps are executed: and analyzing the sentences sequentially through each analysis model contained in the analysis set according to the set model sequence of each analysis model until the analysis is determined to be successful or the analysis is determined to be failed.
Preferably, the set model sequence is determined according to the analysis duration and the model weight of each acquired analysis model.
Preferably, the parsing unit 502 is further configured to:
when the parsing is determined to be successful, recording a parsing model identifier corresponding to a parsing model for successfully parsing the sentence;
when the analysis of each analysis model is determined to be failed, determining the set analysis model identifier as the identifier corresponding to the sentence, and recording the set analysis model identifier;
respectively determining an analysis model for successfully analyzing each statement, comprising the following steps:
and determining the corresponding analytical model according to the analytical model identifier corresponding to each recorded statement.
Preferably, the first determining unit 503 is further configured to:
obtaining the analysis result of each statement, wherein the analysis result is obtained by analyzing the statement through an analysis model;
matching the analysis result of each statement with the acquired corresponding expected result to obtain corresponding matching degree;
screening out sentences the matching degree of which does not accord with preset matching conditions;
and determining the analytical model corresponding to the screened statement as the model to be optimized.
Preferably, the first determining unit 503 is further configured to:
respectively aiming at each analytical model, the following steps are executed:
screening out sentences of which the matching degree does not meet preset matching conditions from the sentences corresponding to the analytic model;
determining a first statement quantity of each statement corresponding to the analysis model and a second statement quantity of the screened statements;
and determining the accuracy of the analytical model according to the first statement quantity and the second statement quantity.
Preferably, the parsing unit 502 further:
acquiring a plurality of analysis models for analyzing sentences;
and classifying the acquired multiple analytical models according to a set classification rule to obtain various analytical models.
In the evaluation method, the evaluation device, the evaluation equipment and the evaluation medium for the analytic models, each statement to be analyzed is analyzed through the analytic models contained in the analytic set; respectively determining an analytic model for successfully analyzing each statement, wherein each statement corresponds to one type of analytic model; respectively determining the analysis number of the sentences corresponding to each type of analysis model; and obtaining the evaluation result of the resolving power of any type of resolving model according to the resolving quantity of each type of resolving model. Therefore, when the sentences are analyzed through the plurality of analysis models, the analysis capability of any type of analysis model is evaluated through the analysis number of the sentences successfully analyzed by various types of analysis models, and the accuracy of the evaluation of the analysis capability of the analysis models is improved.
For convenience of description, the above parts are separately described as modules (or units) according to functional division. Of course, the functionality of the various modules (or units) may be implemented in the same one or more pieces of software or hardware when implementing the present application.
Based on the above embodiments, referring to fig. 6, in an embodiment of the present application, a structural schematic diagram of a control device is shown.
The embodiment of the present application provides a control device, which may include a processor 610 (central processing Unit, CPU), a memory 620, and may further include an input device 630, an output device 640, and the like, where the input device 630 may include a keyboard, a mouse, a touch screen, and the like, and the output device 640 may include a Display device, such as a Liquid Crystal Display (LCD), a Cathode Ray Tube (CRT), and the like.
The processor 610 is configured to perform a method of analytical model evaluation provided by the embodiment shown in fig. 1 by calling program instructions stored in the memory 620 by the processor 610.
In an embodiment of the present application, there is further provided a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the method for evaluating an analytical model in any of the above-mentioned method embodiments.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While the preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all alterations and modifications as fall within the scope of the application.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.
Claims (16)
1. A method for evaluating an analytical model, comprising:
acquiring a plurality of sentences to be analyzed;
analyzing each statement respectively through an analysis model contained in an analysis set, wherein the analysis set comprises a first type of analysis model and a second type of analysis model;
respectively determining an analytic model for successfully analyzing each statement, wherein each statement corresponds to one type of analytic model;
respectively determining the analysis number of the sentences corresponding to each type of analysis model;
and obtaining the evaluation result of the resolving power of any type of resolving model according to the resolving quantity of each type of resolving model.
2. The method of claim 1, wherein parsing each statement separately through a parsing model included in a parsing set comprises:
respectively aiming at each statement, the following steps are executed: and analyzing the statement sequentially through each analysis model contained in the analysis set according to the set model sequence of each analysis model until the analysis is determined to be successful or the analysis fails.
3. The method of claim 2, wherein the set model order is determined based on the analysis duration and the model weight of each of the acquired analysis models.
4. The method of claim 2, wherein each statement is parsed separately by a parsing model included in a parsing set, further comprising:
when the sentence is successfully analyzed, recording an analysis model identifier corresponding to an analysis model successfully analyzed by the sentence;
when the analysis of each analysis model is determined to be failed, determining a set analysis model identifier as an identifier corresponding to the statement, and recording the set analysis model identifier;
respectively determining an analysis model for successfully analyzing each statement, comprising the following steps:
and determining the corresponding analytical model according to the analytical model identifier corresponding to each recorded statement.
5. The method of any of claims 2-4, after separately determining a parsing model that is successful for each statement, further comprising:
obtaining an analysis result of each statement, wherein the analysis result is obtained by analyzing the statement through an analysis model;
matching the analysis result of each statement with the acquired corresponding expected result to obtain corresponding matching degree;
screening out sentences the matching degree of which does not accord with preset matching conditions;
and determining the analytical model corresponding to the screened statement as the model to be optimized.
6. The method of claim 5, further comprising:
respectively aiming at each analytical model, the following steps are executed:
screening out sentences the matching degree of which does not accord with the preset matching condition from the sentences corresponding to the analysis model;
determining a first statement quantity of each statement corresponding to the analysis model and a second statement quantity of the screened statements;
and determining the accuracy of the analytical model according to the first statement quantity and the second statement quantity.
7. The method of any of claims 2-4, prior to parsing each statement separately through the parsing models contained in the parsing set, further comprising:
acquiring a plurality of analysis models for analyzing sentences;
and classifying the acquired multiple analytical models according to a set classification rule to obtain various analytical models.
8. An apparatus for evaluating an analytical model, comprising:
the system comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring a plurality of sentences to be analyzed;
the analysis unit is used for analyzing each statement respectively through analysis models contained in an analysis set, and the analysis set comprises a first type of analysis model and a second type of analysis model;
the first determining unit is used for respectively determining an analytic model which is successfully analyzed for each statement, and each statement corresponds to one type of analytic model;
the second determining unit is used for respectively determining the analysis quantity of the sentences corresponding to each type of analysis model;
and the obtaining unit is used for obtaining the evaluation result of the analysis capability of any type of analysis model according to the analysis quantity of each type of analysis model.
9. The apparatus of claim 8, wherein the parsing unit is to:
respectively aiming at each statement, the following steps are executed: and analyzing the statement sequentially through each analysis model contained in the analysis set according to the set model sequence of each analysis model until the analysis is determined to be successful or the analysis fails.
10. The apparatus of claim 9, wherein the set model order is determined based on the analysis duration and the model weight of each of the acquired analysis models.
11. The apparatus of claim 9, wherein the parsing unit is further to:
when the sentence is successfully analyzed, recording an analysis model identifier corresponding to an analysis model successfully analyzed by the sentence;
when the analysis of each analysis model is determined to be failed, determining a set analysis model identifier as an identifier corresponding to the statement, and recording the set analysis model identifier;
respectively determining an analysis model for successfully analyzing each statement, comprising the following steps:
and determining the corresponding analytical model according to the analytical model identifier corresponding to each recorded statement.
12. The apparatus of any of claims 9-11, wherein the first determination unit is further to:
obtaining an analysis result of each statement, wherein the analysis result is obtained by analyzing the statement through an analysis model;
matching the analysis result of each statement with the acquired corresponding expected result to obtain corresponding matching degree;
screening out sentences the matching degree of which does not accord with preset matching conditions;
and determining the analytical model corresponding to the screened statement as the model to be optimized.
13. The apparatus of claim 12, wherein the first determining unit is further configured to:
respectively aiming at each analytical model, the following steps are executed:
screening out sentences the matching degree of which does not accord with the preset matching condition from the sentences corresponding to the analysis model;
determining a first statement quantity of each statement corresponding to the analysis model and a second statement quantity of the screened statements;
and determining the accuracy of the analytical model according to the first statement quantity and the second statement quantity.
14. The apparatus of any of claims 9-11, wherein the parsing unit is further to:
acquiring a plurality of analysis models for analyzing sentences;
and classifying the acquired multiple analytical models according to a set classification rule to obtain various analytical models.
15. A control apparatus, characterized by comprising:
at least one memory for storing program instructions;
at least one processor for calling program instructions stored in said memory and for executing the steps of the method according to any one of the preceding claims 1 to 7 in accordance with the program instructions obtained.
16. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010095964.4A CN111339748B (en) | 2020-02-17 | 2020-02-17 | Evaluation method, device, equipment and medium of analytical model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010095964.4A CN111339748B (en) | 2020-02-17 | 2020-02-17 | Evaluation method, device, equipment and medium of analytical model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111339748A true CN111339748A (en) | 2020-06-26 |
| CN111339748B CN111339748B (en) | 2023-11-17 |
Family
ID=71181528
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010095964.4A Active CN111339748B (en) | 2020-02-17 | 2020-02-17 | Evaluation method, device, equipment and medium of analytical model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111339748B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107733705A (en) * | 2017-10-10 | 2018-02-23 | 锐捷网络股份有限公司 | A kind of user experience quality assessment models method for building up and equipment |
| CN109408628A (en) * | 2019-01-15 | 2019-03-01 | 深兰人工智能芯片研究院（江苏）有限公司 | A kind of method and relevant device parsing semantic structure of sentences |
| CN109784352A (en) * | 2017-11-14 | 2019-05-21 | 北京京东尚科信息技术有限公司 | A kind of method and apparatus for assessing disaggregated model |
| CN110363243A (en) * | 2019-07-12 | 2019-10-22 | 腾讯科技（深圳）有限公司 | The appraisal procedure and device of disaggregated model |
| US20190332678A1 (en) * | 2017-01-11 | 2019-10-31 | Panasonic Intellectual Property Management Co., Ltd. | Sentence evaluation apparatus and sentence evaluation method |
| CN110489740A (en) * | 2019-07-12 | 2019-11-22 | 深圳追一科技有限公司 | Semantic analytic method and Related product |
-
2020
- 2020-02-17 CN CN202010095964.4A patent/CN111339748B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190332678A1 (en) * | 2017-01-11 | 2019-10-31 | Panasonic Intellectual Property Management Co., Ltd. | Sentence evaluation apparatus and sentence evaluation method |
| CN107733705A (en) * | 2017-10-10 | 2018-02-23 | 锐捷网络股份有限公司 | A kind of user experience quality assessment models method for building up and equipment |
| CN109784352A (en) * | 2017-11-14 | 2019-05-21 | 北京京东尚科信息技术有限公司 | A kind of method and apparatus for assessing disaggregated model |
| CN109408628A (en) * | 2019-01-15 | 2019-03-01 | 深兰人工智能芯片研究院（江苏）有限公司 | A kind of method and relevant device parsing semantic structure of sentences |
| CN110363243A (en) * | 2019-07-12 | 2019-10-22 | 腾讯科技（深圳）有限公司 | The appraisal procedure and device of disaggregated model |
| CN110489740A (en) * | 2019-07-12 | 2019-11-22 | 深圳追一科技有限公司 | Semantic analytic method and Related product |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111339748B (en) | 2023-11-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9038027B2 (en) | Systems and methods for identifying software performance influencers | |
| CN109684008B (en) | Card rendering method, device, terminal and computer readable storage medium | |
| CN111475494A (en) | Mass data processing method, system, terminal and storage medium | |
| CN109240916B (en) | Information output control method, information output control device and computer readable storage medium | |
| CN110990285A (en) | UI automation test method and device | |
| CN110473530A (en) | Instruction classification method, apparatus, electronic equipment and computer readable storage medium | |
| CN112905451B (en) | Automatic testing method and device for application program | |
| CN113609008A (en) | Test result analysis method and device and electronic equipment | |
| CN111339748B (en) | Evaluation method, device, equipment and medium of analytical model | |
| CN111127223A (en) | Insurance product testing method and device and storage medium | |
| CN108304310B (en) | Log analysis method and computing device | |
| CN114238048B (en) | Automatic testing method and system for Web front-end performance | |
| CN113849484A (en) | Big data component upgrading method and device, electronic equipment and storage medium | |
| CN115080433A (en) | Testing method and device based on flow playback | |
| CN113419957A (en) | Rule-based big data offline batch processing performance capacity scanning method and device | |
| CN111008140B (en) | Cross-platform UI (user interface) automatic testing method and device | |
| CN114491044A (en) | Log processing method and device | |
| CN113010339A (en) | Method and device for automatically processing fault in online transaction test | |
| CN113344023A (en) | Code recommendation method, device and system | |
| CN112035425B (en) | Log storage method and device and computer system | |
| CN117555813B (en) | Method, device, equipment and medium for determining common test case | |
| CN109992505B (en) | Application program testing method and device, computer equipment and storage medium | |
| CN115481045A (en) | Method and device for testing adaptability of banking business page, storage medium and equipment | |
| CN114356619A (en) | Project processing method and device | |
| CN115687078A (en) | Test case evaluation method, test case evaluation apparatus, test case evaluation device, test case evaluation medium, and program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |