CN111310829A - Confusion matrix-based classification result detection method and device and storage medium - Google Patents
Confusion matrix-based classification result detection method and device and storage medium Download PDFInfo
- Publication number
- CN111310829A CN111310829A CN202010095296.5A CN202010095296A CN111310829A CN 111310829 A CN111310829 A CN 111310829A CN 202010095296 A CN202010095296 A CN 202010095296A CN 111310829 A CN111310829 A CN 111310829A
- Authority
- CN
- China
- Prior art keywords
- classification result
- accuracy
- confusion matrix
- class
- classification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 239000011159 matrix material Substances 0.000 title claims abstract description 102
- 238000001514 detection method Methods 0.000 title claims description 26
- 238000000034 method Methods 0.000 claims abstract description 40
- 238000012360 testing method Methods 0.000 claims description 19
- 238000013145 classification model Methods 0.000 claims description 12
- 238000004364 calculation method Methods 0.000 claims description 11
- 238000012549 training Methods 0.000 abstract description 7
- 238000011156 evaluation Methods 0.000 description 13
- 238000010586 diagram Methods 0.000 description 10
- 230000000694 effects Effects 0.000 description 8
- 238000010801 machine learning Methods 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001680 brushing effect Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000007635 classification algorithm Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method for detecting classification results based on a confusion matrix, which comprises the following steps: obtaining a confusion matrix of the classification result; calculating the accuracy of each category classification result according to the confusion matrix; and calculating the accuracy of the whole classification result according to the accuracy of each classification result. By the method, the accuracy of the whole classification result of the classifier can be obtained, the accuracy of the classification result of each class can be given, and the method can be used for model selection and has certain significance for better guiding the training process.
Description
Technical Field
The present invention relates to the field of machine learning technologies, and in particular, to a method and an apparatus for detecting classification results based on a confusion matrix, and a storage medium.
Background
The classification task is one of the most common tasks in the field of machine learning. In the field of machine learning, classifying a sample instance into one of two categories is referred to as a two-classification task, and a multi-classification task refers to a problem of classifying a sample instance into one of three or more categories.
Classification result evaluation is one of the basic problems of the machine learning domain classification task. Due to the limitations of a classification algorithm and a model, reasonable evaluation of a classification result of a classifier is a necessary problem, and on the other hand, due to the over-fitting phenomenon of the classifier, it is very important to properly select an evaluation method, and the quality of the evaluation method directly influences the evaluation and selection of the model, thereby influencing the cost of a model training process and practical application.
The confusion matrix is a matrix formed by the classification result of the classifier and the real label of the sample, is also called an error matrix, an error matrix and a possibility table, and is a summary of the prediction result of the classification problem. Existing evaluation indexes such as accuracy rate AP, Kappa coefficient, F1 value, etc. are based on confusion matrix to evaluate the effect of the whole classification result, and it is difficult to provide evaluation of the classification result of a single category, which is not enough to meet the user requirement in some practical applications, for example, in the MNIST handwritten character recognition task, the probability and importance of the occurrence of the number 0 are often larger and higher than those of other numbers.
Disclosure of Invention
The embodiment of the disclosure provides a method and a device for detecting a classification result based on a confusion matrix and a storage medium. The following presents a simplified summary in order to provide a basic understanding of some aspects of the disclosed embodiments. This summary is not an extensive overview and is intended to neither identify key/critical elements nor delineate the scope of such embodiments. Its sole purpose is to present some concepts in a simplified form as a prelude to the more detailed description that is presented later.
In a first aspect, an embodiment of the present disclosure provides a method for detecting a classification result based on a confusion matrix, including:
obtaining a confusion matrix of the classification result;
calculating the accuracy of each category classification result according to the confusion matrix;
and calculating the accuracy of the overall classification result according to the accuracy of each classification result.
Further, before obtaining the confusion matrix of the classification result, the method further includes:
and constructing a confusion matrix of the classification result according to the prediction label and the real label of the test sample.
Further, before calculating the accuracy of each classification result according to the confusion matrix, the method further comprises:
and calculating the classification efficiency of each class classification result according to the confusion matrix.
Further, calculating the accuracy of each class classification result according to the confusion matrix, comprising:
and calculating the accuracy of each class classification result according to the classification efficiency of each class classification result and the probability of the class appearing in the total sample.
Further, calculating the accuracy of the overall classification result according to the accuracy of each classification result, comprising:
according to the formula: calculating the accuracy of the overall classification result;
wherein, R is the accuracy of the overall classification result, R (i) is the accuracy of the classification result of the ith class, and N is the total number of classes.
Further, after calculating the accuracy of the overall classification result according to the accuracy of each classification result, the method further comprises:
and determining a classification model according to the accuracy of each classification result and the accuracy of the overall classification result.
Further, determining a classification model according to the accuracy of each classification result and the accuracy of the overall classification result, comprising:
and determining the model with higher accuracy of the whole classification result or the classification result of a certain class as the classification model.
In a second aspect, an embodiment of the present disclosure provides a device for detecting a classification result based on a confusion matrix, including:
the obtaining module is used for obtaining a confusion matrix of the classification result;
the first calculation module is used for calculating the accuracy of each category classification result according to the confusion matrix;
the second calculation module is used for calculating the accuracy of the whole classification result according to the accuracy of each classification result;
in a third aspect, the present disclosure provides a confusion matrix-based classification result detection system, which includes a processor and a memory storing program instructions, where the processor is configured to execute the confusion matrix-based classification result detection method provided in the foregoing embodiments when executing the program instructions.
In a fourth aspect, the disclosed embodiments provide a computer-readable medium, on which computer-readable instructions are stored, where the computer-readable instructions are executable by a processor to implement a confusion matrix-based classification result detection method provided in the foregoing embodiments.
The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects:
1) the detection method of the invention not only can obtain the accuracy of the whole classification result of the classifier, but also can give the accuracy of the classification result of each category, and the method can be used for model selection and has certain significance for better guiding the training process.
2) The method can be expanded to the problem of multi-classification tasks in any scene, and has wide application prospect.
3) And the inference is carried out based on the confusion matrix, and the method is simple in calculation.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
FIG. 1 is a flow diagram illustrating a confusion matrix-based classification result detection method according to an exemplary embodiment;
FIG. 2 is a flow diagram illustrating a confusion matrix-based classification result detection method according to an exemplary embodiment;
FIG. 3 is a schematic diagram illustrating a confusion matrix according to an exemplary embodiment;
FIG. 4 is a block diagram of an apparatus for detecting a classification result based on a confusion matrix according to an exemplary embodiment;
fig. 5 is a schematic structural diagram illustrating a confusion matrix-based classification result detection system according to an exemplary embodiment.
Detailed Description
So that the manner in which the features and elements of the disclosed embodiments can be understood in detail, a more particular description of the disclosed embodiments, briefly summarized above, may be had by reference to the embodiments, some of which are illustrated in the appended drawings. In the following description of the technology, for purposes of explanation, numerous details are set forth in order to provide a thorough understanding of the disclosed embodiments. However, one or more embodiments may be practiced without these details. In other instances, well-known structures and devices may be shown in simplified form in order to simplify the drawing.
The embodiment of the present disclosure provides a confusion matrix-based classification result detection method, and fig. 1 is a schematic flow chart of a confusion matrix-based classification result detection method according to an exemplary embodiment.
As shown in fig. 1, a method for detecting a classification result based on a confusion matrix includes:
s101, obtaining a confusion matrix of a classification result;
the confusion matrix is a matrix formed by the classification result of the classifier and the real label of the sample, and is also called an error matrix, an error matrix and a possibility table, and is a summary of the prediction result of the classification problem, each column of the confusion matrix represents a prediction class, the total number of each column represents the number of data predicted to be in the class, each row represents a real attribution class of the data, the total number of data in each row represents the number of data instances of the class, and the value in each column represents the number of real data predicted to be in the class.
Specifically, before obtaining a confusion matrix of a classification result, firstly, obtaining test sample data, then inputting the test sample data into a classifier to obtain a prediction label of the test sample, and constructing the confusion matrix of the classification result according to the prediction label and the real label of the test sample. And testing the test sample by the classifier to obtain a prediction label of the test sample, and further combining a real label of the test sample to construct a confusion matrix.
Taking a binary task as an example, a confusion matrix is constructed, and fig. 3 is a schematic diagram illustrating a confusion matrix according to an exemplary embodiment, wherein each column of the confusion matrix represents a prediction class, the total number of each column represents the number of data predicted as the class, each row represents a true belonging class of data, the total number of data in each row represents the number of data instances of the class, and the value in each column represents the number of true data predicted as the class. As shown in fig. 3, the confusion matrix of the classification result includes class 1 and class 2, where the number of true labels of class 1 is m, the number of samples predicted as class 1 in the prediction result is s, the number of samples predicted as class 1 correctly is a, and c samples in class 1 are incorrectly predicted as class 2; the number of the real labels of the class 2 is n, the number of the samples predicted to be the class 2 in the prediction result is t, the number of the samples predicted to be the class 2 correctly is b, d samples in the class 2 are predicted to be the class 1 incorrectly, and the total number of the samples is w.
By the method, the confusion matrix can be constructed based on the classification result.
Step S102, calculating the accuracy of each category classification result according to a confusion matrix;
before the accuracy of the classification result of each category is calculated according to the confusion matrix, the classification efficiency of each category is calculated according to the acquired confusion matrix.
Taking class 1 as an example, the number of the real labels of class 1 is m, the number of samples predicted to be class 1 in the prediction result is s, the number of correctly predicted samples is a, and the total number of samples is w. The classification efficiency E for this class is defined as follows: the difference between the probability of correctly classifying the class and the probability of the sample being predicted as this class is:
E(1)＝P(c)-P(y)
wherein, p (c) represents the probability that the category is correctly classified, and the calculation method is as follows: the ratio of the number of correctly classified samples to the total number of samples of that class is:
p (y) represents the probability that the sample is predicted to be of the class, i.e.:
taking class 2 as an example, the number of the real labels of class 2 is n, the number of samples predicted as class 2 in the prediction result is t, the number of correctly predicted samples is b, and the total number of samples is w. The classification efficiency E for this class is defined as follows: the difference between the probability of correctly classifying the class and the probability of the sample being predicted as this class is:
E(2)＝P(x)-P(z)
wherein, p (x) represents the probability that the category is correctly classified, and the calculation method is as follows: the ratio of the number of correctly classified samples to the total number of samples of that class is:
p (z) represents the probability that the sample is predicted to be of that class, i.e.:
and after the classification efficiency of each category is obtained through calculation, calculating the accuracy of the classification result of each category according to the classification efficiency of each category.
On the basis of obtaining the classification efficiency of the class 1, combining the probability P (a) of the class 1 appearing in the total sample, namely:
calculate accuracy R (1) for category 1:
R(1)＝E(1)+P(a)
on the basis of obtaining the classification efficiency of the class 2, combining the probability P (b) of the class 2 appearing in the total sample, namely:
calculate accuracy for category 2R (2):
R(2)＝E(2)+P(b)
according to the method, the accuracy of each classification result can be obtained, the greater the accuracy R value is, the better the classification effect is, and some existing evaluation indexes such as the accuracy AP, the Kappa coefficient, the F1 value and the like are all based on the evaluation of the effect of the whole classification result by the confusion matrix, and the evaluation indexes are difficult to give the evaluation of the classification result of a single category, which is not enough to meet the requirement of a user in some practical applications, for example, in the MNIST handwritten character body recognition task, the probability and the importance of the occurrence of the numeral 0 are often greater and higher than those of other numerals. According to the method, the evaluation of the classification result effect of each category can be given, and the method not only can be used for model selection, but also has certain significance for better guiding the training process.
And step S103, calculating the accuracy of the whole classification result according to the accuracy of each classification result.
The accuracy R value of the overall classification result is defined as follows:
R＝∑R(i)/N
where R (i) represents the accuracy of the classification result for category i and N is the total number of categories.
Taking the two-classification task as an example, the accuracy of the overall classification result is as follows:
according to the method, the accuracy of the overall classification result can be obtained, and the overall classification effect can be evaluated.
Further, before obtaining the confusion matrix of the classification result, the method further includes:
and constructing a confusion matrix of the classification result according to the prediction label and the real label of the test sample.
Specifically, before obtaining the confusion matrix of the classification result, firstly, the confusion matrix of the classification result is constructed according to the prediction label and the real label of the test sample. And testing the test sample by the classifier to obtain a prediction label of the test sample, and further combining a real label of the test sample to construct a confusion matrix.
Taking a binary task as an example, a confusion matrix is constructed, and fig. 3 is a schematic diagram illustrating a confusion matrix according to an exemplary embodiment, wherein each column of the confusion matrix represents a prediction class, the total number of each column represents the number of data predicted as the class, each row represents a true belonging class of data, the total number of data in each row represents the number of data instances of the class, and the value in each column represents the number of true data predicted as the class. As shown in fig. 3, the confusion matrix of the classification result includes class 1 and class 2, where the number of true labels of class 1 is m, the number of samples predicted as class 1 in the prediction result is s, the number of samples predicted as class 1 correctly is a, and c samples in class 1 are incorrectly predicted as class 2; the number of the real labels of the class 2 is n, the number of the samples predicted to be the class 2 in the prediction result is t, the number of the samples predicted to be the class 2 correctly is b, d samples in the class 2 are predicted to be the class 1 incorrectly, and the total number of the samples is w.
Further, before calculating the accuracy of each classification result according to the confusion matrix, the method further comprises:
and calculating the classification efficiency of each class classification result according to the confusion matrix.
Before the accuracy of the classification result of each category is calculated according to the confusion matrix, the classification efficiency of each category is calculated according to the acquired confusion matrix.
Taking class 1 as an example, the number of the real labels of class 1 is m, the number of samples predicted to be class 1 in the prediction result is s, the number of correctly predicted samples is a, and the total number of samples is w. The classification efficiency E for this class is defined as follows: the difference between the probability of correctly classifying the class and the probability of the sample being predicted as this class is:
E(1)＝P(c)-P(y)
wherein, p (c) represents the probability that the category is correctly classified, and the calculation method is as follows: the ratio of the number of correctly classified samples to the total number of samples of that class is:
p (y) represents the probability that the sample is predicted to be of the class, i.e.:
taking class 2 as an example, the number of the real labels of class 2 is n, the number of samples predicted as class 2 in the prediction result is t, the number of correctly predicted samples is b, and the total number of samples is w. The classification efficiency E for this class is defined as follows: the difference between the probability of correctly classifying the class and the probability of the sample being predicted as this class is:
E(2)＝P(x)-P(z)
wherein, p (x) represents the probability that the category is correctly classified, and the calculation method is as follows: the ratio of the number of correctly classified samples to the total number of samples of that class is:
p (z) represents the probability that the sample is predicted to be of that class, i.e.:
further, calculating the accuracy of each class classification result according to the confusion matrix, comprising:
and calculating the accuracy of each class classification result according to the classification efficiency of each class classification result and the probability of the class appearing in the total sample.
And after the classification efficiency of each category is obtained through calculation, calculating the accuracy of the classification result of each category according to the classification efficiency of each category.
On the basis of obtaining the classification efficiency of the class 1, combining the probability P (a) of the class 1 appearing in the total sample, namely:
calculate accuracy R (1) for category 1:
R(1)＝E(1)+P(a)
on the basis of obtaining the classification efficiency of the class 2, combining the probability P (b) of the class 2 appearing in the total sample, namely:
calculate accuracy for category 2R (2):
R(2)＝E(2)+P(b)
further, calculating the accuracy of the overall classification result according to the accuracy of each classification result, comprising:
according to the formula: calculating the accuracy of the overall classification result;
wherein, R is the accuracy of the overall classification result, R (i) is the accuracy of the classification result of the ith class, and N is the total number of classes.
Taking the two-classification task as an example, the accuracy of the overall classification result is as follows:
according to the method, the accuracy of the overall classification result can be obtained, and the overall classification effect can be evaluated.
Further, after calculating the accuracy of the overall classification result according to the accuracy of each classification result, the method further comprises:
and determining a classification model according to the accuracy of each classification result and the accuracy of the overall classification result.
Further, determining a classification model according to the accuracy of each classification result and the accuracy of the overall classification result, comprising: and determining the model with higher accuracy of the whole classification result or the classification result of a certain class as the classification model.
The method can be used for classification model selection and has certain significance for better guiding the training process.
FIG. 2 is a flow diagram illustrating a confusion matrix-based classification result detection method according to an exemplary embodiment;
as shown in fig. 2, a method for detecting a classification result based on a confusion matrix includes:
step S201, constructing a confusion matrix of the classification result according to the prediction label and the real label of the test sample;
step S202, obtaining a confusion matrix of the classification result;
step S203, calculating the classification efficiency of each class classification result according to the confusion matrix;
step S204, calculating the accuracy of each class classification result according to the classification efficiency of each class classification result and the probability of the class appearing in the total sample;
step S205, calculating the accuracy of the whole classification result according to the accuracy of each classification result;
and S206, determining a classification model according to the accuracy of each classification result and the accuracy of the whole classification result.
By the method, the accuracy of each class classification result and the accuracy of the whole classification result are obtained based on the confusion matrix, and the accuracy is used as an evaluation index of the classification result to select and optimize the model parameters so as to guide the training process and reduce the training cost.
In some exemplary scenarios, a reverse brushing single application scenario of a wind control system in the e-commerce field is taken as an example, and the confusion matrix-based classification result detection method according to the embodiment of the invention is introduced. The test sample data set is input as a predetermined total order set U, a positive case set obtained by classifying the total order set is a set of order-swiped orders, a corresponding positive case is an order-swiped order (namely, an order with a single row exists), a negative case set obtained by classifying is a set of normal orders, and a corresponding negative case is a normal order (namely, an order without an order-swiped behavior).
Constructing a confusion matrix according to the positive case set and the negative case set predicted by the classifier and the real positive case set and the real negative case set of the test sample data set;
calculating the classification efficiency E (1) of the positive example set according to the constructed confusion matrix;
calculating the classification efficiency E (2) of the negative example set according to the constructed confusion matrix;
calculating the accuracy R (1) of the regular example set according to the classification efficiency E (1) of the regular example set;
calculating the accuracy R (2) of the negative example set according to the classification efficiency E (2) of the negative example set;
and calculating the accuracy R of the overall classification result according to the accuracy R (1) of the positive example set and the accuracy R (2) of the negative example set.
And detecting the classification result according to the accuracy of the classified positive case set, the accuracy of the classified negative case set and the accuracy of the overall classification result, evaluating the overall classification effect and the classification effect of each category, and selecting a classification model.
In a second aspect, an embodiment of the present disclosure provides a confusion matrix-based classification result detection apparatus, fig. 4 is a schematic structural diagram of a confusion matrix-based classification result detection apparatus according to an exemplary embodiment, and as shown in fig. 4, a confusion matrix-based classification result detection apparatus includes:
s401, an obtaining module for obtaining a confusion matrix of the classification result;
s402, a first calculating module used for calculating the accuracy of each category classification result according to the confusion matrix;
s403, a second calculating module, configured to calculate accuracy of the overall classification result according to the accuracy of each classification result;
in a third aspect, the present disclosure provides a confusion matrix-based classification result detection system, which includes a processor and a memory storing program instructions, where the processor is configured to execute the confusion matrix-based classification result detection method provided in the foregoing embodiments when executing the program instructions.
Fig. 5 is a schematic structural diagram illustrating a confusion matrix-based classification result detection system according to an exemplary embodiment. In some embodiments, a confusion matrix-based classification result detection system includes a processor 51 and a memory 52 storing program instructions, and may further include a communication interface 53 and a bus 54. The processor 51, the communication interface 53 and the memory 52 may communicate with each other through the bus 54. The communication interface 53 may be used for information transfer. The processor 51 may call logic instructions in the memory 52 to perform the confusion matrix-based classification result detection method provided by the above-described embodiments.
The embodiment of the present disclosure provides a computer-readable medium, on which computer-readable instructions are stored, where the computer-readable instructions can be executed by a processor to implement the confusion matrix-based classification result detection method provided by the above-mentioned embodiment.
It should be noted that:
the algorithms and displays presented herein are not inherently related to any particular computer, virtual machine, or other apparatus. Various general purpose devices may be used with the teachings herein. The required structure for constructing such a device will be apparent from the description above. Moreover, the present invention is not directed to any particular programming language. It is appreciated that a variety of programming languages may be used to implement the teachings of the present invention as described herein, and any descriptions of specific languages are provided above to disclose the best mode of the invention.
Those skilled in the art will appreciate that the steps in the devices of the embodiments may be adaptively changed and disposed in one or more devices other than the embodiments. Steps or components in the embodiments may be combined into one step or component, and further, may be divided into a plurality of steps or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or steps of any system or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or steps are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.
Claims (10)
1. A method for detecting a classification result based on a confusion matrix is characterized by comprising the following steps:
obtaining a confusion matrix of the classification result;
calculating the accuracy of each category classification result according to the confusion matrix;
and calculating the accuracy of the whole classification result according to the accuracy of each classification result.
2. The method of claim 1, wherein before obtaining the confusion matrix of the classification result, further comprising:
and constructing a confusion matrix of the classification result according to the prediction label and the real label of the test sample.
3. The method of claim 1, wherein before calculating the accuracy of each class classification result according to the confusion matrix, further comprising:
and calculating the classification efficiency of each class classification result according to the confusion matrix.
4. The method of claim 3, wherein calculating the accuracy of each class classification result from the confusion matrix comprises:
and calculating the accuracy of each class classification result according to the classification efficiency of each class classification result and the probability of the class appearing in the total sample.
5. The method of claim 1, wherein said calculating the accuracy of the overall classification result from the accuracy of said each class classification result comprises:
according to the formula: calculating the accuracy of the overall classification result;
wherein, R is the accuracy of the overall classification result, R (i) is the accuracy of the classification result of the ith class, and N is the total number of classes.
6. The method of claim 1, wherein after calculating the accuracy of the overall classification result according to the accuracy of each classification result, further comprising:
and determining a classification model according to the accuracy of each classification result and the accuracy of the whole classification result.
7. The method of claim 6, wherein determining a classification model based on the accuracy of the classification result for each category and the accuracy of the overall classification result comprises:
and determining the model with higher accuracy of the whole classification result or the classification result of a certain class as the classification model.
8. A confusion matrix-based classification result detection apparatus, comprising:
the obtaining module is used for obtaining a confusion matrix of the classification result;
the first calculation module is used for calculating the accuracy of each category classification result according to the confusion matrix;
and the second calculation module is used for calculating the accuracy of the whole classification result according to the accuracy of each classification result.
9. A confusion matrix based classification result detection system comprising a processor and a memory storing program instructions, wherein the processor is configured to execute the confusion matrix based classification result detection method of any of claims 1-7 when executing the program instructions.
10. A computer readable medium having computer readable instructions stored thereon which are executable by a processor to implement a confusion matrix-based classification result detection method as claimed in any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010095296.5A CN111310829A (en) | 2020-02-14 | 2020-02-14 | Confusion matrix-based classification result detection method and device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010095296.5A CN111310829A (en) | 2020-02-14 | 2020-02-14 | Confusion matrix-based classification result detection method and device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111310829A true CN111310829A (en) | 2020-06-19 |
Family
ID=71147066
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010095296.5A Pending CN111310829A (en) | 2020-02-14 | 2020-02-14 | Confusion matrix-based classification result detection method and device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111310829A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111866004A (en) * | 2020-07-27 | 2020-10-30 | 中国工商银行股份有限公司 | Security assessment method, apparatus, computer system, and medium |
| CN113721182A (en) * | 2021-11-02 | 2021-11-30 | 武汉格蓝若智能技术有限公司 | Method and system for evaluating reliability of online performance monitoring result of power transformer |
| CN116204670A (en) * | 2023-04-27 | 2023-06-02 | 菲特（天津）检测技术有限公司 | Management method and system of vehicle target detection data and electronic equipment |
| CN117235270A (en) * | 2023-11-16 | 2023-12-15 | 中国人民解放军国防科技大学 | Text classification method and device based on belief confusion matrix and computer equipment |
-
2020
- 2020-02-14 CN CN202010095296.5A patent/CN111310829A/en active Pending
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111866004A (en) * | 2020-07-27 | 2020-10-30 | 中国工商银行股份有限公司 | Security assessment method, apparatus, computer system, and medium |
| CN111866004B (en) * | 2020-07-27 | 2022-09-27 | 中国工商银行股份有限公司 | Security assessment method, apparatus, computer system, and medium |
| CN113721182A (en) * | 2021-11-02 | 2021-11-30 | 武汉格蓝若智能技术有限公司 | Method and system for evaluating reliability of online performance monitoring result of power transformer |
| CN113721182B (en) * | 2021-11-02 | 2022-02-01 | 武汉格蓝若智能技术有限公司 | Method and system for evaluating reliability of online performance monitoring result of power transformer |
| CN116204670A (en) * | 2023-04-27 | 2023-06-02 | 菲特（天津）检测技术有限公司 | Management method and system of vehicle target detection data and electronic equipment |
| CN117235270A (en) * | 2023-11-16 | 2023-12-15 | 中国人民解放军国防科技大学 | Text classification method and device based on belief confusion matrix and computer equipment |
| CN117235270B (en) * | 2023-11-16 | 2024-02-02 | 中国人民解放军国防科技大学 | Text classification method and device based on belief confusion matrix and computer equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111310829A (en) | Confusion matrix-based classification result detection method and device and storage medium | |
| US11488055B2 (en) | Training corpus refinement and incremental updating | |
| CN108491406B (en) | Information classification method and device, computer equipment and storage medium | |
| CN110110726A (en) | Power equipment nameplate identification method and device, computer equipment and storage medium | |
| CN1421013A (en) | Property coverage in formal verification | |
| CN110991474A (en) | Machine learning modeling platform | |
| KR20210062687A (en) | Image classification model training method, image processing method and apparatus | |
| CN110633989A (en) | Method and device for determining risk behavior generation model | |
| US20200364216A1 (en) | Method, apparatus and storage medium for updating model parameter | |
| CN109685104B (en) | Determination method and device for recognition model | |
| CN109615080B (en) | Unsupervised model evaluation method and device, server and readable storage medium | |
| CN111125658A (en) | Method, device, server and storage medium for identifying fraudulent users | |
| CN110705622A (en) | Decision-making method and system and electronic equipment | |
| CN113780365A (en) | Sample generation method and device | |
| EP4227855A1 (en) | Graph explainable artificial intelligence correlation | |
| CN111738290A (en) | Image detection method, model construction and training method, device, equipment and medium | |
| CN111709475A (en) | Multi-label classification method and device based on N-grams | |
| CN116245630A (en) | Anti-fraud detection method and device, electronic equipment and medium | |
| CN113836297B (en) | Training method and device for text emotion analysis model | |
| CN113705201B (en) | Text-based event probability prediction evaluation algorithm, electronic device and storage medium | |
| CN113010687B (en) | Exercise label prediction method and device, storage medium and computer equipment | |
| CN112989040A (en) | Dialog text labeling method and device, electronic equipment and storage medium | |
| CN114428720A (en) | Software defect prediction method and device based on P-K, electronic equipment and medium | |
| CN111612023A (en) | Classification model construction method and device | |
| CN109063702A (en) | Licence plate recognition method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200619 |
|
| RJ01 | Rejection of invention patent application after publication |