CN115114966B - Method, device and equipment for determining operation strategy of model and storage medium - Google Patents
Method, device and equipment for determining operation strategy of model and storage medium Download PDFInfo
- Publication number
- CN115114966B CN115114966B CN202211037370.3A CN202211037370A CN115114966B CN 115114966 B CN115114966 B CN 115114966B CN 202211037370 A CN202211037370 A CN 202211037370A CN 115114966 B CN115114966 B CN 115114966B
- Authority
- CN
- China
- Prior art keywords
- precision
- image data
- data processing
- processing model
- ratio
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/18—Complex mathematical operations for evaluating statistical data, e.g. average values, frequency distributions, probability functions, regression analysis
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Optimization (AREA)
- Pure & Applied Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Mathematical Physics (AREA)
- Computational Mathematics (AREA)
- Mathematical Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Operations Research (AREA)
- Probability & Statistics with Applications (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Algebra (AREA)
- Evolutionary Biology (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Computational Biology (AREA)
- Image Analysis (AREA)
Abstract
The application relates to a method, a device, equipment and a storage medium for determining an operation strategy of a model, in particular to the technical field of artificial intelligence. The method comprises the following steps: obtaining a PR curve of the image data processing model, wherein the PR curve is formed according to value taking points formed by precision ratio and recall ratio of the image data processing model under different prediction times; in the process of traversing the precision ratio list corresponding to the PR curve from back to front, recording subscripts corresponding to the precision ratios under the condition that the water level value is updated in the precision ratio list, wherein the updated water level value is the maximum precision ratio value in the current traversing process, and the subscripts represent the prediction times corresponding to the precision ratios; calculating the average precision of the image data processing model according to the precision index and the recall index corresponding to the recorded subscript; based on the average accuracy, a subsequent operating strategy for the image data processing model is determined.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to a method, a device, equipment and a storage medium for determining an operation strategy of a model.
Background
In the field of artificial intelligence technology, average Precision (AP) may be used as an index for measuring performance of a model, and after the Average Precision is calculated, subsequent operation strategies for the model are determined, such as: applying the model, continuing to train the model, and abandoning the model.
Generally, two indexes of Precision (Precision) and Recall (Recall) of a model during training are mapped on a two-dimensional coordinate axis, and a PR curve is formed by a set of coordinate points located by the Precision and the Recall.
In the related art, the average accuracy is calculated by two traversal processes of the PR curve: traversing the precision ratio list corresponding to the PR curve from back to front, comparing two adjacent elements each time, if the back element is larger than the front element, assigning the value of the back element to the front element, otherwise, continuously traversing forwards, thereby smoothing the PR curve; and traversing the recall ratio list corresponding to the smoothed PR curve again, finding subscripts of two adjacent unequal elements in all the recall ratios, and calculating the area of a polygon formed by the smoothed PR curve and a coordinate axis by using the information to serve as a numerical value of average precision.
Based on the technical scheme, the calculation efficiency of the average precision is not high, so that the efficiency of determining the operation strategy of the model based on the average precision is influenced.
Disclosure of Invention
The application provides a method, a device, equipment and a storage medium for determining an operation strategy of a model, which can improve the efficiency of determining the operation strategy of the model by improving the efficiency of average precision in calculation. The technical scheme is as follows.
In one aspect, a method for determining an operation strategy of a model is provided, and the method includes:
the method comprises the steps of obtaining a PR curve of an image data processing model, wherein the PR curve is a curve formed by using precision of the image data processing model as a vertical coordinate, using recall of the image data processing model as a horizontal coordinate and according to a value taking point formed by the precision and the recall of the image data processing model under different prediction times;
in the process of traversing the precision ratio list corresponding to the PR curve from back to front, recording subscripts corresponding to the precision ratios of the water level value updating condition in the precision ratio list, wherein the updated water level value is the maximum precision ratio value in the current traversing process, and the subscripts represent the prediction times corresponding to the precision ratios;
calculating the average precision of the image data processing model according to the precision index and the recall index corresponding to the recorded subscript;
and determining a subsequent operation strategy of the image data processing model based on the average precision.
In yet another aspect, an apparatus for determining an operation strategy of a model is provided, the apparatus comprising:
the PR curve acquisition module is used for acquiring a PR curve of an image data processing model, wherein the PR curve is a curve formed by using the precision of the image data processing model as a vertical coordinate, using the recall of the image data processing model as a horizontal coordinate and according to a value taking point formed by the precision and the recall of the image data processing model under different prediction times;
the subscript recording module is used for recording subscripts corresponding to precision of the condition that the water level value is updated in the precision list in the process of traversing the precision list corresponding to the PR curve from back to front, wherein the updated water level value is the maximum precision value in the current traversing process, and the subscripts represent the prediction times corresponding to the precision;
the average precision calculation module is used for calculating the average precision of the image data processing model according to the precision ratio and the recall ratio corresponding to the recorded subscript;
and the operation strategy determination module is used for determining a subsequent operation strategy of the image data processing model based on the average precision.
In one possible implementation, the subscript recording module is configured to:
setting an initial value of the water level value to 0;
and traversing the precision ratio list corresponding to the PR curve from back to front, updating the water level value to be the target precision ratio under the condition that the current target precision ratio in the precision ratio list is greater than the current water level value, and recording the subscript corresponding to the target precision ratio.
In one possible implementation, the subscript recording module is configured to:
and under the condition that the current target precision in the precision ratio list is not greater than the current water level value, keeping the water level value unchanged, and continuously traversing the precision ratio list corresponding to the PR curve forwards.
In a possible implementation manner, the average precision calculation module is configured to:
obtaining a recall ratio difference value, wherein the recall ratio difference value is the difference value between recall ratios corresponding to two adjacent recorded subscripts;
multiplying the recall ratio difference value by a prior precision ratio to obtain a product value, wherein the prior precision ratio is the precision ratio corresponding to the prior recorded subscript in the two adjacent recorded subscripts;
in one possible implementation, the calculation formula of the average accuracy of the image data processing model is as follows:
wherein AP is the average accuracy, P _ ch is the precision corresponding to the recorded subscript, R _ ch is the recall corresponding to the recorded subscript, and L is the number of all recorded subscripts.
In yet another aspect, a computer device is provided, which comprises a processor and a memory, wherein the memory stores at least one instruction, at least one program, a set of codes, or a set of instructions, which is loaded and executed by the processor to implement the method for determining an operation strategy of the model.
In yet another aspect, a computer-readable storage medium is provided, having at least one instruction stored therein, the at least one instruction being loaded and executed by a processor to implement the method for determining an operating strategy of a model as described above.
In yet another aspect, a computer program product is provided, as well as a computer program product or a computer program comprising computer instructions stored in a computer readable storage medium. The computer instructions are read by a processor of the computer device from a computer-readable storage medium, and the computer instructions are executed by the processor to cause the computer device to perform the method for determining the operation strategy of the model.
The technical scheme provided by the application can comprise the following beneficial effects:
in the process of traversing the precision ratio list corresponding to the PR curve from back to front, the maximum precision ratio value in the traversing process is recorded by utilizing the water level value, the corresponding subscript is recorded, and the average precision corresponding to the image data processing model is calculated by using the precision ratio and the recall ratio corresponding to the recorded subscript subsequently. Since the water level value represents the maximum precision value in the traversal process, the smoothing of the PR curve and the recording of the subscript value required for the average precision calculation based on the smoothed PR curve can be completed by determining the update condition of the water level value in one traversal process of the precision table. Compared with the scheme that two traversal processes need to be executed in the related technology, the traversal times are reduced, so that the efficiency of the average precision in calculation is improved, and the efficiency of the operation strategy based on the average precision determination model is improved.
Drawings
In order to more clearly illustrate the detailed description of the present application or the technical solutions in the prior art, the drawings needed to be used in the detailed description of the present application or the prior art description will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a schematic diagram illustrating a PR curve according to an exemplary embodiment.
FIG. 2 is a schematic diagram illustrating a smoothed PR curve according to an exemplary embodiment.
FIG. 3 is a method flow diagram illustrating a method of determining an operating strategy for a model in accordance with an exemplary embodiment.
FIG. 4 is a method flow diagram illustrating a method of determining an operating strategy for a model in accordance with an exemplary embodiment.
FIG. 5 is a schematic diagram illustrating a calculation flow of average accuracy according to an exemplary embodiment.
Fig. 6 is a block diagram illustrating an architecture of a device for determining an operational strategy of a model according to an exemplary embodiment.
FIG. 7 is a schematic diagram of a computer device provided in accordance with an exemplary embodiment of the present application.
Detailed Description
The technical solutions of the present application will be described clearly and completely with reference to the accompanying drawings, and it should be understood that the described embodiments are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It should be understood that "indication" mentioned in the embodiments of the present application may be a direct indication, an indirect indication, or an indication of an association relationship. For example, a indicates B, which may mean that a directly indicates B, e.g., B may be obtained by a; it may also mean that a indicates B indirectly, e.g. a indicates C, by which B may be obtained; it can also mean that there is an association between a and B.
In the description of the embodiments of the present application, the term "correspond" may indicate that there is a direct correspondence or an indirect correspondence between the two, may also indicate that there is an association between the two, and may also indicate and be indicated, configure and configured, and so on.
In the embodiment of the present application, "predefining" may be implemented by saving a corresponding code, table, or other manners that may be used to indicate related information in advance in a device (for example, including a terminal device and a network device), and the present application is not limited to a specific implementation manner thereof.
First, a calculation method of an AP provided in the related art will be explained.
The average accuracy is an index used for measuring the performance of the detection model in the current mainstream. The average Precision can be obtained by mapping two indexes of Precision and Recall onto a two-dimensional coordinate axis, then calculating the area wrapped by a PR curve formed by a set of coordinate points positioned by Precision and Recall and the two coordinate axes, and taking the value of the area as the value of the average Precision.
Wherein Precision is defined as Precision, for example, the explanation is that the model predicts n times, wherein the correct prediction is m times, and then Precision is m/n.
Wherein, recall is defined as Recall, for example, the total number of the positive samples is n, m of the positive samples are accurately predicted by the model, and Recall is m/n.
It will be appreciated that, on the one hand, precision and Recall are the same length, and the model predicts n times, then Precision and Recall each produce n results; on the other hand, precision and Recall generally have an inverse relationship, and as Recall increases, precision tends to decrease, and vice versa.
To demonstrate the computational flow of the AP provided in the related art, the following hypothetical cases are exemplified: assuming that there are 7 positive samples in total, the model performs 10 predictions, and then the confidence scores are ranked from high to low to obtain the following results:
1, time: the model correctly predicts one positive sample, when Precision =1/1=1.0, recall =1/7=0.14.
And (2) time: the model correctly predicts one remaining positive sample, when Precision =2/2=1.0, recall =2/7=0.29.
And (3) time: model prediction error, when Precision =2/3=0.66 and recall =0.29.
4, time: the model predicts the error, when Precision =0.5 and recall =0.29.
And 5, time: the model predicts the error, when Precision =0.4 and recall =0.29.
The 6 th time: the model correctly predicted one remaining positive sample, when Precision =0.5 and recall =0.43.
And (7) time: the model predicts the error, when Precision =0.43 and recall =0.43.
And 8, time: the model predicts the error, when Precision =0.38 and recall =0.43.
And (9) time: the model correctly predicted one remaining positive sample, when Precision =0.44 and recall =0.57.
10 th time: the model correctly predicted one remaining positive sample, when Precision =0.5 and recall =0.71.
Based on the above data example, a PR curve is drawn with Precision as the y-axis and Recall as the x-axis, as shown in fig. 1.
The PR curve is smoothed before calculating AP, as follows:
wherein, r represents the preceding subscript,
one index after the previous index is indicated and P indicates the Precision value.
The specific PR curve smoothing algorithm process is to traverse the Precision list from back to front, compare two adjacent elements each time, if the back element is larger than the front element, assign the value of the back element to the front element, otherwise, continue the forward traversal. The PR curve smoothing algorithm intuitively looks to smooth out the "saw-teeth" and the smoothed PR curve is shown in fig. 2.
The AP value is the area of a polygon formed by the smoothed PR curve and two coordinate axes. The specific algorithm is as follows: traversing the Recall list once, finding the subscript positions of two adjacent unequal elements in all recalls, for example, according to the above example, when the subscript list of the adjacent elements of Recall is index = [1,2,6,9, 10], and finally, the AP calculates the difference by accumulating a group of elements obtained by Recall with respect to two adjacent subscripts in index, and then multiplies the difference by an element in index corresponding to Precision.
According to this example, the specific calculation method is:
AP = (0.14-0) * 1 + (0.29-0.14) * 1 + (0.43-0.29) * 0.5 + (0.57-0.43) * 0.5 + (0.71-0.57) * 0.5 = 0.5。
therefore, in the related art, the average accuracy is calculated through two traversal processes of the PR curve, and the calculation of the average accuracy is not efficient.
In order to solve the above problem, an embodiment of the present application provides a method for determining an operation strategy of a model, in a process of traversing a precision list corresponding to a PR curve from back to front, a maximum precision value in the traversing process is recorded by using a water level value, corresponding subscripts are recorded, and then, the average precision corresponding to an image data processing model is calculated by using the precision and the recall corresponding to the recorded subscripts. Since the water level value represents the maximum precision value in the traversal process, the smoothing of the PR curve and the recording of the subscript value required for the average precision calculation based on the smoothed PR curve can be completed by determining the update condition of the water level value in one traversal process of the precision list. Compared with the scheme that two traversal processes need to be executed in the related technology, the traversal times are reduced, so that the efficiency of the average precision in calculation is improved, and the efficiency of the operation strategy based on the average precision determination model is improved.
The determination method of the operation strategy of the model provided in the present application is further explained below.
FIG. 3 is a method flow diagram illustrating a method of determining an operating strategy for a model in accordance with an exemplary embodiment. The method is performed by a computer device. As shown in fig. 3, the method for determining the operation strategy of the model may include the following steps:
The image data processing model is a neural network model for processing image data. It is understood that the method for determining the operation strategy of the model shown in the present application can also be applied to other types of neural network models, and the application to the image data processing model is only exemplarily described herein.
After the image data processing model is built, the image data processing model is used for conducting multiple times of prediction on test data, and then a PR curve of the image data processing model is drawn according to precision ratio and recall ratio of the image data processing model under different prediction times.
Specifically, the PR curve is drawn based on a sampling point including the precision and the recall at different prediction times, with the precision of the image data processing model as the ordinate and the recall of the image data processing model as the abscissa.
Illustratively, the PR curve of the acquired image data processing model is similarly referred to fig. 1.
Step 302, in the process of traversing the precision list corresponding to the PR curve from back to front, recording subscripts corresponding to the precision under the condition that the water level value is updated in the precision list, wherein the updated water level value is the maximum precision value in the current traversal process, and the subscripts represent the prediction times corresponding to the precision.
After the PR curve of the image data processing model is obtained, a process of traversing from back to front is carried out on the precision ratio list corresponding to the PR curve. In the traversing process, if the maximum value of the precision ratio in the current traversing process is changed, the current water level value is correspondingly updated, the PR curve is smoothed in a water level value updating mode, and the subscript corresponding to the precision ratio under the condition that the water level value is updated is recorded.
The precision ratio list corresponding to the PR curve refers to a list formed by precision ratios in the PR curve.
Wherein the water level value is a parameter value introduced for determining a maximum precision value in a traversal process of the precision list corresponding to the PR curve. The water level value is updated along with the traversal process of the precision ratio list corresponding to the PR curve, and the updated water level value is the maximum precision ratio value in the current traversal process.
For example, according to the example described above, the precision list is [1.0, 0.66,0.5,0.4,0.5,0.43,0.38,0.44,0.5]. According to the definition of precision, the precision list is generally descending. In the process of traversing the precision list corresponding to the PR curve from back to front, if the maximum precision value is 0.5,0.66 and 1.0 in sequence, the water level value is updated to 0.5,0.66 and 1.0 in sequence, and the subscripts in the precision list corresponding to the precision are 10,3 and 2, then the subscript is recorded as [10,3,2] by the computer device.
And step 303, calculating the average precision of the image data processing model according to the precision ratio and the recall ratio corresponding to the recorded subscript.
Because the subscript represents the prediction times, the computer device can screen out a part of the prediction times based on the recorded subscript, and inquire the precision ratio and the recall ratio under the prediction times, the value-taking points formed by the precision ratio and the recall ratio can be regarded as smooth points corresponding to the original PR curve, a smooth PR curve can be formed based on the smooth points, and the computer device calculates the area enclosed by the smooth PR curve and the coordinate axis based on the smooth points, so that the average precision of the image data processing model is calculated.
For example, according to the above-described example, the precision ratios [0.5,0.66,1.0] and the recall ratios [0.71, 0.29, 0.29] corresponding to the index indices [10,3,2] are obtained from the record indices, and the average precision of the image data processing model is calculated from these data.
And step 304, determining a subsequent operation strategy for the image data processing model based on the average precision.
Wherein, the subsequent operation strategy of the image data processing model refers to: and after the image data processing model is established, carrying out subsequent processing on the image data processing model. The subsequent operation strategy of the image data processing model comprises the following steps: applying the image data processing model directly; or, continuing to train the image data processing model; alternatively, the image data processing model is discarded.
Illustratively, at least one threshold value is prestored in the computer device, and the calculated average precision is compared with the threshold value, so as to determine the subsequent operation strategy of the image data processing model. Such as: in case the average accuracy is higher than a first threshold, then the image data processing model is directly applied; under the condition that the average precision is lower than a first threshold and higher than a second threshold, continuing to train the image data processing model; in case the average accuracy is below a second threshold, the image data processing model is discarded.
In summary, in the method for determining the operation strategy of the model provided in this embodiment, in the process of traversing the precision list corresponding to the PR curve from back to front, the maximum precision value in the traversal process is recorded by using the water level value, the corresponding subscript is recorded, and the average precision corresponding to the image data processing model is calculated by using the precision and the recall corresponding to the recorded subscript subsequently. Since the water level value represents the maximum precision value in the traversal process, the smoothing of the PR curve and the recording of the subscript value required for the average precision calculation based on the smoothed PR curve can be completed by determining the update condition of the water level value in one traversal process of the precision list. Compared with the scheme that two traversal processes need to be executed in the related technology, the traversal times are reduced, so that the efficiency of the average precision in calculation is improved, and the efficiency of the operation strategy based on the average precision determination model is improved.
In an exemplary embodiment, during the backward-forward traversal of the precision ratio list corresponding to the PR curve, the water level value is updated by comparing the current water level value with the currently traversed precision ratio.
FIG. 4 is a method flow diagram illustrating a method of determining an operating strategy for a model in accordance with an exemplary embodiment. The method is performed by a computer device. As shown in fig. 4, the method for determining the operation strategy of the model may include the following steps:
The specific implementation manner of this step may refer to step 301, which is not described herein again.
That is, the current water level value is compared with the currently traversed target precision, if the current target precision is greater than the current water level value, the water level is raised, and the water level value is updated to be the target precision.
In an alternative implementation, in the case that the current target precision in the precision list is not greater than the current water level value, the water level value is kept unchanged, and the PR curve corresponding to the precision list is continuously traversed forward.
For example, according to the above-described example, the precision table is [1.0, 0.66,0.5,0.4,0.5,0.43,0.38,0.44,0.5], and the "water level" is raised to [0.5,0.66,1.0] from the rear to the front at the time of the subscript = [10,3,2], respectively.
And step 404, obtaining a recall ratio difference value, wherein the recall ratio difference value is a difference value between recall ratios corresponding to two adjacent recorded subscripts.
For example, according to the above example, the recorded subscripts correspond to recall ratios of [0.71, 0.29, 0.29], for convenience of calculation, 0 is inserted at the end, and the recall ratios are [0.71, 0.29,0], then the precision difference values include: (0.29-0), (0.29-0.29), (0.71-0.29).
For example, according to the example described above, the precision difference value includes: (0.29-0), (0.29-0.29), (0.71-0.29), the product value includes: (0.29-0) × 1.0, (0.29-0.29) × 0.66, (0.71-0.29) × 0.5.
And 406, accumulating the product values to obtain the average precision of the image data processing model.
For example, according to the example described above, the average accuracy of the image data processing model = (0.29-0) × 1.0+ (0.29-0.29) × 0.66+ (0.71-0.29) × 0.5.
In an alternative implementation, the average accuracy of the image data processing model is calculated as follows:
wherein, AP is the average precision, P _ ch is the precision corresponding to the recorded subscript, R _ ch is the recall corresponding to the recorded subscript, and L is the number of all recorded subscripts.
The specific implementation manner of this step may refer to step 304, which is not described herein again.
In summary, in the method for determining the operation strategy of the model provided in this embodiment, in the process of traversing the precision list corresponding to the PR curve from back to front, the maximum precision value in the traversal process is recorded by using the water level value, the corresponding subscript is recorded, and the average precision corresponding to the image data processing model is calculated by using the precision and the recall corresponding to the recorded subscript subsequently. Since the water level value represents the maximum precision value in the traversal process, the smoothing of the PR curve and the recording of the subscript value required for the average precision calculation based on the smoothed PR curve can be completed by determining the update condition of the water level value in one traversal process of the precision list. Compared with the scheme that two traversal processes need to be executed in the related technology, the traversal times are reduced, so that the efficiency of the average precision in calculation is improved, and the efficiency of the operation strategy based on the average precision determination model is improved.
Meanwhile, the method for determining the operation strategy of the model provided by the embodiment can calculate the average precision by applying the precision ratio and the recall ratio corresponding to the recorded subscript and substituting into a formula, thereby simplifying the code implementation of the algorithm.
Please refer to fig. 5, which is a flowchart illustrating a calculation process of the average accuracy according to an exemplary embodiment. As shown in fig. 5, the overall steps of the calculation flow of the average accuracy are as follows.
If the precision list has not been traversed, go to step 504; if the precision list has been traversed, go to step 506.
If the current precision value is greater than the water level value, go to step 505; if the current precision value is not greater than the water level value, go to step 503.
And 505, changing the water level value into the current precision value, and simultaneously recording the current subscript.
Wherein, AP is the average precision, P _ ch is the precision corresponding to the recorded subscript, R _ ch is the recall corresponding to the recorded subscript, and L is the number of all recorded subscripts.
It should be noted that the above method embodiments may be implemented alone or in combination, and the present application is not limited thereto.
Fig. 6 is a block diagram illustrating an architecture of a device for determining an operational strategy of a model according to an exemplary embodiment. The device comprises:
a PR curve obtaining module 601, configured to obtain a PR curve of an image data processing model, where the PR curve is a curve formed by using precision of the image data processing model as a vertical coordinate and using recall of the image data processing model as a horizontal coordinate, and according to a value-taking point formed by precision and recall of the image data processing model at different prediction times;
an index recording module 602, configured to record, in a process of traversing a precision ratio list corresponding to the PR curve from back to front, an index corresponding to a precision ratio at which a water level value is updated in the precision ratio list, where the updated water level value is a maximum precision ratio value in a current traversal process, and the index represents a prediction number corresponding to the precision ratio;
an average precision calculation module 603, configured to calculate an average precision of the image data processing model according to the precision ratio and the recall ratio corresponding to the recorded subscript;
an operation strategy determination module 604 for determining a subsequent operation strategy for the image data processing model based on the average accuracy.
In one possible implementation manner, the subscript recording module 602 is configured to:
setting an initial value of the water level value to 0;
and traversing the precision ratio list corresponding to the PR curve from back to front, updating the water level value to be the target precision ratio under the condition that the current target precision ratio in the precision ratio list is greater than the current water level value, and recording the subscript corresponding to the target precision ratio.
In one possible implementation manner, the subscript recording module 602 is configured to:
and under the condition that the current target precision in the precision ratio list is not larger than the current water level value, keeping the water level value unchanged, and continuously traversing the precision ratio list corresponding to the PR curve forwards.
In a possible implementation manner, the average precision calculation module 603 is configured to:
obtaining a recall ratio difference value, wherein the recall ratio difference value is the difference value between recall ratios corresponding to two recorded subscripts;
multiplying the recall ratio difference value by a prior precision ratio to obtain a product value, wherein the prior precision ratio is the precision ratio corresponding to the first recorded subscript in the two adjacent recorded subscripts;
in one possible implementation, the calculation formula of the average accuracy of the image data processing model is as follows:
wherein AP is the average accuracy, P _ ch is the precision corresponding to the recorded subscript, R _ ch is the recall corresponding to the recorded subscript, and L is the number of all recorded subscripts.
It should be noted that: the determining apparatus for determining the operation policy of the model provided in the foregoing embodiment is only illustrated by the division of the functional modules, and in practical applications, the function allocation may be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided by the above embodiments belong to the same concept, and specific implementation processes thereof are described in the method embodiments for details, which are not described herein again.
Please refer to fig. 7, which is a schematic diagram of a computer device according to an exemplary embodiment of the present application, the computer device includes a memory and a processor, the memory is used for storing a computer program, and the computer program, when executed by the processor, implements the method for determining the operation policy of the model.
The processor may be a Central Processing Unit (CPU). The Processor may also be other general purpose processors, digital Signal Processors (DSPs), application Specific Integrated Circuits (ASICs), field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, or a combination thereof.
The memory, which is a non-transitory computer readable storage medium, may be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules corresponding to the methods of the embodiments of the present invention. The processor executes various functional applications and data processing of the processor by executing non-transitory software programs, instructions and modules stored in the memory, that is, the method in the above method embodiment is realized.
The memory may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created by the processor, and the like. Further, the memory may include high speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory optionally includes memory located remotely from the processor, and such remote memory may be coupled to the processor via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
In an exemplary embodiment, a computer-readable storage medium is also provided for storing at least one computer program, which is loaded and executed by a processor to implement all or part of the steps of the above method. For example, the computer-readable storage medium may be a Read-Only Memory (ROM), a Random Access Memory (RAM), a Compact Disc Read-Only Memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, and the like.
Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.
It will be understood that the present application is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.
Claims (8)
1. A method for determining an operating strategy of a model, the method comprising:
the method comprises the steps of obtaining a PR curve of an image data processing model, wherein the PR curve is a curve formed by using precision of the image data processing model as a vertical coordinate, using recall of the image data processing model as a horizontal coordinate and according to a value taking point formed by the precision and the recall of the image data processing model under different prediction times;
in the process of traversing the precision ratio list corresponding to the PR curve from back to front, recording subscripts corresponding to the precision ratios of the water level value updating condition in the precision ratio list, wherein the updated water level value is the maximum precision ratio value in the current traversing process, and the subscripts represent the prediction times corresponding to the precision ratios;
calculating the average precision of the image data processing model according to the precision index and the recall index corresponding to the recorded subscript;
determining a subsequent operating strategy for the image data processing model based on the average precision;
wherein the determining a subsequent operation strategy for the image data processing model based on the average accuracy comprises:
in the event that the average accuracy is above a first threshold, then applying the image data processing model; under the condition that the average precision is lower than a first threshold and higher than a second threshold, continuing to train the image data processing model; in case the average accuracy is below a second threshold, the image data processing model is discarded.
2. The method of claim 1, wherein the recording the index corresponding to the precision at which the water level value is updated in the precision list during the backward-forward traversal of the precision list corresponding to the PR curve comprises:
setting an initial value of the water level value to 0;
and traversing the precision ratio list corresponding to the PR curve from back to front, updating the water level value to be the target precision ratio under the condition that the current target precision ratio in the precision ratio list is greater than the current water level value, and recording the subscript corresponding to the target precision ratio.
3. The method of claim 2, further comprising:
and under the condition that the current target precision in the precision ratio list is not larger than the current water level value, keeping the water level value unchanged, and continuously traversing the precision ratio list corresponding to the PR curve forwards.
4. The method of claim 1, wherein calculating the average accuracy of the image data processing model from the precision ratio and the recall ratio corresponding to the recorded index comprises:
obtaining a recall ratio difference value, wherein the recall ratio difference value is the difference value between recall ratios corresponding to two adjacent recorded subscripts;
multiplying the recall ratio difference value by a prior precision ratio to obtain a product value, wherein the prior precision ratio is the precision ratio corresponding to the first recorded subscript in the two adjacent recorded subscripts;
and accumulating the product values to obtain the average precision of the image data processing model.
5. The method of claim 4, wherein the average accuracy of the image data processing model is calculated as follows:
wherein AP is the average accuracy, P _ ch is the precision corresponding to the recorded subscript, R _ ch is the recall corresponding to the recorded subscript, L is the number of all subscripts recorded, and i is the subscript.
6. An apparatus for determining an operation strategy of a model, the apparatus comprising:
the PR curve acquisition module is used for acquiring a PR curve of the image data processing model, the PR curve is a curve which is formed by using the precision of the image data processing model as a vertical coordinate, using the recall ratio of the image data processing model as a horizontal coordinate and according to an evaluation point formed by the precision and the recall ratio of the image data processing model under different prediction times;
the subscript recording module is used for recording subscripts corresponding to precision of the condition that the water level value is updated in the precision list in the process of traversing the precision list corresponding to the PR curve from back to front, wherein the updated water level value is the maximum precision value in the current traversing process, and the subscripts represent the prediction times corresponding to the precision;
the average precision calculation module is used for calculating the average precision of the image data processing model according to the precision ratio and the recall ratio corresponding to the recorded subscripts;
an operation strategy determination module for determining a subsequent operation strategy for the image data processing model based on the average accuracy: in the event that the average accuracy is above a first threshold, then applying the image data processing model; under the condition that the average precision is lower than a first threshold and higher than a second threshold, continuing to train the image data processing model; in case the average accuracy is below a second threshold, the image data processing model is discarded.
7. A computer device comprising a processor and a memory, the memory having stored therein at least one instruction, at least one program, set of codes, or set of instructions, the at least one instruction, at least one program, set of codes, or set of instructions being loaded and executed by the processor to implement the method of determining an operating strategy of a model according to any one of claims 1 to 5.
8. A computer-readable storage medium, having stored therein at least one instruction, at least one program, a set of codes, or a set of instructions, which is loaded and executed by a processor to implement a method of determining an operating strategy of a model according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211037370.3A CN115114966B (en) | 2022-08-29 | 2022-08-29 | Method, device and equipment for determining operation strategy of model and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211037370.3A CN115114966B (en) | 2022-08-29 | 2022-08-29 | Method, device and equipment for determining operation strategy of model and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115114966A CN115114966A (en) | 2022-09-27 |
| CN115114966B true CN115114966B (en) | 2023-04-07 |
Family
ID=83335509
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211037370.3A Active CN115114966B (en) | 2022-08-29 | 2022-08-29 | Method, device and equipment for determining operation strategy of model and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115114966B (en) |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102176208B (en) * | 2011-02-28 | 2012-12-26 | 西安电子科技大学 | Robust video fingerprint method based on three-dimensional space-time characteristics |
| CN103530403B (en) * | 2013-10-23 | 2016-09-28 | 天津大学 | A kind of structurized Image Description Methods |
| CN104200240B (en) * | 2014-09-24 | 2017-07-07 | 梁爽 | A kind of Sketch Searching method based on content-adaptive Hash coding |
| CN107169518A (en) * | 2017-05-18 | 2017-09-15 | 北京京东金融科技控股有限公司 | Data classification method, device, electronic installation and computer-readable medium |
| CN110276754B (en) * | 2019-06-21 | 2021-08-20 | 厦门大学 | Surface defect detection method, terminal device and storage medium |
| US11947633B2 (en) * | 2020-11-30 | 2024-04-02 | Yahoo Assets Llc | Oversampling for imbalanced test data |
| CN113627229B (en) * | 2021-05-31 | 2023-08-25 | 中国兵器工业计算机应用技术研究所 | Target detection method, system, device and computer storage medium |
| CN114494943A (en) * | 2021-12-29 | 2022-05-13 | 以萨技术股份有限公司 | Novel video target detection and evaluation method, device, product and storage medium |
-
2022
- 2022-08-29 CN CN202211037370.3A patent/CN115114966B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN115114966A (en) | 2022-09-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110299136A (en) | A kind of processing method and its system for speech recognition | |
| CN109902641B (en) | Semantic alignment-based face key point detection method, system and device | |
| CN112949519B (en) | Target detection method, device, equipment and storage medium | |
| CN113033719B (en) | Target detection processing method, device, medium and electronic equipment | |
| CN117632905B (en) | Database management method and system based on cloud use records | |
| CN110121171A (en) | Trust prediction technique based on exponential smoothing and gray model | |
| CN111832693B (en) | Neural network layer operation and model training method, device and equipment | |
| CN115114966B (en) | Method, device and equipment for determining operation strategy of model and storage medium | |
| CN109711555B (en) | Method and system for predicting single-round iteration time of deep learning model | |
| CN115587545B (en) | Parameter optimization method, device and equipment for photoresist and storage medium | |
| CN115163266A (en) | Method, device, equipment and medium for determining ash load of particle catcher | |
| CN115550190A (en) | Method, device and equipment for determining network topological graph and storage medium | |
| CN112612784B (en) | River basin calculation unit automatic dividing method and device and computer equipment | |
| CN111368931B (en) | Method for determining learning rate of image classification model | |
| CN113971454A (en) | Deep learning model quantification method and related device | |
| CN112785000A (en) | Machine learning model training method and system for large-scale machine learning system | |
| CN112700270A (en) | Grading data processing method, device, equipment and storage medium | |
| CN115543490B (en) | Flash firmware starting method and system | |
| CN113765607B (en) | Frequency spectrum detection method and device | |
| CN112232068B (en) | Intention recognition method and device, electronic equipment and storage medium | |
| CN116409326A (en) | Lane line tracking method and lane line tracking system | |
| CN118690761A (en) | Parameter distribution method, device, equipment and medium of multilingual machine translation model | |
| CN118195945A (en) | Video frame processing method, device, equipment and storage medium | |
| CN115660053A (en) | Method, device, equipment and storage medium for determining CT value of PCR instrument | |
| CN116662660A (en) | Service recommendation method and device based on space-time hypergraph |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |