WO2023179133A1 - Target algorithm selection method and apparatus, and electronic device and storage medium - Google Patents

Abstract

The present invention relates to the technical field of the Internet, and particularly relates to a target algorithm selection method and apparatus, and an electronic device and a storage medium. The method comprises: acquiring a plurality of candidate algorithms from an algorithm bin, and respectively identifying a test set to obtain identification data corresponding to the candidate algorithms; correspondingly comparing identified coordinate information and an identified event quantity, which are obtained by means of each candidate algorithm identifying an identification object in each test sample, with target coordinate information and a target event quantity in an annotation file of each test sample of the test set, respectively, and calculating a coordinate accuracy and an event identification rate; reading a response time of the candidate algorithm to the identification of each test sample; and selecting a target algorithm from among the plurality of candidate algorithms on the basis of the coordinate accuracy, the event identification rate and the response time. In the present application for the invention, candidate algorithms are screened in view of a plurality of dimensions, and when a selected target algorithm executes a video identification task, the identification rate and the accuracy of the video task can be improved.

Description

A target algorithm selection method, device, electronic equipment and storage medium Technical field

This application requests the priority of the Chinese patent application submitted to the China Patent Office on March 22, 2022, with the application number 202210282517.9 and the invention title "A target algorithm selection method, device, electronic equipment and storage medium", all of which The contents are incorporated into this application by reference.

The present invention relates to the field of Internet technology, and in particular to a target algorithm selection method, device, electronic equipment and storage medium.

Background technique

The algorithm warehouse is compatible with internal algorithms or algorithms provided by external manufacturers. There are differences in performance or accuracy between different algorithms with the same requirements. Therefore, when a video detection task needs to be performed, the differences in performance indicators of the recognition results obtained through different algorithms will affect the accuracy of the calculation results and the recognition efficiency. It can be seen that in the existing technology, there is a problem that poor selection of algorithms in the algorithm warehouse leads to low recognition rate and accuracy rate of video tasks.

Technical solutions

Embodiments of the present invention provide a target algorithm selection method, aiming to solve the existing problem of low recognition rate and accuracy of video tasks caused by poor selection of algorithms in algorithm bins.

In a first aspect, embodiments of the present invention provide a method for selecting a target algorithm. The method includes the following steps:

Obtain multiple candidate algorithms in the algorithm warehouse and perform identification processing on the test set to obtain identification data corresponding to each candidate algorithm. The test set includes multiple test samples, and the identification data includes the candidate algorithm's identification of each candidate algorithm. The identified coordinate information obtained by identifying the identification objects in the test sample and the number of identified events;

Compare the identified coordinate information with the target coordinate information in the annotation file of the test sample, and calculate the coordinate accuracy;

Compare the number of identified events with the number of target events in the annotation file of the test sample, and calculate the event recognition rate;

Read the response time when the candidate algorithm identifies each test sample in the test set;

Based on the coordinate accuracy, the event recognition rate and the response time, a target algorithm is selected from a plurality of candidate algorithms in the algorithm bin.

In a second aspect, embodiments of the present invention also provide a device for selecting a target algorithm, including:

The acquisition module is used to acquire multiple candidate algorithms in the algorithm warehouse and perform identification processing on the test set to obtain identification data corresponding to each candidate algorithm. The test set includes multiple test samples, and the identification data includes the candidate algorithm. The identified coordinate information and the number of identified events obtained by identifying the identified objects in each test sample respectively;

The first calculation module is used to compare the identified coordinate information with the target coordinate information in the annotation file of the test sample, and calculate the coordinate accuracy;

The second calculation module is used to compare the number of identified events with the number of target events in the annotation file of the test sample, and calculate the event recognition rate;

A reading module, configured to read the response time when the candidate algorithm identifies each test sample in the test set;

An algorithm selection module is configured to select a target algorithm from a plurality of candidate algorithms in the algorithm bin based on the coordinate accuracy, the event recognition rate, and the response time.

In a third aspect, embodiments of the present invention further provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the computer program. When implementing the steps in the target algorithm selection method provided by the embodiment of the present invention.

In a fourth aspect, embodiments of the present invention also provide a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, the target algorithm provided by the embodiment of the present invention is implemented. Select the steps in the method.

In the embodiment of the present invention, identification data corresponding to each candidate algorithm is obtained by obtaining multiple candidate algorithms in the algorithm warehouse and performing identification processing on the test set respectively. The test set includes multiple test samples, and the identification data includes all The candidate algorithm separately identifies the identified coordinate information and the number of identified events obtained by identifying the identified objects in each test sample; compares the identified coordinate information with the target coordinate information in the annotation file of the test sample. Compare and calculate the coordinate accuracy rate; compare the number of identified events with the number of target events in the annotation file of the test sample, calculate the event recognition rate; read the candidate algorithm's results for each test in the test set The response time when the sample is identified; based on the coordinate accuracy, the event recognition rate and the response time, select a target algorithm from a plurality of candidate algorithms in the algorithm bin. It can be seen that the embodiment of the present invention obtains the multi-dimensional (coordinate accuracy rate, event recognition rate and response time) data of each candidate algorithm by calculating the identification data calculated by the algorithm bin candidate algorithm and the data in the test set, and based on Select the target algorithm for multi-dimensional data, and the selected target algorithm has the highest recognition rate and accuracy. In this way, when used in video recognition tasks, the selected target algorithm can improve the recognition rate and accuracy of the video task.

Description of the drawings

The drawings needed to be used in the embodiments of this application will be introduced below.

Figure 1 is a flow chart of a target algorithm selection method provided by an embodiment of the present invention;

Figure 2 is a flow chart of step S102 in Figure 1 provided by an embodiment of the present invention;

Figure 3 is a flow chart of step S103 in Figure 1 provided by an embodiment of the present invention;

Figure 4 is a flow chart of step S105 in Figure 1 provided by an embodiment of the present invention;

Figure 5 is a flow chart of another target algorithm selection method provided by an embodiment of the present invention;

Figure 6 is a module structure diagram of a target algorithm selection device provided by an embodiment of the present invention;

Figure 7 is a module structure diagram of the first computing module in Figure 6 provided by an embodiment of the present invention;

Figure 8 is a module structure diagram of the second computing module in Figure 6 provided by an embodiment of the present invention;

Figure 9 is a module structure diagram of the algorithm selection module in Figure 6 provided by an embodiment of the present invention;

Figure 10 is a partial module structure diagram of another target algorithm selection device provided by an embodiment of the present invention;

Figure 11 is a partial module structure diagram of another target algorithm selection device provided by an embodiment of the present invention;

Figure 12 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention.

Embodiments of the invention

The embodiments of the present application are described below with reference to the accompanying drawings.

As shown in Figure 1, Figure 1 is a flow chart of a target algorithm selection method provided by an embodiment of the present invention. As shown in Figure 1, it includes the following steps:

S101. Obtain multiple candidate algorithms in the algorithm warehouse, perform recognition processing on the test set respectively, and obtain the recognition data corresponding to each candidate algorithm. The test set includes multiple test samples, and the recognition data includes the recognition of each test sample by the candidate algorithm. The identified coordinate information obtained by object recognition and the number of identified events.

Among them, the scenarios for using electronic devices used in the target algorithm selection method provided in this embodiment include but are not limited to urban governance, such as road monitoring, face recognition, environmental monitoring, etc. through cameras. And the electronic device on which the above-mentioned target algorithm selection method runs can obtain identification data and perform data transmission through wired connection or wireless connection. Among them, wireless connection methods may include but are not limited to 3G/4G connection, WiFi (Wireless-Fidelity) connection, Bluetooth connection, WiMAX (Worldwide Interoperability for Microwave Access) connection, Zigbee (Low Power Consumption LAN Protocol, also known as Zifeng Protocol) connection, UWB (ultra wideband) connection, and other wireless connection methods now known or developed in the future.

Among them, the above-mentioned algorithm warehouse can include internally provided algorithms and algorithmic methods provided by external suppliers, and different algorithms can be calculated and processed for the same requirement. The above-mentioned candidate algorithms may include algorithms for face recognition, algorithms for vehicle recognition, algorithms for garbage detection and recognition, and so on. The above-mentioned recognition objects may include faces, human features, vehicle information, garbage types, etc. The above test set is an existing data set. The test set includes multiple test samples, and the test samples include recognition objects. First, you can first determine the recognition object that needs to be detected, select multiple candidate algorithms from the algorithm warehouse that can calculate and process the recognition object based on the recognition object, and then output a corresponding recognition data based on each candidate algorithm, so there are n candidate algorithms When calculating, n pieces of identification data will be output, and the calculation result of each identification data can be different. Each identification data includes identified coordinate information calculated through the corresponding candidate algorithm and the number of identified events. The identified coordinate information may refer to the location of the identification object identified through the algorithm, and the number of identified events may refer to the The number of events in which the algorithm identifies the above identified objects. Therefore, in the algorithm warehouse, after the same identification object is identified through multiple candidate algorithms, the identified coordinate information of the identification object returned by each candidate algorithm and the number of identified events can be obtained.

S102. Compare the identified coordinate information with the target coordinate information in the annotation file of the test sample, and calculate the coordinate accuracy rate.

Among them, the above-mentioned test sets can be provided by various manufacturers, and the test sets also include annotation files. The test samples can be different types of pictures collected through cameras. The annotation file will mark the target coordinate information of the recognized object and the number of target events. The recognition data calculated by multiple candidate algorithms can be compared with the data of each test sample in the test set. Specifically, the identified coordinate information returned by each candidate algorithm can be compared with the target coordinate information corresponding to multiple test samples, and the coordinate accuracy can be determined based on the coincidence of the coordinate information. Among them, the coordinate accuracy rate can refer to the proportion of the number of results whose coordinates meet expectations to the total number of returned results, specifically:

Coordinate accuracy = total number of coordinates that meet expectations/total number of detected coordinates × 100%

Among them, the total number of detected coordinates is the total number of identified coordinate information returned by multiple candidate algorithms in the algorithm warehouse provided by the algorithm manufacturer, and the total number of coordinates that meet the expectations is the number of identified coordinate information returned by each candidate algorithm that reaches the preset coordinate threshold. total.

S103. Compare the number of identified events with the number of target events in the annotation file of the test sample, and calculate the event recognition rate.

Among them, the number of identified events returned by the candidate algorithm can also be compared with the number of target events in the annotation file of the test sample in the test set, and the event recognition rate can be calculated. The event recognition rate can be the expected number of events in the recognition data of the candidate algorithm. The proportion of the total number of all events is as follows:

Event recognition rate = total number of detected events that meet expectations/total number of events × 100%

Among them, the total number of detected events that meets expectations is the total number of events returned by the algorithm manufacturer, and the total number of events is the total number of target objects in all test sample entries.

S104. Read the response time when the candidate algorithm identifies each test sample in the test set.

Among them, each test sample in the test set corresponds to a response time. The shorter the response time, the faster the response speed, and vice versa. Statistics are performed on all test samples to obtain the response time of each test sample.

S105. Based on the coordinate accuracy, event recognition rate and response time, select the target algorithm from multiple candidate algorithms in the algorithm warehouse.

Specifically, after calculating the above-mentioned coordinate accuracy rate, event recognition rate, and candidate Xuanda's response time when identifying each test sample, the above-mentioned multiple dimensions can be combined to select from multiple candidate algorithms in the algorithm warehouse. The candidate algorithm with the highest coordinate accuracy, highest event recognition rate and fast response time is used as the target algorithm.

In the embodiment of the present invention, identification data corresponding to each candidate algorithm is obtained by obtaining multiple candidate algorithms in the algorithm warehouse and performing identification processing on the test set respectively. The test set includes multiple test samples, and the identification data includes candidate algorithms that identify each candidate algorithm respectively. The identified coordinate information and the number of identified events obtained by identifying the identified objects in the test samples; compare the identified coordinate information with the target coordinate information in the annotation file of the test sample, and calculate the coordinate accuracy; calculate the number of identified events Compare with the number of target events in the annotation file of the test sample to calculate the event recognition rate; read the response time when the candidate algorithm identifies each test sample in the test set; based on the coordinate accuracy, event recognition rate and response time, Select the target algorithm from multiple candidate algorithms in the algorithm bin. The embodiment of the present invention calculates the recognition data calculated by multiple candidate algorithms in the algorithm warehouse and the data in the test set to obtain the multi-dimensional (coordinate accuracy rate, event recognition rate and response time) index data of each algorithm, and based on Multi-dimensional data selection target algorithm, the selected target algorithm has the highest recognition rate and accuracy. In this way, when used in video recognition tasks, the selected target algorithms can improve the recognition rate and accuracy of video tasks.

As shown in Figure 2, Figure 2 is a flow chart of step S102 in Figure 1 provided by an embodiment of the present invention. As shown in Figure 2, it includes the following steps:

S201. Based on the identified coordinate information and the target coordinate information in the annotation file of each test sample, calculate the accuracy rate of the candidate algorithm for identifying a single test sample.

Among them, because the test set includes multiple test samples, the identified coordinate information and the target coordinate information of each test sample can be calculated to obtain the accuracy of identifying each test sample by each candidate algorithm.

S202. Based on the accuracy of the candidate algorithm in identifying each test sample, determine the coordinate accuracy of the candidate algorithm in identifying the test set.

Among them, after obtaining the accuracy of each candidate algorithm for identifying each test sample, the coordinate accuracy of each candidate algorithm for identifying the test set can be calculated. For example, the above coordinate accuracy can be calculated by averaging. .

As a specific implementation, the above step S202 may specifically include:

Determine the object type to which the recognition object belongs when each candidate algorithm performs recognition, wherein the candidate algorithm corresponding to each object type is assigned a coincidence threshold.

Among them, the candidate algorithm can correspond to different coincidence thresholds for different object types. For example, the face recognition algorithm requires higher accuracy, and the coincidence threshold ratio is set to 90%. The garbage detection algorithm requires lower coincidence thresholds. The threshold ratio is set to 50%.

Each identified coordinate information is compared with the target coordinate information in the annotation file of each test sample, the identified coordinate information is marked based on the coincidence threshold, and the total number of identified coordinate information that meets the coincidence threshold is selected.

Based on the above coincidence threshold, the identified coordinate information returned by the candidate algorithm can be filtered, and the identified coordinate information that meets the coincidence threshold will be marked. Those that do not meet the coincidence threshold will not be marked. For example: face recognition through algorithm A, If it is recognized that the coincidence degree of the recognized coordinate information of face a and the target coordinate information is 98%, and the coincidence threshold is 95%, then the recognized coordinate information of face a recognized by candidate algorithm A is marked. Because the test set includes multiple test samples, multiple coincidence degree comparisons will be performed for the same candidate algorithm. After all comparisons are completed, the total number of identified coordinate information corresponding to each candidate algorithm that meets the coincidence degree threshold can be counted.

Based on the total number of identified coordinate information and the total number of identified coordinate information that meets the coincidence threshold, the coordinate accuracy of the corresponding algorithm is calculated, where the coordinate accuracy includes weighting the accuracy calculated by the same candidate algorithm on different test samples.

Among them, after counting the total number of the above-mentioned identified coordinate information and the total number of the identified coordinate information satisfying the coincidence threshold, the above-mentioned coordinate accuracy rate can be calculated. Specifically, under the same algorithm calculation, it can be a weighted sum between the accuracy of each test sample, and of course it can also be a weighted sum between the accuracy of a single test sample and the total accuracy of all test sample entries.

In the embodiment of the present invention, the accuracy of identifying each test sample by each candidate algorithm is calculated in advance, and then the coordinate accuracy of each candidate algorithm for identifying the test set is calculated, specifically by judging the identification of the algorithm. The object type of the object is matched with different coincidence thresholds according to different object types. This can improve the accuracy of detection for different object types, and the accuracy of the final coordinates filtered and calculated will be higher, which is beneficial to Select the target algorithm.

As shown in Figure 3, Figure 3 is a flow chart of step S103 in Figure 1 provided by an embodiment of the present invention. As shown in Figure 3, it includes the following steps:

S301. Based on the number of identified events and the number of target events in the annotation file of each test sample, calculate the recognition rate of the candidate algorithm for identifying a single test sample.

Among them, the accuracy rate can be calculated based on the number of identified events in the recognition data and the number of target events for each test sample, and the recognition rate of each candidate algorithm for identifying each test sample can be obtained.

S302. According to the recognition rate of each test sample recognized by the candidate algorithm, determine the event recognition rate when the candidate algorithm recognizes the test set.

Among them, after obtaining the recognition rate of each candidate algorithm for recognizing each test sample, the event recognition rate of each candidate algorithm for recognizing the test set can be calculated. The above event recognition rate can also be calculated by averaging. .

As a specific implementation, the above step S302 may specifically include:

Based on the preset event quantity threshold, the number of identified events that meet the preset event quantity threshold is selected.

Among them, the event quantity threshold can be set in advance, the number of identified events can be filtered based on the event quantity threshold, and the number of identified events that meet the event quantity threshold can be filtered out.

The event recognition rate is calculated based on the total number of identified events that meet the preset event number threshold and the total number of target events in the annotation files of the test set. The event recognition rate includes the recognition rate calculated by the same algorithm on different test samples. Be weighted.

Among them, after filtering out the number of identified events that meet the event quantity threshold, the total number can be counted, and then the event recognition rate is calculated based on the total number of identified events that meet the event quantity threshold and the total number of target events in the annotation file. Similarly, when the same candidate algorithm calculates the event recognition rate, the final event recognition rate can be obtained by calculating the weighted sum between the recognition rates. Of course, it can also be the weighted sum between the recognition rate of a single test sample and all accuracy rates.

In the embodiment of the present invention, by calculating the recognition rate of each candidate algorithm for identifying each test sample, and then calculating the event recognition rate of each candidate algorithm for identifying the test set, specifically by presetting the event number threshold, Filter out the data whose number of identified events meets the event number threshold, and count the total number, and then combine it with the total number of target events in the annotation file to calculate the event recognition rate. This can improve the event recognition rate and facilitate the selection of target algorithms.

As shown in Figure 4, Figure 4 is a specific flow chart of step S105 in Figure 1 provided by an embodiment of the present invention. As shown in Figure 4, it includes the following steps:

401. Generate an instruction library based on coordinate accuracy, event recognition rate, and response time, and assign the first weight ratio to each dimension.

Among them, after running and evaluating the algorithms provided by each manufacturer, an instruction library can be generated based on the results obtained after comparing the algorithm with the test set. The instruction library can be used to screen algorithms that meet the requirements. The instruction library can include multiple dimensions. Specifically, the dimensions include coordinate accuracy, event recognition rate, response time, etc., and each dimension is assigned a corresponding first weight ratio. For example: when the dimension includes coordinates In terms of accuracy, event recognition rate and response time, the corresponding first weight ratio may be 4:4:2.

402. Create an algorithm selection task and send it to the algorithm warehouse, and select the target algorithm from multiple candidate algorithms in the algorithm warehouse based on the instruction library.

Among them, when algorithm selection is required, an algorithm selection task is created in advance and the algorithm selection task is sent to the algorithm warehouse. In the algorithm warehouse, the algorithm selection is performed based on the generated instruction library, and the target algorithm is finally selected.

In this embodiment, by allocating a first weight ratio to the dimensions in the instruction library after creating the instruction library, the distribution ratio can distinguish the emphasis, so that the target algorithm can be selected more accurately.

As shown in Figure 5, Figure 5 is a flow chart of another target algorithm selection method provided by an embodiment of the present invention. As shown in Figure 5, it includes the following steps:

S501. Obtain multiple candidate algorithms in the algorithm warehouse, perform recognition processing on the test set respectively, and obtain the recognition data corresponding to each candidate algorithm. The test set includes multiple test samples, and the recognition data includes the recognition of each test sample by the candidate algorithm. The identified coordinate information obtained by object recognition and the number of identified events.

S502. Compare the identified coordinate information with the target coordinate information in the annotation file of the test sample, and calculate the coordinate accuracy rate.

S503. Compare the number of identified events with the number of target events in the annotation file of the test sample, and calculate the event recognition rate.

S504. Read the response time when the candidate algorithm identifies each test sample in the test set.

S505, select the candidate algorithm with the largest coordinate accuracy and event recognition rate.

Among them, after each candidate algorithm calculates the coordinate accuracy and event recognition rate, all coordinate accuracy and event recognition rates can be screened to find the maximum coordinate accuracy and event recognition rate.

S506. Identify and mark the scene information of the test sample corresponding to the candidate algorithm whose coordinate accuracy and event recognition rate are both the largest.

Among them, the test sample corresponding to the candidate algorithm with the largest coordinate accuracy and event recognition rate is then determined, and the scene information of the test sample is marked. Among them, the test samples can contain a large number of pictures. Based on the pictures, the test samples can be artificially classified into scenes in advance. For example, the test samples can be divided into test samples during the day and test samples at night. You can distinguish between day and night by setting a time. Value distinction, if the time is after 18:00, the scene corresponding to the task is considered to be at night. Of course, the scene can also include underground floors, urban main roads, highways, national highways, etc.

S507. Generate an instruction library based on the scene information, coordinate accuracy, event recognition rate and response time of the marked test sample, and assign a second weight ratio to each dimension. Based on the instruction library, select from multiple candidate algorithms in the algorithm warehouse. Choose the target algorithm.

Among them, when the scene information dimension is added, the instruction library can be generated based on the scene information, coordinate accuracy, event recognition rate and response time, and the weight of each dimension can be adjusted. The assigned weight is the above-mentioned second weight. Proportion. When the scene dimension is added, the scene dimension has the highest weight, and the second weight ratios of scene, coordinate accuracy, event recognition rate, and response time correspond to 4:2:2:2 respectively.

After generating the instruction library, you can create an algorithm selection task and send it to the algorithm warehouse, and then match the optimal algorithm (target algorithm) based on the instruction library in the evaluation. Specifically, the camera information of the task can be first selected based on the algorithm to query the scene where the device is, for example, the scene is underground, or at night. Prioritizing screening based on scenarios can eliminate more options, and then calculate the target algorithm based on the above-mentioned second weight ratio.

As a possible embodiment, the target algorithm selection method may also include the following steps:

Create an algorithm to select priorities.

Among them, the algorithm selection priority can mean selection based on higher priority conditions. In the algorithm selection priority, the second weight ratio is higher than the first weight ratio.

Determine whether to perform scene classification on the test sample.

If the test sample has been scene classified, based on the second weight ratio, an instruction library example is generated according to the scene, coordinate accuracy, event recognition rate, and response time of the test sample.

Among them, if it is determined that the test sample needs to be scene classified, that is, if the existing dimension is scene, then when the algorithm selection task is executed, instructions will be generated based on the scene, coordinate accuracy, event recognition rate, and response time of the test sample. For library examples, the second weight ratio is first selected for calculation to select the target algorithm.

If the test sample is not scene classified, an instruction library is generated based on the first weight ratio and the coordinate accuracy, event recognition rate, and response time.

Among them, when there is no dimension as a scene, an instruction library is generated based on the first weight ratio, coordinate accuracy, event recognition rate and response time. When executing the algorithm selection task, the target is selected in the algorithm bin according to the first weight ratio. algorithm.

In the embodiment of the present invention, by adding the dimension of scene information, combining the coordinate accuracy, event recognition rate and response time to generate an instruction library, and redistributing the second weight ratio to the above four dimensions, when executing the algorithm selection task, In the algorithm warehouse, the target algorithm is selected based on the above four dimensions and the corresponding second weight ratio. After increasing the scene dimension, the weight of the scene dimension will be adjusted to the maximum, and the scene will be selected first. The selected target algorithm has the highest recognition rate and accuracy. When used in video recognition tasks, the selected target algorithm can improve the recognition rate and accuracy of video tasks.

As shown in Figure 6, Figure 6 is a module structure diagram of a target algorithm selection device provided by an embodiment of the present invention. The device 600 includes:

The acquisition module 601 is used to obtain multiple candidate algorithms in the algorithm warehouse and perform identification processing on the test set to obtain identification data corresponding to each candidate algorithm. The test set includes multiple test samples, and the identification data includes candidate algorithms that identify each test separately. The identified coordinate information obtained by identifying the identified objects in the sample and the number of identified events;

The first calculation module 602 is used to compare the identified coordinate information with the target coordinate information in the annotation file of the test sample, and calculate the coordinate accuracy;

The second calculation module 603 is used to compare the number of identified events with the number of target events in the annotation file of the test sample, and calculate the event recognition rate;

The reading module 604 is used to read the response time when the candidate algorithm identifies each test sample in the test set;

The algorithm selection module 605 is used to select a target algorithm from multiple candidate algorithms in the algorithm warehouse based on coordinate accuracy, event recognition rate and response time.

Optionally, as shown in Figure 7, Figure 7 is a module structure diagram of the first calculation module in Figure 6 provided by an embodiment of the present invention, wherein the first calculation module 602 includes:

The first calculation sub-module 6021 is used to calculate the accuracy of identifying a single test sample by the candidate algorithm based on the identified coordinate information and the target coordinate information in the annotation file of each test sample;

The second calculation sub-module 6022 is used to determine the coordinate accuracy of the candidate algorithm when it identifies the test set based on the accuracy of the candidate algorithm in identifying each test sample.

Optionally, as shown in Figure 8, Figure 8 is a module structure diagram of the second calculation module in Figure 6 provided by an embodiment of the present invention, where the second calculation module 603 includes:

The third calculation sub-module 6031 is used to calculate the recognition rate of a single test sample by the candidate algorithm based on the number of identified events and the number of target events in the annotation file of each test sample;

The fourth calculation sub-module 6032 is used to determine the event recognition rate when the candidate algorithm recognizes the test set based on the recognition rate of each test sample by the candidate algorithm.

Optionally, as shown in Figure 9, Figure 9 is a module structure diagram of another target algorithm selection device provided by an embodiment of the present invention. The algorithm selection module 605 includes:

Generating sub-module 6051, used to generate an instruction library based on coordinate accuracy, event recognition rate and response time, and assign a first weight ratio to each dimension;

The selection sub-module 6052 is used to create an algorithm selection task and send it to the algorithm warehouse, and select the target algorithm from multiple candidate algorithms in the algorithm warehouse based on the instruction library.

Optionally, the test sample also includes scene information, as shown in Figure 10. Figure 10 is a partial module structure diagram of another target algorithm selection device provided by an embodiment of the present invention. The device 600 also includes:

The screening module 606 is used to screen candidate algorithms with the highest coordinate accuracy and event recognition rate;

The identification module 607 is used to identify and mark the scene information of the test sample corresponding to the candidate algorithm with the largest coordinate accuracy and event recognition rate.

Optionally, the algorithm selection module 605 is also used to generate an instruction library based on the scene information, coordinate accuracy, event recognition rate and response time of the marked test sample, and assign a second weight ratio to each dimension, based on the instruction library from Select the target algorithm from multiple candidate algorithms in the algorithm bin.

Optionally, as shown in Figure 11, Figure 11 is a module structure diagram of another target algorithm selection device provided by an embodiment of the present invention. The device 600 also includes:

The creation module 608 is used to create an algorithm selection priority. In the algorithm selection priority, the second weight ratio is higher than the first weight ratio;

Determination module 609, used to determine whether to perform scene classification on the test sample;

The algorithm selection module 605 is also used to generate an instruction library based on the scene, coordinate accuracy, event recognition rate and response time of the test sample based on the second weight ratio if the test sample has been scene classified;

The algorithm selection module 605 is also used to generate an instruction library based on the coordinate accuracy, event recognition rate, and response time based on the first weight ratio if the test sample is not scene classified.

A device for selecting a target algorithm provided by an embodiment of the present invention can realize various implementations of the above-mentioned method for selecting a target algorithm, as well as corresponding beneficial effects. To avoid duplication, they will not be described again here.

As shown in Figure 12, Figure 12 is a structural diagram of an electronic device provided by an embodiment of the present invention. As shown in Figure 12, it includes: a processor 1201, a memory 1202, a network interface 1203, and a computer program stored on the memory 1202 and executable on the processor 1201, wherein:

The processor 1201 is used to call the computer program stored in the memory 1202 and perform the following steps:

Obtain multiple candidate algorithms in the algorithm warehouse and perform recognition processing on the test set respectively to obtain the recognition data corresponding to each candidate algorithm. The test set includes multiple test samples. The recognition data includes candidate algorithms that perform recognition on the recognition objects in each test sample. The identified coordinate information obtained and the number of identified events;

Compare the identified coordinate information with the target coordinate information in the annotation file of the test sample to calculate the coordinate accuracy;

Compare the number of identified events with the number of target events in the annotation file of the test sample to calculate the event recognition rate;

Read the response time of the candidate algorithm as it identifies each test sample in the test set.

Optionally, the processor 1201 compares the identified coordinate information with the target coordinate information in the annotation file of the test sample, and calculates the coordinate accuracy, including:

Based on the identified coordinate information and the target coordinate information in the annotation file of each test sample, calculate the accuracy rate of the candidate algorithm for identifying a single test sample;

According to the accuracy of the candidate algorithm in identifying each test sample, the coordinate accuracy of the candidate algorithm in identifying the test set is determined.

Optionally, the processor 1201 compares the number of identified events with the number of target events in the annotation file of each test sample of the test set, and calculates the event recognition rate, including:

Based on the number of identified events and the number of target events in the annotation file of each test sample, calculate the recognition rate of the candidate algorithm for identifying a single test sample;

According to the recognition rate of each test sample recognized by the candidate algorithm, the event recognition rate when the candidate algorithm recognizes the test set is determined.

Optionally, the processor 1201 selects a target algorithm from multiple candidate algorithms in the algorithm bin based on coordinate accuracy, event recognition rate and response time, including:

Generate an instruction library based on coordinate accuracy, event recognition rate and response time, and assign the first weight ratio to each dimension;

Create an algorithm selection task and send it to the algorithm warehouse, and select the target algorithm from multiple candidate algorithms in the algorithm warehouse based on the instruction library.

Optionally, the test sample also includes scene information, and the processor 1201 is also used to execute:

Screen candidate algorithms with the highest coordinate accuracy and event recognition rate;

The scene information of the test sample corresponding to the candidate algorithm with the largest recognition coordinate accuracy and event recognition rate is marked and marked.

Optionally, processor 1201 is also used to execute:

Generate an instruction library based on the scene information, coordinate accuracy, event recognition rate and response time of the marked test samples, assign a second weight ratio to each dimension, and select targets from multiple candidate algorithms in the algorithm warehouse based on the instruction library algorithm.

Optionally, processor 1201 is also used to execute:

Create an algorithm selection priority. In the algorithm selection priority, the second weight ratio is higher than the first weight ratio;

Determine whether to perform scene classification on test samples;

If the test sample has been scene classified, an instruction library is generated based on the scene information, coordinate accuracy, event recognition rate and response time of the marked test sample based on the second weight ratio;

If the test sample is not scene classified, an instruction library is generated based on the first weight ratio and the coordinate accuracy, event recognition rate, and response time.

Embodiments of the present invention also provide a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, each process of the target algorithm selection method embodiment provided by the embodiment of the present invention is implemented. , and can achieve the same technical effect, so to avoid repetition, they will not be described again here.

It should be noted that only 1201-1203 with components is shown in the figure, but it should be understood that implementation of all the components shown is not required, and more or less components may be implemented instead. Among them, those skilled in the art can understand that the electronic device here is a device that can automatically perform numerical calculations and/or information processing according to preset or stored instructions. Its hardware includes but is not limited to microprocessors, special-purpose Application Specific Integrated Circuit, ASIC), Programmable Gate Array (Field-Programmable GateArray, FPGA), Digital Signal Processor (DSP), embedded devices, etc.

The electronic device 1200 may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The electronic device 1200 can perform human-computer interaction with the customer through a keyboard, mouse, remote control, touch pad, or voice-activated device.

The memory 1202 includes at least one type of readable storage medium. The readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory ( SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disks, optical disks, etc. In some embodiments, memory 1202 may be an internal storage unit of the electronic device, such as a hard drive or memory of the electronic device. In other embodiments, the memory 1202 may also be an external storage device of the electronic device, such as a plug-in hard disk or smart memory card (Smart Media card) equipped on the electronic device. Card, SMC), Secure Digital (SD) card, Flash Card, etc. Of course, the memory 1202 may also include both the internal storage unit of the electronic device and its external storage device. In this embodiment, the memory 1202 is usually used to store the operating system and various application software installed on the electronic device, such as the program code of the target algorithm selection method, etc. In addition, the memory 1202 can also be used to temporarily store various types of data that have been output or will be output.

The processor 1201 may be a central processing unit (Central Processing Unit) in some embodiments. Processing Unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. The processor 1201 is typically used to control the overall operation of the electronic device. In this embodiment, the processor 1201 is used to run the program code stored in the memory 1201 or process data, for example, run the program code of the selection method of the target algorithm.

The network interface 1203 may include a wireless network interface or a wired network interface. The network interface 1203 is generally used to establish a communication connection between the electronic device 1200 and other electronic devices.

An embodiment of the present invention also provides a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is executed by the processor 1201, it implements each of the target algorithm selection method embodiments provided by the embodiment of the present invention. The process can achieve the same technical effect. To avoid repetition, it will not be described again here.

Those of ordinary skill in the art can understand that all or part of the process of selecting the method for implementing the target algorithm of the embodiment can be completed by instructing relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it may include processes such as the embodiments of each method. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory (Read-Only Memory, ROM) or random access memory 1202 (Random Access Memory, RAM for short), etc.

The terms "first", "second", etc. in the description and claims of this application or the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific sequence. Reference herein to "an embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearances of this phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those skilled in the art understand, both explicitly and implicitly, that the embodiments described herein may be combined with other embodiments.

What is disclosed above is only the preferred embodiment of the present invention. Of course, it cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention still fall within the scope of the present invention.

Claims (10)

1. A method for selecting a target algorithm, characterized in that the method includes the following steps:

   Obtain multiple candidate algorithms in the algorithm warehouse and perform identification processing on the test set to obtain identification data corresponding to each candidate algorithm. The test set includes multiple test samples, and the identification data includes the candidate algorithm's identification of each candidate algorithm. The identified coordinate information obtained by identifying the identification objects in the test sample and the number of identified events;

   Compare the identified coordinate information with the target coordinate information in the annotation file of the test sample, and calculate the coordinate accuracy;

   Compare the number of identified events with the number of target events in the annotation file of the test sample, and calculate the event recognition rate;

   Read the response time when the candidate algorithm identifies each test sample in the test set;

   Based on the coordinate accuracy, the event recognition rate and the response time, a target algorithm is selected from a plurality of candidate algorithms in the algorithm bin.
2. The method of claim 1, wherein comparing the identified coordinate information with the target coordinate information in the annotation file of the test sample and calculating the coordinate accuracy includes:

   Calculate the accuracy of identifying a single test sample by the candidate algorithm based on the identified coordinate information and the target coordinate information in the annotation file of each test sample;

   According to the accuracy of the candidate algorithm in identifying each of the test samples, the coordinate accuracy of the candidate algorithm in identifying the test set is determined.
3. The method of claim 1, wherein comparing the number of identified events with the number of target events in the annotation file of each test sample of the test set and calculating the event recognition rate includes:

   Based on the number of identified events and the number of target events in the annotation file of each test sample, calculate the recognition rate of the candidate algorithm for identifying a single test sample;

   According to the recognition rate of each test sample recognized by the candidate algorithm, the event recognition rate when the candidate algorithm performs recognition processing on the test set is determined.
4. The method of claim 1, wherein the target algorithm is selected from a plurality of candidate algorithms in the algorithm bin based on the coordinate accuracy, the event recognition rate and the response time. ,include:

   Generate an instruction library based on the coordinate accuracy, the event recognition rate, and the response time, and assign a first weight ratio to each dimension;

   An algorithm selection task is created and sent to the algorithm warehouse, and the target algorithm is selected from a plurality of candidate algorithms in the algorithm warehouse based on the instruction library.
5. The method of claim 4, wherein the test sample further includes scene information, and the method further includes:

   Screen the candidate algorithm whose coordinate accuracy rate and event recognition rate are both the largest;

   The scene information of the test sample corresponding to the candidate algorithm whose coordinate accuracy rate and event recognition rate are both the largest is identified and marked.
6. The method of claim 5, comprising: generating the instruction library based on the marked scene information of the test sample, the coordinate accuracy, the event recognition rate and the response time, and A second weight ratio is assigned to each dimension, and the target algorithm is selected from a plurality of candidate algorithms in the algorithm bin based on the instruction library.
7. The method of claim 6, further comprising:

   Create an algorithm selection priority in which the second weight ratio is higher than the first weight ratio;

   Determine whether to perform scene classification on the test sample;

   If the test sample has been scene classified, based on the second weight ratio, the scene information of the marked test sample, the coordinate accuracy, the event recognition rate and the response time are generated. Described instruction library;

   If the test sample is not scene classified, the instruction library is generated based on the coordinate accuracy, the event recognition rate, and the response time based on the first weight ratio.
8. A target algorithm selection device, which is characterized by including:

   The acquisition module is used to acquire multiple candidate algorithms in the algorithm warehouse and perform identification processing on the test set to obtain identification data corresponding to each candidate algorithm. The test set includes multiple test samples, and the identification data includes the candidate algorithm. The identified coordinate information and the number of identified events obtained by identifying the identified objects in each test sample respectively;

   The first calculation module is used to compare the identified coordinate information with the target coordinate information in the annotation file of the test sample, and calculate the coordinate accuracy;

   The second calculation module is used to compare the number of identified events with the number of target events in the annotation file of the test sample, and calculate the event recognition rate;

   A reading module, configured to read the response time when the candidate algorithm identifies each test sample in the test set;

   An algorithm selection module is configured to select a target algorithm from a plurality of candidate algorithms in the algorithm bin based on the coordinate accuracy, the event recognition rate, and the response time.
9. An electronic device, characterized in that it includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements claim 1 to the steps in the method for selecting a target algorithm described in any one of 7.
10. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the method of any one of claims 1 to 7 is implemented. Steps in the selection method of the target algorithm.