WO2020114136A1

WO2020114136A1 - Method and device for evaluating algorithm performance

Info

Publication number: WO2020114136A1
Application number: PCT/CN2019/112914
Authority: WO
Inventors: 刘若鹏; 栾琳; 季春霖; 赵盟盟
Original assignee: 西安光启未来技术研究院
Priority date: 2018-12-06
Filing date: 2019-10-24
Publication date: 2020-06-11
Also published as: CN111292359A

Abstract

The present application provides a method and device for evaluating algorithm performance. The method comprises: determining a tracking-system algorithm to be evaluated, and obtaining one or more parameters of the tracking-system algorithm with respect to a tracked target object in a video, wherein the one or more parameters are used to evaluate the tracking-system algorithm comprehensively, including evaluating various aspects such as the stability and accuracy of the tracking-system algorithm. The solution resolves the issue in the related art in which performance evaluation of a target-tracking-system algorithm is not comprehensive enough, and can fully and objectively evaluate the performance of the target-tracking-system algorithm.

Description

Algorithm performance measurement method and device

Technical field

This application relates to, but not limited to, the field of electronic tracking, and in particular, to an algorithm performance measurement method and device.

Background technique

In the related technology, the most common target tracking evaluation method is the mean square error (Mean-Squre Error, referred to as MSE). MSE refers to the expected difference between the true value and the estimated value. In fact, since expectations are often difficult to obtain, it is very difficult to calculate MSE indicators directly. Therefore, a commonly used indicator is the root mean square error RMSE, which uses Monte Carlo simulation of the sampled values to statistically approximate the expected value. RMSE is one of the most commonly used indicators in the field of multi-target tracking. However, the RMSE indicator has several shortcomings: first, it is not the concept of distance in Euclidean space; second, when the number of targets is large, such as hundreds Targets, using RMSE as a multi-target tracking evaluation index is too redundant.

technical problem

In fact, in the case of multiple targets, the tracking performance index of a single target is increasingly weakened. In this case, people pay more attention to the evaluation of the tracking performance of the overall target group rather than the tracking performance of a single target. . Therefore, the application of RMSE in complex target situations is very cumbersome and incomplete. In view of the problem that the performance evaluation scheme for the target tracking system algorithm in the related art is not comprehensive enough, there is currently no effective solution.

Technical solution

The embodiments of the present application provide an algorithm performance measurement method and device, so as to at least solve the problem that the performance evaluation scheme for the target tracking system algorithm in the related art is insufficient.

According to an embodiment of the present application, an algorithm performance measurement method is provided, which includes: acquiring, from video stream data, prediction data of the target object's action trajectory predicted by a first algorithm, and the trueness of the target object's action trajectory Data, wherein the first algorithm is used to track the target object; based on the predicted data and the real data, at least one of the following parameters is obtained: an average difference, multiple frame data of the video stream data The average value of the position difference of each frame, where the position difference of each frame of data is the difference between the predicted position and the true position; the first value is the number of real data that does not correspond to the predicted data; the second value is not the corresponding real data The number of prediction data of the third; the third value is the number of changes in the correspondence between the prediction data and the real data after the real walking trajectories of multiple target objects cross; the first algorithm is measured according to at least one of the parameters Performance.

According to another embodiment of the present application, there is also provided an algorithm performance measurement device, including: a first acquisition module, configured to acquire the prediction of the target object's action trajectory predicted by the first algorithm for video stream data Data, and real data of the target object's action trajectory, wherein the first algorithm is used to track the target object; the second acquisition module is used to acquire the following parameters based on the predicted data and the real data At least one of: the average difference, the average value of the position difference of multiple frame data of the video stream data, wherein the position difference of each frame data is the difference between the predicted position and the true position; the first value, no The number of real data corresponding to the predicted data; the second value, the number of predicted data that does not correspond to the real data; the third value, after the real walking trajectories of multiple target objects cross, the correspondence between the predicted data and the real data occurs The number of changes; a measurement module, configured to measure the performance of the first algorithm according to at least one of the parameters, wherein the first algorithm is used to track the target object.

According to yet another embodiment of the present application, there is also provided a storage medium in which a computer program is stored, wherein the computer program is set to execute the steps in any one of the above method embodiments during runtime.

According to yet another embodiment of the present application, there is also provided an electronic device including a memory and a processor, the memory stores a computer program, the processor is configured to run the computer program to perform any of the above The steps in the method embodiment.

Beneficial effect

Through this application, determine the tracking system algorithm to be evaluated and obtain one or more parameters of the tracking system algorithm to track the target object in the video. The one or more parameters are used to measure the tracking system algorithm in all aspects, including the measurement tracking system algorithm The stability, accuracy and other aspects of the above, using the above scheme, solves the problem of insufficient performance evaluation scheme for the target tracking system algorithm in the related technology, and completely and objectively measures the performance of the target tracking system algorithm.

BRIEF DESCRIPTION

The drawings described herein are used to provide a further understanding of the present application and form a part of the present application. The schematic embodiments and descriptions of the present application are used to explain the present application and do not constitute an undue limitation on the present application. In the drawings:

1 is a block diagram of a hardware structure of a computer terminal of an algorithm performance measurement method according to an embodiment of the present application;

2 is a flowchart of an algorithm performance measurement method according to an embodiment of the present application;

3 is a schematic diagram of a file format according to a specific embodiment of the present application;

4 is a structural block diagram of an algorithm performance measurement device according to an embodiment of the present application.

Embodiments of the invention

Hereinafter, the present application will be described in detail with reference to the drawings and in conjunction with the embodiments. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other if there is no conflict.

It should be noted that the terms “first” and “second” in the description and claims of the present application and the above drawings are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence.

The scheme of this application document is used to measure the performance of the algorithm of the target tracking system, and it can be a pedestrian intelligent tracking system, which is applied to urban security and other fields.

Example one

The method embodiment provided in Embodiment 1 of the present application may be executed in a computer terminal or a similar computing device. Taking running on a computer terminal as an example, FIG. 1 is a block diagram of a hardware structure of a computer terminal of an algorithm performance measurement method according to an embodiment of the present application. As shown in FIG. 1, the computer terminal 10 may include one or more (FIG. 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the computer terminal A transmission device 106 for communication functions and an input and output device 108 may be included. Persons of ordinary skill in the art may understand that the structure shown in FIG. 1 is merely an illustration, which does not limit the structure of the computer terminal described above. For example, the computer terminal 10 may further include more or fewer components than those shown in FIG. 1, or have a configuration different from that shown in FIG.

The memory 104 may be used to store software programs and modules of application software, such as program instructions/modules corresponding to the algorithm performance measurement method in the embodiments of the present application. The processor 102 executes the software programs and modules stored in the memory 104 to execute Various functional applications and data processing, namely to achieve the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memories remotely provided with respect to the processor 102, and these remote memories may be connected to the computer terminal 10 through a network. Examples of the above network include but are not limited to the Internet, intranet, local area network, mobile communication network, and combinations thereof.

The transmission device 106 is used to receive or send data via a network. The above specific example of the network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller (NIC), which can be connected to other network devices through the base station to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (Radio Frequency, RF) module, which is used to communicate with the Internet in a wireless manner.

In this embodiment, a method for measuring performance of an algorithm running on the computer terminal is provided. FIG. 2 is a flowchart of a method for measuring performance of an algorithm according to an embodiment of the present application. As shown in FIG. 2, the process includes the following steps :

Step S202: Obtain the predicted data of the action trajectory of the target object predicted by the first algorithm and the real data of the action trajectory of the target object for the video stream data, wherein the first algorithm is used to perform track;

The above video is the video of the tracking system application. The tracking system tracks the walking path of the target object in the video. Optionally, videos recorded by multiple cameras can be combined to achieve target tracking in a certain range or scene. For example, the video of a camera can only track the trajectory of the target object in the square, and the video combined with the surrounding environment can track the trajectory of the target object in the urban area.

Step S204: Acquire at least one of the following parameters according to the predicted data and the real data: an average difference value, an average value of position difference values of multiple frame data of the video stream data, where the position difference of each frame data The value is the difference between the predicted position and the true position; the first value, the number of real data that does not correspond to the predicted data; the second value, the number of predicted data that does not correspond to the real data; the third value, in multiple target objects After the real walking trajectory crosses, the number of correspondences between the predicted data and the real data changes;

The above average difference may be equivalent to the position accuracy index in subsequent embodiments, the above first value may be equivalent to the miss index of the number of statistical losses in subsequent embodiments, the second value may be equivalent to the false alarm index fp, and the third value It may be equivalent to the cross misjudgment index mix, and the average value of the first value, the second value, and the third value may be called a statistical accuracy index.

The solution of the above embodiment calculates the average difference of all frames from the difference of each frame image, and uses the average difference to measure the accuracy of the tracking system algorithm to be more accurate and complete.

Step S206: Measure the performance of the first algorithm according to at least one of the parameters.

Through the above steps, determine the tracking system algorithm to be evaluated and obtain one or more parameters of the tracking system algorithm to track the target object in the video. The one or more parameters are used to measure the tracking system algorithm in all aspects, including the measurement tracking system algorithm The stability, accuracy and other aspects of the above, using the above scheme, solves the problem of insufficient performance evaluation scheme for the target tracking system algorithm in the related technology, and completely and objectively measures the performance of the target tracking system algorithm.

Optionally, obtaining the predicted data of the target object's action trajectory predicted by the first algorithm and the real data of the target object's action trajectory include: acquiring the first algorithm prediction for the frame data of the video stream data The predicted position of the target object and the true position of the target object; determining that the predicted data includes the predicted position, and the true data includes the true position.

Optionally, obtaining the predicted data of the target object's action trajectory predicted by the first algorithm and the real data of the target object's action trajectory, including: in the video stream data, the first information of the target object A storage mark, wherein the first information includes at least one of the following: the number of video frames, the identification ID of the target object in each frame of the data image, the face size of the target object, and the coordinate position of the center point of the face of the target object; The marked first information is stored to obtain predicted data and the real data about the target object.

In the above optional embodiments, the target object's activity process is fully marked and recorded, and the above marking information is unified in the entire video, for example, the characters in the entire video have a unique ID from beginning to end, which is convenient for later data analysis .

Optionally, the method for obtaining the first value and/or the second value includes: obtaining the predicted data and real data for each frame of the video stream data; there will be no real data corresponding to the predicted data The number of data, as a third value, obtains the second average difference of the video stream data according to the third value of the multiple frame data, and uses the second average difference as the first value; there will be no corresponding The number of the prediction data of the real data is taken as the fourth value, and the third average difference of the video stream data is obtained according to the fourth value of the plurality of frame data, and the third average difference is taken as the first Two numerical values. The solution of the above embodiment calculates the second average difference or the third average difference of all frames from the difference of each frame image, and uses the second average difference or the third average difference to measure the accuracy of the tracking system algorithm More accurate and complete.

Optionally, the method for obtaining the third value includes: for the video stream data, after the real walking trajectories of a plurality of the target objects cross, acquiring the plurality of target objects predicted by the first algorithm Prediction data; obtaining the number of changes in the correspondence between the prediction data and the real data before and after the real walking track crosses; obtaining the fourth average of the video stream data according to the number of changes in multiple frame data For the difference, the fourth average difference is used as the third value. The solution of the above embodiment calculates a fourth average difference value of all frames from the difference value of each frame image, and uses the fourth average difference value to measure the accuracy of the tracking system algorithm to be more accurate and complete.

The target tracking evaluation methods in related technologies are mostly single-target or multi-target simple path scenarios, and can only measure the performance of the tracking algorithm itself. There is no reasonable evaluation method for the overall performance of pedestrian detection, tracking and recognition in complex scenes. To measure the overall performance of the pedestrian tracking system, it is necessary to completely measure the synthetic performance of each module, rather than only considering the tracking accuracy as traditional methods. In response to this problem, combined with the actual application, the proposal of this application is proposed.

This application provides a complete method system for measuring the accuracy of a pedestrian tracking and identification system. The system of this system may include a target detection module, a face ID recognition module, and a target trajectory tracking module. The system can be used in the field of urban security. This application can comprehensively and objectively measure the accuracy and performance of the tracking and identification system, and has very important application value for the optimization and evaluation of product development.

The complete pedestrian intelligent tracking system in this application should include key functional modules such as target detection, path tracking, and face recognition. This application provides a complete set of methods for measuring the accuracy of pedestrian tracking and identification systems. The system is divided into three parts: data preprocessing methods, position accuracy indicators, and statistical accuracy indicators. Data preprocessing refers to the interception and marking of the video data to be evaluated, including the interception of face, uniform name, frame number alignment and file format uniformity; position accuracy index refers to the error value of the target tracking prediction trajectory and the actual video trajectory , Which reflects the ability of the tracking algorithm of the system to predict the target position; statistical accuracy indicators are divided into three types, which describe the performance of the system target detection module, face recognition module and tracking module when the target crosses.

This evaluation system can completely and objectively measure the performance of the pedestrian intelligent tracking system. Make up for the lack of traditional tracking and evaluation methods.

The following is further described in conjunction with specific embodiments of the present application.

Example 1: Data preprocessing method

The tracking evaluation methods in related technologies mostly preprocess the data for simple location marking, and the purpose of this system is to comprehensively measure the detection, tracking, and recognition effects. Therefore, it is necessary to fully mark the pedestrian target activity process. The content to be recorded includes: the number of video frames, the ID of different people in each frame (the same person ID must be the same in different frames), the width of the pedestrian face, the coordinates of the center point of the pedestrian face, etc. After obtaining the above records, it is organized into a program-readable .json format file. FIG. 3 is a schematic diagram of the file format according to the specific embodiment 1 of the present application. As shown in FIG. 3, the specific content format is shown in FIG. 3, of which No. 1 The box is the outer field of the complete data; the box 2 is the digital field of the video frame; the box 3 is the fixed field; the box 4 is the pedestrian name ID field; the box 5 is the file path field. This data preprocessing method can completely record the activity trajectories of all pedestrians at all times to achieve an objective measurement of the performance of the entire system.

Example 2: Position accuracy index

The position accuracy index can measure the accuracy of the tracking algorithm. On the basis of completing the data preprocessing, first of all, we must establish the corresponding relationship between the real target and the prediction result of the system, which can be realized according to the number of frames and the ID name. After establishing the target correspondence, you can calculate the error between the predicted position coordinate and the real position coordinate of each target system in each frame, sum all the errors in all frames, and then log the relationship between the predicted position and the real position. The average error can be calculated by averaging. This indicator can measure the performance of the system tracking module.

Example 3: Statistical accuracy index

The statistical accuracy index can measure the comprehensive performance of the system detection target module, ID recognition module and tracking module. It is specifically divided into three indicators: the number of missing statistics, the false alarm indicator fp, and the cross-fault indicator mix, and the statistical accuracy indicator It is the average of the above three indicators. As with the position accuracy index, the statistical accuracy index needs to be calculated on the basis of establishing the corresponding relationship between the real target and the system prediction result. The statistical missing number miss indicator refers to the loss of the target number in the real data in the predicted data. The reason for the loss is the missed detection of the detection module or the loss of the tracking module; the false alarm indicator fp refers to the fact that the true error appears in the predicted data There is no target in the data, the reason for fp is the multi-detection of the detection module or the false recognition of the face recognition module; the cross misjudgment index mix refers to the prediction of the system at the intersection of the trajectories is completely opposite to the real target. Tracking module error tracking or identification module misidentification. The above indicators should first sum all the targets in the frame, and then average the logarithm of the relationship between the predicted position and the true position.

Using the above-mentioned index system, the performance of pedestrian tracking and identification system can be measured and evaluated, which is of great significance to system development and testing.

This application provides a complete set of methods for measuring the accuracy of pedestrian tracking and identification systems. Due to the complexity of the system modules, the evaluation effect cannot be achieved using traditional evaluation indicators. This application comprehensively and objectively measures the comprehensive performance of each module of the system by establishing data preprocessing and two categories of indicators, which provides important evaluation significance for product development and performance evaluation.

Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware, but in many cases the former is Better implementation. Based on this understanding, the technical solution of the present application can essentially be reflected in the form of a software product that contributes to the existing technology. The computer software product is stored in a storage medium (such as ROM/RAM, magnetic disk, The CD-ROM includes several instructions to enable a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to perform the methods described in the embodiments of the present application.

Example 2

In this embodiment, an algorithm performance measurement device is also provided. The device is used to implement the above-mentioned embodiments and preferred implementation modes, and descriptions that have already been described will not be repeated. As used below, the term "module" may implement a combination of software and/or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation of hardware or a combination of software and hardware is also possible and conceived.

FIG. 4 is a structural block diagram of an algorithm performance measurement device according to an embodiment of the present application. As shown in FIG. 4, the device includes:

The first obtaining module 42 is used to obtain the predicted data of the target object's action trajectory predicted by the first algorithm and the real data of the target object's action trajectory for the video stream data, wherein the first algorithm is used to Tracking the target object;

The second obtaining module 44 is configured to obtain at least one of the following parameters according to the predicted data and the real data: an average difference value, an average value of position difference values of a plurality of frame data of the video stream data, where each The position difference of the frame data is the difference between the predicted position and the true position; the first value, the number of real data that does not correspond to the predicted data; the second value, the number of predicted data that does not correspond to the real data; the third value, After the real walking trajectories of multiple target objects cross, the number of correspondences between the predicted data and the real data changes;

The measurement module 46 is configured to measure the performance of the first algorithm according to at least one of the parameters, wherein the first algorithm is used to track the target object.

Through the above device, determine the tracking system algorithm to be evaluated and obtain one or more parameters of the tracking system algorithm to track the target object in the video. The one or more parameters are used to measure the tracking system algorithm in all aspects, including the tracking system algorithm The stability, accuracy and other aspects of the above, using the above scheme, solves the problem of insufficient performance evaluation scheme for the target tracking system algorithm in the related technology, and completely and objectively measures the performance of the target tracking system algorithm.

Optionally, the first obtaining module 42 is configured to obtain the predicted position of the target object predicted by the first algorithm and the real position of the target object for the frame data of the video stream data; and It is used to determine that the predicted data includes the predicted position, and the real data includes the true position.

Optionally, the first obtaining module 42 is further configured to store a mark for the first information of the target object in the video stream data, where the first information includes at least one of the following: the number of video frames , The identification ID of the target object in each frame of the data image, the size of the face of the target object, the coordinate position of the center point of the face of the target object; and used to obtain information about the target object based on the first information in which the marker is stored Forecast data and the real data.

It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented by the following methods, but not limited to this: the above modules are all located in the same processor; or, the above modules can be combined in any combination The forms are located in different processors.

Example Three

The embodiments of the present application also provide a storage medium. Optionally, in this embodiment, the above storage medium may be set to store program code for performing the following steps:

S1. For the video stream data, obtain the predicted data of the target object's action trajectory predicted by the first algorithm and the real data of the target object's action trajectory, where the first algorithm is used to track the target object ;

S2, obtaining at least one of the following parameters according to the predicted data and the real data: an average difference value, an average value of position difference values of a plurality of frame data of the video stream data, wherein the position difference value of each frame data Is the difference between the predicted position and the true position; the first value is the number of real data that does not correspond to the predicted data; the second value is the number of predicted data that does not correspond to the real data; the third value is the value of multiple target objects After the real walking trajectory crosses, the number of correspondences between the predicted data and the real data changes;

S3, measuring the performance of the first algorithm according to at least one of the parameters.

Optionally, in this embodiment, the above storage medium may include, but is not limited to: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic Various media such as discs or optical discs that can store program codes.

An embodiment of the present application further provides an electronic device, including a memory and a processor, where the computer program is stored in the memory, and the processor is configured to run the computer program to perform the steps in any one of the foregoing method embodiments.

Optionally, the electronic device may further include a transmission device and an input-output device, where the transmission device is connected to the processor, and the input-output device is connected to the processor.

Optionally, in this embodiment, the foregoing processor may be configured to perform the following steps through a computer program:

Optionally, for specific examples in this embodiment, reference may be made to the examples described in the foregoing embodiments and optional implementation manners, and details are not repeated in this embodiment.

Obviously, those skilled in the art should understand that the above-mentioned modules or steps of the present application can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed in a network composed of multiple computing devices Above, optionally, they can be implemented with program code executable by the computing device, so that they can be stored in the storage device to be executed by the computing device, and in some cases, can be in a different order than here The steps shown or described are performed, or they are made into individual integrated circuit modules respectively, or multiple modules or steps among them are made into a single integrated circuit module to achieve. In this way, this application is not limited to any specific combination of hardware and software.

Industrial applicability

The above are only preferred embodiments of the present application, and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of this application shall be included in the scope of protection of this application.

Claims

An algorithm performance measurement method, characterized by including:

For the video stream data, obtain the predicted data of the target object's action trajectory predicted by the first algorithm and the real data of the target object's action trajectory, where the first algorithm is used to track the target object;

Acquire at least one of the following parameters according to the prediction data and the real data: average difference, the average value of the position difference of multiple frames of the video stream data, where the position difference of each frame of data is the prediction The difference between the position and the real position; the first value, the number of real data that does not correspond to the predicted data; the second value, the number of predicted data that does not correspond to the real data; the third value, the real walking on multiple target objects After the trajectories cross, the number of correspondences between the predicted data and the real data changes;

The performance of the first algorithm is measured according to at least one of the parameters.
The method according to claim 1, wherein obtaining the predicted data of the target object's action trajectory predicted by the first algorithm and the real data of the target object's action trajectory include:

For the frame data of the video stream data, obtain the predicted position of the target object predicted by the first algorithm and the real position of the target object;

It is determined that the predicted data includes the predicted position, and the real data includes the true position.
The method according to claim 1, wherein obtaining the predicted data of the target object's action trajectory predicted by the first algorithm and the real data of the target object's action trajectory include:

In the video stream data, a mark is stored for the first information of the target object, wherein the first information includes at least one of the following: the number of video frames, the identification ID of the target object in each frame of the data image, and the target The size of the subject's face and the coordinate position of the center point of the target's face;

According to the first information in which the mark is stored, the predicted data and the real data about the target object are acquired.
The method according to claim 1, wherein the method of obtaining the first value and/or the second value comprises:

For each frame of the video stream data, obtain the predicted data and real data;

The number of the real data without corresponding prediction data is taken as the third value, and the second average difference value of the video stream data is obtained according to the third value of the multiple frame data, and the second average difference value is taken as The first value;

The number of the prediction data without corresponding real data is taken as the fourth value, and the third average difference value of the video stream data is obtained according to the fourth value of the multiple frame data, and the third average difference value is taken as The second value.
The method according to claim 1, wherein the method of obtaining the third value comprises:

For the video stream data, after the real walking trajectories of a plurality of the target objects cross, obtain the prediction data of the plurality of target objects predicted by the first algorithm;

Acquiring the number of changes in the correspondence between the predicted data and the real data before and after the real walking track crosses;

A fourth average difference value of the video stream data is obtained according to the changed number of multiple frame data, and the fourth average difference value is used as the third numerical value.
An algorithm performance measurement device, characterized in that it includes:

The first obtaining module is used to obtain the predicted data of the target object's action trajectory predicted by the first algorithm and the real data of the target object's action trajectory for the video stream data, wherein the first algorithm is used to To track the target object;

The second obtaining module is configured to obtain at least one of the following parameters according to the predicted data and the real data: an average difference value, an average value of position difference values of a plurality of frame data of the video stream data, where each frame The position difference of the data is the difference between the predicted position and the true position; the first value, the number of real data that does not correspond to the predicted data; the second value, the number of predicted data that does not correspond to the real data; the third value, in After the real walking trajectories of multiple target objects cross, the number of correspondences between the predicted data and the real data changes;

The measurement module is used to measure the performance of the first algorithm according to at least one of the parameters, wherein the first algorithm is used to track the target object.
The apparatus according to claim 6, wherein the first acquiring module is configured to acquire the predicted position of the target object predicted by the first algorithm for the frame data of the video stream data, and State the true location of the target object;

And for determining that the predicted data includes the predicted location, and the real data includes the true location.
The apparatus according to claim 6, wherein the first acquisition module is further configured to store a mark for the first information of the target object in the video stream data, wherein the first information includes At least one of the following: the number of video frames, the identification ID of the target object in each frame of the data image, the face size of the target object, and the coordinate position of the center point of the face of the target object;

And for acquiring the predicted data and the real data about the target object according to the first information in which the mark is stored.
A storage medium characterized in that a computer program is stored in the storage medium, wherein the computer program is configured to execute the method described in any one of claims 1 to 5 when it is run.
An electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute any one of claims 1 to 5. Described method.