CN115410072A - Method and system for testing video event detection algorithm - Google Patents
Method and system for testing video event detection algorithm Download PDFInfo
- Publication number
- CN115410072A CN115410072A CN202211360771.2A CN202211360771A CN115410072A CN 115410072 A CN115410072 A CN 115410072A CN 202211360771 A CN202211360771 A CN 202211360771A CN 115410072 A CN115410072 A CN 115410072A
- Authority
- CN
- China
- Prior art keywords
- event
- test
- alarm
- video
- detection algorithm
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/98—Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/18—Complex mathematical operations for evaluating statistical data, e.g. average values, frequency distributions, probability functions, regression analysis
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Pure & Applied Mathematics (AREA)
- Mathematical Analysis (AREA)
- Mathematical Optimization (AREA)
- Mathematical Physics (AREA)
- Computational Mathematics (AREA)
- Quality & Reliability (AREA)
- Evolutionary Biology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Multimedia (AREA)
- Algebra (AREA)
- Probability & Statistics with Applications (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Operations Research (AREA)
Abstract
The invention discloses a method and a system for testing a video event detection algorithm, wherein the video event detection algorithm is tested by simulating one or more test event types for multiple times, the detection result of each test is evaluated, meanwhile, the obtained multiple evaluation results are counted to obtain a technical parameter index corresponding to each test event type, the test score of the video event detection algorithm is calculated according to the weight of each test event type and the weight of the technical parameter index, and the application reliability of the video event detection algorithm is evaluated according to the test score. The invention provides a test flow and a working method of a whole set of algorithms aiming at a reliability verification non-systematic test method for a video event detection algorithm in the current market, and the test method and the system can simultaneously carry out comparison test on a plurality of sets of algorithms and have guiding significance for promoting the popularization and the use of an algorithm model and a tunnel video event detection system.
Description
Technical Field
The invention belongs to the field of video event detection, and particularly relates to a method and a system for testing a video event detection algorithm.
Background
With the annual increase of the number of highway tunnels and the continuous increase of traffic volume in China, serious traffic accidents and fire accidents occur in the tunnels, and due to the semi-closed structural form of the tunnels, the accidents easily cause great economic loss and social influence, and the pressure of highway tunnel operation safety management is gradually highlighted.
The old generation video event detection technology based on adjacent steady-state video image difference utilizes digital video and image processing technology to analyze and calculate gray level video sequence, and in practical application, the video event detection technology is basically idle at present due to high missing report rate and poor accuracy. With the development and introduction of artificial intelligence technology and the improvement of industrial equipment level, the artificial intelligence monitoring technology based on machine vision is separate, on the basis of a set of artificial neural network, the artificial neural network is trained by using image materials of massive vehicles, pedestrians, sprinklers, smoke and the like of different types in advance, and the characteristics and rules of different targets are automatically summarized by the neural network, so that the targets in a video are accurately identified and classified in actual use, and accurate identification of various abnormal traffic events, such as vehicle break-down, retrograde motion, congestion, pedestrian/non-motor vehicle entering, sprinklers and the like, is realized.
The new generation of tunnel video event detection algorithm based on machine vision provides a key technical means for a high-speed management unit, can effectively solve the problem of monitoring tunnel traffic events by manually polling videos, and has important effects on preventing secondary accidents and controlling event development due to intellectualization, timeliness and accuracy.
In addition, other new technical algorithms are proposed, and an important technical means can be provided for the tunnel safety operation management. However, how to scientifically and effectively evaluate the reliability and applicability of one or more sets of new technical algorithms is a premise that new technologies can be popularized and applied.
Disclosure of Invention
Aiming at the blank of reliability evaluation of the application of the current new technology algorithm, the disclosed embodiment of the invention at least provides a method and a system for testing a video event detection algorithm.
In a first aspect, an embodiment of the present invention provides a method for testing a video event detection algorithm, including: acquiring a video of a random simulation event as an input of a video event detection algorithm, wherein the simulation event belongs to one and/or more of a plurality of preset test event types; executing the video event detection algorithm and outputting a detection result; evaluating the detection result according to the information of the simulation event, wherein the evaluation result comprises positive report, false report, missing report, timely alarm and/or untimely alarm; repeating the steps according to preset times, and counting the obtained multiple evaluation results to obtain technical parameter indexes corresponding to each test event type; and calculating the test score of the video event detection algorithm according to the weight of each test event type and the weight of the technical parameter index.
Optionally, the evaluating the detection result according to the information of the simulated event includes: storing the detection result according to a preset format, comparing the detection result with information in the event simulation, if the information is consistent, the positive report is given, and if the information is inconsistent, the false report is given, if the alarm is not detected, the alarm is missed, if the alarm is positive within the preset time, the alarm is in time, and if the detection result is not obtained within the preset time, the alarm is not in time.
Optionally, the information of the simulated event includes event type characteristics, time and space information.
Optionally, the technical parameter indicators include event capture rate, accuracy, and timeliness; wherein, capture rate reaction system automatic capture event's ability, the accuracy indicates whether the event that system automated inspection detected is correct, and in time indicates after the event takes place, whether the system sends an alarm within the specified time, and the computational formula is as follows: capture rate = number of positive reports/number of analog events; accuracy = positive counts/(positive counts + false counts); timeliness = timely alarm number/(positive alarm number + untimely alarm number).
Optionally, the method further comprises: and determining the weight of each test event type and the weight of the technical parameter index by adopting an AHP analytic hierarchy process.
Optionally, the determining the weight of each test event type and the weight of the technical parameter index by using the AHP analytic hierarchy process includes the following steps: establishing a hierarchical structure model, constructing a judgment matrix, and checking the single hierarchical ordering and the consistency thereof, and checking the total hierarchical ordering and the consistency thereof; wherein, the hierarchical structure model is established as follows: target layer: evaluating a video event detection algorithm; a criterion layer: testing the event type; an index layer: a technical parameter index; scheme layer: at least 1 video event detection algorithm.
Optionally, when the scheme layer is a plurality of video event detection algorithms, respectively calculating test scores of the plurality of video event detection algorithms according to the weight of each test event type and the weight of the technical parameter index determined by the AHP analytic hierarchy process, and comparing the advantages and disadvantages of different video event detection algorithms according to the test scores.
In a second aspect, an embodiment of the present invention further provides a system for testing a video event detection algorithm, including a test platform, an algorithm server, and an event alarm platform; the testing platform is used for acquiring videos of random simulation events as input of a video event detection algorithm, and the simulation events belong to one and/or more of a plurality of preset testing event types; the algorithm server is used for executing the video event detection algorithm and outputting a detection result to an event alarm platform; the event alarm platform is used for alarming according to the detection result and feeding back the detection result to the test platform; the test platform is also used for receiving a detection result from the event alarm platform and evaluating the detection result according to the information of the simulation event, wherein the evaluation result comprises positive report, false report, missing report, timely alarm and/or non-timely alarm; and the test platform repeats the steps according to preset times, counts the obtained multiple evaluation results to obtain a technical parameter index corresponding to each test event type, and calculates the test score of the video event detection algorithm according to the weight of each test event type and the weight of the technical parameter index.
In a third aspect, an embodiment of the present invention further provides a computer device, including: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating over the bus when a computer device is running, the machine-readable instructions when executed by the processor performing the method of the first aspect.
In a fourth aspect, the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and the computer program is executed by a processor to perform the steps in the first aspect or any one of the possible implementation manners of the first aspect.
In the method and system for testing the video event detection algorithm provided by the embodiment of the invention, the video event detection algorithm is tested by simulating one or more of the test event types for multiple times, the detection result of each test is evaluated, meanwhile, the obtained multiple evaluation results are counted to obtain the technical parameter index corresponding to each test event type, the test score of the video event detection algorithm is calculated according to the weight of each test event type and the weight of the technical parameter index, and the application reliability of the video event detection algorithm is evaluated according to the test score. Meanwhile, the method for determining the test content and the weight of the test index by the AHP analytic hierarchy process is adopted, the recognition degree is high, and a scientific quantitative calculation mode is provided for evaluating the superiority of the algorithm; moreover, the scheme of the invention not only provides a whole set of test flow and working method aiming at the reliability verification of a specific video detection algorithm, has strong operability and has guiding significance for the popularization and the use of the algorithm, but also can simultaneously carry out parallel test on a plurality of different algorithms in the same environment.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a flow chart illustrating a method for testing a video event detection algorithm according to a disclosed embodiment of the invention;
FIG. 2 is a flow chart illustrating another method for testing a video event detection algorithm provided by the disclosed embodiment of the present invention;
FIG. 3 illustrates an AHP analytic hierarchy process-based build hierarchy model provided by the disclosed embodiments;
FIG. 4 illustrates a parking event simulated for tunnel video event detection algorithm testing provided by a disclosed embodiment of the invention;
FIG. 5 illustrates a projectile event simulated for tunnel video event detection algorithm testing provided by a disclosed embodiment of the invention;
FIG. 6 illustrates simulation of pedestrian events for testing of a tunnel video event detection algorithm provided by a disclosed embodiment of the invention;
FIG. 7 illustrates a simulation of smoke events for a tunnel video event detection algorithm test provided by a disclosed embodiment of the invention;
FIG. 8 illustrates a two-wheeled motorcycle event simulated for tunnel video event detection algorithm testing provided by the disclosed embodiment of the present invention;
FIG. 9 illustrates a test system topology for a video event detection algorithm provided by a disclosed embodiment of the invention;
fig. 10 shows a schematic structural diagram of a computer device according to an embodiment of the present disclosure.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to specific embodiments and the accompanying drawings. It should be noted that in the drawings or the description, the undescribed contents and parts of the english letters are well known to those skilled in the art. The factors of the comparison matrix constructed in this embodiment are derived by expert back-to-back scoring.
Example 1
As shown in fig. 1, a method for testing a video event detection algorithm according to an embodiment of the present disclosure is characterized by comprising:
s101: acquiring a video of a random simulation event as an input of a video event detection algorithm, wherein the simulation event belongs to one and/or more preset test event types;
s102: executing the video event detection algorithm and outputting a detection result;
s103: evaluating the detection result according to the information of the simulation event, wherein the evaluation result comprises positive report, false report, missing report, timely alarm and/or untimely alarm;
s104: repeating the steps according to preset times, and counting the obtained multiple evaluation results to obtain technical parameter indexes corresponding to each test event type;
s105: and calculating the test score of the video event detection algorithm according to the weight of each test event type and the weight of the technical parameter index.
Further, the evaluating the detection result according to the information of the simulated event includes: and storing the detection result according to a preset format, comparing the detection result with the information during the simulation event, judging that the information is in accordance with the information and is positive, judging that the information is not in accordance with the information and is false, judging that the information is false, if the information is not detected, the information is missed, judging that the positive alarm is timely alarm within preset time, and judging that the detection result is not timely alarm within preset time.
Further, the information of the simulated event comprises event type characteristics, time and space information.
Further, the technical parameter indexes comprise event capture rate, accuracy and timeliness; wherein, capture rate reaction system automatic capture event's ability, the accuracy indicates whether the event that system automated inspection detected is correct, and in time indicates after the event takes place, whether the system sends an alarm within the specified time, and the computational formula is as follows:
capture rate = number of positive reports/number of analog events;
accuracy = positive count/(positive count + false count);
timeliness = timely alarm number/(positive alarm number + untimely alarm number).
Further, the AHP analytic hierarchy process is adopted to determine the weight of each test event type and the weight of the technical parameter index.
Further, the determining the weight of each test event type and the weight of the technical parameter index by using the AHP analytic hierarchy process includes the following steps:
s1051: establishing a hierarchical structure model; wherein, the hierarchical structure model is established as follows:
and (4) target layer: evaluating a video event detection algorithm; a criterion layer: testing the event type; an index layer: a technical parameter index; scheme layer: at least 1 video event detection algorithm. Further, when the scheme layer is a plurality of video event detection algorithms, respectively calculating the test scores of the plurality of video event detection algorithms according to the weight of each test event type and the weight of the technical parameter index determined by the AHP analytic hierarchy process, and comparing the advantages and disadvantages of different video event detection algorithms according to the test scores.
S1052: constructing a judgment matrix;
s1053: checking the hierarchical list ordering and consistency thereof;
s1054: and (5) checking the total hierarchical ordering and the consistency thereof.
In the method for testing the video event detection algorithm provided by the embodiment of the invention, the video event detection algorithm is tested by simulating one or more of the test event types for multiple times, the detection result of each test is evaluated, meanwhile, the obtained multiple evaluation results are counted to obtain the technical parameter index corresponding to each test event type, the test score of the video event detection algorithm is calculated according to the weight of each test event type and the weight of the technical parameter index, and the application reliability of the video event detection algorithm is evaluated according to the test score. Meanwhile, the embodiment of the invention adopts the AHP analytic hierarchy process to determine the test content and the weight of the test index, has higher recognition degree and provides a scientific quantitative calculation mode for evaluating the superiority of the algorithm. The embodiment of the invention not only provides a whole set of test flow and working method aiming at the reliability verification of a specific video detection algorithm, has strong operability and has guiding significance for the popularization and the use of the algorithm, but also sets more than 1 video event detection algorithm at a scheme layer in the process of constructing a hierarchical structure model when the AHP analytic hierarchy process is adopted to determine the weight of each test event type and the weight of a technical parameter index, thereby simultaneously carrying out parallel test on a plurality of different algorithms under the same environment.
Example 2
As shown in fig. 2, on the basis of embodiment 1 of the present invention, the present invention is further described in detail by simulating test steps and test results of a test in a tunnel test field in real time by using 3 sets of algorithms:
the set test event types comprise abnormal parking events, throwing events, pedestrian events, smoke events and motorcycle events; the standard events simulated are defined as:
(1) abnormal parking: the vehicle stops on the road for more than 10 seconds.
(2) Throwing, namely, displaying an obstacle on the road surface and staying for 1 minute, carrying out the heavy year at the obstacle volume of not less than 20 x 20cm, and carrying out the heavy year at the pixel volume of not less than 30 x 30 in the monitoring picture.
(3) Pedestrian events: and (5) monitoring that pedestrians appear in the picture and stay for more than 10S.
(4) And (4) smoke, namely, the smoke can be identified by naked eyes in the monitoring picture and stays for more than 3 minutes.
(5) Motorcycle events: and monitoring the motorcycle running on the lane in the picture.
The technical parameter indicators include event capture rate, accuracy, and timeliness.
(1) The capture rate: the capability of the reaction system for automatically capturing events is expressed by the ratio of the correct alarm times (namely positive counts) of the events to the real times (namely the number of simulation events);
(2) the accuracy is as follows: whether the event automatically detected by the system is correct or not is indicated;
(3) timeliness: after the standard event occurs, the system sends out an alarm within the specified time.
Further, the technical parameter index calculation mode is as follows:
capture rate = number of positive reports/number of analog events;
accuracy = positive counts/(positive counts + false counts);
timeliness = timely alarm number/(positive alarm number + untimely alarm number).
The number of correct alarms (i.e. positive alarm count) is the number of repeated alarms with correct alarms removed, and the number of false alarms contains non-relevant false alarms.
And step 2, determining the test event type and the weight of the technical parameter index by adopting an AHP analytic hierarchy process. The method specifically comprises the following steps:
s21: and establishing a hierarchical structure model.
As shown in fig. 3, the hierarchical structure model is established based on the AHP analytic hierarchy process, and the target layer: evaluating a video event detection algorithm; a criterion layer: testing the content; an index layer: testing indexes; scheme layer: a video event detection algorithm 1, a video event detection algorithm 2 and a video event detection algorithm 3.
S22: firstly, a criterion layer judgment (pair comparison) matrix is constructedAElements ofa ij Is shown asiA factor relative tojComparing the factors; an index layer decision (pairwise comparison) matrix is then constructedB 1 ，B 2 ，B 3 ，B 4 ，B 5 ，Element(s)b ij Is shown asiA factor relative tojThe result of the comparison of the individual factors.
Determining elements of a matrixa ij Andb ij by usingSantyThe method of scale 1-9. The comparison results of the factors are given by the expert back-to-back scoring, and the scoring results are as follows:
A=
s23: judging matrix to criterion layerAAnd (5) carrying out hierarchical single-sequencing and consistency check. If the paired comparison matrix is non-uniform matrix, consistency test is carried out, and the normalized eigenvector corresponding to the maximum characteristic root lambda of the paired comparison matrix is used
As the vector of the weights,
then, then
。
I.e. the weight value of the criterion layer or the index layer. Defining a consistency indicator
，CIThe larger the inconsistency, the more severe the inconsistency; is a measure ofCISize of (2), introducing a random consistency indexRI(obtainable by table lookup), definitionConsistency ratio
When the consistency ratio isCR﹤0.1When the consistency of the comparison matrix is within the allowable range, the consistency is satisfied, and the normalized characteristic vector can be used by the consistency test
And if not, reconstructing the weight vector into a comparison matrix, and adjusting elements of the judgment matrix. Judging matrix for index layer by same methodB 1 ，B 2 ， B 3 ，B 4 ，B 5 ，And carrying out hierarchical single-sequencing and consistency check. By calculation, all passed the consistency test.
The calculation results are shown in the following table:
table 1: checking result of consistency of hierarchical single ordering
S24: the judgment (pair comparison) matrix is subjected to total sorting and consistency check.ALayer(s)nThe total target is ordered by the factorsa 1 、a 2 、a 3 、a 4 、a 5 ，BLayer(s)3One factor pair to the upper layerAThe middle factor isA j Is ordered in a hierarchy ofb 1j ，b 2j ，b 3j ，BThe total hierarchical ordering of the layersBThe weight of the ith factor of the layer to the total target is
。
Table 2: b total weight calculation result to total target
BLayer to upper layer (ALayer) has a hierarchical single ordering consistency index ofCI j The random consistency index isRI j Then the consistency ratio of the overall hierarchical ordering is
And when CR < 0.1, the total hierarchical ranking is considered to have satisfactory consistency, otherwise, the values of the elements of the judgment matrix with high consistency ratio need to be readjusted. And after the consistency check is passed, the normalized feature vector is the weight value of the layer to the upper layer. Calculated, the consistency ratio of the overall ranking of the hierarchy:
and the total sorting consistency of the layers is met if the total sorting consistency is less than 0.1.
Step 3, defining an event alarm data output format, comprising: event type, alarm time, position information (such as tunnel name, advancing direction, camera number, stake number and the like), and corresponding screenshot and video of the event.
And 4, simulating a real-time event video stream required by the tunnel video event detection algorithm test in a local area network environment, and pushing the real-time event video stream to an algorithm server, wherein the algorithm server adopts an RTSP stream taking mode, records the time and position information of the event while simulating the event, and the simulation mode is shown in the figures 4-8.
Step 5, the algorithm server pushes the recognition result to an event alarm platform according to a defined data format, and compares the recognition result with the event type, time and spatial information characteristics recorded during the event simulation, if the information is consistent, the result is positive report, if the information is inconsistent, the result is false report, if the information is not consistent, the result is false report, if the information is not detected, the result is false report, and if the result is detected within the specified alarm time, the result is timely alarm;
and 6, performing data statistics on the N times of algorithm detection results in a mode of repeating random simulation events, and calculating technical parameter indexes.
In this embodiment, the common mode simulates an abnormal parking event 305; a projectile 164; 146 pedestrians; 18 pieces of smoke; 150 pieces of motorcycle; through data statistics, the data statistics of 3 sets of video event detection algorithms are as follows:
table 3: algorithm 1, algorithm 2 and Algorithm 3 detection data statistical results
And 7, calculating a test score according to the test type determined by the AHP analytic hierarchy process and the weight of the parameter technical index, comparing the superiorities of different algorithms, and taking 100 points as a reference, wherein the score result is shown in the following table, and the result shows that the algorithm 1 is optimal.
Table 4: scoring results of Algorithm 1, algorithm 2 and Algorithm 3
The result of the embodiment shows that the testing method of the tunnel video event detection algorithm based on the machine vision is feasible in operation method, reliable in testing result and scientific in scoring basis. The method has guiding significance for popularization and application of the algorithm.
Example 3
As shown in fig. 9, a system for testing a video event detection algorithm according to an embodiment of the disclosure includes: the system comprises a test platform, an algorithm server and an event alarm platform;
the testing platform is used for randomly acquiring videos of simulated events as input of a video event detection algorithm, and the simulated events belong to one and/or more types of preset testing event types;
the algorithm server is used for executing the video event detection algorithm and outputting a detection result to an event alarm platform;
the event alarm platform is used for alarming according to the detection result and sending the detection result to the test platform;
the test platform is also used for receiving a detection result from the event alarm platform and evaluating the detection result according to the information of the simulation event, wherein the evaluation result comprises positive report, false report, missing report, timely alarm and/or untimely alarm;
and the test platform repeats the steps according to preset times, counts the obtained multiple evaluation results to obtain a technical parameter index corresponding to each test event type, and calculates the test score of the video event detection algorithm according to the weight of each test event type and the weight of the technical parameter index.
Example 4
Based on the same technical concept, an embodiment of the present application further provides a computer device, which includes a memory 1 and a processor 2, as shown in fig. 10, where the memory 1 stores a computer program, and the processor 2 implements any one of the methods described above when executing the computer program.
The memory 1 includes at least one type of readable storage medium, which includes a flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, and the like. The memory 1 may in some embodiments be an internal storage unit of the test system of the video event detection algorithm, e.g. a hard disk. The memory 1 may also be an external storage device of the test system for video event detection algorithms in other embodiments, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like. Further, the memory 1 may also comprise both an internal storage unit of the test system of the video event detection algorithm and an external storage device. The memory 1 may be used not only to store application software installed in a test system of a video event detection algorithm and various kinds of data, such as codes of a test program of the video event detection algorithm, etc., but also to temporarily store data that has been output or will be output.
The processor 2 may be, in some embodiments, a Central Processing Unit (CPU), controller, microcontroller, microprocessor or other data Processing chip for running program code stored in the memory 1 or Processing data, such as a test program executing a video event detection algorithm, etc.
The disclosed embodiments also provide a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the method described in the method embodiments above. The storage medium may be a volatile or non-volatile computer-readable storage medium.
The computer program product of the method for testing a video event detection algorithm according to the embodiments of the present disclosure includes a computer-readable storage medium storing a program code, where instructions included in the program code may be used to execute the steps of the method described in the above method embodiments, which may be referred to in detail in the above method embodiments, and are not described herein again.
The disclosed embodiments also provide a computer program which, when executed by a processor, implements any one of the methods of the preceding embodiments. The computer program product may be embodied in hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied in a computer storage medium, and in another alternative embodiment, the computer program product is embodied in a Software product, such as a Software Development Kit (SDK) or the like.
It is understood that the same or similar parts in the above embodiments may be mutually referred to, and the same or similar parts in other embodiments may be referred to for the content which is not described in detail in some embodiments.
It should be noted that the terms "first," "second," and the like in the description of the present invention are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. Further, in the description of the present invention, the meaning of "a plurality" means at least two unless otherwise specified.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and alternate implementations are included within the scope of the preferred embodiment of the present invention in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present invention.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present invention may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a separate product, may also be stored in a computer-readable storage medium.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.
Claims (10)
1. A method for testing a video event detection algorithm, comprising:
acquiring a video of a random simulation event as an input of a video event detection algorithm, wherein the simulation event belongs to one and/or more of a plurality of preset test event types;
executing the video event detection algorithm and outputting a detection result;
evaluating the detection result according to the information of the simulation event, wherein the evaluation result comprises positive report, false report, missing report, timely alarm and/or untimely alarm;
repeating the steps according to preset times, and counting the obtained multiple evaluation results to obtain technical parameter indexes corresponding to each test event type;
and calculating the test score of the video event detection algorithm according to the weight of each test event type and the weight of the technical parameter index.
2. The testing method according to claim 1, wherein the evaluating the detection result according to the information of the simulation event comprises: storing the detection result according to a preset format, comparing the detection result with information in the event simulation, if the information is consistent, the positive report is given, and if the information is inconsistent, the false report is given, if the alarm is not detected, the alarm is missed, if the alarm is positive within the preset time, the alarm is given in time, and if the detection result is not obtained within the preset time, the alarm is not given in time.
3. The test method of claim 2, wherein the information of the simulated events comprises event type characteristics, time and space information.
4. The test method of claim 2, wherein the technical parameter indicators include event capture rate, accuracy, and timeliness; wherein, capture rate reaction system automatic capture event's ability, the accuracy indicates whether the event that system automated inspection detected is correct, and in time indicates after the event takes place, whether the system sends an alarm within the specified time, and the computational formula is as follows:
capture rate = number of positive reports/number of analog events;
accuracy = positive counts/(positive counts + false counts);
timeliness = timely alarm number/(positive alarm number + untimely alarm number).
5. The test method according to claim 1,
and determining the weight of each test event type and the weight of the technical parameter index by adopting an AHP analytic hierarchy process.
6. The test method according to claim 5, wherein the determining the weight of each test event type and the weight of the technical parameter indicator by AHP analytic hierarchy process comprises the steps of:
establishing a hierarchical structure model, constructing a judgment matrix, and checking the single hierarchical ordering and the consistency thereof, and checking the total hierarchical ordering and the consistency thereof;
wherein, the hierarchical structure model is established as follows:
and (4) target layer: evaluating a video event detection algorithm;
a criterion layer: testing the event type;
an index layer: a technical parameter index;
scheme layer: at least 1 video event detection algorithm.
7. The method according to claim 5, wherein when the scheme layer comprises a plurality of video event detection algorithms, the test scores of the plurality of video event detection algorithms are respectively calculated according to the weight of each test event type determined by the AHP analytic hierarchy process and the weight of the technical parameter index, and the advantages and disadvantages of different video event detection algorithms are compared according to the test scores.
8. A test system of a video event detection algorithm is characterized by comprising a test platform, an algorithm server and an event alarm platform;
the test platform is used for acquiring a video of a random simulation event as an input of a video event detection algorithm, wherein the simulation event belongs to one and/or more preset test event types;
the algorithm server is used for executing the video event detection algorithm and outputting a detection result to an event alarm platform;
the event alarm platform is used for alarming according to the detection result and feeding back the detection result to the test platform;
the test platform is also used for receiving a detection result from the event alarm platform and evaluating the detection result according to the information of the simulation event, wherein the evaluation result comprises positive report, false report, missing report, timely alarm and/or untimely alarm;
and the test platform repeats the steps according to preset times, counts the obtained multiple evaluation results to obtain a technical parameter index corresponding to each test event type, and calculates the test score of the video event detection algorithm according to the weight of each test event type and the weight of the technical parameter index.
9. A computer device, comprising: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating over the bus when a computer device is running, the machine-readable instructions when executed by the processor performing the method of any of claims 1 to 7.
10. A computer-readable storage medium, having stored thereon a computer program which, when executed by a processor, performs the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211360771.2A CN115410072A (en) | 2022-11-02 | 2022-11-02 | Method and system for testing video event detection algorithm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211360771.2A CN115410072A (en) | 2022-11-02 | 2022-11-02 | Method and system for testing video event detection algorithm |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115410072A true CN115410072A (en) | 2022-11-29 |
Family
ID=84169124
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211360771.2A Pending CN115410072A (en) | 2022-11-02 | 2022-11-02 | Method and system for testing video event detection algorithm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115410072A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113627229A (en) * | 2021-05-31 | 2021-11-09 | 中国兵器工业计算机应用技术研究所 | Object detection method, system, device and computer storage medium |
| CN113672500A (en) * | 2021-07-27 | 2021-11-19 | 浙江大华技术股份有限公司 | Deep learning algorithm testing method and device, electronic device and storage medium |
| CN115269438A (en) * | 2022-08-23 | 2022-11-01 | 中国电信股份有限公司 | Automatic testing method and device for image processing algorithm |
-
2022
- 2022-11-02 CN CN202211360771.2A patent/CN115410072A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113627229A (en) * | 2021-05-31 | 2021-11-09 | 中国兵器工业计算机应用技术研究所 | Object detection method, system, device and computer storage medium |
| CN113672500A (en) * | 2021-07-27 | 2021-11-19 | 浙江大华技术股份有限公司 | Deep learning algorithm testing method and device, electronic device and storage medium |
| CN115269438A (en) * | 2022-08-23 | 2022-11-01 | 中国电信股份有限公司 | Automatic testing method and device for image processing algorithm |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110364008B (en) | Road condition determining method and device, computer equipment and storage medium | |
| Park et al. | Application of microscopic simulation model calibration and validation procedure: Case study of coordinated actuated signal system | |
| US9111442B2 (en) | Estimating incident duration | |
| CN102881162A (en) | Data processing and fusion method for large-scale traffic information | |
| JP5992681B2 (en) | Unusual condition detection system for congestion | |
| CN107885795A (en) | A kind of data verification method, system and the device of bayonet socket data | |
| CN103473459A (en) | Method of processing and fusing multisystem big data | |
| CN109979193B (en) | Data anomaly diagnosis method based on Markov model | |
| CN112711757B (en) | Data security centralized management and control method and system based on big data platform | |
| CN110032947A (en) | A kind of method and device that monitor event occurs | |
| CN113255439B (en) | Obstacle identification method, device, system, terminal and cloud | |
| CN110782670A (en) | Scene restoration method based on data fusion, vehicle cloud platform and storage medium | |
| EP4020428A1 (en) | Method and apparatus for recognizing lane, and computing device | |
| CN112633262B (en) | Channel monitoring method and device, electronic equipment and medium | |
| CN112233428A (en) | Traffic flow prediction method, traffic flow prediction device, storage medium and equipment | |
| CN114323042A (en) | Map data processing method and device, nonvolatile storage medium and processor | |
| CN115907639A (en) | CIM-based block chain engineering collaborative supervision system and method | |
| CN113851001B (en) | Multi-lane merging illegal automatic auditing method, system, device and storage medium | |
| CN117238112A (en) | Bridge structure health monitoring and early warning method and related equipment | |
| CN115410072A (en) | Method and system for testing video event detection algorithm | |
| CN116461546A (en) | Vehicle early warning method, device, storage medium and processor | |
| CN110826897A (en) | Regional safety evaluation method, device, equipment and storage medium | |
| CN114066288B (en) | Intelligent data center-based emergency detection method and system for operation road | |
| CN113192340B (en) | Method, device, equipment and storage medium for identifying highway construction vehicles | |
| CN115359658A (en) | Method, device, equipment, storage medium and program product for detecting traffic incident |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |