CN112135121B - Intelligent video monitoring and identifying performance evaluation system and method - Google Patents
Intelligent video monitoring and identifying performance evaluation system and method Download PDFInfo
- Publication number
- CN112135121B CN112135121B CN202010977778.3A CN202010977778A CN112135121B CN 112135121 B CN112135121 B CN 112135121B CN 202010977778 A CN202010977778 A CN 202010977778A CN 112135121 B CN112135121 B CN 112135121B
- Authority
- CN
- China
- Prior art keywords
- video
- image
- recognition
- identification
- result
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N17/00—Diagnosis, testing or measuring for television systems or their details
- H04N17/002—Diagnosis, testing or measuring for television systems or their details for television cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
Abstract
The invention discloses an intelligent video monitoring and identifying performance evaluation system and method, wherein the system comprises the following steps: the camera optical test platform projects the video/image in the reference data set acquired from the reference video/image recognition and evaluation platform onto a projection curtain through a projector, and reflects video/image optical signals to the optical lens of the tested sample through the projection curtain; the standard video/image recognition evaluation platform receives the recognition result of the tested sample optical lens after the video/image optical signals are acquired and analyzed, compares the recognition result with the recognition calibration correct result of the video/image in the standard data set, and evaluates the monitoring recognition performance of the tested sample optical lens based on the comparison result. The invention overcomes the defect that the traditional objective reference data set method cannot evaluate the intelligent video monitoring system which can not provide a video/image data model calling interface. Compared with the traditional field live-action test evaluation method, the method has objectivity and high evaluation precision.
Description
Technical Field
The invention relates to the technical field of video monitoring and recognition, in particular to an intelligent video monitoring and recognition performance evaluation system and method.
Background
This section is intended to provide a background or context to the embodiments of the invention that are recited in the claims. The description herein is not admitted to be prior art by inclusion in this section.
With the development of society, the safety awareness is continuously improved, and video monitoring is widely applied to important parts and places in various fields such as road traffic, urban public security monitoring, airports, subways, financial banks, customs, security inspection of frontier defense and the like, and goes deep into the production and life of people. In the face of mass data uploading and storage pressure brought by widely deployed cameras, intelligent video monitoring analyzes images through an identification algorithm, converts information into valuable data, provides warnings for users through key information, removes a large amount of redundant information, and provides an efficient and accurate informatization and intelligent means for target monitoring management. The video data uploading network bandwidth and storage space are saved, the traditional video monitoring is changed, manual detection and processing are relied on, the monitoring difficulty is reduced, faults can be processed in time, and the operation and maintenance cost is greatly reduced.
The traditional evaluation method of intelligent video monitoring recognition performance is mainly classified into 2 types.
The method is to evaluate the performance of the recognition algorithm from a software level, namely, taking a reference image data set as a software input parameter, verifying the robustness, accuracy and recognition speed of the recognition algorithm, and not considering factors such as imaging distortion of an optical lens of an intelligent video monitoring product/system, quantization error of A/D conversion, pixel filling error and the like, and not comprehensively evaluating the recognition performance of the whole intelligent video monitoring product/system. In particular, for intelligent video monitoring products/systems that cannot provide a video/image signal data model call interface, since the benchmark test video/image signal cannot be an input parameter to the algorithm, the identification performance evaluation cannot be performed using this method.
The other is to judge whether the identification result is correct or not through a subjective mode of artificial viewing by field live-action test and subjectively evaluate the identification performance of the whole intelligent video monitoring product/system. The method has the following problems: on the one hand, the test procedure is not reproducible, the test sample space is limited, the video/image dataset lacks completeness, and the test sample space cannot be used for evaluating the robustness of the identification algorithm. On the other hand, by means of lagging and low-efficiency manual evaluation, the test period is long, the cost is high, and the evaluation precision is low.
Disclosure of Invention
The embodiment of the invention provides an intelligent video monitoring and identifying performance evaluation system, which is used for realizing the evaluation and objectivity evaluation of an intelligent video monitoring system which can not provide a video/image data model calling interface, and comprises the following steps: the system comprises a camera optical test platform and a reference video/image recognition evaluation platform, wherein the camera optical test platform comprises a projector and a projection curtain;
the camera optical test platform is used for: projecting the video/image in the reference data set acquired from the reference video/image recognition and evaluation platform onto a projection curtain through a projector, and reflecting video/image optical signals to an optical lens of a sample to be tested through the projection curtain;
the reference video/image recognition evaluation platform is used for: and receiving an identification result obtained by collecting and analyzing the video/image optical signals by the optical lens of the tested sample, comparing the identification result with an identification calibration correct result of the video/image in the reference data set, and evaluating the monitoring identification performance of the optical lens of the tested sample based on the comparison result.
The embodiment of the invention also provides an intelligent video monitoring identification performance evaluation method, which is used for realizing the evaluation and objectivity evaluation of an intelligent video monitoring system which can not provide a video/image data model calling interface, and comprises the following steps:
the reference video/image recognition evaluation platform inputs the video/images in the reference data set into a projector of the camera optical test platform;
the projector projects the video/image in the reference data set onto a projection curtain;
the projection curtain reflects the video/image light signals to the optical lens of the tested sample;
the optical lens of the tested sample collects and analyzes the video/image optical signals to obtain a recognition result, and transmits the recognition result to the reference video/image recognition evaluation platform;
the standard video/image recognition evaluation platform receives the recognition result, compares the recognition result with a recognition calibration correct result of the video/image in the standard data set, and evaluates the monitoring recognition performance of the optical lens of the tested sample based on the comparison result;
the camera optical test platform comprises a projector and a projection curtain.
The embodiment of the invention also provides computer equipment, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the intelligent video monitoring and identifying performance evaluation method is realized when the processor executes the computer program.
The embodiment of the invention also provides a computer readable storage medium which stores a computer program for executing the intelligent video monitoring and identifying performance evaluation method.
In the embodiment of the invention, based on the video/image in the reference data set projected by the projector onto the projection curtain, the video/image optical signal is reflected to the optical lens of the tested sample by the projection curtain, so that the evaluation of the intelligent video monitoring system which can not provide a video/image data model calling interface can be realized; the method comprises the steps of acquiring and analyzing a video/image optical signal by a tested sample optical lens to obtain an identification result, receiving the identification result by a reference video/image identification evaluation platform, comparing the identification result with an identification calibration correct result of a video/image in a reference data set, and evaluating the monitoring identification performance of the tested sample optical lens based on the comparison result.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art. In the drawings:
FIG. 1 is a block diagram of an intelligent video monitoring and recognition performance evaluation system in an embodiment of the invention;
FIG. 2 is a block diagram II of an intelligent video monitoring and recognition performance evaluation system in an embodiment of the invention;
FIG. 3 is a block diagram of a system for evaluating the performance of intelligent video monitoring and recognition in the embodiment of the invention;
FIG. 4 is a flowchart of a method for evaluating the performance of intelligent video monitoring and recognition according to an embodiment of the present invention;
fig. 5 is a flowchart of a second method for evaluating intelligent video monitoring and identifying performance in an embodiment of the invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the embodiments of the present invention will be described in further detail with reference to the accompanying drawings. The exemplary embodiments of the present invention and their descriptions herein are for the purpose of explaining the present invention, but are not to be construed as limiting the invention.
Based on the problems existing in the prior art, the invention provides an intelligent video monitoring and identifying performance evaluation system, as shown in fig. 1 to 3, comprising: the camera optical test platform 1 and the reference video/image recognition evaluation platform 2, wherein the camera optical test platform comprises a projector 11 and a projection curtain 12;
the camera optical test platform is used for: the video/image in the reference data set acquired from the reference video/image recognition and evaluation platform is projected onto a projection curtain through a projector, played/displayed on the projection curtain, and video/image optical signals are reflected to a tested sample optical lens 3 (namely a tested product) through the projection curtain;
the reference video/image recognition evaluation platform is used for: and receiving an identification result obtained by collecting and analyzing the video/image optical signals by the optical lens of the tested sample, comparing the identification result with an identification calibration correct result of the video/image in the reference data set, and evaluating the monitoring identification performance of the optical lens of the tested sample based on the comparison result.
In an embodiment of the present invention, as shown in fig. 2, the camera optical testing platform further includes an optical darkroom 13, and the projector, the projection curtain and the optical lens of the sample to be tested are placed in the optical darkroom. This reduces interference of external background light noise.
In an embodiment of the present invention, as shown in fig. 2, the camera optical test platform further includes a test bench 14, and the optical lens of the tested sample is placed on the test bench. The test bench can be a camera bracket or a column body, and only the optical lens of the tested sample can be placed.
In an embodiment of the present invention, as shown in fig. 3, the optical testing platform for the camera further includes a projector control device 15, configured to control the projector, and may control some projection parameters of the projector, and so on. The projector control apparatus 15 may be a single apparatus provided separately, instead of the camera optical test platform.
In the embodiment of the present invention, as shown in fig. 2 and 3, the reference video/image recognition evaluation platform 2 includes a reference video/image server 21 and a recognition evaluation server 22;
the reference video/image server is configured to: storing the reference data set and the correct identification and calibration result of the video/image in the reference data set; inputting the video/image in the reference dataset to a projector;
the identification evaluation server is used for: receiving an identification result obtained after the optical lens of the tested sample collects and analyzes the video/image optical signals; acquiring an identification calibration correct result of the video/image in the reference data set from the reference video/image server; and comparing the identification result with the identification calibration correct result of the video/image in the reference data set, and evaluating the monitoring identification performance of the optical lens of the tested sample based on the comparison result. Whether the identification result is correct or not is judged, and four indexes with accurate identification are shown in table 1:
TABLE 1
TP: true positive, the recognition result outputs positive number/ratio, i.e., detection rate.
FN: false negative, actually positive, the recognition result outputs negative number/proportion, i.e. the omission factor.
FP: false positive, actually negative, the recognition result outputs a positive number/ratio, i.e., false detection rate.
TN: true negative, the recognition result outputs a negative number/ratio.
The embodiment of the invention also provides an intelligent video monitoring and identifying performance evaluation method, which is described in the following embodiment. Because the principle of solving the problem of the method is similar to that of the intelligent video monitoring and identifying performance evaluation system, the implementation of the method can be referred to the implementation of the intelligent video monitoring and identifying performance evaluation system, and the repetition is not repeated.
Fig. 4 is a flowchart of a method for evaluating intelligent video monitoring and identifying performance according to an embodiment of the present invention, as shown in fig. 4, the method includes:
step 401: the reference video/image recognition evaluation platform inputs the video/images in the reference data set into a projector of the camera optical test platform;
step 402: the projector projects the video/image in the reference data set onto a projection curtain;
step 403: the projection curtain reflects the video/image light signals to the optical lens of the tested sample;
step 404: the optical lens of the tested sample collects and analyzes the video/image optical signals to obtain a recognition result, and transmits the recognition result to the reference video/image recognition evaluation platform;
step 405: the standard video/image recognition evaluation platform receives the recognition result, compares the recognition result with a recognition calibration correct result of the video/image in the standard data set, and evaluates the monitoring recognition performance of the optical lens of the tested sample based on the comparison result;
the camera optical test platform comprises a projector and a projection curtain.
In an embodiment of the present invention, the method further includes:
placing the projector, the projection curtain and the optical lens of the tested sample in an optical darkroom;
the camera optical test platform further comprises an optical darkroom.
In an embodiment of the present invention, the method further includes:
a projector control apparatus controls the projector;
the camera optical test platform further comprises projector control equipment.
Fig. 5 is a flowchart of an intelligent video monitoring and identifying performance evaluation method in an embodiment of the present invention, as shown in fig. 5, a process of collecting and analyzing video/image optical signals by an optical lens (existing in an intelligent video monitoring product/system) of a sample to be tested is provided as follows:
after the video/image is collected by the optical lens, an initial video/image signal is formed through optical imaging, A/D conversion and image preprocessing, an image signal which is convenient to detect is generated by utilizing a feature extraction algorithm, an intelligent detection result is output, recognition is carried out based on the intelligent detection result and an existing feature library, and an intelligent recognition result is output.
The embodiment of the invention also provides computer equipment, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the intelligent video monitoring and identifying performance evaluation method is realized when the processor executes the computer program.
The embodiment of the invention also provides a computer readable storage medium which stores a computer program for executing the intelligent video monitoring and identifying performance evaluation method.
In the embodiment of the invention, based on the video/image in the reference data set projected by the projector onto the projection curtain, the video/image optical signal is reflected to the optical lens of the tested sample by the projection curtain, so that the evaluation of the intelligent video monitoring system which can not provide a video/image data model calling interface can be realized; the method comprises the steps of acquiring and analyzing a video/image optical signal by a tested sample optical lens to obtain an identification result, receiving the identification result by a reference video/image identification evaluation platform, comparing the identification result with an identification calibration correct result of a video/image in a reference data set, and evaluating the monitoring identification performance of the tested sample optical lens based on the comparison result.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The foregoing description of the embodiments has been provided for the purpose of illustrating the general principles of the invention, and is not meant to limit the scope of the invention, but to limit the invention to the particular embodiments, and any modifications, equivalents, improvements, etc. that fall within the spirit and principles of the invention are intended to be included within the scope of the invention.
Claims (9)
1. An intelligent video monitoring recognition performance evaluation system, which is characterized by comprising: the system comprises a camera optical test platform and a reference video/image recognition evaluation platform, wherein the camera optical test platform comprises a projector and a projection curtain;
the camera optical test platform is used for: projecting the video/image in the reference data set acquired from the reference video/image recognition and evaluation platform onto a projection curtain through a projector, and reflecting video/image optical signals to an optical lens of a sample to be tested through the projection curtain;
the reference video/image recognition evaluation platform is used for: receiving an identification result obtained by collecting and analyzing the video/image optical signals by the optical lens of the tested sample, comparing the identification result with an identification calibration correct result of the video/image in the reference data set, and evaluating the monitoring identification performance of the optical lens of the tested sample based on the comparison result;
the reference video/image recognition evaluation platform comprises a reference video/image server and a recognition evaluation server;
the reference video/image server is configured to: storing the reference data set and the correct identification and calibration result of the video/image in the reference data set; inputting the video/image in the reference dataset to a projector;
the identification evaluation server is used for: receiving an identification result obtained after the optical lens of the tested sample collects and analyzes the video/image optical signals; acquiring an identification calibration correct result of the video/image in the reference data set from the reference video/image server; and comparing the identification result with the identification calibration correct result of the video/image in the reference data set, and evaluating the monitoring identification performance of the optical lens of the tested sample based on the comparison result.
2. The intelligent video surveillance identification performance evaluation system of claim 1, wherein the camera optical test platform further comprises an optical darkroom, wherein the projector, projection curtain and sample optical lens under test are placed within the optical darkroom.
3. The intelligent video surveillance identification performance evaluation system of claim 1, wherein the camera optical test platform further comprises a test stand, the sample optical lens under test being placed on the test stand.
4. The intelligent video surveillance identification performance evaluation system of claim 1 wherein the camera optical test platform further comprises a projector control device for controlling the projector.
5. The intelligent video monitoring and identifying performance evaluation method is characterized by comprising the following steps of:
the reference video/image recognition evaluation platform inputs the video/images in the reference data set into a projector of the camera optical test platform;
the projector projects the video/image in the reference data set onto a projection curtain;
the projection curtain reflects the video/image light signals to the optical lens of the tested sample;
the optical lens of the tested sample collects and analyzes the video/image optical signals to obtain a recognition result, and transmits the recognition result to the reference video/image recognition evaluation platform;
the standard video/image recognition evaluation platform receives the recognition result, compares the recognition result with a recognition calibration correct result of the video/image in the standard data set, and evaluates the monitoring recognition performance of the optical lens of the tested sample based on the comparison result;
the camera optical test platform comprises a projector and a projection curtain;
the reference video/image recognition evaluation platform comprises a reference video/image server and a recognition evaluation server;
the reference video/image server is configured to: storing the reference data set and the correct identification and calibration result of the video/image in the reference data set; inputting the video/image in the reference dataset to a projector;
the identification evaluation server is used for: receiving an identification result obtained after the optical lens of the tested sample collects and analyzes the video/image optical signals; acquiring an identification calibration correct result of the video/image in the reference data set from the reference video/image server; and comparing the identification result with the identification calibration correct result of the video/image in the reference data set, and evaluating the monitoring identification performance of the optical lens of the tested sample based on the comparison result.
6. The intelligent video monitoring and recognition performance evaluation method according to claim 5, further comprising:
placing the projector, the projection curtain and the optical lens of the tested sample in an optical darkroom;
the camera optical test platform further comprises an optical darkroom.
7. The intelligent video monitoring and recognition performance evaluation method according to claim 5, further comprising:
a projector control apparatus controls the projector;
the camera optical test platform further comprises projector control equipment.
8. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the intelligent video surveillance recognition performance evaluation method of claim 5 when the computer program is executed by the processor.
9. A computer-readable storage medium, wherein the computer-readable storage medium stores a computer program which, when executed by a processor, implements the intelligent video surveillance recognition performance evaluation method of claim 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010977778.3A CN112135121B (en) | 2020-09-17 | 2020-09-17 | Intelligent video monitoring and identifying performance evaluation system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010977778.3A CN112135121B (en) | 2020-09-17 | 2020-09-17 | Intelligent video monitoring and identifying performance evaluation system and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112135121A CN112135121A (en) | 2020-12-25 |
| CN112135121B true CN112135121B (en) | 2023-04-28 |
Family
ID=73846050
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010977778.3A Active CN112135121B (en) | 2020-09-17 | 2020-09-17 | Intelligent video monitoring and identifying performance evaluation system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112135121B (en) |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4556873B2 (en) * | 2003-10-21 | 2010-10-06 | 日本電気株式会社 | Image collation system and image collation method |
| CN101500177B (en) * | 2009-02-20 | 2010-10-13 | 重庆市计量质量检测研究院 | Video monitoring system detection method and apparatus thereof |
| CN102262348A (en) * | 2010-05-24 | 2011-11-30 | 深圳富泰宏精密工业有限公司 | Optical detection device |
| CN108172156A (en) * | 2018-01-18 | 2018-06-15 | 安徽三弟电子科技有限责任公司 | A kind of projector intelligent projection control system and method |
-
2020
- 2020-09-17 CN CN202010977778.3A patent/CN112135121B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN112135121A (en) | 2020-12-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109949305B (en) | Product surface defect detection method and device and computer equipment | |
| US7872584B2 (en) | Analyzing smoke or other emissions with pattern recognition | |
| AU2007231552B2 (en) | Testing surveillance camera installations | |
| US20100002142A1 (en) | System and method for video-processing algorithm improvement | |
| JP4999794B2 (en) | Still region detection method and apparatus, program and recording medium | |
| CN112135121B (en) | Intelligent video monitoring and identifying performance evaluation system and method | |
| CN110602481B (en) | Video quality detection method and device in video monitoring system | |
| CN108257117B (en) | Image exposure evaluation method and device | |
| JP2002032872A (en) | Fire detector | |
| CN112016509B (en) | Personnel station abnormality reminding method and device | |
| CN116189063B (en) | Key frame optimization method and device for intelligent video monitoring | |
| Zhang et al. | Texture information boosts video quality assessment | |
| CN114550129B (en) | Machine learning model processing method and system based on data set | |
| KR102160583B1 (en) | Method and Apparatus for Detecting Object of Event | |
| CN113808116A (en) | Intelligent detection method and system based on image recognition and product detection system | |
| JP2014142213A (en) | Photographing parameter determination device and control method of the same | |
| CN116910489B (en) | Wall seepage prevention detection method based on artificial intelligence and related device | |
| JP2022112341A (en) | Oil spill monitoring system and oil spill monitoring method | |
| JP2024032264A (en) | Method of testing molten metal cleanliness | |
| KR20230092604A (en) | Intelligent cloud volume measurement apparatus and method | |
| CN115761578A (en) | Smoke and fire detection method, system, device and medium | |
| GB2521267A (en) | A method of assessing sensor performance | |
| KR20220057025A (en) | Fine dust detecting solution and system by computing RGB channel volume residual based on AI | |
| CN116189312A (en) | Living body detection method and living body detection device | |
| CN114913459A (en) | Group passenger flow detection system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |