CN115509739A - High-concurrency scheduling and analyzing system for real-time intelligent perception of videos - Google Patents
High-concurrency scheduling and analyzing system for real-time intelligent perception of videos Download PDFInfo
- Publication number
- CN115509739A CN115509739A CN202211071915.2A CN202211071915A CN115509739A CN 115509739 A CN115509739 A CN 115509739A CN 202211071915 A CN202211071915 A CN 202211071915A CN 115509739 A CN115509739 A CN 115509739A
- Authority
- CN
- China
- Prior art keywords
- video
- analysis
- data
- module
- intelligent
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000008447 perception Effects 0.000 title claims abstract description 6
- 238000004458 analytical method Methods 0.000 claims abstract description 168
- 238000012545 processing Methods 0.000 claims abstract description 55
- 238000013500 data storage Methods 0.000 claims abstract description 17
- 238000007689 inspection Methods 0.000 claims abstract description 17
- 238000013079 data visualisation Methods 0.000 claims abstract description 15
- 238000000034 method Methods 0.000 claims abstract description 13
- 238000007781 pre-processing Methods 0.000 claims abstract description 13
- 238000012544 monitoring process Methods 0.000 claims abstract description 11
- 230000006870 function Effects 0.000 claims description 18
- 230000002159 abnormal effect Effects 0.000 claims description 16
- 238000010191 image analysis Methods 0.000 claims description 14
- 238000007405 data analysis Methods 0.000 claims description 10
- 238000013523 data management Methods 0.000 claims description 7
- 238000007619 statistical method Methods 0.000 claims description 5
- 238000001514 detection method Methods 0.000 claims description 4
- 238000005516 engineering process Methods 0.000 claims description 4
- 230000000007 visual effect Effects 0.000 claims description 4
- 230000000903 blocking effect Effects 0.000 claims description 3
- 230000006835 compression Effects 0.000 claims description 3
- 238000007906 compression Methods 0.000 claims description 3
- 238000000926 separation method Methods 0.000 claims description 3
- 238000012916 structural analysis Methods 0.000 abstract description 5
- 238000011161 development Methods 0.000 description 10
- 238000010586 diagram Methods 0.000 description 4
- 239000002699 waste material Substances 0.000 description 3
- 238000012952 Resampling Methods 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000013480 data collection Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000002950 deficient Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a high concurrency scheduling and analyzing system for video real-time intelligent perception, which comprises: and the data acquisition and distribution module is responsible for acquiring monitoring videos and distributing video resources. And the video scheduling module is responsible for processing the received high-concurrency video stream. And the video processing queue module is responsible for caching a large amount of acquired video stream resources. And the intelligent analysis module is used for preprocessing the video by video quality inspection and calling a computer vision algorithm to analyze and process the video and the image. And the data storage module is used for storing the video and the image before and after analysis and is responsible for managing data. And the data visualization module is used for displaying the videos and images processed by the algorithm and displaying the statistical data and the structural analysis data. The invention provides technical support for the fields of video monitoring, high-concurrency scheduling, video intelligent analysis and the like.
Description
Technical Field
The invention belongs to the field of monitoring security, relates to a high-concurrency scheduling and analyzing system for video real-time intelligent sensing, and particularly relates to a system for completing high-concurrency intelligent analysis and monitoring videos by using a new intelligent scheduling algorithm.
Background
In recent years, with the development of computer vision algorithms, the monitoring video intelligent analysis system is increasingly perfected, and the technology is more and more approved and used by various industries. However, in actual development, due to access of a large number of monitoring cameras, limitation of network speed, capacity of a server, performance of the server and other various factors, the system is limited. Under certain hardware equipment conditions, how to play the hardware performance can better improve the performance of a software system under the same hardware condition, so that innovation and breakthrough are needed on the whole architecture and algorithm.
The traditional approach to solve the problem of high concurrency of large amounts of video data is to connect multiple servers responsible for actual processing to a load balancing server. The load balancing server then performs load distribution on the servers for processing the services, once the client sends a network request to the load balancing server, the load balancing server forwards the request to different servers connected with the load balancing server according to a certain algorithm (polling, weight, ip _ hash, minimum connection number and load distribution strategy of a third party), so that the pressure of each server is reduced, the servers can balance the services, and the concurrency and throughput of the whole system are improved.
At present, intelligent scheduling algorithms of distribution servers are more and more mature, but problems exist in speed and allocation of processing resources, and processing of large amounts of data with high concurrency is still a very complex and difficult problem. In particular, in terms of items related to the intelligent analysis monitoring system, not only hardware devices but also bandwidth, resolution and current operating state of the intelligent analysis server need to be considered. How to achieve real-time or near real-time intelligent analysis data processing in the environment of limited computing resources of edge computing equipment such as a camera is a very significant problem in the field of security protection. Outstanding and practical intelligent scheduling algorithm, better intelligent analysis technology, shareable stored information processing result and visual display result can improve the intelligent analysis monitoring system.
In addition, most of the current video preprocessing projects have or not exist, and do not pay attention to the existence of the video preprocessing projects, and although the final intelligent analysis service can be completed without the video preprocessing projects, precious computing resources and unnecessary time consumption are wasted, and the video preprocessing projects are defective in a high-concurrency service scene. How to utilize the video preprocessing flow of the video quality inspection, reduce the unnecessary waste of computing resources and time, and is also one of the development core problems of the project related to the high-concurrency intelligent analysis monitoring system at present.
The invention focuses on the overall framework of the system, improves and improves the load distribution method of the distribution server and the video preprocessing flow operation of the video quality inspection, so as to improve the response capability of the intelligent analysis system to data processing in the scene facing a large amount of data. By utilizing an excellent load distribution algorithm and an important video quality inspection process, an efficient solution is established, and the method can meet the requirement of a monitoring system in a high-concurrency big-data scene.
Disclosure of Invention
The invention solves the problems: the defects of the prior art are overcome, and a high-concurrency scheduling and analyzing system for video real-time intelligent perception is provided.
A high concurrency scheduling and analyzing system facing video real-time intelligent perception comprises: the system comprises a data acquisition and distribution module, a video scheduling module, a video processing queue module, an intelligent analysis module, a data storage module and a data visualization module;
the data acquisition and distribution module: the module takes a video image of a network camera as input, and the network camera, a streaming media server, a load balancing server and a distribution server output the video image to a video processing queue of each intelligent analysis server after carrying out video acquisition, video coding and decoding, load balancing and intelligent scheduling distribution. The network video camera (NVR or IPC) has the function of uploading the collected video to the streaming media server, the video signal collected in real time is transmitted to a video encoder to carry out video encoding and compression, and then the video after encoding and compression is uploaded to the streaming media server; the streaming media server has a function of converging the multiple paths of network cameras, and achieves the summary of the streaming media data in a local area; the load balancing server has the function of a load distribution server, and the distribution server with shorter response time can be preferentially designated as the current target distribution server by a software-level IP load balancing method; the distribution server has a video scheduling function, and calculates the distribution weight of each intelligent analysis server connected at a subordinate level in real time by calling an intelligent scheduling algorithm, so that the videos are intelligently distributed, and the performance of the servers is utilized to the maximum extent.
And the video scheduling module consists of four sub items, namely video request, video scheduling, video decoding and streaming media forwarding. The video request sub-item firstly obtains the first-level directories of all the cameras, then progresses layer by layer, uses a recursive algorithm, and finally obtains the keywords (IDs) of all the cameras. And after all the camera IDs are acquired, sending a large number of network requests for all the camera video sources, pushing streams by the target streaming media server, and pushing the corresponding camera video sources to the video scheduling sub-items of the distribution server. The video scheduling sub-item is characterized in that firstly, according to parameters set manually, a designed intelligent scheduling algorithm (the algorithm considers bandwidth, resolution, data amount processed by an intelligent analysis server and data amount not processed by the intelligent analysis server) is used for obtaining a weighted value of each intelligent analysis server, then videos are processed according to priorities of the intelligent analysis servers, the videos are sequentially placed into video processing queues corresponding to the intelligent analysis servers with the priorities from high to low according to comparison of the weighted values of the priorities of the different intelligent analysis servers, after one video in one queue in a video processing queue module is processed by the intelligent analysis server, the weighted value of each intelligent analysis server is recalculated by the intelligent scheduling algorithm, the order of placing the videos into the video processing queues of the intelligent analysis servers is redetermined, and the previous operation is repeated. The intelligent scheduling algorithm realizes intelligent forwarding and task allocation of high-concurrency videos, and fully utilizes the performance of the server to accelerate the video processing speed. And when additional development and extension are needed, an external third party can send a network request to the video scheduling module, the sub-item performs decoding operation on the received data stream, and pushes the decoded video stream media to a specified port. And when additional development requirements are needed, an external third party can send a network request to forward the streaming media video source scheduled by the video to the third party, and the third party can perform other operations by utilizing the video source.
And the intelligent analysis module consists of four sub-items, namely video quality inspection, video analysis, image analysis and external calling. The video quality inspection sub-items realize operations such as lens stain judgment, image frame blurring, abnormal visual angle, too low video code rate, image resampling, video frame rate conversion and the like. The method adopts a mature video preprocessing algorithm, performs different video quality inspection operations on each video according to local conditions, and reduces unnecessary computing resources and time waste. And the video analysis subentry takes the video source of the video processing queue module as input, calls a computer vision algorithm to analyze the video distributed by intelligent scheduling in the video scheduling module and outputs the analysis result to the data visualization module, and various intelligent algorithms are also built in the subentry to analyze events under various different scenes. The image analysis sub item takes a video source of the video processing queue module as input, one frame of the video is continuously extracted as a picture at a certain time interval, a computer vision algorithm is called to analyze the picture, an analysis result is output to the data visualization module, and various intelligent algorithms are also built in the sub item, so that events under various different scenes can be analyzed. And when additional development requirements are needed, an external third party can send a network request to call various different intelligent analysis algorithms provided in the intelligent analysis module.
The data storage module takes video results and picture results in the intelligent analysis module as input, writes and reads data by using a relational database, and separates database reading operation and database writing operation by using a master-slave read-write separation method in high-concurrency processing of the database, so that the problem of high concurrency encountered when results of a plurality of intelligent analysis servers are written into the database or when a client frequently reads a large amount of database data is solved. The module consists of four sub-items, namely video storage, image storage, analysis data storage and data management. And the video storage sub-item stores the original video processed by the intelligent analysis module of each intelligent analysis server and the video processed by the algorithm into a database. And the image storage sub-item is used for storing the original image processed by each intelligent analysis server intelligent analysis module and the image processed by the algorithm into a database. And analyzing the data storage sub-items, wherein when additional development requirements are needed, an external third party can request the database and provide video analysis results and picture analysis results in the intelligent analysis module so as to meet other additional requirements. And the data management sub-item can perform basic operations such as adding, inquiring, updating and deleting on the stored pictures, videos and analysis data by using a database language, and also provides a function of exporting data at regular time to meet other additional requirements.
And the data visualization module takes the intelligent analysis video result and the picture result as input, provides a preview display function of the video analysis result and the image analysis result of the intelligent analysis module, and has the function of carrying out data statistics and analysis on the video analysis result and the image analysis result in the intelligent analysis module. The module carries out statistical analysis on the video analysis result and the image analysis result through counting the structural analysis data output by the intelligent analysis module, and visually displays the intelligent analysis result in the form of using a statistical chart and a statistical table. The module consists of four sub-items, namely video preview, image preview, statistical data analysis and structured data analysis. And the video previewing sub-item displays the original video processed by the intelligent analysis module of each intelligent analysis server and the video processed by the algorithm on the data visualization platform. And (4) image previewing sub-items, and displaying the original image processed by each intelligent analysis server intelligent analysis module and the image processed by the algorithm on a data visualization platform. And the statistical data analysis sub-item carries out statistical analysis on the video analysis result and the image analysis result through counting the structural analysis data output by the intelligent analysis module, and visually displays the intelligent analysis result in the form of a statistical chart or a statistical table. The structured data analysis sub-item provides external structured analysis data for visualizing the data in multiple aspects. When additional development requirements are needed, an external third party can send a network request, and data of the picture analysis result and data of the video analysis result can be obtained to meet other additional requirements.
The intelligent scheduling algorithm in the video scheduling module is specifically realized as follows:
(1) Giving n video processing queue initial parameters alpha corresponding to n intelligent analysis servers according to hardware goodness degree of the intelligent analysis servers 1 ...α n . Initial parameter alpha k The smaller the weight value omega k The larger the priority, the higher k ∈ [1, n ]]；
(2) According to the designed weight formula
And calculating the weight values of the n intelligent analysis servers. Wherein alpha is k Is the initial value given in (1).
The amount of data (bit) to be processed for the kth video processing queue, n is the number of videos to be processed in the corresponding queue, X ki And = resolution of the video, bit depth, fps, video length. ε is a constant to ensure that the denominator is not zero. Y is k Data processing speed (b/s), Y, of the intelligent analysis server corresponding to the kth video processing queue k = amount of data of one completed video/total time used;
(3) According to the weight values omega of the n intelligent analysis servers 1 ...ω n Assigning a video source from a video scheduling module, a weight value ω k The larger, the higher the priority, k ∈ [1, n ]]And putting the video source into the video processing queue module corresponding to each intelligent analysis server. If the intelligent analysis server is idle, the video source at the head of the corresponding queue is dequeued and is placed into the intelligent analysis server for processing;
(4) And (4) repeatedly executing the operations (2) and (3) to finish high-concurrency large-quantity video data processing work.
The video quality inspection is used in the intelligent analysis module to carry out video preprocessing as follows:
(1) Judging whether the angle of the video is abnormal (the video is intentionally rotated by people or the holder fails, and the shot picture is not a scene needing to be monitored);
(2) If the video is abnormal, the video is not processed, and the information of the abnormal video angle is prompted;
(3) Otherwise, continuously judging whether the network is blocked;
(4) If the video is abnormal, the video is not processed, and the information of network blocking is prompted;
(5) Otherwise, continuously judging whether the lens is stained or not;
(6) If the lens is stained, calling an image restoration algorithm;
(7) Otherwise, continuously judging whether the video image is blurred;
(8) If there is picture blur, an image enhancement algorithm is invoked.
Compared with the prior art, the invention has the advantages that:
1) The invention provides a new intelligent dispatching algorithm of a distribution server, which utilizes information such as bandwidth, resolution, current running state condition of an intelligent analysis server and the like to design the algorithm, calculate and obtain the weight of each intelligent analysis server, and set the priority of business resource distribution, so that under the same hardware condition, the pressure of each intelligent analysis server can be better reduced, the intelligent analysis servers can process business resources in a balanced manner, the concurrency and the throughput of the whole system are improved, and the system performance is fully exerted.
2) The invention provides an excellent video quality inspection process, and realizes operations such as lens stain judgment, image frame blurring, abnormal visual angle, low video code rate, image resampling, video frame rate conversion and the like. Specifically, the mature video preprocessing algorithm is adopted, and different video quality inspection operations are performed on each video according to local conditions, so that unnecessary waste of computing resources and time is reduced. The method is suitable for scenes with high concurrency and a large number of service resources, and improves the stability and the processing speed of the whole system.
3) The system provided by the invention designs the maintainable interface and the extensible interface. So that the system can replace a module without affecting the use of other modules. The modules can be expanded, and the work of other modules is not influenced by adding new functions.
Drawings
FIG. 1 is a system flow diagram of the present invention;
FIG. 2 is a flow diagram of a data collection and distribution module of the present invention;
FIG. 3 is a flow diagram of a video scheduling module of the present invention;
FIG. 4 is a flow chart of an intelligent scheduling algorithm of the video scheduling module of the present invention;
FIG. 5 is a flow diagram of an intelligent analysis module of the present invention;
FIG. 6 is a flow chart of video pre-processing for video quality inspection in the intelligent analysis module of the present invention;
Detailed Description
The present invention will be described in detail below with reference to the drawings and examples.
As shown in fig. 1, it is a high concurrency scheduling and analyzing system facing video real-time intelligent sensing of the present invention: the system comprises a data acquisition and distribution module, a video scheduling module, an intelligent analysis module, a data storage module and a data visualization module;
the specific implementation process of each module is as follows:
1. the data collection and distribution module, as shown in figure 2,
(1) The streaming media server receives a data stream sent by an acquisition end;
(2) The load balancing server forwards the data stream to each distribution server;
(3) The distribution server decodes the data stream and then distributes the data stream to each intelligent analysis server according to an intelligent scheduling algorithm;
(4) The intelligent analysis server receives the video stream and puts it into a video processing queue.
2. The video scheduling module, as shown in figure 3,
(1) The video request sub item initiates a video resource request to the streaming media server;
(2) Calculating the distribution weight of a subordinate intelligent analysis server by using the received video stream by using an intelligent scheduling algorithm;
(3) Decoding the video stream;
(4) If the decoding is successful, distributing the decoded video stream to an intelligent analysis server with the highest distribution weight;
(5) If it fails, no processing is performed.
3. The intelligent scheduling algorithm in the video scheduling module, as shown in figure 4,
(1) Giving n video processing queue initial parameters alpha corresponding to n intelligent analysis servers according to the hardware goodness of the intelligent analysis servers 1 ...α n . Initial parameter alpha k The smaller the weight value omega k The larger, the higher the priority, k ∈ [1, n ]]；
(2) According to the designed weight formula
And calculating the weight values of the n intelligent analysis servers. Wherein alpha is k Is the initial value given in (1).
The amount of data (bit) to be processed for the kth video processing queue, n is the number of videos to be processed in the corresponding queue, X ki = video resolution bit depth fps video length. ε is a constant to ensure that the denominator is not zero. Y is k Data processing speed (b/s), Y, of the intelligent analysis server corresponding to the kth video processing queue k = amount of data of one video completed/total time used;
(3) According to the weight values omega of the n intelligent analysis servers 1 ...ω n Assigning a video source from a video scheduling module, a weight value ω k The larger, the higher the priority, k ∈ [1, n ]]And putting the video source into the video processing queue module corresponding to each intelligent analysis server. If the intelligent analysis server is idle, the pairThe video source at the head of the queue is dequeued and is put into an intelligent analysis server for processing;
(4) And (3) repeatedly executing the operations (2) and (3) to finish high-concurrency large-amount video data processing work.
4. The intelligent analysis module, as shown in figure 5,
(1) Performing video quality inspection on the head video of the video processing queue module;
(2) If not, calling the image quality enhancement algorithm for processing, and then executing (4);
(3) If yes, executing (4);
(4) Judging whether the quality-tested video needs static image detection or not;
(5) If yes, extracting the first frame image of the video stream, putting the first frame image into an image queue, and executing (7);
(6) If not, extracting a plurality of frame images of the video stream at intervals of frames, putting the frame images into an image queue, and executing (7);
(7) And calling a computer vision algorithm to process the images in the image queue.
5. The video pre-processing flow of the video quality inspection in the intelligent analysis module is shown in fig. 6
(1) Judging whether the angle of the video is abnormal (the video is intentionally rotated by a person or the holder fails, and a shot picture is not a scene needing to be monitored);
(2) If the video is abnormal, the video is not processed, and the information of the abnormal video angle is prompted;
(3) Otherwise, continuously judging whether the network is blocked;
(4) If the video is abnormal, the video is not processed, and the information of network blocking is prompted;
(5) Otherwise, continuously judging whether the lens is stained or not;
(6) If the lens is stained, calling an image restoration algorithm;
(7) Otherwise, continuously judging whether the video image is blurred;
(8) If there is picture blur, an image enhancement algorithm is invoked.
6. Data storage module
The module takes video results and picture results in an intelligent analysis module as input, writes in and reads data by using a relational database, and separates database reading operation and database writing operation by using a master-slave read-write separation method in high-concurrency processing of the database, so that the problem of high concurrency encountered when results of a plurality of intelligent analysis servers are written in the database or when a client frequently reads a large amount of database data is solved. The module consists of four sub-items, namely video storage, image storage, analysis data storage and data management. And the video storage sub-item stores the original video processed by the intelligent analysis module of each intelligent analysis server and the video processed by the algorithm into a database. And the image storage sub-item is used for storing the original image processed by each intelligent analysis server intelligent analysis module and the image processed by the algorithm into a database. And analyzing the data storage sub-items, wherein when additional development requirements are needed, an external third party can request the database and provide video analysis results and picture analysis results in the intelligent analysis module so as to meet other additional requirements. And the data management sub-item can perform basic operations such as adding, inquiring, updating and deleting on the stored pictures, videos and analysis data by using a database language, and also provides a function of exporting data regularly to meet other additional requirements.
7. Data visualization module
The module takes an intelligent analysis video result and a picture result as input, provides a preview display function of the video analysis result and the image analysis result of the intelligent analysis module, and has a function of carrying out data statistics and analysis on the video analysis result and the image analysis result in the intelligent analysis module. The module carries out statistical analysis on the video analysis result and the image analysis result through counting the structural analysis data output by the intelligent analysis module, and visually displays the intelligent analysis result in the form of using a statistical chart and a statistical table. The module consists of four sub-items, namely video preview, image preview, statistical data analysis and structured data analysis. And the video previewing sub-item displays the original video processed by the intelligent analysis module of each intelligent analysis server and the video processed by the algorithm on a data visualization platform. And (4) image previewing sub-items, and displaying the original image processed by each intelligent analysis server intelligent analysis module and the image processed by the algorithm on a data visualization platform. And the statistical data analysis sub-item carries out statistical analysis on the video analysis result and the image analysis result through counting the structural analysis data output by the intelligent analysis module, and visually displays the intelligent analysis result in the form of a statistical chart or a statistical table. The structured data analysis sub-item provides external structured analysis data for visualizing the data in multiple aspects. When additional development requirements are needed, an external third party can send a network request, and data of the picture analysis result and data of the video analysis result can be obtained to meet other additional requirements.
The present invention is not limited to the above-described examples, and various changes can be made without departing from the spirit and scope of the present invention within the knowledge of those skilled in the art.
Claims (3)
1. A high concurrency scheduling and analyzing system facing video real-time intelligent perception comprises: a data acquisition and distribution module, a video scheduling module, a video processing queue module, an intelligent analysis module, a data storage module and a data visualization module,
the data acquisition and distribution module takes a video image of a network camera as input, and the network camera, the streaming media server, the load balancing server and the distribution server perform video acquisition, video encoding and decoding, load balancing and intelligent scheduling distribution and then output the video images to a video processing queue of each intelligent analysis server, wherein the network camera uploads the acquired video to the streaming media server, carries out video encoding and compression by transmitting a real-time acquired video signal into a video encoder, and uploads the encoded and compressed video to the streaming media server; the streaming media server gathers the multiple network cameras to realize the streaming media data summarization of a local area; the load balancing server distributes and distributes, and the distribution server with shorter response time is preferentially designated as the current target distribution server by a software-level IP load balancing method; the distribution server carries out video scheduling, calculates the distribution weight of each intelligent analysis server connected at a lower level in real time by calling an intelligent scheduling algorithm, and utilizes the performance of the server to the maximum;
the video scheduling module takes the video stream forwarded by the load balancing server as input, and outputs the video to the video processing queues of the intelligent analysis servers after the video request and the video scheduling are used for sending the network request and distributing the video, wherein: the video request sends a large amount of dense network requests, and the videos of all the target streaming media servers are continuously obtained by sending the dense http network requests; video scheduling allocates each video to corresponding queues of different intelligent analysis servers, calculates priority weight of each intelligent analysis server by considering bandwidth, resolution, data amount processed by the intelligent analysis server and data amount not processed by the intelligent analysis server, compares the priority weight of each intelligent analysis server, and sends corresponding video streams to video processing queue modules of the corresponding intelligent analysis servers according to the sequence of the priority from high to low, thereby realizing intelligent scheduling of a large number of video streams; the video scheduling module has the functions of video decoding and video streaming media forwarding;
the video processing queue module transmits the video segments to a video processing queue in a designated intelligent analysis server based on the scheduling result of the video scheduling module, and all the video segments to be processed and the basic information of each video segment, including a source address, a resolution, a code rate and a frame rate, are stored in the queue;
the intelligent analysis module takes the video stream in the video processing queue module as input, performs video quality detection and pretreatment, video segment analysis and image segment analysis by video quality inspection, video analysis and image analysis, and outputs the result of computer vision algorithm analysis to the data storage module and the data visualization module, wherein the video quality inspection function is used for performing quality detection on the video stream distributed to each intelligent analysis server, thereby realizing real-time monitoring of lens stains, image blurring, abnormal visual angle and too low code stream; the video analysis and the picture analysis have the functions of performing computer vision algorithm analysis on video clips conforming to video quality inspection, and performing target detection, target identification and target tracking on videos and images in different scenes; the intelligent analysis algorithm of the intelligent analysis module provides a calling interface for other platforms or systems to call as required;
the data storage module takes video results and picture results of intelligent analysis as input, and performs original video and image storage, video and image storage after algorithm processing, result data storage after analysis and basic data management by video storage, image storage, analysis data storage and data management, wherein the video, image and analysis data storage has the functions of storing the original video and picture, the video and picture after algorithm processing and the data results after analysis; the data storage module separates database reading operation from database writing operation by using a master-slave read-write separation method for high concurrency processing of a relational database so as to solve the high concurrency problem when results of a plurality of intelligent analysis servers need to be written into the database or when a client frequently reads a large amount of database data; the data storage module also comprises a data management function, and the operations of adding, inquiring, updating, deleting and regularly exporting data are realized by using a database language;
the data visualization module takes an intelligent analysis video result and a picture result as input, performs original video, original image and video and image display after algorithm processing by video preview, image preview and statistical data analysis, and counts data results; video preview and picture preview are provided for previewing video analysis results and image analysis results in the intelligent analysis module, and videos and images are displayed in a webpage by using a webpage front-end technology; the statistical data analysis carries out data statistics and analysis on video analysis results and image analysis results in the intelligent analysis module, the video analysis results and the image analysis results are subjected to statistical analysis by using a webpage front-end technology, and the intelligent analysis results are visually displayed by using the forms of statistical graphs and statistical tables; the data visualization module has the function of providing structured data results.
2. The video-real-time intelligent perception-oriented high-concurrency scheduling and analyzing system according to claim 1, wherein: the intelligent scheduling algorithm in the video scheduling module is specifically realized as follows:
(1) Giving n video processing queue initial parameters alpha corresponding to n intelligent analysis servers according to hardware goodness degree of the intelligent analysis servers 1 ...α n (ii) a Initial parameter alpha k The smaller the weight value omega k The larger, the higher the priority, k ∈ [1, n ]]；
(2) According to the designed weight formula
Calculating the weighted values of n intelligent analysis servers, wherein alpha k Is the initial value given in step (1),
the amount of bit data to be processed for the kth video processing queue, n is the amount of video to be processed in the corresponding queue, X ki = video resolution bit depth fps video length; epsilon is a constant to ensure that the denominator is not zero; y is k Data processing speed (b/s), Y, of the intelligent analysis server corresponding to the kth video processing queue k = amount of data of one completed video/total time used;
(3) According to the weight values omega of the n intelligent analysis servers 1 ...ω n Assigning a video source from a video scheduling module, a weight value ω k The larger, the higher the priority, k ∈ [1, n ]]Putting video sources into video processing queue modules corresponding to the intelligent analysis servers; if the intelligent analysis server is idle, the video source at the head of the corresponding queue is dequeued and is placed into the intelligent analysis server for processing;
(4) And (4) repeatedly executing the operations in the steps (2) and (3) to finish high-concurrency massive video data processing work.
3. The video real-time intelligent perception-oriented high concurrency scheduling and analysis system according to claim 1, wherein: the preprocessing process of the video quality inspection function in the intelligent analysis module is as follows:
(1) Judging whether the angle of the video is abnormal or not, wherein the abnormal angle comprises intentional rotation of a person or failure of a holder, and a shot picture is not a scene needing to be monitored;
(2) If the video is abnormal, the video is not processed, and the information of the abnormal video angle is prompted;
(3) Otherwise, continuously judging whether the network is blocked;
(4) If the video is abnormal, the video is not processed, and the information of network blocking is prompted;
(5) Otherwise, continuously judging whether the lens is stained or not;
(6) If the lens is stained, calling an image restoration algorithm;
(7) Otherwise, continuously judging whether the video image is blurred;
(8) If there is picture blur, an image enhancement algorithm is invoked.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211071915.2A CN115509739A (en) | 2022-09-02 | 2022-09-02 | High-concurrency scheduling and analyzing system for real-time intelligent perception of videos |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211071915.2A CN115509739A (en) | 2022-09-02 | 2022-09-02 | High-concurrency scheduling and analyzing system for real-time intelligent perception of videos |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115509739A true CN115509739A (en) | 2022-12-23 |
Family
ID=84502595
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211071915.2A Pending CN115509739A (en) | 2022-09-02 | 2022-09-02 | High-concurrency scheduling and analyzing system for real-time intelligent perception of videos |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115509739A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117750040A (en) * | 2024-02-20 | 2024-03-22 | 浙江宇视科技有限公司 | Video service balancing method, device, equipment and medium of intelligent server cluster |
-
2022
- 2022-09-02 CN CN202211071915.2A patent/CN115509739A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117750040A (en) * | 2024-02-20 | 2024-03-22 | 浙江宇视科技有限公司 | Video service balancing method, device, equipment and medium of intelligent server cluster |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7697026B2 (en) | Pipeline architecture for analyzing multiple video streams | |
| Ali et al. | RES: Real-time video stream analytics using edge enhanced clouds | |
| JP5213105B2 (en) | Video network system and video data management method | |
| US7751683B1 (en) | Scene change marking for thumbnail extraction | |
| CN108062499B (en) | Face recognition method, system and platform | |
| Sun et al. | VU: Edge computing-enabled video usefulness detection and its application in large-scale video surveillance systems | |
| CN110505534B (en) | Monitoring video processing method, device and storage medium | |
| EP2956891A2 (en) | Segmenting objects in multimedia data | |
| JP6217888B1 (en) | Analysis device, analysis method, and program | |
| US7663661B2 (en) | Feed-customized processing of multiple video streams in a pipeline architecture | |
| CN115509739A (en) | High-concurrency scheduling and analyzing system for real-time intelligent perception of videos | |
| Kaseb et al. | Cloud resource management for image and video analysis of big data from network cameras | |
| CN114223189A (en) | Duration statistical method and device, electronic equipment and computer readable medium | |
| CN111223011A (en) | Food safety supervision method and system for catering enterprises based on video analysis | |
| CN106844727B (en) | Mass image characteristic data distributed acquisition processing and grading application system and method | |
| CN112804219A (en) | Low-delay real-time video analysis method based on edge calculation | |
| US7667732B1 (en) | Event generation and camera cluster analysis of multiple video streams in a pipeline architecture | |
| US7664183B2 (en) | Correlation processing among multiple analyzers of video streams at stages of a pipeline architecture | |
| CN113992948A (en) | Video management system, method and device based on cloud platform | |
| CN114049523A (en) | Distributed leaf occlusion recognition system and method for video surveillance camera | |
| Ierodiaconou et al. | Implementation and optimisation of a video object segmentation algorithm on an embedded DSP platform | |
| CN113836972A (en) | Security audit method, device, equipment and storage medium based on OCR | |
| CN115033457B (en) | Multi-source data real-time acquisition method and system capable of monitoring and early warning | |
| CN112188235A (en) | Media processing mode selection method and media processing method | |
| CN117424936A (en) | Video edge gateway autonomous scheduling monitoring method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |