Disclosure of Invention
In order to solve the problems, the invention provides a video analysis method and a video analysis system based on edge cloud collaborative computing in a smart pole scene, the video streams are classified according to different pressure requirements of the video streams on computing resources, and video stream forwarding threshold values of different levels are set according to the computing resources of an edge gateway and a cloud server, so that the effective collaboration of the intelligent edge gateway and the cloud server is ensured, the occupation of network bandwidth is reduced, and data congestion is not formed.
In order to achieve the purpose, the invention adopts the following technical scheme:
in a first aspect, the invention provides a video analysis method based on edge cloud collaborative computing in a smart pole scene, which includes:
acquiring video data;
dividing video data into continuous frame-extracting videos and discontinuous frame-extracting videos according to the requirements on computing resources;
and dynamically distributing the continuous frame-extracting video and the discontinuous frame-extracting video to the edge gateway and the cloud server for video analysis according to the computing resource occupancy rates of the edge gateway and the cloud server and the set threshold value rule.
As an alternative embodiment, the process of dynamic allocation includes: if the occupancy rate of the computing resources of the edge gateway is smaller than the first threshold value, the edge gateway performs video analysis processing on the continuous frame extraction video and the discontinuous frame extraction video.
As an alternative embodiment, the process of dynamic allocation includes: and if the computing resource occupancy rate of the edge gateway exceeds a first threshold value and is less than a second threshold value, forwarding the discontinuous frame-extracting video to a cloud server, performing video analysis processing by the cloud server, and feeding back an analysis result to the edge gateway.
As an alternative embodiment, the process of dynamic allocation includes: and if the occupancy rate of the computing resources of the edge gateway exceeds a second threshold and is less than a third threshold, forwarding part of the continuous frame-extracting videos to the cloud server until the occupancy rate of the computing resources of the edge gateway is recovered to the second threshold or below the first threshold.
As an alternative embodiment, the process of dynamic allocation includes: and if the computing resource occupancy rates of the edge gateway and the cloud server exceed a third threshold value at the same time, stopping receiving the new video data until the computing resource occupancy rates of the edge gateway and the cloud server are reduced to be below the third threshold value.
As an alternative embodiment, the process of acquiring video data includes: the method comprises the steps that video data under a smart bar scene are obtained, a camera device is arranged on the smart bar, and the camera device collects video data around the smart bar; and simultaneously sending the video data to the edge gateway.
As an alternative implementation mode, the discontinuous frame-extraction-type video is subjected to segmented transmission or frame skipping transmission.
In a second aspect, the present invention provides a video analysis system based on edge cloud collaborative computing in a smart bar scene, including:
a video acquisition module configured to acquire video data;
the video classification module is configured to divide the video data into continuous frame-extraction videos and discontinuous frame-extraction videos according to the demand on computing resources;
and the video distribution module is configured to dynamically distribute the continuous frame-extraction video and the discontinuous frame-extraction video to the edge gateway and the cloud server for video analysis according to the computing resource occupancy rates of the edge gateway and the cloud server and the set threshold value rule.
In a third aspect, the present invention provides an electronic device comprising a memory and a processor, and computer instructions stored on the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method of the first aspect is performed.
In a fourth aspect, the present invention provides a computer readable storage medium for storing computer instructions which, when executed by a processor, perform the method of the first aspect.
Compared with the prior art, the invention has the beneficial effects that:
the invention provides a video analysis method and a video analysis system based on edge cloud collaborative computing in a smart pole scene, wherein a method for performing video analysis at an edge gateway is adopted, a front-end camera device is not bound with an algorithm, the front-end camera device can be flexibly configured in the edge gateway, the computing resources of a cloud server are dynamically called according to monitoring of the computing resources, data congestion is not generated at the smart pole side, the computing pressure of the cloud server is effectively shared, and efficient processing and analysis of video data are realized.
According to the method, the video streams are classified according to different pressure requirements of the video streams on the computing resources, and when the computing resources of the intelligent edge gateway are in shortage, the video streams can be forwarded according to different classes, so that the computing pressure of the intelligent edge gateway and the cloud service can be relieved, and the computing capacity of the intelligent edge gateway and the cloud service can be efficiently utilized.
According to the invention, the computing resources of the intelligent edge gateway and the cloud server are monitored in real time, and a video stream scheduling processing mechanism meeting the requirements of an intelligent pole application scene is formed by setting video stream forwarding thresholds of different levels, so that the effective cooperation of the intelligent edge gateway and the cloud server is ensured, the occupation of network bandwidth is reduced, and data congestion is not formed.
The method adopts a cooperative computing mode of the edge gateway and the cloud server in the intelligent pole, and compared with a mode of directly carrying out video analysis in the camera, the method can provide stronger computing capability, flexibly configures an algorithm for the camera at the background through a service system, and better meets the requirement of a city management scene.
Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Detailed Description
The invention is further described with reference to the following figures and examples.
It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the invention. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the invention. As used herein, the singular is intended to include the plural unless the context clearly dictates otherwise, and furthermore, it should be understood that the terms "comprises" and "comprising," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
The embodiments and features of the embodiments of the present invention may be combined with each other without conflict.
Example 1
As shown in fig. 1, the present embodiment provides a video analysis method based on edge cloud collaborative computing in a smart pole scene, including:
s1: acquiring video data;
s2: dividing video data into continuous frame-extracting videos and discontinuous frame-extracting videos according to the requirements on computing resources;
s3: and dynamically distributing the continuous frame-extracting video and the discontinuous frame-extracting video to the edge gateway and the cloud server for video analysis according to the computing resource occupancy rates of the edge gateway and the cloud server and the set threshold value rule.
In this embodiment, in step S1, video data in a smart bar scene is obtained, a camera device is disposed on the smart bar, and the camera device collects video data around the smart bar;
the camera device is in communication connection with the edge gateway and is used for sending the acquired video data to the edge gateway;
the edge gateway is in communication connection with the cloud server and is used for acquiring video data and sending the video data to be analyzed to the cloud server according to the computing resource occupancy rate.
In an alternative embodiment, the communication connection is a wired communication or a wireless communication.
In this embodiment, after sending the acquired video data to the edge gateway, the video data is divided into a continuous frame-extracting video and a discontinuous frame-extracting video in the edge gateway;
in an alternative embodiment, the continuous frame-extraction video refers to a video stream that a video analysis algorithm needs to analyze each frame of a video stream, and the overall analysis result is determined by the analysis result of each frame in the video. Such as monitoring vehicle parking violation identification (continuous parking over 3 minutes), fighting identification (continuous multiple actions), personnel fall identification (continuous falling of personnel for more than 20 seconds), and the like; that is, for example, for monitoring and analyzing the falling of the person, the content of all frames within 20 consecutive seconds is required to be analyzed, and the person can be determined as the falling without standing up when falling on the ground.
The non-continuous frame extraction type video is a video stream which can generate a causal relationship conclusion only by judging the content of a certain frame in the video and without comparing and analyzing the content of the certain frame with the front frame and the rear frame of the video; such video streams can be frame-skipping (frame-dropping) analyzed or segmented to analyze the video content without affecting the analysis result. As long as a frame of judgment results exist, the result of the overall outward output of the algorithm is established. Such as face recognition, mask recognition, riding without safety helmet recognition, road garbage falling recognition, etc.
Therefore, the continuous frame-extracting video needs continuous frame-extracting analysis, and occupies higher computing resource occupancy rate and data transmission bandwidth compared with the discontinuous frame-extracting video, so that the continuous frame-extracting video has much more computing pressure on computing resources (such as CPU, GPU and memory) of video analysis equipment than the discontinuous frame-extracting video.
Therefore, in the embodiment, the continuous frame-extraction video is preferentially locally calculated on the edge gateway side, so as to save network transmission flow; and the discontinuous frame-extracting video is transmitted in a segmented mode or frame skipping mode, occupies lower network bandwidth and has lower requirements, and the network bandwidth is preferentially sent to the cloud server for calculation.
The difference between the video distribution and calculation strategy of this embodiment and the existing policy for performing edge cloud collaborative calculation in the car networking or other services is that videos collected by the car networking do not distinguish continuous or discontinuous frame extraction types, the car networking needs to analyze and calculate each frame of the video, and cannot skip frames or omit unanalyzed videos, that is, there is no classification, so there is no priority order for forwarding video streams, that is, there is no traffic distribution mechanism involved in reducing network bandwidth traffic occupancy.
In the embodiment, the video data are classified according to different requirements of the video data on computing resources, and when the edge gateway computing resources are in shortage, the video data are forwarded according to different classes, so that the computing pressure of the video data is relieved.
Meanwhile, the front-end camera device is not bound with the algorithm, the algorithm is flexibly configured in the edge gateway, cloud resources are dynamically called according to the monitoring of the edge gateway on the computing capacity of the edge gateway, and the high-efficiency processing and analysis of the video data are realized.
Specifically, in step S3, if the occupancy rate of the computing resource of the edge gateway is less than the first threshold, the intelligent edge gateway processes the continuous frame-extracting video and the discontinuous frame-extracting video, and generates the alarm information according to the processing result.
If the computing resource occupancy rate of the edge gateway exceeds a first threshold and is smaller than a second threshold, controlling the edge gateway to forward the discontinuous frame-extracting video to a cloud server so as to reduce the computing pressure of the edge gateway; and after receiving the discontinuous frame-extracting video, the cloud server performs video analysis processing and feeds back an analysis result to the edge gateway.
And if the occupancy rate of the computing resources of the edge gateway continues to increase and exceeds the second threshold and is less than the third threshold, controlling the intelligent edge gateway to forward part of the continuous frame-extracting videos to the cloud server so as to further reduce the computing pressure of the intelligent edge gateway until the occupancy rate of the computing resources is recovered to the second threshold or below the first threshold.
And if the computing resource occupancy rates of the edge gateway and the cloud server exceed a third threshold value at the same time, controlling the edge gateway to stop receiving new video data so as to relieve the overall computing pressure and avoid system congestion, and controlling the edge gateway to receive the video data to be analyzed and processed again until the overall computing resource is reduced below the third threshold value.
Continuously monitoring the computing resource occupancy rates of the edge gateway and the cloud server, and dynamically distributing the video stream from the edge gateway to the cloud server for processing; the edge gateway moves the relevant terminal equipment on the intelligent pole to link according to the analysis result of the video stream or the received video analysis result sent back by the cloud server, and if the operations such as video broadcasting, broadcast prompting, camera follow shooting and the like are carried out.
As an alternative embodiment, the values of the first threshold, the second threshold, and the third threshold are set according to actual requirements.
The embodiment provides a complete solution method and a complete solution system for video analysis and flow allocation based on edge cloud cooperative computing in a smart pole scene. The cloud server computing pressure is reduced, the network bandwidth occupation is reduced, data blockage is not formed at the edge gateway side, and the advantage of edge cloud collaborative video analysis is fully played.
Example 2
As shown in fig. 2, the present embodiment provides a video analysis system based on edge cloud collaborative computing in a smart pole scenario, integrated at an edge gateway, including:
a video acquisition module configured to acquire video data;
the video classification module is configured to divide the video data into continuous frame-extraction videos and discontinuous frame-extraction videos according to the demand on computing resources;
and the video distribution module is configured to dynamically distribute the continuous frame-extraction video and the discontinuous frame-extraction video to the edge gateway and the cloud server for video analysis according to the computing resource occupancy rates of the edge gateway and the cloud server and the set threshold value rule.
In this embodiment, the system further includes a cloud system in communication with the edge gateway, and a camera device; as shown in fig. 3, the camera device is installed on the smart stick, collects video data around the smart stick, and sends the video data to the edge gateway, and the edge gateway classifies and distributes the video data and then sends the video to be analyzed to the cloud server.
In addition, it should be noted that the modules correspond to the steps described in embodiment 1, and the modules are the same as the corresponding steps in the implementation examples and application scenarios, but are not limited to the contents disclosed in embodiment 1. It should be noted that the modules described above as part of a system may be implemented in a computer system such as a set of computer-executable instructions.
In further embodiments, there is also provided:
an electronic device comprising a memory and a processor and computer instructions stored on the memory and executed on the processor, the computer instructions when executed by the processor performing the method of embodiment 1. For brevity, no further description is provided herein.
It should be understood that in this embodiment, the processor may be a central processing unit CPU, and the processor may also be other general purpose processors, a graphics processor GPU, a digital signal processor DSP, an application specific integrated circuit ASIC, an off-the-shelf programmable gate array FPGA or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, and so on. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory may include both read-only memory and random access memory, and may provide instructions and data to the processor, and a portion of the memory may also include non-volatile random access memory. For example, the memory may also store device type information.
A computer readable storage medium storing computer instructions which, when executed by a processor, perform the method described in embodiment 1.
The method in embodiment 1 may be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor. The software modules may be located in ram, flash, rom, prom, or eprom, registers, among other storage media as is well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and completes the steps of the method in combination with hardware of the processor. To avoid repetition, it is not described in detail here.
Those of ordinary skill in the art will appreciate that the various illustrative elements, i.e., algorithm steps, described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
Although the embodiments of the present invention have been described with reference to the accompanying drawings, it is not intended to limit the scope of the present invention, and it should be understood by those skilled in the art that various modifications and variations can be made without inventive efforts by those skilled in the art based on the technical solution of the present invention.