Disclosure of Invention
Aiming at the problems, the invention provides a safety supervision visualization system based on streaming media and AI technology.
The purpose of the invention is realized by adopting the following technical scheme:
a safety supervision visualization system based on streaming media and AI technology, the safety supervision visualization system comprising: the system comprises a video data acquisition platform, a data forwarding server and a visual management terminal computer which are sequentially connected in a data communication manner;
the video data acquisition platform is arranged in the electric power operation site and used for acquiring video images of the electric power operation site and transmitting the acquired video images to the data forwarding server in real time; the video image includes: a first video image regarding a scene of the electric power work site and a second video image regarding each electric power equipment in the electric power work site;
the data forwarding server is used for caching the received video image;
the visual management terminal computer is used for acquiring a first video image from the data forwarding server and displaying the first video image through a video window; and the video window is also used for acquiring a second video image from the data forwarding server, processing the second video image, judging whether each electric power device has defects and corresponding defect types, and presenting the defect condition of each electric power device through the video window.
In an alternative embodiment, the video data collection platform comprises: the monitoring camera is arranged at a specified position in the electric power operation field, and the image sensor is arranged near each electric power device.
In an optional embodiment, the safety supervision visualization system further comprises: and the terminal control equipment is in communication connection with the visual management terminal computer and is used for sending an instruction to the visual management terminal computer so as to enable the visual management terminal computer to execute corresponding operation.
In an optional embodiment, the visualization management terminal computer is further configured to obtain weather information from a weather distribution center and present the weather information through the video window.
In an optional embodiment, the visualization management terminal computer includes: the system comprises a first video image acquisition module, a second video image acquisition module, a meteorological information acquisition module, a second video image processing module, a power equipment defect monitoring module and a video window;
the first video image acquisition module is used for acquiring a first video image from the data forwarding server and presenting the first video image through the video window;
the second video image acquisition module is used for acquiring a second video image from the data forwarding server and transmitting the second video image to the second video image processing module;
the weather information acquisition module is used for acquiring weather information from a weather release center and displaying the weather information through the video window;
the second video image processing module is used for processing the second video image to acquire characteristic data of each power device;
and the power equipment defect monitoring module is used for judging whether each power equipment has a defect and a corresponding defect type according to the characteristic data of each power equipment and the pre-stored characteristic data about the defect type of each power equipment, and presenting the judgment result through the video window.
In an optional embodiment, the second video image processing module includes an image denoising unit, an edge detection unit, an image enhancement unit and a feature extraction unit;
the image denoising unit is used for denoising the second video image;
the edge detection unit is used for carrying out edge detection and segmentation on the denoised second video image to obtain a video image only containing the power equipment;
the image enhancement unit is used for enhancing the video image obtained by segmentation;
the feature extraction unit is used for extracting feature data describing the electric power equipment in the video image from the enhanced video image.
In an optional embodiment, the denoising the second video image includes:
(1) carrying out noise detection on pixel points in the second video image to obtain a noise point set of the second video image
And a set of non-noise points { Ψ }
N;
(2) Graying the second video image, and estimating the gray value of each noise point to obtain the estimated value of the gray value of each noise point; wherein, the estimation value of the gray value of the noise point A is obtained by the following formula:
in the formula (I), the compound is shown in the specification,
is an estimated value of the gray value of the noise point A, G (A) is the gray value of the noise point A, gamma is a sliding window with the noise point A as the center, Z
Γ、W
ΓRespectively the number of non-noise points and the number of noise points in the sliding window gamma, G
z、G
wRespectively the gray value of the non-noise point z and the gray value of the noise point w in the sliding window Γ,
is the mean value of the gray values of all the non-noise points in the grayed second video image, gamma
1、γ
2Is a preset weight coefficient;
(3) and the set formed by the estimation value of the gray value of each noise point and the gray value of the non-noise point is the denoised second video image data.
The invention has the beneficial effects that: the invention provides a safety supervision visualization system based on streaming media and AI (artificial intelligence) technology, which can acquire a video image of an electric power operation site through a video data acquisition platform and transmit the video image of the electric power operation site acquired by the video data acquisition platform to a visualization management terminal computer through a data forwarding server, so that a maintainer can remotely know the operation condition of the electric power operation site and whether each electric power device has defects through the visualization management terminal computer, thereby being convenient to take effective measures in time and eliminating hidden dangers; meanwhile, a maintainer does not need to go deep into an electric power operation field to carry out manual inspection, inspection efficiency is improved, and the situations of insufficient inspection, incomplete inspection and the like caused by manual inspection are avoided.
Detailed Description
The invention is further described with reference to the following examples.
Fig. 1 shows a safety supervision visualization system based on streaming media and AI technology, which includes: the video data acquisition platform 100, the data forwarding server 200 and the visualization management terminal computer 300 are sequentially connected in data communication.
The video data acquisition platform 100 is arranged in an electric power operation site, and is used for acquiring a video image of the electric power operation site and transmitting the acquired video image to the data forwarding server 200 in real time; the video image includes: a first video image regarding a scene of the electric power work site and a second video image regarding each electric power equipment in the electric power work site;
the data forwarding server 200 is configured to cache the received video image;
the visualization management terminal computer 300 is configured to obtain a first video image from the data forwarding server 200, and present the first video image through a video window 360; the video processing module is further configured to obtain a second video image from the data forwarding server 200, process the second video image, determine whether each electrical device has a defect and a corresponding defect type, and present the defect condition of each electrical device through the video window 360.
The embodiment of the invention has the following beneficial effects: the invention provides a safety supervision visualization system based on streaming media and AI (Artificial intelligence) technology, which can acquire a video image of an electric power operation field through a video data acquisition platform 100, and transmit the video image of the electric power operation field acquired by the video data acquisition platform to a visualization management terminal computer 300 through a data forwarding server 200, so that a maintainer can remotely know the operation condition of the electric power operation field and whether each electric power device has defects through the visualization management terminal computer 300, thereby being convenient to take effective measures in time and eliminating hidden dangers; meanwhile, a maintainer does not need to go deep into an electric power operation field to carry out manual inspection, inspection efficiency is improved, and the situations of insufficient inspection, incomplete inspection and the like caused by manual inspection are avoided.
In an alternative embodiment, the video data capture platform 100 comprises: the monitoring camera is arranged at a specified position in the electric power operation field, and the image sensor is arranged near each electric power device.
In an optional embodiment, the safety supervision visualization system further comprises: and a terminal control device 400 which is connected with the visualization management terminal computer 300 in a communication way. The terminal control device 400 is configured to issue an instruction to the visualization management terminal computer 300, so that the visualization management terminal computer 300 executes a corresponding operation.
In an alternative embodiment, the visualization management terminal computer 300 is further configured to obtain weather information from a weather distribution center and to present the weather information through the video window 360.
In an alternative embodiment, the following contents may also be presented through the video window 360 of the visualization management terminal computer 300:
1) operation information: and displaying the on-site operation information which is being developed. Including the geographic location and distribution of the site operation, the specific tasks and implementation units of the site operation, etc.
2) Supervising and checking the operation information: and displaying the ongoing supervision operation information. Including supervising the geographic location and distribution of the jobs.
3) Integrating the risk information of the power grid: the current power grid risk taking effect and management and control conditions are displayed.
4) Defect information: the defect types of the current power equipment and the processing conditions of the current power equipment by a maintainer are displayed.
5) Video information: a. the video window 360 can select videos collected by the monitoring camera of the designated power operation site to view at any time; b. video window 360 may select a designated live law enforcement video for viewing at any time.
6) Photo and short video information: the photo and short video information of the field operation can be displayed, and the checking of a plurality of photos is supported.
7) And (3) statistical analysis information: background data model calculation for supporting the video window 360 is achieved, the background data model calculation comprises a field operation information analysis model, various levels of safety supervision plan analysis models, violation record analysis models, a power grid risk analysis model, a defect data analysis model, an equipment evaluation analysis model and an operation data analysis model, calculation results are displayed on a large screen in a chart form to facilitate decision analysis, and specifically displayed charts comprise operation risk statistics, operation condition statistics, operation in-place condition statistics, equipment defect condition statistics, equipment state evaluation condition statistics, violation condition statistics and the like.
In an optional embodiment, the visualization management terminal computer 300 may further integrate all the obtained data in a unified manner, and may use a customized interface to classify and process the data related to the service concerned by the security supervision departments, such as field operation, supervision plan, and defects, instead of the conventional data visualization interface, to implement the highly intuitive visualization of the content of the video window 360 in a three-dimensional graphical manner, so that the boring data becomes more flexible and has a more magnificent visual effect.
In an alternative embodiment, the visualization management terminal computer 300 includes: a first video image acquisition module 310, a second video image acquisition module 320, a meteorological information acquisition module 330, a second video image processing module 340, an electrical equipment defect monitoring module 350 and a video window 360;
the first video image obtaining module 310 is configured to obtain a first video image from the data forwarding server 200, and present the first video image through the video window 360;
the second video image obtaining module 320 is configured to obtain a second video image from the data forwarding server 200, and transmit the second video image to the second video image processing module 340;
the weather information acquiring module 330 is configured to acquire weather information from a weather distribution center and present the weather information through the video window 360;
the second video image processing module 340 is configured to process the second video image to obtain feature data of each power device;
the power device defect monitoring module 350 is configured to determine whether each power device has a defect and a corresponding defect type according to the feature data of each power device and pre-stored feature data about the defect type of each power device, and present a determination result through the video window 360.
In an alternative embodiment, referring to fig. 2, the second video image processing module 340 includes an image denoising unit 341, an edge detection unit 342, an image enhancement unit 343, and a feature extraction unit 344;
the image denoising unit 341 is configured to denoise the second video image;
the edge detection unit 342 is configured to perform edge detection on the denoised second video image and segment the second video image to obtain a video image only including the power device;
the image enhancement unit 343 is configured to perform enhancement processing on the video image obtained by segmentation;
the feature extraction unit 344 is configured to extract feature data describing the power devices in the video image from the enhanced video image.
In an optional embodiment, the denoising the second video image includes:
(1) carrying out noise detection on pixel points in the second video image to obtain a noise point set of the second video image
And a set of non-noise points { Ψ }
N;
(2) Graying the second video image, and estimating the gray value of each noise point to obtain the estimated value of the gray value of each noise point; wherein, the estimation value of the gray value of the noise point A is obtained by the following formula:
in the formula (I), the compound is shown in the specification,
is an estimated value of the gray value of the noise point A, G (A) is the gray value of the noise point A, gamma is a sliding window with the noise point A as the center, Z
Γ、W
ΓRespectively the number of non-noise points and the number of noise points in the sliding window gamma, G
z、G
wRespectively the gray value of the non-noise point z and the gray value of the noise point w in the sliding window Γ,
is the mean value of the gray values of all the non-noise points in the grayed second video image, gamma
1、γ
2Is a preset weight coefficient;
(3) and the set formed by the estimation value of the gray value of each noise point and the gray value of the non-noise point is the denoised second video image data.
Has the advantages that: the applicant innovatively proposes the above embodiment based on which noise is required to be reduced due to the influence of factors such as environment, damage of an image sensor, packet loss in a video image transmission process, and the like, so as to improve image quality of the second video image and facilitate accurate subsequent identification of various defects of the power equipment
The method has the advantages that the method can accurately estimate the gray value of the noise point due to the influence of the absolute value difference of the gray value of each noise point in the sliding window, improves the denoising effect, improves the image quality of the image, and reduces the influence caused by factors such as environment, self damage of an image sensor, packet loss in the video image transmission process and the like, so that the subsequent processing of the image is facilitated.
In an optional embodiment, the noise detection is performed on the pixel points in the second video image to obtain a noise point set of the second video image
And a set of non-noise points { Ψ }
NThe method specifically comprises the following steps:
(1) calculating the brightness value of the pixel point p and the lighting of other pixels in the neighborhood thereof by using the following formulaRelative difference of values, wherein pixel point p (x)p,yp) The calculation formula of the relative difference value with the brightness values of other pixel points in the neighborhood is as follows:
in the formula, Qq(p) is the relative difference between the brightness value of the pixel point p and the brightness value of the pixel point q in the neighborhood, L (p), L (q) are the brightness values of the pixel point p and the pixel point q, LmaxIs the maximum value, sigma, of the brightness values of other pixels in the neighborhood1、σ2For a set adjustment factor, ΘpThe method is a set formed by removing a pixel point p from a window with the size of (2B +1) × (2B +1) by taking the pixel point p as a center and remaining pixel points;
(2) arranging the obtained relative difference values of the pixel point p and the brightness values of other pixel points in the neighborhood in a descending order, calculating the accumulated value of K relative differences in the front order,
In the formula, RLK(p) is the accumulated value of the K relative difference values in the top order, Rk(p) is the K-th value after sequencing, wherein the value of K can be set according to the actual requirement;
(3) RL to be obtained
K(p) comparing the noise point with a preset noise threshold value T, if the noise point is larger than T, taking the pixel point p as a noise point, and adding the noise point into the set
Otherwise, add it to the set { Ψ }
N;
(4) Traversing all the pixel points to obtain the noise point set of the second video image
And a set of non-noise points { Ψ }
N。
Has the advantages that: if the noise reduction processing is directly performed on all the pixel points of the second video image, not only is the burden of the image denoising unit 341 increased, but also the denoising efficiency of the image denoising unit 341 is reduced, the applicant innovatively performs noise point detection on the second video image first, divides the pixel points in the second video image, and obtains a noise point set of the second video image
And a set of non-noise points { Ψ }
NTherefore, when the subsequent denoising is performed, only the denoising processing is performed on the noise point, so that the denoising efficiency of the image denoising unit 341 is improved, the burden of the image denoising unit 341 is reduced, and the service life of the image denoising unit 341 is prolonged. When dividing the pixel points in the second video image, calculating relative difference values of the pixel points and brightness values of other pixel points in the neighborhood of the pixel points, descending the obtained relative difference values, calculating an accumulated value of K relative difference values arranged in the front, comparing the accumulated value with a preset noise threshold value, and further judging whether the noise point is a non-noise point or a noise point. The value of K is set according to actual requirements, when the value of K is larger, the accumulated value of K is larger, the probability of dividing the pixel point into noise points is larger, and although noise reduction processing needs to be carried out on more pixel points to a certain extent, the noise reduction effect is improved, the image quality of the image is better improved, the image is more beneficial to follow-up processing of the image, and whether each power device has defects or not and the defect type of each power device is analyzed.
In an optional embodiment, the edge detection and segmentation are performed on the denoised second video image to obtain a video image only including the power equipment, specifically;
(1) by pixel point s (x)s,ys) Selecting a sliding window with the size of (2t +1) × (2t +1) as the center, and respectively calculating the transverse edge characteristic value and the longitudinal edge of the pixel point s by using a following formulaAn edge characteristic value, a left diagonal edge characteristic value and a right diagonal edge characteristic value;
in the formula (I), the compound is shown in the specification,
respectively representing a transverse edge characteristic value, a longitudinal edge characteristic value, a left oblique diagonal edge characteristic value and a right oblique diagonal edge characteristic value of a pixel point; g (-) is the gray value of the pixel point, and sigma is a preset Gaussian weighting coefficient;
(3) calculating a comprehensive value reflecting the edge feature based on the calculation result of the step (2) by using the following formula
If it is
If the pixel point s is an edge point, otherwise, the pixel point is a non-edge point;
(4) traversing all pixel points in the denoised second video image, and segmenting the denoised second video image according to the obtained edge detection result to obtain a video image only containing the power equipment.
Has the advantages that: in the embodiment of the present invention, the denoised second video image is subjected to edge detection and segmentation to obtain a video image only including the power device, so that only the segmented video image needs to be processed during the subsequent feature extraction, which reduces the burden of the subsequent image enhancement unit 343 and the feature extraction unit 344 on the one hand, and improves the processing efficiency of the subsequent image enhancement unit 343 and the feature extraction unit 344 on the other hand, thereby enabling faster identification of the defect condition of each power device and facilitating an overhaul personnel to overhaul the power device with defects more quickly. When the edge detection is carried out, the applicant innovatively introduces the transverse edge characteristic value, the longitudinal edge characteristic value, the left oblique diagonal edge characteristic value and the right oblique diagonal edge characteristic value of each pixel point, firstly calculates the transverse edge characteristic value, the longitudinal edge characteristic value, the left oblique diagonal edge characteristic value and the right oblique diagonal edge characteristic value of each pixel point respectively, and then obtains a comprehensive value capable of reflecting the edge characteristic of each pixel point according to the obtained transverse edge characteristic value, the longitudinal edge characteristic value, the left oblique diagonal edge characteristic value and the right oblique diagonal edge characteristic value, thereby accurately detecting whether each pixel point belongs to an edge point.
Finally, it should be noted that the above embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the protection scope of the present invention, although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions can be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.