CN110136210B - Intelligent early warning system for leakage of monitoring device - Google Patents
Intelligent early warning system for leakage of monitoring device Download PDFInfo
- Publication number
- CN110136210B CN110136210B CN201810119998.5A CN201810119998A CN110136210B CN 110136210 B CN110136210 B CN 110136210B CN 201810119998 A CN201810119998 A CN 201810119998A CN 110136210 B CN110136210 B CN 110136210B
- Authority
- CN
- China
- Prior art keywords
- leakage
- video
- video image
- module
- early warning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012806 monitoring device Methods 0.000 title claims description 15
- 238000012544 monitoring process Methods 0.000 claims abstract description 41
- 238000000034 method Methods 0.000 claims abstract description 39
- 239000007788 liquid Substances 0.000 claims abstract description 37
- 238000012545 processing Methods 0.000 claims abstract description 34
- 238000003062 neural network model Methods 0.000 claims abstract description 24
- 230000008569 process Effects 0.000 claims abstract description 18
- 238000013528 artificial neural network Methods 0.000 claims abstract description 16
- 238000012549 training Methods 0.000 claims abstract description 13
- 238000010276 construction Methods 0.000 claims abstract description 12
- 238000013524 data verification Methods 0.000 claims abstract description 9
- 238000007726 management method Methods 0.000 claims description 16
- 208000032750 Device leakage Diseases 0.000 claims description 14
- 238000012706 support-vector machine Methods 0.000 claims description 12
- 238000012795 verification Methods 0.000 claims description 12
- 238000012546 transfer Methods 0.000 claims description 10
- 238000010586 diagram Methods 0.000 claims description 9
- 238000002347 injection Methods 0.000 claims description 9
- 239000007924 injection Substances 0.000 claims description 9
- 238000013135 deep learning Methods 0.000 claims description 7
- 239000011159 matrix material Substances 0.000 claims description 6
- 230000009467 reduction Effects 0.000 claims description 6
- 230000003595 spectral effect Effects 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000003672 processing method Methods 0.000 claims description 4
- 238000002310 reflectometry Methods 0.000 claims description 4
- 230000000052 comparative effect Effects 0.000 claims description 3
- 230000002708 enhancing effect Effects 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims description 3
- 230000035515 penetration Effects 0.000 claims description 3
- 230000003044 adaptive effect Effects 0.000 claims description 2
- 230000004044 response Effects 0.000 abstract 1
- 238000005507 spraying Methods 0.000 abstract 1
- 230000006870 function Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000007792 addition Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 239000002360 explosive Substances 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
- G08B21/12—Alarms for ensuring the safety of persons responsive to undesired emission of substances, e.g. pollution alarms
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Toxicology (AREA)
- Environmental & Geological Engineering (AREA)
- Quality & Reliability (AREA)
- Business, Economics & Management (AREA)
- Emergency Management (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention provides an intelligent early warning system for monitoring leakage of a device, which realizes rapid early warning processing of automatically identifying and sending alarm information after liquid leakage occurs in the device at night and under severe weather conditions. The system comprises a video acquisition module, a video image enhancement module, a data annotation module, a model construction module, a model learning module, a data verification module and a leakage early warning module, wherein the video image enhancement module respectively processes video images under the conditions of night, rainy days and haze weather by adopting a self-adaptive enhancement algorithm based on closed operation, a fuzzy C-means algorithm and a Retinex algorithm; the data marking module divides the liquid leakage state into three types of permeation, dripping and spraying; the model construction module constructs a five-layer deep neural network model, takes supervised training to obtain characteristic parameters, and verifies and optimizes the deep neural network through the data verification module. The system also has the advantages of timely response, high intelligent degree and the like.
Description
Technical Field
The invention relates to an intelligent video monitoring system, in particular to an intelligent early warning system for monitoring device leakage based on a deep learning neural network model.
Background
Petrochemical equipment has equipment such as conveying pipelines, towers, tanks and the like for inflammable, explosive or corrosive liquid in the processes of mass production, storage and transportation, and the equipment is easy to leak in the use process, so that fire and explosion are possible, and loss is brought to enterprises. According to relevant standards, a certain amount of monitoring equipment is arranged on site by each enterprise, but aiming at massive video data, the attention of staff of a management and control platform is difficult to keep high concentration for a long time, the manual searching efficiency is low, and the judgment of the staff on the monitoring video is influenced by the influence of the intensity of on-site light, rainy and snowy weather and haze, so that the accuracy of early warning by utilizing the leakage of a video monitoring on-site device is greatly influenced.
Currently, there are many techniques for device leakage monitoring, such as CN103596824B "night pressure generating device and hydraulic brake system", TW200820940a "system and method for locating liquid leakage at the coverage of reduced pressure delivery system", etc. Most of the monitoring technologies adopt technologies such as a sensing circuit and ultrasonic waves, and with the development of a video monitoring system, the artificial intelligence video technology has a wide application prospect. How to realize the uninterrupted monitoring of the site for 24 hours and quickly and accurately identify the leakage points of the site device, in particular to the leakage points of the site with dim light (night, cloudy day or dim light), rainy day and haze, and prompt or alarm the alarm condition of the site in time so as to avoid the expansion of accidents; meanwhile, the video of the accident occurrence in the scene can be extracted, and the method has great effect on analyzing the reasons of the accident occurrence.
Disclosure of Invention
In order to solve the problem of uninterrupted real-time monitoring and accurately identifying the leakage condition of a device, especially under the conditions of dim light, rainy days and haze, the invention provides an intelligent early warning system for monitoring the leakage of the device and an intelligent early warning method for monitoring the leakage of the device, which comprises the following specific technical scheme:
an intelligent early warning system for monitoring leakage of a device comprises a video acquisition module, a video image enhancement module, a data annotation module, a model construction module, a model learning module, a data verification module and a leakage early warning module; the video acquisition module converts an input device site monitoring video into a picture or a video frame; the video image enhancement module is used for processing video images under the conditions of dim light, rain and snow weather and haze weather; the data marking module marks three liquid leakage states of permeation, drip and injection in an image by using a boundary frame according to the picture or video frame of liquid leakage by using an image marking tool; the model construction module constructs a five-layer deep neural network model; the model learning module adopts supervised training to obtain characteristic parameters of the liquid leakage state; the data verification module utilizes the identification capability of the input liquid leakage depth neural network model and optimizes the depth neural network structure; the leakage early warning module diagnoses the leakage state by using the successfully trained liquid leakage depth neural network model and issues a control command through the video control platform.
Preferably, the video image enhancement module specifically processes the video image under the condition of dim light by adopting a self-adaptive enhancement algorithm based on a closed operation to obtain an enhanced video image; processing the video image under the weather conditions of rain and snow by adopting a fuzzy C-means algorithm to obtain an enhanced video image; and processing the video image under the condition of haze weather by adopting a Retinex algorithm to obtain an enhanced video image.
It is also preferred that the adaptive enhancement algorithm based on the closed-loop algorithm processes video images in dim light conditions, where the processing results in an enhanced video image expressed as
The processing method specifically comprises the following steps:
step one: performing low-pass filtering processing on the video image under the condition of dim light to obtain a low-frequency image A (x, y) containing contour information;
step two: acquiring high-frequency information containing video image details, in particular subtracting a low-frequency image A (x, y) from an original image F (x, y), namely F (x, y) ΘA (x, y);
step three: enhancing contrast, in particular by multiplying the high-frequency image by an enhancement factor a, i.e
Step four: conditional image brightness, in particular, a factor b is added that adjusts the image brightness.
It is also preferable that the fuzzy C-means algorithm processes video images in rainy and snowy weather conditions, including the steps of:
step one: the video image is composed of a plurality of pixels, the pixels are set to be I (x, y), and the light intensity sequences I (t) =I (1), I (2), … and I (k) in the image are set, wherein t=1, 2, … and k;
step two: setting two categories of a background category and a raindrop category, and setting the average brightness of the background category as Bmenter through brightness comparison; setting the average brightness of raindrops as Rcenter;
step three: initializing a membership matrix U, so that the sum of the membership satisfied by U is 1;
step four: calculating average brightness of two pixel points, namely a background pixel point and a raindrop pixel point, wherein the average brightness is specifically bcenter=min (I), and rcenter=max (I);
step five: calculating the absolute value of the average brightness of each pixel point I (t) and two categories of the background category and the raindrop category, and setting J B ＝|I(t)-Bcenter|，J R Comparative, if J B <J R The pixel point I (t) is summarized into the background class, if J B ≥J R The pixel point I (t) is summarized into raindrops;
step six: calculation of
If the value of J is smaller than the set threshold value, the classification is finished; if the value of J is not less than the set valueCalculating a new matrix U according to the threshold value, and starting from the third step;
step seven: and (3) finishing classification, namely respectively carrying out arithmetic average on the brightness of the background class and the raindrop class, and calculating the Center brightness Center of the background class and the raindrop class.
It is further preferable that the fuzzy C-means algorithm sets C as a pixel for estimating background brightness in the process of processing the video image in the rainy or snowy weather, and sets α as a percentage of the total number k in the background class, and sets c=αbcenter+ (1- α) Rcenter.
It is also preferred that the Retinex algorithm processes video images in case of hazy weather, comprising the steps of:
step one: the atmospheric light intensity estimation, the brightness values of all pixel points of the video image are ordered according to descending order, the position of the brightest point with the brightness value reaching the first 0.1% is selected, the brightness value is compared with the original video image, and the maximum value is selected as the value of the atmospheric light intensity;
step two: calculating a transfer diagram, calculating and estimating to obtain an irradiation component, and converting a video image into a gray image to obtain a transfer diagram t (x);
step three: and solving an equation, and solving by using the atmospheric defogging model to obtain a defogging video image.
Preferably, the five-layer deep neural network model constructed by the model construction module specifically comprises a first layer adopting a sparse automatic encoder, a second layer and a third layer adopting universal automatic encoders, a fourth layer adopting a noise reduction automatic encoder, a fifth layer adopting an SVM support vector machine, input data being pictures or video frames converted by the video acquisition module, and output data being device leakage positions and liquid leakage states.
It is also preferable that the model learning module acquires characteristic parameters of the liquid leakage state including a color layout description feature, a color structure description feature, a texture description feature, an edge histogram description feature, and a spectral reflectance feature.
It is further preferable that when the leakage early warning module diagnoses that the leakage state is penetration, the video management and control platform displays general risks, alarms in the platform, automatically pops up leakage images and marks the leakage positions, and displays the leakage positions in a text state; when the leakage early warning module diagnoses that the leakage state is drip, the video management and control platform displays medium risk, alarms in the platform, automatically pops up a leakage image and marks the leakage position, and simultaneously displays the leakage position in a character state; and pushing the message to a responsible person in real time in a short message or WeChat mode; when the leakage early warning module diagnoses that the leakage state is jetting, the video management and control platform displays a major risk, the platform alarms, automatically pops up a leakage image and marks the leakage position, and simultaneously displays the leakage position in a text state; and the message is immediately pushed to a responsible person and a supervisor in a short message or WeChat mode.
In addition, preferably, the early warning method of the intelligent early warning system for monitoring leakage of the device specifically comprises the following steps:
step one: the method comprises the steps that a video is collected, and a video collection module converts a video shot by field monitoring of a device into a picture or a video frame;
step two: the method comprises the steps of carrying out sharpening processing on a video image, wherein a video image enhancement module is used for processing the video image under the condition of dim light by adopting a self-adaptive enhancement algorithm based on closed operation to obtain a sharpened video image; processing the video image under the weather conditions of rain and snow by adopting a fuzzy C-means algorithm to obtain a clear video image; processing the video image under the condition of haze weather by adopting a Retinex algorithm to obtain a clear video image;
step three: marking the video image, namely marking the obtained clear video image by using a boundary box by adopting an image marking tool, and marking three liquid leakage states of permeation, drip and injection in the boundary box;
step four: dividing the video image into a training video image and a verification video image;
step five: constructing a deep learning neural network, and constructing a five-layer deep neural network model, wherein the first layer adopts a sparse automatic encoder, the second layer and the third layer adopt universal automatic encoders, the fourth layer adopts a noise reduction automatic encoder, and the fifth layer adopts an SVM support vector machine;
step six: determining input and output data, selecting a training video image in the fourth step, inputting a picture or a video frame converted by the video acquisition module by an input layer, and outputting a result by an output layer as a device leakage position and a liquid leakage state;
step seven: training a deep learning neural network, and acquiring characteristic parameters of a liquid leakage state by using a model learning module, wherein the characteristic parameters comprise color layout description characteristics, color structure description characteristics, texture description characteristics, edge histogram description characteristics and spectral reflectivity characteristics;
step eight: adjusting the deep neural network, and adjusting a fifth layer of the deep neural network according to the characteristic parameters obtained in the step seven;
step nine: model verification, namely inputting the video image for verification in the fourth step into the deep neural network model obtained in the eighth step, and outputting a result;
step ten: model optimization, namely optimizing a deep neural network model according to the verification result of the video image in the step nine;
step eleven: and (3) leakage early warning, namely accessing a monitoring video into a video acquisition module, inputting a system identification result into a video management and control platform, and respectively early warning three liquid leakage states of permeation, drip leakage and injection by the video management and control platform.
The invention has the beneficial effects that: (1) The system automatic intelligent recognition device combining the video acquisition module, the video image enhancement module, the data annotation module, the model construction module, the model learning module, the data verification module and the leakage early warning module is utilized to leak, so that the problem that the device cannot be discovered at the first time in the leakage process of the video monitoring device is solved, and the problem of misjudgment caused by long-time concentration of personnel is avoided; (2) The video image enhancement module respectively processes video images under the conditions of night, rainy days and haze weather by using a self-adaptive enhancement algorithm, a fuzzy C-means algorithm and a Retinex algorithm based on closed operation, so that a monitoring system can automatically identify video monitoring conditions under different severe weather influences; (3) The intelligent early warning method also has a highly intelligent and automatic monitoring function, and realizes the safe and automatic monitoring of the leakage condition of the device; (4) The invention also has the advantages that the leakage early warning module can prompt or alarm the on-site monitoring condition in time, so as to avoid the expansion of accidents, and the video of the on-site accident occurrence is extracted to be used as an important basis for analyzing the cause of the accident occurrence.
Drawings
FIG. 1 is a schematic diagram of a warning device;
FIG. 2 is a flow chart of the operation of the early warning system.
Detailed Description
With reference to fig. 1 and fig. 2, the intelligent early warning system for leakage of the monitoring device and the intelligent early warning method for leakage of the monitoring device of the present invention are further specifically described.
Example 1
Referring to fig. 1, the embodiment is a detailed description of an intelligent early warning system structure and an early warning method for monitoring leakage of a device.
The intelligent early warning system for monitoring device leakage specifically comprises a video acquisition module, a video image enhancement module, a data labeling module, a model construction module, a model learning module, a data verification module and a leakage early warning module, wherein the automatic intelligent recognition device leakage is realized through the combined system of the modules, the problem that the device leakage cannot be discovered at the first time in the process of utilizing the video monitoring device leakage is solved, and the problem that judgment errors are caused by long-time concentrated attention of monitoring operators is avoided.
The video acquisition module is used for realizing real-time transmission of monitoring data between the video acquisition module and the device video monitoring, and the video acquisition module is mainly used for converting an input device field monitoring video into a picture or a video frame. The video image enhancement module processes video images under the conditions of dim light, rainy and snowy weather and haze weather, and specifically processes the video images under the conditions of dim light by adopting a self-adaptive enhancement algorithm based on closed operation to obtain enhanced video images; processing the video image under the weather conditions of rain and snow by adopting a fuzzy C-means algorithm to obtain an enhanced video image; and processing the video image under the condition of haze weather by adopting a Retinex algorithm to obtain an enhanced video image. And the data marking module respectively marks three liquid leakage states of permeation, drip and injection in the image by using a boundary box according to the picture or video frame of liquid leakage by using an image marking tool. The video monitoring conditions under different bad weather influences can be automatically identified by the aid of clear processing of the fuzzy video images, and the clear processing of the video images through image automation has important significance for analysis of leakage reasons.
The model construction module is used for constructing a five-layer deep neural network model, the five-layer deep neural network model constructed by the model construction module is specifically characterized in that a sparse automatic encoder is adopted in a first layer, a universal automatic encoder is adopted in a second layer and a third layer, a noise reduction automatic encoder is adopted in a fourth layer, an SVM (support vector machine) is adopted in a fifth layer, input data are pictures or video frames converted by the video acquisition module, and output data are device leakage positions and liquid leakage states. The model learning module acquires characteristic parameters of the liquid leakage state by adopting supervised training, and the characteristic parameters of the liquid leakage state acquired by the model learning module comprise color layout description features, color structure description features, texture description features, edge histogram description features and spectral reflectivity features. The data verification module utilizes the identification capability of the input liquid leakage deep neural network model and optimizes the deep neural network structure.
The leakage early warning module diagnoses the leakage state by using the successfully trained liquid leakage depth neural network model, issues a control command through the video control platform, adopts different treatment measures according to different leakage conditions, and informs related personnel according to the leakage position. When the leakage early warning module diagnoses that the leakage state is penetration, the video management and control platform displays the general risk, alarms in the platform, automatically pops up a leakage image and marks the leakage position, and simultaneously displays the leakage position in a text state; when the leakage early warning module diagnoses that the leakage state is drip, the video management and control platform displays medium risk, alarms in the platform, automatically pops up a leakage image and marks the leakage position, and simultaneously displays the leakage position in a character state; and pushing the message to a responsible person in real time in a short message or WeChat mode; when the leakage early warning module diagnoses that the leakage state is jetting, the video management and control platform displays a major risk, the platform alarms, automatically pops up a leakage image and marks the leakage position, and simultaneously displays the leakage position in a text state; and the message is immediately pushed to a responsible person and a supervisor in a short message or WeChat mode.
Example 2
The embodiment is further described with reference to fig. 2, which is a method for intelligent early warning of leakage of a monitoring device.
The early warning method of the intelligent early warning system for monitoring the leakage of the device comprises the following steps:
step one: the method comprises the steps that a video is collected, a video collection module converts a video shot by field monitoring of a device into a picture or a video frame, and video images are transmitted to a video image enhancement module through a data transmission line;
step two: the method comprises the steps of carrying out sharpening processing on a video image, wherein a video image enhancement module is used for processing the video image under the condition of dim light by adopting a self-adaptive enhancement algorithm based on closed operation to obtain a sharpened video image; processing the video image under the weather conditions of rain and snow by adopting a fuzzy C-means algorithm to obtain a clear video image; processing the video image under the condition of haze weather by adopting a Retinex algorithm to obtain a clear video image;
step three: marking the video image, namely marking the obtained clear video image by using a boundary box by adopting an image marking tool, and marking three liquid leakage states of permeation, drip and injection in the boundary box;
step four: dividing the video image into a training video image and a verification video image;
step five: constructing a deep learning neural network, and constructing a five-layer deep neural network model, wherein the first layer adopts a sparse automatic encoder, the second layer and the third layer adopt universal automatic encoders, the fourth layer adopts a noise reduction automatic encoder, and the fifth layer adopts an SVM support vector machine;
step six: determining input and output data, selecting a training video image in the fourth step, inputting a picture or a video frame converted by the video acquisition module by an input layer, and outputting a result by an output layer as a device leakage position and a liquid leakage state;
step seven: training a deep learning neural network, and acquiring characteristic parameters of a liquid leakage state by using a model learning module, wherein the characteristic parameters comprise color layout description characteristics, color structure description characteristics, texture description characteristics, edge histogram description characteristics and spectral reflectivity characteristics;
step eight: adjusting the deep neural network, selecting proper characteristic parameters according to the characteristic parameters obtained in the step seven, and adjusting the fifth layer of the deep neural network according to the proper characteristic parameters;
step nine: model verification, namely inputting the video image for verification in the fourth step into the deep neural network model obtained in the eighth step, and outputting a result;
step ten: model optimization, namely optimizing a deep neural network model according to the verification result of the video image in the step nine;
step eleven: and (3) leakage early warning, namely accessing a monitoring video into a video acquisition module, inputting a system identification result into a video management and control platform, and respectively early warning three liquid leakage states of permeation, drip leakage and injection by the video management and control platform.
The intelligent early warning method has the highly intelligent and automatic monitoring function, realizes the safe and automatic monitoring of the leakage condition of the device, adjusts and optimizes according to the deep neural network model, has learning capability, and improves the early warning accuracy by continuously identifying the leakage condition of the device.
Example 3
The embodiment is a further detailed description of a video image processing method in the intelligent early warning method for leakage of the monitoring device in embodiment 2.
In the sharpness processing video image, for the video image under the condition of dim light processed by the self-adaptive enhancement algorithm based on the closed operation, the expression of the enhanced video image obtained by processing is
The processing method specifically comprises the following steps:
step one: performing low-pass filtering processing on the video image under the condition of dim light to obtain a low-frequency image A (x, y) containing contour information;
step two: acquiring high-frequency information containing video image details, in particular subtracting a low-frequency image A (x, y) from an original image F (x, y), namely F (x, y) ΘA (x, y);
step three: enhancing contrast, in particular by multiplying the high-frequency image by an enhancement factor a, i.e
Step four: conditional image brightness, in particular, a factor b is added that adjusts the image brightness.
In the process of sharpening a video image, for the video image under the condition of rain and snow, the fuzzy C-means algorithm is used for processing the video image under the condition of rain and snow, wherein a pixel estimated by background brightness is set as C, the percentage of the number in the background class accounting for the total number k is set as alpha, and C=alpha Bcentre+ (1-alpha) Rcenter is set, and the method comprises the following steps:
step one: the video image is composed of a plurality of pixels, the pixels are set to be I (x, y), and the light intensity sequences I (t) =I (1), I (2), … and I (k) in the image are set, wherein t=1, 2, … and k;
step two: setting two categories of a background category and a raindrop category, and setting the average brightness of the background category as Bmenter through brightness comparison; setting the average brightness of raindrops as Rcenter;
step three: initializing a membership matrix U, so that the sum of the membership satisfied by U is 1;
step four: calculating average brightness of two pixel points, namely a background pixel point and a raindrop pixel point, wherein the average brightness is specifically bcenter=min (I), and rcenter=max (I);
step five: calculating the absolute value of the average brightness of each pixel point I (t) and two categories of the background category and the raindrop category, and setting J B ＝|I(t)-Bcenter|，J R Comparative, if J B <J R The pixel point I (t) is summarized into the background class, if J B ≥J R The pixel point I (t) is summarized into raindrops;
step six: calculation of
If the value of J is smaller than the set threshold value, the classification is finished; if the value of J is not smaller than the set threshold value, calculating a new matrix U, and starting from the step three;
step seven: and (3) finishing classification, namely respectively carrying out arithmetic average on the brightness of the background class and the raindrop class, and calculating the Center brightness Center of the background class and the raindrop class.
In the process of sharpening the video image, the Retinex algorithm is used for processing the video image under the condition of haze weather, and the method specifically comprises the following steps of:
step one: the atmospheric light intensity estimation, the brightness values of all pixel points of the video image are ordered according to descending order, the position of the brightest point with the brightness value reaching the first 0.1% is selected, the brightness value is compared with the original video image, and the maximum value is selected as the value of the atmospheric light intensity A;
step two: calculating a transfer diagram, calculating and estimating to obtain an irradiation component, and converting a video image into a gray image to obtain a transfer diagram t (x);
step three: and solving an equation, and solving by using the atmospheric defogging model to obtain a defogging video image.
Wherein the expression of the atmospheric defogging model processed by the Retinex algorithm is F (x, y) =I (x, y) R (x, y), and F (x, y) represents an original video image; i (x, y) is represented as a light component, i.e., a transfer map t (x) of the atmospheric transfer model; r (x, y) represents the reflected component, including detail information in the video image.
The expression for solving the transfer diagram t (x) is:
wherein t (x) is a transfer diagram, and 1- ρ (x) represents an inverse albedo C (x); wherein, the calculation method of the albedo C (x) is C (x) =L (x, y) =I O (x, y) G, where L (x, y) is the illumination component to be estimated, I O (x, y) is a gray scale image into which the original video image I (x, y) is convertedLike, G is a gaussian function, which represents convolution.
The Gaussian function G is defined as follows
Wherein->
The defogging image J (x) has the expression:
it should be understood that the above description is not intended to limit the invention to the particular embodiments disclosed, but to limit the invention to the particular embodiments disclosed, and that the invention is not limited to the particular embodiments disclosed, but is intended to cover modifications, adaptations, additions and alternatives falling within the spirit and scope of the invention.
Claims (10)
1. The intelligent early warning system for monitoring leakage of the device is characterized by comprising a video acquisition module, a video image enhancement module, a data annotation module, a model construction module, a model learning module, a data verification module and a leakage early warning module;
the video acquisition module converts an input device site monitoring video into a picture or a video frame;
the video image enhancement module is used for processing video images under the conditions of dim light, rain and snow weather and haze weather;
the data marking module marks three liquid leakage states of permeation, drip and injection in an image by using a boundary frame according to the picture or video frame of liquid leakage by using an image marking tool;
the model construction module constructs a five-layer deep neural network model;
the model learning module adopts supervised training to obtain characteristic parameters of the liquid leakage state;
the data verification module utilizes the identification capability of the input liquid leakage depth neural network model and optimizes the depth neural network structure;
the leakage early warning module diagnoses the leakage state by using the successfully trained liquid leakage depth neural network model and issues a control command through the video control platform.
2. The intelligent early warning system for monitoring leakage of a device according to claim 1, wherein the video image enhancement module is specifically configured to obtain an enhanced video image by processing a video image under a condition of dim light by adopting a self-adaptive enhancement algorithm based on a closed operation; processing the video image under the weather conditions of rain and snow by adopting a fuzzy C-means algorithm to obtain an enhanced video image; and processing the video image under the condition of haze weather by adopting a Retinex algorithm to obtain an enhanced video image.
3. The intelligent early warning system for leakage of monitoring device according to claim 2, wherein the adaptive enhancement algorithm based on the closed operation processes the video image in the case of dim light, and the expression of the processed enhanced video image is that
The processing method specifically comprises the following steps:
step one: performing low-pass filtering processing on the video image under the condition of dim light to obtain a low-frequency image A (x, y) containing contour information;
step two: acquiring high-frequency information containing video image details, in particular subtracting a low-frequency image A (x, y) from an original image F (x, y), namely F (x, y) ΘA (x, y);
step three: enhancing contrast, in particular by multiplying the high-frequency image by an enhancement factor a, i.e
Step four: the brightness of the image is adjusted, in particular by adding a factor b that adjusts the brightness of the image.
4. The intelligent early warning system for monitoring device leakage according to claim 2, wherein the fuzzy C-means algorithm processes video images in rainy and snowy weather conditions comprising the steps of:
step one: the video image is composed of a plurality of pixels, the pixels are set to be I (x, y), and the light intensity sequences I (t) =I (1), I (2), … and I (k) in the image are set, wherein t=1, 2, … and k;
step two: setting two categories of a background category and a raindrop category, and setting the average brightness of the background category as Bmenter through brightness comparison; setting the average brightness of raindrops as Rcenter;
step three: initializing a membership matrix U, so that the sum of the membership satisfied by U is 1;
step four: calculating average brightness of two pixel points, namely a background pixel point and a raindrop pixel point, wherein the average brightness is specifically bcenter=min (I), and rcenter=max (I);
step five: calculating the absolute value of the average brightness of each pixel point I (t) and two categories of the background category and the raindrop category, and setting J B ＝|I(t)-Bcenter|，J R Comparative, if J B <J R The pixel point I (t) is summarized into the background class, if J B ≥J R The pixel point I (t) is summarized into raindrops;
step six: calculation of
If the value of J is smaller than the set threshold value, the classification is finished; if the value of J is not smaller than the set threshold value, calculating a new matrix U, and starting from the step three; wherein the Center value is the average background brightness Bcenter or the average raindrop brightness Rcenter;
step seven: and (3) finishing classification, respectively carrying out arithmetic average on the brightness of the background class and the raindrop class, and calculating the central brightness of the background class and the raindrop class.
5. The intelligent early warning system for leakage of monitoring apparatus according to claim 4, wherein the pixel for estimating background brightness is set as C, the percentage of the number in the background class to the total number k is set as α, and c=αbcenter+ (1- α) Rcenter is set during the processing of the video image in the case of rain and snow weather by the fuzzy C-means algorithm.
6. The intelligent early warning system for monitoring device leakage according to claim 2, wherein the Retinex algorithm processes video images in case of hazy weather, comprising the steps of:
step one: the atmospheric light intensity estimation, the brightness values of all pixel points of the video image are ordered according to descending order, the position of the brightest point with the brightness value reaching the first 0.1% is selected, the brightness value is compared with the original video image, and the maximum value is selected as the value of the atmospheric light intensity;
step two: calculating a transfer diagram, calculating and estimating to obtain an irradiation component, and converting a video image into a gray image to obtain a transfer diagram t (x);
step three: and solving an equation, and solving by using the atmospheric defogging model to obtain a defogging video image.
7. The intelligent early warning system for monitoring leakage of a device according to claim 1, wherein the five-layer deep neural network model constructed by the model construction module is characterized in that a sparse automatic encoder is adopted in a first layer, a universal automatic encoder is adopted in a second layer and a third layer, a noise reduction automatic encoder is adopted in a fourth layer, an SVM (support vector machine) is adopted in a fifth layer, input data are pictures or video frames converted by the video acquisition module, and output data are device leakage positions and liquid leakage states.
8. The intelligent early warning system for leakage of a monitoring device of claim 1, wherein the model learning module obtains characteristic parameters of the liquid leakage state including a color layout description feature, a color structure description feature, a texture description feature, an edge histogram description feature, and a spectral reflectance feature.
9. The intelligent warning system for leakage of monitoring apparatus according to claim 2, wherein,
when the leakage early warning module diagnoses that the leakage state is penetration, the video management and control platform displays general risks, alarms in the platform, automatically pops up leakage images and marks the leakage positions, and displays the leakage positions in a text state;
when the leakage early warning module diagnoses that the leakage state is drip, the video management and control platform displays medium risk, alarms in the platform, automatically pops up a leakage image and marks the leakage position, and simultaneously displays the leakage position in a character state; and pushing the message to a responsible person in real time in a short message or WeChat mode;
when the leakage early warning module diagnoses that the leakage state is jetting, the video management and control platform displays a major risk, the platform alarms, automatically pops up a leakage image and marks the leakage position, and simultaneously displays the leakage position in a text state; and the message is immediately pushed to a responsible person and a supervisor in a short message or WeChat mode.
10. The intelligent early warning system for leakage of monitoring device according to any one of claims 1 to 8, wherein the early warning method specifically comprises:
step one: the method comprises the steps that a video is collected, and a video collection module converts a video shot by field monitoring of a device into a picture or a video frame;
step two: the method comprises the steps of carrying out sharpening processing on a video image, wherein a video image enhancement module is used for processing the video image under the condition of dim light by adopting a self-adaptive enhancement algorithm based on closed operation to obtain a sharpened video image; processing the video image under the weather conditions of rain and snow by adopting a fuzzy C-means algorithm to obtain a clear video image; processing the video image under the condition of haze weather by adopting a Retinex algorithm to obtain a clear video image;
step three: marking the video image, namely marking the obtained clear video image by using a boundary box by adopting an image marking tool, and marking three liquid leakage states of permeation, drip and injection in the boundary box;
step four: dividing the video image into a training video image and a verification video image;
step five: constructing a deep learning neural network, and constructing a five-layer deep neural network model, wherein the first layer adopts a sparse automatic encoder, the second layer and the third layer adopt universal automatic encoders, the fourth layer adopts a noise reduction automatic encoder, and the fifth layer adopts an SVM support vector machine;
step six: determining input and output data, selecting a training video image in the fourth step, inputting a picture or a video frame converted by the video acquisition module by an input layer, and outputting a result by an output layer as a device leakage position and a liquid leakage state;
step seven: training a deep learning neural network, and acquiring characteristic parameters of a liquid leakage state by using a model learning module, wherein the characteristic parameters comprise color layout description characteristics, color structure description characteristics, texture description characteristics, edge histogram description characteristics and spectral reflectivity characteristics;
step eight: adjusting the deep neural network, and adjusting a fifth layer of the deep neural network according to the characteristic parameters obtained in the step seven;
step nine: model verification, namely inputting the video image for verification in the fourth step into the deep neural network model obtained in the eighth step, and outputting a result;
step ten: model optimization, namely optimizing a deep neural network model according to the verification result of the video image in the step nine;
step eleven: and (3) leakage early warning, namely accessing a monitoring video into a video acquisition module, inputting a system identification result into a video management and control platform, and respectively early warning three liquid leakage states of permeation, drip leakage and injection by the video management and control platform.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810119998.5A CN110136210B (en) | 2018-02-07 | 2018-02-07 | Intelligent early warning system for leakage of monitoring device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810119998.5A CN110136210B (en) | 2018-02-07 | 2018-02-07 | Intelligent early warning system for leakage of monitoring device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110136210A CN110136210A (en) | 2019-08-16 |
| CN110136210B true CN110136210B (en) | 2023-06-09 |
Family
ID=67567446
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810119998.5A Active CN110136210B (en) | 2018-02-07 | 2018-02-07 | Intelligent early warning system for leakage of monitoring device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110136210B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110470669B (en) * | 2019-08-23 | 2020-05-26 | 吉林大学 | Leakage detection method and system for underwater pipeline and related device |
| CN111327790B (en) * | 2020-03-27 | 2022-02-08 | 武汉烛照科技有限公司 | Video processing chip |
| CN111696320B (en) * | 2020-06-15 | 2021-11-26 | 中国石油大学(华东) | Intelligent visual early warning system for early gas leakage |
| CN112686864A (en) * | 2020-12-30 | 2021-04-20 | 中山嘉明电力有限公司 | Liquid and gas leakage and overflow identification method and system based on image identification |
| CN114436086B (en) * | 2022-01-24 | 2023-09-19 | 浙江新再灵科技股份有限公司 | Method and system for identifying and alarming darkness of elevator car |
| CN116259012B (en) * | 2023-05-16 | 2023-07-28 | 新疆克拉玛依市荣昌有限责任公司 | Monitoring system and method for embedded supercharged diesel tank |
| CN116402823B (en) * | 2023-06-09 | 2023-08-04 | 深圳市思拓通信系统有限公司 | Road surface garbage drip detection method based on image enhancement |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2004044683A2 (en) * | 2002-11-06 | 2004-05-27 | Simmonds Precision Products, Inc. | Method for detection and recognition of fog presence within an aircraft compartment using video images |
| CN101008992A (en) * | 2006-12-30 | 2007-08-01 | 北京市劳动保护科学研究所 | Method for detecting leakage of pipeline based on artificial neural network |
| CN106296612A (en) * | 2016-08-09 | 2017-01-04 | 南京工业大学 | The stagewise monitor video sharpening system and method that a kind of image quality evaluation and weather conditions guide |
| CN107061996A (en) * | 2017-03-10 | 2017-08-18 | 内蒙古大学 | A kind of water supply line leakage detecting and locating method |
-
2018
- 2018-02-07 CN CN201810119998.5A patent/CN110136210B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2004044683A2 (en) * | 2002-11-06 | 2004-05-27 | Simmonds Precision Products, Inc. | Method for detection and recognition of fog presence within an aircraft compartment using video images |
| CN101008992A (en) * | 2006-12-30 | 2007-08-01 | 北京市劳动保护科学研究所 | Method for detecting leakage of pipeline based on artificial neural network |
| CN106296612A (en) * | 2016-08-09 | 2017-01-04 | 南京工业大学 | The stagewise monitor video sharpening system and method that a kind of image quality evaluation and weather conditions guide |
| CN107061996A (en) * | 2017-03-10 | 2017-08-18 | 内蒙古大学 | A kind of water supply line leakage detecting and locating method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110136210A (en) | 2019-08-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110136210B (en) | Intelligent early warning system for leakage of monitoring device | |
| CN111553929B (en) | Mobile phone screen defect segmentation method, device and equipment based on converged network | |
| US10628890B2 (en) | Visual analytics based vehicle insurance anti-fraud detection | |
| CN101430195B (en) | Method for computing electric power line ice-covering thickness by using video image processing technology | |
| CN104217404A (en) | Video image sharpness processing method in fog and haze day and device thereof | |
| CN110378865A (en) | A kind of greasy weather visibility intelligence hierarchical identification method and system under complex background | |
| CN110060508B (en) | Automatic ship detection method for inland river bridge area | |
| CN111598098B (en) | Water gauge water line detection and effectiveness identification method based on full convolution neural network | |
| CN110108704A (en) | A kind of automatic monitoring and pre-alarming method of cyanobacteria and its automatic monitoring and alarming system | |
| CN113269040A (en) | Driving environment sensing method combining image recognition and laser radar point cloud segmentation | |
| US20230222643A1 (en) | Semantic deep learning and rule optimization for surface corrosion detection and evaluation | |
| CN112365468A (en) | AA-gate-Unet-based offshore wind power tower coating defect detection method | |
| CN111310592B (en) | Detection method based on scene analysis and deep learning | |
| CN112258490A (en) | Low-emissivity coating intelligent damage detection method based on optical and infrared image fusion | |
| CN112284619B (en) | Oil leakage real-time monitoring device and detection method | |
| CN111833369A (en) | Alum image processing method, system, medium and electronic device | |
| CN115311623A (en) | Equipment oil leakage detection method and system based on infrared thermal imaging | |
| CN111598793A (en) | Method and system for defogging image of power transmission line and storage medium | |
| CN112883969B (en) | Rainfall intensity detection method based on convolutional neural network | |
| CN113516652A (en) | Battery surface defect and adhesive detection method, device, medium and electronic equipment | |
| CN116229404A (en) | Image defogging optimization method based on distance sensor | |
| CN115578693A (en) | Construction safety early warning method and device based on significance neural network model | |
| CN114998801A (en) | Forest fire smoke video detection method based on contrast self-supervision learning network | |
| CN113781388A (en) | Image enhancement-based power transmission line channel hidden danger image identification method and device | |
| CN112115767A (en) | Tunnel foreign matter detection method based on Retinex and YOLOv3 models |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20230515 Address after: No. 22, Chaoyangmen street, Chaoyang District, Beijing 100020 Applicant after: CHINA PETROLEUM & CHEMICAL Corp. Applicant after: Sinopec Safety Engineering Research Institute Co.,Ltd. Applicant after: Sinopec National Petrochemical Project Risk Assessment Technology Center Co.,Ltd. Address before: Yanan City, Shandong province Qingdao City three road 266071 No. 218 Applicant before: CHINA PETROLEUM & CHEMICAL Corp. Applicant before: SINOPEC Research Institute OF SAFETY ENGINEERING |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |