CN113343923A - Real-time river drainage port drainage state identification method based on video images - Google Patents
Real-time river drainage port drainage state identification method based on video images Download PDFInfo
- Publication number
- CN113343923A CN113343923A CN202110746229.XA CN202110746229A CN113343923A CN 113343923 A CN113343923 A CN 113343923A CN 202110746229 A CN202110746229 A CN 202110746229A CN 113343923 A CN113343923 A CN 113343923A
- Authority
- CN
- China
- Prior art keywords
- drainage
- state
- motion vector
- image
- deep learning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 38
- 239000013598 vector Substances 0.000 claims abstract description 80
- 238000013136 deep learning model Methods 0.000 claims abstract description 45
- 238000012549 training Methods 0.000 claims abstract description 18
- 238000002372 labelling Methods 0.000 claims abstract description 10
- 238000001914 filtration Methods 0.000 claims abstract description 8
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 claims description 32
- 238000013527 convolutional neural network Methods 0.000 claims description 7
- 238000004422 calculation algorithm Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 3
- 230000006835 compression Effects 0.000 claims description 3
- 238000007906 compression Methods 0.000 claims description 3
- 238000004445 quantitative analysis Methods 0.000 claims description 3
- 238000012544 monitoring process Methods 0.000 abstract description 5
- 230000009286 beneficial effect Effects 0.000 abstract description 4
- 238000013135 deep learning Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 3
- 238000005286 illumination Methods 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 239000010865 sewage Substances 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 230000005856 abnormality Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 239000007788 liquid Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000010223 real-time analysis Methods 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 231100000331 toxic Toxicity 0.000 description 1
- 230000002588 toxic effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
- G06F30/27—Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2113/00—Details relating to the application field
- G06F2113/08—Fluids
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Medical Informatics (AREA)
- Computer Hardware Design (AREA)
- Geometry (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a real-time recognition method of drainage state of a drainage port of a river channel based on video images, which comprises the following steps of firstly establishing a deep learning model of the drainage state, wherein the step of establishing the deep learning model of the drainage state comprises the following steps: collecting drainage and non-drainage state videos of a drainage port of a river channel; splitting a video into a frame of image; calculating to obtain a motion vector of each frame of image; labeling the drainage state label information of the motion vector; establishing a drainage state deep learning model; and training a drainage state deep learning model. And then acquiring a real-time video stream of a river drainage port through a camera, splitting the video stream into a frame of image, calculating to obtain a motion vector of each frame of image, analyzing the motion vector of each frame of image by using the established drainage state deep learning model to output a preliminary drainage state result, analyzing and filtering the preliminary result, and outputting final drainage state information. The invention realizes the real-time unmanned monitoring and automatic identification of the drainage state of the drainage port of the river channel, is beneficial to the timely discovery of problems of related parts, eliminates hidden dangers and solves the problem of river channel pollution from the source.
Description
Technical Field
The invention relates to the field of intelligent water affairs, in particular to a real-time river discharge port drainage state identification method based on video images, which is suitable for real-time video identification of river discharge port running water.
Background
With the development of society, especially the progress of science and technology, the rapid development of social productivity is greatly promoted; especially, the widespread application of communication network technology makes most fields develop from traditional artificial statistical analysis to intelligence.
The river discharge outlet flow identification technology is not mature at present and can be realized through a video monitoring technology, if abnormal drainage exists at a discharge outlet, the field situation can be conveniently called and checked, but the video monitoring has the disadvantage that the abnormality can be found only by staring at a person. Therefore, research of an automatic river discharge outlet flow identification method is imperative to improve the intelligent river discharge outlet flow identification precision and efficiency.
In recent years, Deep Learning (Deep Learning) has made an important and successful breakthrough in the field of artificial intelligence, and has become a new and popular research direction for machine Learning, and has strong Learning and efficient feature expression capability, and has made great success in many fields such as computer vision, image and video analysis, voice recognition, multimedia and the like. The automatic river discharge outlet flow identification method based on the video images has high identification precision and high video stream data analysis efficiency.
In the prior art, the drainage condition of a river channel drainage port is complex, for example, the drainage port is different in shape and can be round, square, polygonal and the like; the water flow at the discharge port is greatly changed, sometimes the water flow is very small, and a large error exists when the measurement is carried out by a liquid level meter; the color of the discharged water flow changes greatly and is sometimes mixed with moss at the discharge port into a color; therefore, the method for measuring the drainage state of the drainage port and further controlling the problem of stealing sewage at the drainage port of the river channel, particularly the problem of small-amount discharge of toxic sewage is a problem which needs to be solved urgently.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides a method for identifying the drainage state of a drainage port of a river channel based on a video image, which is characterized in that the analysis of the drainage video data of the drainage port of the river channel is realized by taking a deep learning technology as a core, a deep convolutional neural network is adopted for training, various scene data are added into the data set, a judgment model of the drainage port and the drainage scene is obtained after the training, and the real-time identification is realized and the abnormal drainage alarm information with time sequence property is given out by the quasi real-time analysis of the video stream.
The purpose of the invention is realized as follows:
a method for identifying drainage state of a drainage port of a river channel in real time based on video images comprises the following steps of firstly establishing a drainage state deep learning model, and establishing the drainage state deep learning model: collecting drainage and non-drainage state videos of a drainage port of a river channel; splitting a video into a frame of image; calculating to obtain a motion vector of each frame of image; labeling the drainage state label information of the motion vector; establishing a drainage state deep learning model; training a drainage state deep learning model; then, acquiring a real-time video stream of a river drainage port through a camera, splitting the video stream into a frame of image, calculating to obtain a motion vector of each frame of image, analyzing the motion vector of each frame of image by using an established drainage state deep learning model to output a preliminary drainage state result, analyzing and filtering the preliminary result, and outputting final drainage state information; the method comprises the following specific steps:
(1) acquiring a real-time video stream of a river channel discharge port through a camera;
(2) splitting a video stream into a frame of image;
(3) calculating to obtain a motion vector of each frame of image;
(4) analyzing the motion vector of each frame of image by using the established drainage state deep learning model and outputting a preliminary drainage state result;
(5) analyzing the result of the preliminary drainage state of the drainage outlet given by the filtering model;
the method for establishing the drainage model of the drainage port of the river channel and setting the initial conditions comprises the following steps:
establishing drainage model of river channel drainage port
ut+uux＝0
To describe the drain state;
t represents time, x represents water surface position, and u represents velocity wave; x is 0 and represents the position of water which will be discharged from the pipe orifice but not discharged from the pipe orifice, when x is less than 0, the speed of water is lower than that after being discharged from the pipe, so if u is 0, after being discharged from the pipe, the speed is suddenly increased, the water flow is smoothly discharged through fall, and if the average speed is v0Thus, the initial condition of equation (1) can be defined as
Let δ be the minimum drainage velocity, so when velocity v is0>δ>At 0, we can determine the drainage and carry out the subsequent quantitative analysis including the flow;
the solution of equation (1) is in the region { x }>0,t>x, the speed of the flowing water at the discharge port is
The change of the discharge speed along with time is illustrated; the change of the water discharge state can be described by a motion vector, and the water discharge state of the water discharge port can be obtained by finding the motion vector; a motion vector (motion vector) is a difference between two frames of images stored in the image compression, and is a vector for describing the change of the spatial position of an object;
v calculation method of motion vector of imageThe method comprises the following steps: draining two adjacent frames of gray images (g) by using drain outleti，gj) To define a gray scale map of the motion vector as
gv＝gi-gj，
The motion vector v is a one-dimensional expansion of its grayscale map; the motion vector can be further improved to be a plurality of adjacent motion vectors viIs equal to Σ αivi(ii) a Alternatively, v can be determined by a correlation algorithm, such as the OptionalFlowFarneback methodiAnd using v ═ Σ αiviObtaining a motion vector by weighted average; the above weighted average may also be an arithmetic average or a gaussian weighted average;
the method for establishing the drainage state deep learning model comprises the following specific steps:
(1) collecting a large amount of drainage videos and non-drainage videos;
(2) calculating a motion vector of each frame of image in a video to form a motion vector gray image; the motion vector can be determined according to the difference value of the two frames of images before and after or according to an algorithm including related OptionalFlowFarneback;
(3) labeling drainage state label information for each motion vector gray level graph, wherein the label value is the state of the water flow corresponding to the motion vector, and the specific labeling method comprises the following steps: the label information types are divided into a no-flow state, a low-speed flow state, a medium-speed flow state and a high-speed flow state, the specific flow speed values of the low-speed flow state, the medium-speed flow state and the high-speed flow state are actually adjusted according to specific conditions, and generally 0 m/s, 1 m/s, 2 m/s and 5 m/s can be adopted;
(4) establishing a drainage state deep learning model, training and establishing the drainage state deep learning model by combining a plurality of convolution maps and a plurality of full-connection maps, and setting a convolution layer and a full-connection layer of the model according to conditions;
(5) training a model: by adopting a conventional CNN convolutional neural network training method, the recognition accuracy reaches over 99% during convergence, and the training learning rate is 0.0001.
Has the positive and beneficial effects that: 1. the invention realizes the real-time unmanned monitoring and automatic identification of the drainage state of the drainage port of the river channel, is beneficial to the timely discovery of problems of related parts, eliminates hidden dangers and solves the problem of river channel pollution from the source.
2. In order to filter the preliminary result judged by the deep learning model of the drainage state, the preliminary judgment result of the deep learning model of the drainage state is filtered, analyzed and processed due to the complex environment of a drainage port, poor quality of a drainage image and high noise interference, and the condition that the drainage quantity suddenly changes unreasonably is filtered according to the current condition so as to improve the recognition rate of practical application. For example, if a frame or several consecutive frames is determined to be drained but a large number of adjacent frames are determined to be drained, the preliminary result is considered to be caused by noise interference of the pictures, and finally the drainage is not reported.
3. Judging the drainage state by using a drainage state deep learning model, and judging the average velocity v of the water flow at the drainage outlet0Due to the influence of factors including illumination, environment, weather change, brightness and day and night interference, the extraction in the video is difficult, and because of the video image, the precision of directly judging the drainage state of the drainage port by using a motion vector calculated by the image is not high; the drain pipeline shape also causes large difference of the motion vector gray-scale map, so that the pipeline state is difficult to be directly judged through the motion vector gray-scale map; the drainage state deep learning model is established by using a deep learning method, and whether drainage is performed or not is judged by using the model, so that the drainage state identification precision is improved, and the robustness is enhanced.
Drawings
FIG. 1 is a schematic diagram illustrating a real-time river drainage state identification step based on video images according to the present invention;
FIG. 2 is a schematic diagram of a drainage state deep learning model building procedure according to the present invention;
FIG. 3 is an illustration of a port of the present invention;
FIG. 4 is a schematic diagram of an analytic solution of a bezel model according to the present invention;
FIG. 5 is a motion vector grayscale diagram of a bezel image according to the present invention;
FIG. 6 motion vector grayscale map to motion vector
FIG. 7 is a schematic diagram of a drainage state deep learning model architecture according to the present invention.
Detailed Description
The following detailed description is made in conjunction with the accompanying drawings and detailed description:
a real-time recognition method for drainage state of a drainage port of a river channel based on video images comprises the following steps of firstly establishing a deep learning model of the drainage state, wherein the step of establishing the deep learning model of the drainage state comprises the following steps: collecting drainage and non-drainage state videos of a drainage port of a river channel; splitting a video into a frame of image; calculating to obtain a motion vector of each frame of image; labeling the drainage state label information of the motion vector; establishing a drainage state deep learning model; and training a drainage state deep learning model. And then acquiring a real-time video stream of a river drainage port through a camera, splitting the video stream into a frame of image, calculating to obtain a motion vector of each frame of image, analyzing the motion vector of each frame of image by using the established drainage state deep learning model to output a preliminary drainage state result, analyzing and filtering the preliminary result, and outputting final drainage state information.
A drainage model of a river channel drainage port is established and initial conditions are set, and the method comprises the following steps:
establishing drainage model of river channel drainage port
ut+uux＝0……………(1)
To describe the drain state, as shown in fig. 3;
in equation (1), t represents time, x represents water surface position, and u represents velocity wave
x is 0 and represents the position of water which will be discharged from the pipe orifice but not discharged from the pipe orifice, when x is less than 0, the speed of water is lower than that after being discharged from the pipe, so if u is 0, after being discharged from the pipe, the speed is suddenly increased, the water flow is smoothly discharged through fall, and if the average speed is v0Thus, the initial condition of equation (1) can be defined as
Let δ be the minimum drainage velocity, so when velocity v is0>δ>At 0 timeWe can determine the drainage and perform subsequent quantitative analysis, such as flow rate, etc.
The solution of equation (1) is shown in FIG. 4 in the region { x }>0,t>x, the speed of the flowing water at the discharge port is
Illustrating the discharge velocity over time. The change of the drainage state can be described by a motion vector, and the drainage state of the drainage port can be obtained by finding the motion vector. A motion vector (motionvector) is a difference between storing two frames of images in graphics compression, and is a vector describing a change in the spatial position of an object.
The motion vector v of the image is calculated as follows:
draining two adjacent frames of gray images (g) by using drain outleti，gj) The gray scale map (see fig. 5) of the motion vector is defined by the difference values of
gv＝gi-gj，
The motion vector v is a one-dimensional expansion of its grey scale map (see fig. 6). The motion vector can be further improved to be a plurality of adjacent motion vectors viIs equal to Σ αivi. Alternatively, v may be determined by a correlation algorithm, such as the OptionalFlowFarneback methodiAnd using v ═ Σ αiviThe motion vector is obtained by weighted averaging. The above weighted average may be an arithmetic average or a gaussian weighted average.
And judging the drainage state by using a drainage state deep learning model. Average velocity v of water flow at discharge port0The method is difficult to extract in the video, and because the video image is subjected to a lot of noise interferences, such as illumination, environment, weather change, brightness, day and night, and the like, the precision of directly judging the drainage state of the drainage port by using a motion vector calculated by the image is not high; the drain flow shape also causes a large difference in the motion vector grayscale map, so it is difficult to directly determine the flow state from the motion vector grayscale map. The drainage state deep learning model is established by using a deep learning method, and whether drainage is performed or not is judged by using the model, so that the drainage state identification precision is improved, and the robustness is enhanced.
The method for establishing the deep learning model of the drainage state comprises the following specific steps:
(1) collecting a large amount of drainage videos and non-drainage videos;
(2) and calculating the motion vector of each frame of image in the video, wherein the specific calculation method is shown as the 3 rd point in the specific embodiment to form a motion vector gray-scale map. The motion vector can be approximated by the difference between the previous and subsequent frames of images, or can be determined by a correlation algorithm such as optialflowfarneback.
(3) Labeling drainage state label information for each motion vector gray level graph, wherein the label value is the state of the water flow corresponding to the motion vector, and the specific labeling method comprises the following steps: the label information types are divided into a no-flow state, a low-speed flow state, a medium-speed flow state and a high-speed flow state, and the specific flow speed values of the low-speed flow state, the medium-speed flow state and the high-speed flow state are actually adjusted according to specific conditions, and generally 0 m/s, 1 m/s, 2 m/s and 5 m/s can be adopted.
(4) And establishing a drainage state deep learning model, and training and establishing the drainage state deep learning model by using a plurality of convolution maps and a plurality of full-connection maps. The convolution layer and the full link layer of the model can be set according to the situation, and the patent is not limited. A drainage state deep learning model framework schematic (see fig. 7), which is also the model we employ.
(5) And (5) training the model. By adopting a conventional CNN convolutional neural network training method, the recognition accuracy reaches over 99% during convergence, and the training learning rate is 0.0001.
The method for identifying the drainage state of the drainage port of the river channel in real time based on the video image comprises the following steps:
(1) acquiring a real-time video stream of a river channel discharge port through a camera;
(2) splitting a video stream into a frame of image;
(3) calculating to obtain a motion vector of each frame of image;
(4) analyzing the motion vector of each frame of image by using the established drainage state deep learning model and outputting a preliminary drainage state result;
(5) analyzing the result of the preliminary drainage state of the drainage outlet given by the filtering model: the preliminary result judged by the deep learning model of the drainage state is filtered, and because the drainage port environment is complex, the quality of the drainage image is poor and the noise interference is large, the method carries out filtering analysis processing on the preliminary judgment result of the deep learning model of the drainage state, and filters the unreasonable condition of the sudden change of the drainage quantity according to the current condition so as to improve the recognition rate of practical application. For example, if a frame or several consecutive frames is determined to be drained but a large number of adjacent frames are determined to be drained, the preliminary result is considered to be caused by noise interference of the pictures, and finally the drainage is not reported.
The invention realizes the real-time unmanned monitoring and automatic identification of the drainage state of the drainage port of the river channel, is beneficial to the timely discovery of problems of related parts, eliminates hidden dangers and solves the problem of river channel pollution from the source; judging the drainage state by using a drainage state deep learning model, and judging the average velocity v of the water flow at the drainage outlet0Due to the influence of factors including illumination, environment, weather change, brightness and day and night interference, the extraction in the video is difficult, and because of the video image, the precision of directly judging the drainage state of the drainage port by using a motion vector calculated by the image is not high; the drain pipeline shape also causes large difference of the motion vector gray-scale map, so that the pipeline state is difficult to be directly judged through the motion vector gray-scale map; the drainage state deep learning model is established by using a deep learning method, and whether drainage is performed or not is judged by using the model, so that the drainage state identification precision is improved, and the robustness is enhanced.
Claims (4)
1. A real-time river drainage port drainage state identification method based on video images is characterized by comprising the following steps: firstly, establishing a drainage state deep learning model, wherein the step of establishing the drainage state deep learning model comprises the following steps: collecting drainage and non-drainage state videos of a drainage port of a river channel; splitting a video into a frame of image; calculating to obtain a motion vector of each frame of image; labeling the drainage state label information of the motion vector; establishing a drainage state deep learning model; training a drainage state deep learning model; then, acquiring a real-time video stream of a river drainage port through a camera, splitting the video stream into a frame of image, calculating to obtain a motion vector of each frame of image, analyzing the motion vector of each frame of image by using an established drainage state deep learning model to output a preliminary drainage state result, analyzing and filtering the preliminary result, and outputting final drainage state information; the method comprises the following specific steps:
(1) acquiring a real-time video stream of a river channel discharge port through a camera;
(2) splitting a video stream into a frame of image;
(3) calculating to obtain a motion vector of each frame of image;
(4) analyzing the motion vector of each frame of image by using the established drainage state deep learning model and outputting a preliminary drainage state result;
(5) and analyzing the result of the preliminary drainage state of the drainage outlet given by the filtering model.
2. The method for identifying the drainage state of the drainage port of the river channel based on the video image in real time according to claim 1, wherein the method comprises the following steps: the method for establishing the drainage model of the drainage port of the river channel and setting the initial conditions comprises the following steps: establishing drainage model of river channel drainage port
ut+uux＝0
To describe the drain state;
t represents time, x represents water surface position, and u represents velocity wave; x is 0 and represents the position of water which will be discharged from the pipe orifice but not discharged from the pipe orifice, when x is less than 0, the speed of water is lower than that after being discharged from the pipe, so if u is 0, after being discharged from the pipe, the speed is suddenly increased, the water flow is smoothly discharged through fall, and if the average speed is v0Thus, the initial condition of equation (1) can be defined as
Let δ be the minimum drainage velocity, so when velocity v is0>δ>At 0, we can determine the drainage and carry out the subsequent quantitative analysis including the flow;
the solution of equation (1) is in the region { x }>0,t>x, the speed of the flowing water at the discharge port is
The change of the discharge speed along with time is illustrated; the change of the water discharge state can be described by a motion vector, and the water discharge state of the water discharge port can be obtained by finding the motion vector; a motion vector (motionvector) is a difference between storing two frames of images in graphics compression, and is a vector describing a change in the spatial position of an object.
3. The method for identifying the drainage state of the drainage port of the river channel based on the video image in real time according to claim 1, wherein the method comprises the following steps: the v calculation method of the motion vector of the image is as follows: draining two adjacent frames of gray images (g) by using drain outleti，gj) To define a gray scale map of the motion vector as
gv＝gi-gj，
The motion vector v is a one-dimensional expansion of its grayscale map; the motion vector can be further improved to be a plurality of adjacent motion vectors viIs equal to Σ αivi(ii) a Alternatively, v can be determined by a correlation algorithm, such as the OptionalFlowFarneback methodiAnd using v ═ Σ αiviObtaining a motion vector by weighted average; the above weighted average may be an arithmetic average or a gaussian weighted average.
4. The method for identifying the drainage state of the drainage port of the river channel based on the video image in real time according to claim 1, wherein the method for establishing the deep learning model of the drainage state comprises the following specific steps:
(1) collecting a large amount of drainage videos and non-drainage videos;
(2) calculating a motion vector of each frame of image in a video to form a motion vector gray image; the motion vector can be determined according to the difference value of the two frames of images before and after or according to an algorithm including related OptionalFlowFarneback;
(3) labeling drainage state label information for each motion vector gray level graph, wherein the label value is the state of the water flow corresponding to the motion vector, and the specific labeling method comprises the following steps: the label information types are divided into a no-flow state, a low-speed flow state, a medium-speed flow state and a high-speed flow state, the specific flow speed values of the low-speed flow state, the medium-speed flow state and the high-speed flow state are actually adjusted according to specific conditions, and generally 0 m/s, 1 m/s, 2 m/s and 5 m/s can be adopted;
(4) establishing a drainage state deep learning model, training and establishing the drainage state deep learning model by combining a plurality of convolution maps and a plurality of full-connection maps, and setting a convolution layer and a full-connection layer of the model according to conditions;
(5) training a model: by adopting a conventional CNN convolutional neural network training method, the recognition accuracy reaches over 99% during convergence, and the training learning rate is 0.0001.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110746229.XA CN113343923A (en) | 2021-07-01 | 2021-07-01 | Real-time river drainage port drainage state identification method based on video images |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110746229.XA CN113343923A (en) | 2021-07-01 | 2021-07-01 | Real-time river drainage port drainage state identification method based on video images |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113343923A true CN113343923A (en) | 2021-09-03 |
Family
ID=77482159
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110746229.XA Pending CN113343923A (en) | 2021-07-01 | 2021-07-01 | Real-time river drainage port drainage state identification method based on video images |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113343923A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114419556A (en) * | 2022-01-20 | 2022-04-29 | 北京北控悦慧环境科技有限公司 | Abnormal drainage image identification method and system for drainage pipe network drainage port |
Citations (42)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102564508A (en) * | 2011-12-14 | 2012-07-11 | 河海大学 | Method for implementing online tests of stream flow based on video images |
| CN102903051A (en) * | 2012-10-11 | 2013-01-30 | 惠龙港国际钢铁物流股份有限公司 | Automotive/boat member credit evaluating method based on network |
| CN103325216A (en) * | 2012-03-23 | 2013-09-25 | 罗普特（厦门）科技集团有限公司 | Water conservancy flood prevention monitoring early warning method and system based on video monitoring |
| US8547428B1 (en) * | 2006-11-02 | 2013-10-01 | SeeScan, Inc. | Pipe mapping system |
| CN103996171A (en) * | 2014-05-05 | 2014-08-20 | 河海大学 | Fluid motion vector estimation method based on space-time image |
| KR101428531B1 (en) * | 2013-02-19 | 2014-08-26 | 광운대학교 산학협력단 | A Multi-Frame-Based Super Resolution Method by Using Motion Vector Normalization and Edge Pattern Analysis |
| CN105675623A (en) * | 2016-01-29 | 2016-06-15 | 重庆扬讯软件技术有限公司 | Real-time analysis method for sewage color and flow detection on basis of sewage port video |
| CN105842475A (en) * | 2016-03-21 | 2016-08-10 | 山西泫氏实业集团有限公司 | Non-intruding type waterpower test method in building draining system |
| CN106094872A (en) * | 2016-08-24 | 2016-11-09 | 北京大学深圳研究生院 | A kind of sewage based on unmanned plane secretly arranges mouthful detection method and system |
| CN106683114A (en) * | 2016-12-16 | 2017-05-17 | 河海大学 | Fluid motion vector estimation method based on feature optical flow |
| CN107833244A (en) * | 2017-11-02 | 2018-03-23 | 南京市测绘勘察研究院股份有限公司 | A kind of shade tree attribute automatic identifying method based on mobile lidar data |
| CN107886133A (en) * | 2017-11-29 | 2018-04-06 | 南京市测绘勘察研究院股份有限公司 | A kind of underground piping defect inspection method based on deep learning |
| CN109145696A (en) * | 2017-06-28 | 2019-01-04 | 安徽清新互联信息科技有限公司 | A kind of Falls Among Old People detection method and system based on deep learning |
| CN109284753A (en) * | 2018-08-30 | 2019-01-29 | 深圳大学 | A kind of localization method of liquid transmission line and application |
| CN109919372A (en) * | 2019-02-28 | 2019-06-21 | 武汉大学 | A kind of urban storm ponding assessment modeling method based on full-time sky |
| CN110599460A (en) * | 2019-08-14 | 2019-12-20 | 深圳市勘察研究院有限公司 | Underground pipe network detection and evaluation cloud system based on hybrid convolutional neural network |
| CN110633530A (en) * | 2019-09-18 | 2019-12-31 | 南通大学 | Fishway design method based on computational fluid dynamics and convolutional neural network |
| CN110765676A (en) * | 2019-07-18 | 2020-02-07 | 成都信息工程大学 | Watershed water quality simulation method based on stable flow field |
| CN111062316A (en) * | 2019-12-16 | 2020-04-24 | 成都之维安科技股份有限公司 | Pollution source wastewater discharge real-time video analysis system based on deep learning technology |
| CN111080651A (en) * | 2019-12-12 | 2020-04-28 | 西南科技大学 | Automatic monitoring method for petroleum drilling polluted gas based on water flow segmentation |
| CN111089625A (en) * | 2019-12-13 | 2020-05-01 | 国网浙江省电力有限公司紧水滩水力发电厂 | Binocular vision-simulated river flow real-time monitoring system and method |
| US10725438B1 (en) * | 2019-10-01 | 2020-07-28 | 11114140 Canada Inc. | System and method for automated water operations for aquatic facilities using image-based machine learning |
| CN111570331A (en) * | 2020-04-30 | 2020-08-25 | 北京智通云联科技有限公司 | Unqualified product removing device and method under variable-speed assembly line environment |
| CN111626178A (en) * | 2020-05-24 | 2020-09-04 | 中南民族大学 | Compressed domain video motion recognition method and system based on new spatio-temporal feature stream |
| CN111798386A (en) * | 2020-06-24 | 2020-10-20 | 武汉大学 | River channel flow velocity measurement method based on edge identification and maximum sequence density estimation |
| CN112016831A (en) * | 2020-08-27 | 2020-12-01 | 西安易辑数字科技有限公司 | AI intelligent forecast-based urban waterlogging landing area identification method |
| CN112033642A (en) * | 2020-09-17 | 2020-12-04 | 河南省对外科技交流中心 | Device, method and system for detecting motion behavior of bubbles in liquid |
| CN112036043A (en) * | 2020-09-02 | 2020-12-04 | 河海大学 | Method for calculating fracture model of long-distance gravity flow large-pipe-diameter water supply pipeline |
| CN112112240A (en) * | 2020-07-30 | 2020-12-22 | 同济大学 | Urban river network waterlogging prevention optimal scheduling method |
| CN112163481A (en) * | 2020-09-16 | 2021-01-01 | 清华大学合肥公共安全研究院 | Water environment pollution analysis method based on video recognition |
| WO2021016596A1 (en) * | 2019-07-25 | 2021-01-28 | Nvidia Corporation | Deep neural network for segmentation of road scenes and animate object instances for autonomous driving applications |
| CN112287899A (en) * | 2020-11-26 | 2021-01-29 | 山东捷讯通信技术有限公司 | Unmanned aerial vehicle aerial image river drain detection method and system based on YOLO V5 |
| CN112308040A (en) * | 2020-11-26 | 2021-02-02 | 山东捷讯通信技术有限公司 | River sewage outlet detection method and system based on high-definition images |
| US20210041596A1 (en) * | 2019-08-06 | 2021-02-11 | Exxonmobil Upstream Research Company | Petrophysical Inversion With Machine Learning-Based Geologic Priors |
| CN112488020A (en) * | 2020-12-10 | 2021-03-12 | 西安交通大学 | Water environment pollution condition detection and evaluation device based on unmanned aerial vehicle aerial photography data |
| CN212905977U (en) * | 2020-11-04 | 2021-04-06 | 中持水务股份有限公司 | Intelligent-connection well lid drainage monitoring system for industrial park |
| CN112862898A (en) * | 2021-02-05 | 2021-05-28 | 慧目(重庆)科技有限公司 | Flow velocity measuring method based on computer vision |
| CN112861856A (en) * | 2021-02-05 | 2021-05-28 | 慧目(重庆)科技有限公司 | Drainage monitoring method based on computer vision and water body monitoring method |
| CN112884731A (en) * | 2021-02-05 | 2021-06-01 | 慧目(重庆)科技有限公司 | Water level detection method and river channel monitoring method based on machine vision |
| CN112884039A (en) * | 2021-02-05 | 2021-06-01 | 慧目(重庆)科技有限公司 | Water body pollution identification method based on computer vision |
| CN112945208A (en) * | 2021-01-28 | 2021-06-11 | 天地伟业技术有限公司 | River course section velocity of flow water level monitoring spherical camera |
| CN112987751A (en) * | 2021-03-18 | 2021-06-18 | 南京理工大学 | System and method for quickly detecting hidden sewage draining outlet in automatic cruising mode |
-
2021
- 2021-07-01 CN CN202110746229.XA patent/CN113343923A/en active Pending
Patent Citations (42)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8547428B1 (en) * | 2006-11-02 | 2013-10-01 | SeeScan, Inc. | Pipe mapping system |
| CN102564508A (en) * | 2011-12-14 | 2012-07-11 | 河海大学 | Method for implementing online tests of stream flow based on video images |
| CN103325216A (en) * | 2012-03-23 | 2013-09-25 | 罗普特（厦门）科技集团有限公司 | Water conservancy flood prevention monitoring early warning method and system based on video monitoring |
| CN102903051A (en) * | 2012-10-11 | 2013-01-30 | 惠龙港国际钢铁物流股份有限公司 | Automotive/boat member credit evaluating method based on network |
| KR101428531B1 (en) * | 2013-02-19 | 2014-08-26 | 광운대학교 산학협력단 | A Multi-Frame-Based Super Resolution Method by Using Motion Vector Normalization and Edge Pattern Analysis |
| CN103996171A (en) * | 2014-05-05 | 2014-08-20 | 河海大学 | Fluid motion vector estimation method based on space-time image |
| CN105675623A (en) * | 2016-01-29 | 2016-06-15 | 重庆扬讯软件技术有限公司 | Real-time analysis method for sewage color and flow detection on basis of sewage port video |
| CN105842475A (en) * | 2016-03-21 | 2016-08-10 | 山西泫氏实业集团有限公司 | Non-intruding type waterpower test method in building draining system |
| CN106094872A (en) * | 2016-08-24 | 2016-11-09 | 北京大学深圳研究生院 | A kind of sewage based on unmanned plane secretly arranges mouthful detection method and system |
| CN106683114A (en) * | 2016-12-16 | 2017-05-17 | 河海大学 | Fluid motion vector estimation method based on feature optical flow |
| CN109145696A (en) * | 2017-06-28 | 2019-01-04 | 安徽清新互联信息科技有限公司 | A kind of Falls Among Old People detection method and system based on deep learning |
| CN107833244A (en) * | 2017-11-02 | 2018-03-23 | 南京市测绘勘察研究院股份有限公司 | A kind of shade tree attribute automatic identifying method based on mobile lidar data |
| CN107886133A (en) * | 2017-11-29 | 2018-04-06 | 南京市测绘勘察研究院股份有限公司 | A kind of underground piping defect inspection method based on deep learning |
| CN109284753A (en) * | 2018-08-30 | 2019-01-29 | 深圳大学 | A kind of localization method of liquid transmission line and application |
| CN109919372A (en) * | 2019-02-28 | 2019-06-21 | 武汉大学 | A kind of urban storm ponding assessment modeling method based on full-time sky |
| CN110765676A (en) * | 2019-07-18 | 2020-02-07 | 成都信息工程大学 | Watershed water quality simulation method based on stable flow field |
| WO2021016596A1 (en) * | 2019-07-25 | 2021-01-28 | Nvidia Corporation | Deep neural network for segmentation of road scenes and animate object instances for autonomous driving applications |
| US20210041596A1 (en) * | 2019-08-06 | 2021-02-11 | Exxonmobil Upstream Research Company | Petrophysical Inversion With Machine Learning-Based Geologic Priors |
| CN110599460A (en) * | 2019-08-14 | 2019-12-20 | 深圳市勘察研究院有限公司 | Underground pipe network detection and evaluation cloud system based on hybrid convolutional neural network |
| CN110633530A (en) * | 2019-09-18 | 2019-12-31 | 南通大学 | Fishway design method based on computational fluid dynamics and convolutional neural network |
| US10725438B1 (en) * | 2019-10-01 | 2020-07-28 | 11114140 Canada Inc. | System and method for automated water operations for aquatic facilities using image-based machine learning |
| CN111080651A (en) * | 2019-12-12 | 2020-04-28 | 西南科技大学 | Automatic monitoring method for petroleum drilling polluted gas based on water flow segmentation |
| CN111089625A (en) * | 2019-12-13 | 2020-05-01 | 国网浙江省电力有限公司紧水滩水力发电厂 | Binocular vision-simulated river flow real-time monitoring system and method |
| CN111062316A (en) * | 2019-12-16 | 2020-04-24 | 成都之维安科技股份有限公司 | Pollution source wastewater discharge real-time video analysis system based on deep learning technology |
| CN111570331A (en) * | 2020-04-30 | 2020-08-25 | 北京智通云联科技有限公司 | Unqualified product removing device and method under variable-speed assembly line environment |
| CN111626178A (en) * | 2020-05-24 | 2020-09-04 | 中南民族大学 | Compressed domain video motion recognition method and system based on new spatio-temporal feature stream |
| CN111798386A (en) * | 2020-06-24 | 2020-10-20 | 武汉大学 | River channel flow velocity measurement method based on edge identification and maximum sequence density estimation |
| CN112112240A (en) * | 2020-07-30 | 2020-12-22 | 同济大学 | Urban river network waterlogging prevention optimal scheduling method |
| CN112016831A (en) * | 2020-08-27 | 2020-12-01 | 西安易辑数字科技有限公司 | AI intelligent forecast-based urban waterlogging landing area identification method |
| CN112036043A (en) * | 2020-09-02 | 2020-12-04 | 河海大学 | Method for calculating fracture model of long-distance gravity flow large-pipe-diameter water supply pipeline |
| CN112163481A (en) * | 2020-09-16 | 2021-01-01 | 清华大学合肥公共安全研究院 | Water environment pollution analysis method based on video recognition |
| CN112033642A (en) * | 2020-09-17 | 2020-12-04 | 河南省对外科技交流中心 | Device, method and system for detecting motion behavior of bubbles in liquid |
| CN212905977U (en) * | 2020-11-04 | 2021-04-06 | 中持水务股份有限公司 | Intelligent-connection well lid drainage monitoring system for industrial park |
| CN112287899A (en) * | 2020-11-26 | 2021-01-29 | 山东捷讯通信技术有限公司 | Unmanned aerial vehicle aerial image river drain detection method and system based on YOLO V5 |
| CN112308040A (en) * | 2020-11-26 | 2021-02-02 | 山东捷讯通信技术有限公司 | River sewage outlet detection method and system based on high-definition images |
| CN112488020A (en) * | 2020-12-10 | 2021-03-12 | 西安交通大学 | Water environment pollution condition detection and evaluation device based on unmanned aerial vehicle aerial photography data |
| CN112945208A (en) * | 2021-01-28 | 2021-06-11 | 天地伟业技术有限公司 | River course section velocity of flow water level monitoring spherical camera |
| CN112862898A (en) * | 2021-02-05 | 2021-05-28 | 慧目(重庆)科技有限公司 | Flow velocity measuring method based on computer vision |
| CN112861856A (en) * | 2021-02-05 | 2021-05-28 | 慧目(重庆)科技有限公司 | Drainage monitoring method based on computer vision and water body monitoring method |
| CN112884731A (en) * | 2021-02-05 | 2021-06-01 | 慧目(重庆)科技有限公司 | Water level detection method and river channel monitoring method based on machine vision |
| CN112884039A (en) * | 2021-02-05 | 2021-06-01 | 慧目(重庆)科技有限公司 | Water body pollution identification method based on computer vision |
| CN112987751A (en) * | 2021-03-18 | 2021-06-18 | 南京理工大学 | System and method for quickly detecting hidden sewage draining outlet in automatic cruising mode |
Non-Patent Citations (4)
| Title |
|---|
| LILIK EKO WIDODO等: "Development of drain hole design optimisation: a conceptual model for open pit mine slope drainage system with fractured media using a multi-stage genetic algorithm", 《 ENVIRONMENTAL EARTH SCIENCES 》, vol. 77, no. 721, 17 October 2018 (2018-10-17), pages 1 - 16, XP036628677, DOI: 10.1007/s12665-018-7895-3 * |
| MIHIR JAIN等: "Better Exploiting Motion for Better Action Recognition", 《2013 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION（IEEE）》, 3 October 2013 (2013-10-03), pages 2555 * |
| 王兴海等: "基于移动互联网的城市综合地下管线信息服务系统构建", 《城市勘测》, vol. 06, 30 June 2016 (2016-06-30), pages 27 - 31 * |
| 田伟等: "基于机器视觉深度学习的电站渗水检测识别技术研究", 《电子设计工程》, vol. 28, no. 20, 20 October 2020 (2020-10-20), pages 66 - 70 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114419556A (en) * | 2022-01-20 | 2022-04-29 | 北京北控悦慧环境科技有限公司 | Abnormal drainage image identification method and system for drainage pipe network drainage port |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110348376B (en) | Pedestrian real-time detection method based on neural network | |
| CN117078943B (en) | Remote sensing image road segmentation method integrating multi-scale features and double-attention mechanism | |
| CN112394152A (en) | Water quality real-time intelligent monitoring analysis management system based on big data | |
| CN110335294B (en) | Mine water pump house water leakage detection method based on frame difference method and 3D convolutional neural network | |
| CN111860143B (en) | Real-time flame detection method for inspection robot | |
| CN111259827A (en) | Automatic detection method and device for water surface floating objects for urban river supervision | |
| CN112734739B (en) | Visual building crack identification method based on attention mechanism and ResNet fusion | |
| CN110334703B (en) | Ship detection and identification method in day and night image | |
| CN112967271B (en) | Casting surface defect identification method based on improved DeepLabv3+ network model | |
| CN113658131A (en) | Tour type ring spinning broken yarn detection method based on machine vision | |
| CN112417955A (en) | Patrol video stream processing method and device | |
| CN112418124A (en) | Intelligent fish monitoring method based on video images | |
| CN110717921A (en) | Full convolution neural network semantic segmentation method of improved coding and decoding structure | |
| CN114639064B (en) | Water level identification method and device | |
| CN115761563A (en) | River surface flow velocity calculation method and system based on optical flow measurement and calculation | |
| CN113343923A (en) | Real-time river drainage port drainage state identification method based on video images | |
| CN115205363A (en) | Conveyor belt real-time judgment detection method and system based on improved ResNet network | |
| CN116310845A (en) | Intelligent monitoring system for sewage treatment | |
| CN111753693A (en) | Target detection method in static scene | |
| CN114332739A (en) | Smoke detection method based on moving target detection and deep learning technology | |
| CN115063434A (en) | Low-low-light image instance segmentation method and system based on feature denoising | |
| CN113920421B (en) | Full convolution neural network model capable of achieving rapid classification | |
| CN112669269A (en) | Pipeline defect classification and classification method and system based on image recognition | |
| CN112801021B (en) | Method and system for detecting lane line based on multi-level semantic information | |
| CN116152699B (en) | Real-time moving target detection method for hydropower plant video monitoring system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |