CN116228712A - Multi-scale slope disaster monitoring method, system and device - Google Patents
Multi-scale slope disaster monitoring method, system and device Download PDFInfo
- Publication number
- CN116228712A CN116228712A CN202310192166.7A CN202310192166A CN116228712A CN 116228712 A CN116228712 A CN 116228712A CN 202310192166 A CN202310192166 A CN 202310192166A CN 116228712 A CN116228712 A CN 116228712A
- Authority
- CN
- China
- Prior art keywords
- slope
- detection result
- side slope
- video
- deep learning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30232—Surveillance
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Biophysics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Quality & Reliability (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a multi-scale slope disaster monitoring method, a system and a device, wherein the method comprises the following steps: acquiring a side slope video of a side slope to be monitored based on binocular vision, and calculating to obtain a side slope displacement result; processing the side slope video by using an inter-frame difference method to obtain a dynamic detection result; processing the slope video through a multi-scale deep learning algorithm to obtain a deep learning detection result; setting a weight value between a dynamic detection result and a deep learning detection result according to the live-action parameters of the slope video; and obtaining a side slope falling stone result of the side slope to be monitored according to the dynamic detection result, the deep learning detection result and the weight value. The method has the advantages that the side slope video is acquired through binocular vision, the side slope displacement monitoring is carried out, the side slope falling stone monitoring is carried out through dynamic detection and deep learning detection, the real-time displacement and the falling stone monitoring of the side slope are realized, the side slope disaster detection is free from the limitation of manpower observation, and the accuracy of the side slope disaster detection is improved.
Description
Technical Field
The invention belongs to the field of geological monitoring, and particularly relates to a multi-scale slope disaster monitoring method, system and device.
Background
With the increasing engineering projects, such as the excavation of rock mass by ergonomic activities such as house and municipal construction, hydraulic and hydroelectric engineering, railway and highway construction, and strip mining, various artificial slopes are also produced successively.
The geological condition of the side slope is complex, and once instability or sliding occurs, serious threat is caused to national economy. Therefore, development of slope monitoring research is needed to realize early warning and prevention of slope disasters.
The displacement of the side slope is a dynamic physical process, and the occurrence of the instability of the side slope can be found through the process of analyzing the displacement of the side slope, often, the development process from peristaltic deformation to severe deformation to gradual stabilization is carried out, wherein the duration time of the peristaltic deformation stage is longer, the deformation at the stage is difficult to be perceived by naked eyes of people, so that the change of the displacement of the side slope cannot be monitored in real time, and the early warning of the instability disaster of the side slope is carried out.
Disclosure of Invention
In order to solve the problems, the invention provides a multi-scale slope disaster monitoring method, a system and a device.
The technical scheme adopted by the invention is as follows:
in a first aspect, the present invention provides a method for monitoring a multi-scale slope disaster, including:
acquiring a side slope video of a side slope to be monitored based on binocular vision, and calculating to obtain a side slope displacement result, wherein the side slope video is a side slope image of continuous frames;
processing the side slope video by using an inter-frame difference method to obtain a dynamic detection result;
processing the slope video through a multi-scale deep learning algorithm to obtain a deep learning detection result;
setting a weight value between a dynamic detection result and a deep learning detection result according to the live-action parameters of the slope video;
and obtaining a side slope falling stone result of the side slope to be monitored according to the dynamic detection result, the deep learning detection result and the weight value.
Further, based on binocular vision, collecting a side slope video of the side slope to be monitored, and calculating to obtain a side slope displacement result, including:
obtaining calibration information of a binocular camera, and setting targets on a slope to be monitored;
collecting a side slope video of a side slope to be monitored by a binocular camera, wherein the side slope video is a side slope image of continuous frames;
identifying target pixel coordinates of a target in each frame of side slope image;
calculating a target space coordinate of the target by combining the target pixel coordinate and the calibration information based on a space coordinate calculation algorithm of binocular vision;
and comparing the space coordinates of the target with the original space coordinates of the target to obtain a displacement value of the target, and taking the displacement value as a slope displacement result.
Further, the method for processing the side slope video by utilizing the inter-frame difference method to obtain a dynamic detection result comprises the following steps:
carrying out differential operation on two or three continuous frames of slope images in the slope video by using an inter-frame difference method to obtain the gray level difference absolute value of each pixel point;
judging whether the gray level difference absolute value of the target pixel point exceeds a preset absolute value;
if the dynamic detection result exists, determining the target pixel point as a moving target, and obtaining the dynamic detection result;
if not, determining that there is no moving object.
Further, determining the target pixel point as a moving target to obtain a dynamic detection result includes:
determining a target pixel point as a moving target;
performing median filtering and expansion corrosion treatment;
and performing characteristic analysis and experience modeling on the processed moving target, and screening to obtain a target area where the moving target is positioned, thereby obtaining a dynamic detection result.
Further, the slope video is processed through a multi-scale deep learning algorithm to obtain a deep learning detection result, which comprises the following steps:
feature extraction is carried out on the slope image in the slope video through a backbone network Darknet53, so that at least three feature layers are obtained;
constructing a feature pyramid FPN to extract the reinforcing features of at least three feature layers to obtain three reinforcing feature layers;
calculating to obtain a prediction result based on a multi-scale deep learning algorithm and three reinforcement feature layers;
decoding the prediction result to obtain a prediction frame position;
and obtaining a deep learning detection result according to the predicted frame position.
Further, setting a weight value between a dynamic detection result and a deep learning detection result according to a live-action parameter of the slope video, including:
obtaining real scene parameters according to the slope video;
obtaining a definition value according to the live-action parameters;
when the definition degree value exceeds a preset degree value, setting a weight value K1 between the dynamic detection result and the deep learning detection result, wherein the weight value K1 represents that the weight of the deep learning detection result is larger than that of the dynamic detection result;
when the definition degree value does not exceed the preset degree value, a weight value K2 between the dynamic detection result and the deep learning detection result is set, wherein the weight value K2 indicates that the weight of the dynamic detection result is larger than that of the deep learning detection result.
In a second aspect, the present invention provides a multi-scale slope disaster monitoring system comprising:
the data acquisition module is used for acquiring a side slope video of the side slope to be monitored based on binocular vision, and calculating to obtain a side slope displacement result, wherein the side slope video is a side slope image of continuous frames;
the dynamic detection module is used for processing the side slope video by utilizing an inter-frame difference method to obtain a dynamic detection result;
the deep learning module is used for processing the slope video through a multi-scale deep learning algorithm to obtain a deep learning detection result;
the weight setting module is used for setting a weight value between a dynamic detection result and a deep learning detection result according to the live-action parameters of the slope video;
and the comprehensive evaluation module is used for obtaining a side slope falling stone result of the side slope to be monitored according to the dynamic detection result, the deep learning detection result and the weight value.
Further, the dynamic detection module includes:
the inter-frame difference calculation unit is used for carrying out difference operation on the side slope images of two or three continuous frames in the side slope video by utilizing an inter-frame difference method to obtain the gray level difference absolute value of each pixel point;
the judging unit is used for judging whether the gray level difference absolute value of the target pixel point exceeds a preset absolute value;
the dynamic processing unit is used for determining the target pixel point as a moving target when the gray level difference absolute value of the target pixel point exceeds a preset absolute value, performing median filtering and expansion corrosion processing, performing feature analysis and experience modeling on the processed moving target, and screening to obtain a target area where the moving target is positioned to obtain a dynamic detection result;
and the dynamic processing unit is also used for determining that no moving target exists when the absolute value of the gray difference of the target pixel point does not exist exceeds the preset absolute value.
Further, the deep learning module includes:
the feature extraction unit is used for extracting features of the slope images in the slope video through the backbone network Darknet53 to obtain at least three feature layers;
the enhanced feature extraction unit is used for constructing a feature pyramid FPN to extract enhanced features of at least three feature layers to obtain three enhanced feature layers;
the deep learning unit is used for calculating and obtaining a prediction result based on a multi-scale deep learning algorithm and three reinforcement feature layers;
and the prediction processing unit is used for decoding the prediction result to obtain a prediction frame position and obtaining a deep learning detection result according to the prediction frame position.
In a third aspect, the present invention provides a multi-scale slope disaster monitoring device, comprising:
the system comprises an image data acquisition system, an edge computing system, a cloud server and a client;
the image data acquisition system is used for acquiring a side slope video of the side slope to be monitored based on binocular vision, and calculating to obtain a side slope displacement result, wherein the side slope video is a side slope image of continuous frames;
the edge computing system is used for processing the side slope video by utilizing an inter-frame difference method to obtain a dynamic detection result, processing the side slope video by utilizing a multi-scale deep learning algorithm to obtain a deep learning detection result, setting a weight value between the dynamic detection result and the deep learning detection result according to the real scene parameter of the side slope video, and obtaining a side slope falling stone result of the side slope to be monitored according to the dynamic detection result, the deep learning detection result and the weight value;
the cloud server is used for receiving and storing the slope video, the slope displacement result and the slope stone falling result, and performing data interaction with the client.
The invention has the beneficial effects that:
the method comprises the steps of monitoring a side slope in real time through a preset binocular camera, collecting a side slope video of the side slope to be monitored based on binocular vision, calculating to obtain a side slope displacement result, processing the side slope video by utilizing an inter-frame difference method to obtain a dynamic detection result, processing the side slope video through a multi-scale deep learning algorithm to obtain a deep learning detection result, setting a weight value between the dynamic detection result and the deep learning detection result according to real-scene parameters of the side slope video, and obtaining a side slope falling stone result of the side slope to be monitored according to the dynamic detection result, the deep learning detection result and the weight value. The method has the advantages that the side slope video is acquired through binocular vision, the side slope displacement is monitored, the side slope falling rocks are monitored through dynamic detection and deep learning detection, the real-time displacement and falling rocks monitoring of the side slope are realized, and the side slope disaster detection is free from the limitation of manual observation; and the dynamic detection result and the deep learning detection result are combined for comprehensive treatment, so that the accuracy of detecting the side slope disasters is improved.
Drawings
FIG. 1 is a flow chart of a multi-scale slope disaster monitoring method of the present invention;
FIG. 2 is a flow chart of the method for obtaining dynamic detection results by using the inter-frame difference method according to the present invention;
FIG. 3 is a flow chart of the invention for obtaining a deep learning detection result by using a multi-scale deep learning algorithm;
FIG. 4 is a block diagram of a multi-scale slope disaster monitoring system of the present invention;
fig. 5 is a block diagram of the multi-scale slope disaster monitoring device of the present invention.
Detailed Description
The invention is further described below with reference to the accompanying drawings. The following examples are only for more clearly illustrating the technical aspects of the present invention, and are not intended to limit the scope of the present invention.
As shown in fig. 1, an embodiment of the present invention provides a multi-scale slope disaster monitoring method, including the following steps:
101, acquiring a side slope video of a side slope to be monitored based on binocular vision, and calculating to obtain a side slope displacement result;
the binocular vision concept is that image acquisition is carried out through a binocular camera, the binocular camera consists of two industrial cameras, and point location arrangement of the binocular camera is preset at a monitoring position aiming at a slope to be monitored. After the binocular camera is arranged, the binocular camera needs to be calibrated. The main purpose of the calibration of a binocular system is to reduce imaging distortion of a monocular camera and to determine the spatial relationship of the two cameras. The reduction of imaging distortion of the monocular camera is to increase measurement accuracy, and the determination of the spatial relationship of the two cameras is for target spatial coordinate calculation. Based on the principle of binocular cameras, the side slope image data of the side slope to be monitored can be acquired.
The specific implementation process is as follows:
obtaining calibration information of a binocular camera, and setting targets on a slope to be monitored;
the method comprises the steps that a side slope video of a side slope to be monitored is collected through a binocular camera, the side slope video is a side slope image of continuous frames, the collection frequency can be set to be 1 frame/min, the highest collection frequency can be set to be 20 frames/sec, and in practical application, the specific collection frequency can be set according to requirements and does not exceed the function limit of the binocular camera;
identifying target pixel coordinates of a target in each frame of slope image, wherein the target pixel coordinates are two-dimensional coordinates in a two-dimensional plane in the image;
therefore, a binocular vision-based space coordinate calculation algorithm is needed, and the target pixel coordinates and the calibration information are combined to calculate the target space coordinates of the target;
and comparing the space coordinates of the target with the original space coordinates of the target to obtain a displacement value of the target, and taking the displacement value as a slope displacement result.
102, processing the side slope video by using an inter-frame difference method to obtain a dynamic detection result;
the side slope images acquired by the binocular camera are side slope videos with fixed frame numbers or preset frame numbers, so that the side slope videos can be processed by utilizing an inter-frame difference method, if no moving object exists in the side slope videos, the image change of continuous frames is weak, and if the moving object exists, the continuous frames can be obviously changed, and the dynamic detection result can be obtained.
103, processing the slope video through a multi-scale deep learning algorithm to obtain a deep learning detection result;
in this embodiment, the multi-scale deep learning algorithm is based on a YoloV3 network frame, that is, by processing a side slope video, three scale feature layers are extracted to perform target prediction, so as to obtain a deep learning detection result.
104, setting a weight value between a dynamic detection result and a deep learning detection result according to the live-action parameters of the side slope video;
the multi-scale deep learning algorithm is easy to be interfered by complex backgrounds of images, and the accuracy in scenes with complex backgrounds is reduced; the inter-frame difference rule cannot accurately identify the target object, only the target area of the moving target can be obtained, and the identification accuracy is not high, so that the real scene parameters are combined, and the weight value between the dynamic detection result and the deep learning detection result is required to be set. According to the characteristics of the multi-scale deep learning algorithm and the interframe difference method, when the background noise is large, the weight of the dynamic detection result is larger than the weight of the deep learning detection result, and when the background noise is small, the weight of the dynamic detection result is smaller than the weight of the deep learning detection result.
And 105, obtaining a side slope falling stone result of the side slope to be monitored according to the dynamic detection result, the deep learning detection result and the weight value.
The implementation principle of the embodiment of the invention is as follows:
the method comprises the steps of monitoring a side slope in real time through a preset binocular camera, collecting a side slope video of the side slope to be monitored based on binocular vision, calculating to obtain a side slope displacement result, processing the side slope video by utilizing an inter-frame difference method to obtain a dynamic detection result, processing the side slope video through a multi-scale deep learning algorithm to obtain a deep learning detection result, setting a weight value between the dynamic detection result and the deep learning detection result according to real-scene parameters of the side slope video, and obtaining a side slope falling stone result of the side slope to be monitored according to the dynamic detection result, the deep learning detection result and the weight value. The method has the advantages that the side slope video is acquired through binocular vision, the side slope displacement is monitored, the side slope falling rocks are monitored through dynamic detection and deep learning detection, the real-time displacement and falling rocks monitoring of the side slope are realized, and the side slope disaster detection is free from the limitation of manual observation; and the dynamic detection result and the deep learning detection result are combined for comprehensive treatment, so that the accuracy of detecting the side slope disasters is improved.
In the embodiment shown in fig. 1, the steps 102 and 103 are executed in parallel, and there is no sequence of execution.
Based on the embodiment shown in fig. 1, a specific process of obtaining a dynamic detection result by using the inter-frame difference method is shown in fig. 2, and the steps include:
201, performing differential operation on two or three continuous frames of slope images in the slope video by using an inter-frame difference method to obtain the gray level difference absolute value of each pixel point;
wherein, the image that binocular camera gathered has the characteristics of continuity. If the moving object does not exist, the change of the continuous frames is very weak, if the moving object exists, the continuous frames are obviously changed, the gray values of the corresponding pixel points in the images of different frames are subtracted to obtain the gray difference value of each pixel point, and the absolute value of the gray difference value is calculated to obtain the gray difference absolute value of each pixel point.
202, judging whether the gray level difference absolute value of a target pixel point exceeds a preset absolute value;
wherein, the gray level difference absolute value of each pixel point may have a slight difference due to the interference of external factors, and is not actually caused by the moving object, then a preset absolute value is required to be empirically set for eliminating the misjudgment of the moving object caused by the interference of other factors, and the moving object can be determined only when the gray level difference absolute value of the target pixel point exceeds the preset absolute value, and step 203 is executed; otherwise, step 206 is performed.
203, determining a target pixel point as a moving target;
when the gray level difference absolute value of the target pixel point exceeds the preset absolute value, it is indicated that the target pixel point is still judged to be a moving target under the condition that interference of other factors is eliminated, the target pixel point is not just one pixel point, but is a set of gray level difference absolute values exceeding the preset absolute value, in practical application, if a certain stone moves, and under the condition that the volume of the stone is large, a plurality of pixel points are occupied in an image, so that the target pixel point is also a plurality of pixel points.
204, performing median filtering and expansion corrosion treatment;
after two continuous frames are processed by an inter-frame difference method, a frame difference image is actually obtained, the frame difference image is represented by a binarization image, and after a target pixel point with obvious gray level change is determined, median filtering and expansion corrosion processing are carried out. The median filtering is an image smoothing algorithm, and the basic principle is that the median in the neighborhood pixel set around the test pixel replaces the original pixel, so that isolated noise points or finer noise lines can be effectively removed; corrosion and expansion are also an algorithm for smoothing the picture, and generally corrosion and expansion can effectively remove interference lines. And the influence of environmental interference factors such as noise points and the like is eliminated.
205, performing feature analysis and experience modeling on the processed moving target, and screening to obtain a target area where the moving target is located, thereby obtaining a dynamic detection result;
after median filtering and expansion corrosion are performed, characteristic analysis and empirical modeling screening are performed on the moving target to obtain a target area where the moving target is located, and a dynamic detection result is obtained.
206, determining that there is no moving object.
And when the absolute value of the gray level difference of the non-existing target pixel point exceeds a preset absolute value, determining that the moving target is not present.
The implementation principle of the embodiment of the invention is as follows:
the detection of the moving target is realized by an inter-frame difference method, and the median filtering and the expansion corrosion treatment can effectively reduce noise and interference, so that the dynamic detection result is more accurate.
Based on the embodiment shown in fig. 1, a specific process of obtaining a deep learning detection result by using a multi-scale deep learning algorithm is shown in fig. 3, and the steps include:
301, extracting features of a slope image in a slope video through a backbone network Darknet53 to obtain at least three feature layers;
the multi-scale deep learning algorithm is specifically a YoloV3 algorithm framework, the backbone network of the YoloV3 algorithm framework is a dark net53, and residual convolution in the dark net53 is to first perform convolution with a convolution kernel size of 3×3 and a step size of 2, and the convolution compresses the width and the height of an input feature layer, so as to obtain a feature layer (layer). The residual structure is then constructed by performing a 1 x 1 convolution and a 3 x 3 convolution on the feature layer and adding a layer to the result. By constantly convolving 1 x 1 and 3 x 3 and overlapping the residual edges, the network can be greatly deepened. Each convolution portion of the dark 53 uses a unique dark conv2D structure, performs L2 regularization on each convolution, and performs batch normalization and LeakyReLU after the convolution is completed. The normal ReLU is to set all negative values to zero, and the leak ReLU is to assign a non-zero slope to all negative values. Mathematically, this can be expressed as:
wherein a is i Is represented by the general formula (1), + -infinity) interval.
302, constructing a feature pyramid FPN to extract reinforcing features of at least three feature layers to obtain three reinforcing feature layers;
wherein the feature pyramid (Feature Pyramid Networks, FPN) is a fundamental component in an identification system for detecting objects of different dimensions. And carrying out reinforced feature extraction on the plurality of extracted feature layers through the feature pyramid FPN to obtain three reinforced feature layers.
303, calculating to obtain a prediction result based on a multi-scale deep learning algorithm and three reinforcement feature layers;
wherein, three enhancement feature layers are utilized to transmit to the Yolo Head to obtain the prediction result. Yolo Head is essentially a 3 x 3 convolution plus a 1 x 1 convolution, the effect of which is feature integration, and the effect of which is to adjust the number of channels.
304, decoding the prediction result to obtain a prediction frame position;
and then, calculating the width and height of the prediction frame by combining the prior frame with the width value and the height value, so that the position of the whole prediction frame can be obtained.
And 305, obtaining a deep learning detection result according to the predicted frame position.
After the position of the predicted frame is obtained, the position of the predicted frame in the slope image of the corresponding frame can be determined by combining the original image. It should be noted that operations such as score sorting and non-maximal suppression screening are also required to be performed on the deep learning detection result, so as to improve the accuracy of the result.
In combination with the embodiments shown in fig. 2 and fig. 3, after the dynamic detection result and the deep learning detection result are obtained, a weight value between the dynamic detection result and the deep learning detection result needs to be set, which specifically includes the following steps:
obtaining a real scene parameter according to the slope video, wherein the real scene parameter is a parameter with larger influence on background noise of a slope image, and can be specifically the definition;
obtaining a sharpness value according to the real scene parameter, wherein the sharpness value can be determined by the resolution of the slope image, for example, the real scene parameter is 2K resolution, and the corresponding sharpness value is smaller than the sharpness value corresponding to 4K resolution;
according to the influence degree of the definition degree value on the deep learning detection result, a preset degree value is set for the definition degree value according to calculation experience, when the definition degree value exceeds the preset degree value, the image background has no excessive noise, the whole range has no interference of moving objects, the whole recognition degree of the moving objects is higher, the degree of interference of the deep learning detection result is smaller, then a weight value K1 between the dynamic detection result and the deep learning detection result is set, and the weight value K1 represents that the weight of the deep learning detection result is larger than that of the dynamic detection result;
when the sharpness value does not exceed the preset degree value, the background noise of the image is higher, similar interferents are more, for example, a slope stone pile in construction is more, and a rolling threat target is difficult to normally identify, at this time, a weight value K2 between a dynamic detection result and a deep learning detection result is set, and the weight value K2 represents that the weight of the dynamic detection result is greater than that of the deep learning detection result. Therefore, in the special scene of the side slope, the influence of the interference is eliminated.
When the side slope falling stone result and the side slope displacement result of the side slope to be monitored are the side slope displacement or falling stone and other conditions, the side slope disaster is indicated to exist, and early warning information is required to be sent at the moment and used for prompting personnel to perform side slope disaster prevention or removal and other works.
The above embodiments describe a multi-scale slope disaster monitoring method, and the following describes a multi-scale slope disaster monitoring system applying the multi-scale slope disaster monitoring method, as shown in fig. 4, and the embodiment of the present invention provides a multi-scale slope disaster monitoring system, which includes:
the data acquisition module 401 is configured to acquire a slope video of a to-be-monitored slope based on binocular vision, and calculate to obtain a slope displacement result, where the slope video is a slope image of a continuous frame;
the dynamic detection module 402 is configured to process the side slope video by using an inter-frame difference method to obtain a dynamic detection result;
the deep learning module 403 is configured to process the slope video through a multi-scale deep learning algorithm to obtain a deep learning detection result;
the weight setting module 404 is configured to set a weight value between a dynamic detection result and a deep learning detection result according to a live-action parameter of the slope video;
and the comprehensive evaluation module 405 is configured to obtain a side slope falling stone result of the side slope to be monitored according to the dynamic detection result, the deep learning detection result and the weight value.
The implementation principle of the embodiment of the invention is as follows:
the method comprises the steps that a side slope is monitored in real time through a preset binocular camera, a data acquisition module 401 acquires a side slope video of the side slope to be monitored based on binocular vision, a side slope displacement result is obtained through calculation, a dynamic detection module 402 processes the side slope video through an inter-frame difference method to obtain a dynamic detection result, a deep learning module 403 processes the side slope video through a multi-scale deep learning algorithm to obtain a deep learning detection result, a weight setting module 404 sets a weight value between the dynamic detection result and the deep learning detection result according to a real-scene parameter of the side slope video, and a comprehensive evaluation module 405 obtains a side slope falling stone result of the side slope to be monitored according to the dynamic detection result, the deep learning detection result and the weight value. The method has the advantages that the side slope video is acquired through binocular vision, the side slope displacement is monitored, the side slope falling rocks are monitored through dynamic detection and deep learning detection, the real-time displacement and falling rocks monitoring of the side slope are realized, and the side slope disaster detection is free from the limitation of manual observation; and the dynamic detection result and the deep learning detection result are combined for comprehensive treatment, so that the accuracy of detecting the side slope disasters is improved.
Based on the embodiment shown in fig. 4 above, preferably, in some embodiments of the present invention, the dynamic detection module 402 includes:
the inter-frame difference calculation unit is used for carrying out difference operation on the slope image data of two or three continuous frames in the slope video by utilizing an inter-frame difference method to obtain the gray level difference absolute value of each pixel point;
the judging unit is used for judging whether the gray level difference absolute value of the target pixel point exceeds a preset absolute value;
the dynamic processing unit is used for determining the target pixel point as a moving target when the gray level difference absolute value of the target pixel point exceeds a preset absolute value, performing median filtering and expansion corrosion processing, performing feature analysis and experience modeling on the processed moving target, and screening to obtain a target area where the moving target is positioned to obtain a dynamic detection result;
and the dynamic processing unit is also used for determining that no moving target exists when the absolute value of the gray difference of the target pixel point does not exist exceeds the preset absolute value.
The detection of the moving target is realized by an inter-frame difference method, and the median filtering and the expansion corrosion treatment can effectively reduce noise and interference, so that the dynamic detection result is more accurate.
Based on the embodiment shown in fig. 4 above, it is preferable that in some embodiments of the present invention, the deep learning module includes:
the feature extraction unit is used for extracting features of the slope images in the slope video through the backbone network Darknet53 to obtain at least three feature layers;
the enhanced feature extraction unit is used for constructing a feature pyramid FPN to extract enhanced features of at least three feature layers to obtain three enhanced feature layers;
the deep learning unit is used for calculating and obtaining a prediction result based on a multi-scale deep learning algorithm and three reinforcement feature layers;
and the prediction processing unit is used for decoding the prediction result to obtain a prediction frame position and obtaining a deep learning detection result according to the prediction frame position.
The multi-scale side slope disaster monitoring method and the multi-scale side slope disaster monitoring system are described in the above embodiments, and the multi-scale side slope disaster monitoring device is described below by way of embodiments, as shown in fig. 5, including:
an image data acquisition system 501, an edge computing system 502, a cloud server 503 and a client 504;
the image data acquisition system 501 is used for acquiring a side slope video of a side slope to be monitored based on binocular vision, and calculating to obtain a side slope displacement result, wherein the side slope video is a side slope image of continuous frames;
the edge computing system 502 is used for processing the side slope video by utilizing an inter-frame difference method to obtain a dynamic detection result, processing the side slope video by utilizing a multi-scale deep learning algorithm to obtain a deep learning detection result, setting a weight value between the dynamic detection result and the deep learning detection result according to the real scene parameter of the side slope video, and obtaining a side slope falling stone result of the side slope to be monitored according to the dynamic detection result, the deep learning detection result and the weight value;
the cloud server 503 is configured to receive and store a slope video, a slope displacement result, and a slope stone falling result, and perform data interaction with the client 504.
The implementation principle of the embodiment of the invention is as follows:
the image data acquisition system 501 acquires a side slope video of a side slope to be monitored based on binocular vision, calculates a side slope displacement result, the edge calculation system 502 processes the side slope video by using an inter-frame difference method to obtain a dynamic detection result, processes the side slope video by using a multi-scale deep learning algorithm to obtain a deep learning detection result, sets a weight value between the dynamic detection result and the deep learning detection result according to a live-action parameter of the side slope video, and obtains a side slope falling stone result of the side slope to be monitored according to the dynamic detection result, the deep learning detection result and the weight value. The cloud server 503 is configured to receive and store a slope video, a slope displacement result, and a slope stone falling result, and perform data interaction with the client 504. The method has the advantages that the side slope video is acquired through binocular vision, the side slope displacement is monitored, the side slope falling rocks are monitored through dynamic detection and deep learning detection, the real-time displacement and falling rocks monitoring of the side slope are realized, and the side slope disaster detection is free from the limitation of manual observation; and the dynamic detection result and the deep learning detection result are combined for comprehensive treatment, so that the accuracy of detecting the side slope disasters is improved.
Based on the same technical scheme, the invention also discloses a computing device, which comprises one or more processors, one or more memories and one or more programs, wherein the one or more programs are stored in the one or more memories and are configured to be executed by the one or more processors, and the one or more programs comprise instructions for executing the multi-scale slope disaster monitoring method.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The foregoing is illustrative of the present invention and is not to be construed as limiting thereof, but rather as providing for the use of additional embodiments and advantages of all such modifications, equivalents, improvements and similar to the present invention are intended to be included within the scope of the present invention as defined by the appended claims.
Claims (10)
1. The multi-scale slope disaster monitoring method is characterized by comprising the following steps of:
acquiring a side slope video of a side slope to be monitored based on binocular vision, and calculating to obtain a side slope displacement result, wherein the side slope video is a side slope image of continuous frames;
processing the slope video by using an inter-frame difference method to obtain a dynamic detection result;
processing the slope video through a multi-scale deep learning algorithm to obtain a deep learning detection result;
setting a weight value between the dynamic detection result and the deep learning detection result according to the live-action parameters of the slope video;
and obtaining a side slope falling stone result of the side slope to be monitored according to the dynamic detection result, the deep learning detection result and the weight value.
2. The multi-scale slope disaster monitoring method according to claim 1, wherein the capturing the slope video of the slope to be monitored based on binocular vision and calculating the slope displacement result comprises:
obtaining calibration information of a binocular camera, and setting targets on a slope to be monitored;
collecting a side slope video of the side slope to be monitored through the binocular camera, wherein the side slope video is a side slope image of continuous frames;
identifying target pixel coordinates of a target in each frame of side slope image;
calculating a target space coordinate of the target by combining the target pixel coordinate and the calibration information based on a space coordinate calculation algorithm of binocular vision;
and comparing the target space coordinates with the original space coordinates of the target to obtain a displacement value of the target, wherein the displacement value is used as a slope displacement result.
3. The method for monitoring a multi-scale slope disaster according to claim 1, wherein the processing the slope video by using an inter-frame difference method to obtain a dynamic detection result comprises:
performing differential operation on two or three continuous frames of slope images in the slope video by using an inter-frame difference method to obtain the gray level difference absolute value of each pixel point;
judging whether the gray level difference absolute value of the target pixel point exceeds a preset absolute value;
if the dynamic detection result exists, the target pixel point is determined to be a moving target, and a dynamic detection result is obtained;
if not, determining that there is no moving object.
4. The method for monitoring a multi-scale slope disaster according to claim 3, wherein said determining the target pixel point as a moving target, to obtain a dynamic detection result, comprises:
determining the target pixel point as a moving target;
performing median filtering and expansion corrosion treatment;
and performing characteristic analysis and empirical modeling on the processed moving target, and screening to obtain a target area where the moving target is positioned, thereby obtaining a dynamic detection result.
5. The method for monitoring the disaster of the multi-scale side slope according to claim 1, wherein the processing the side slope video by the multi-scale deep learning algorithm to obtain the deep learning detection result comprises the following steps:
feature extraction is carried out on the slope images in the slope video through a backbone network Darknet53, so that at least three feature layers are obtained;
constructing a feature pyramid FPN to extract the reinforced features of the at least three feature layers to obtain three reinforced feature layers;
calculating to obtain a prediction result based on a multi-scale deep learning algorithm and the three reinforcement feature layers;
decoding the prediction result to obtain a prediction frame position;
and obtaining a deep learning detection result according to the predicted frame position.
6. The method for monitoring a multi-scale slope disaster according to claim 1, wherein the step of setting a weight value between the dynamic detection result and the deep learning detection result according to the live-action parameters of the slope video comprises:
obtaining real scene parameters according to the slope video;
obtaining a definition value according to the live-action parameters;
when the definition degree value exceeds a preset degree value, setting a weight value K1 between the dynamic detection result and the deep learning detection result, wherein the weight value K1 represents that the weight of the deep learning detection result is larger than that of the dynamic detection result;
and when the definition degree value does not exceed a preset degree value, setting a weight value K2 between the dynamic detection result and the deep learning detection result, wherein the weight value K2 represents that the weight of the dynamic detection result is larger than that of the deep learning detection result.
7. A multi-scale slope disaster monitoring system, comprising:
the data acquisition module is used for acquiring a side slope video of a side slope to be monitored based on binocular vision, and calculating to obtain a side slope displacement result, wherein the side slope video is a side slope image of continuous frames;
the dynamic detection module is used for processing the slope video by utilizing an inter-frame difference method to obtain a dynamic detection result;
the deep learning module is used for processing the slope video through a multi-scale deep learning algorithm to obtain a deep learning detection result;
the weight setting module is used for setting a weight value between the dynamic detection result and the deep learning detection result according to the live-action parameters of the slope video;
and the comprehensive evaluation module is used for obtaining the side slope falling stone result of the side slope to be monitored according to the dynamic detection result, the deep learning detection result and the weight value.
8. The multi-scale slope disaster monitoring system of claim 7, wherein said dynamic detection module comprises:
the inter-frame difference calculation unit is used for carrying out difference operation on the continuous two-frame or three-frame slope images in the slope video by utilizing an inter-frame difference method to obtain the gray level difference absolute value of each pixel point;
the judging unit is used for judging whether the gray level difference absolute value of the target pixel point exceeds a preset absolute value;
the dynamic processing unit is used for determining the target pixel point as a moving target when the gray level difference absolute value of the target pixel point exceeds a preset absolute value, performing median filtering and expansion corrosion processing, performing feature analysis and empirical modeling on the processed moving target, and screening to obtain a target area where the moving target is positioned to obtain a dynamic detection result;
and the dynamic processing unit is also used for determining that no moving target exists when the absolute value of the gray level difference of the target pixel point does not exist exceeds the preset absolute value.
9. The multi-scale slope disaster monitoring system according to claim 7, wherein said deep learning module comprises:
the feature extraction unit is used for extracting features of the slope images in the slope video through a backbone network Darknet53 to obtain at least three feature layers;
the enhanced feature extraction unit is used for constructing a feature pyramid FPN to extract enhanced features of the at least three feature layers to obtain three enhanced feature layers;
the deep learning unit is used for calculating and obtaining a prediction result based on a multi-scale deep learning algorithm and the three reinforcement feature layers;
and the prediction processing unit is used for decoding the prediction result to obtain a prediction frame position, and obtaining a deep learning detection result according to the prediction frame position.
10. A multi-scale slope disaster monitoring device, comprising:
the system comprises an image data acquisition system, an edge computing system, a cloud server and a client;
the image data acquisition system is used for acquiring a side slope video of a side slope to be monitored based on binocular vision, and calculating to obtain a side slope displacement result, wherein the side slope video is a side slope image of continuous frames;
the edge computing system is used for processing the side slope video by utilizing an interframe difference method to obtain a dynamic detection result, processing the side slope video by utilizing a multi-scale deep learning algorithm to obtain a deep learning detection result, setting a weight value between the dynamic detection result and the deep learning detection result according to the live-action parameters of the side slope video, and obtaining a side slope falling stone result of the side slope to be monitored according to the dynamic detection result, the deep learning detection result and the weight value;
the cloud server is used for receiving and storing the slope video, the slope displacement result and the slope rockfall result, and performing data interaction with the client.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310192166.7A CN116228712A (en) | 2023-03-02 | 2023-03-02 | Multi-scale slope disaster monitoring method, system and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310192166.7A CN116228712A (en) | 2023-03-02 | 2023-03-02 | Multi-scale slope disaster monitoring method, system and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116228712A true CN116228712A (en) | 2023-06-06 |
Family
ID=86569221
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310192166.7A Pending CN116228712A (en) | 2023-03-02 | 2023-03-02 | Multi-scale slope disaster monitoring method, system and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116228712A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117423224A (en) * | 2023-09-27 | 2024-01-19 | 深圳市地质环境研究院有限公司 | Data acquisition method of slope monitoring internet of things equipment |
-
2023
- 2023-03-02 CN CN202310192166.7A patent/CN116228712A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117423224A (en) * | 2023-09-27 | 2024-01-19 | 深圳市地质环境研究院有限公司 | Data acquisition method of slope monitoring internet of things equipment |
| CN117423224B (en) * | 2023-09-27 | 2024-08-23 | 深圳市地质环境研究院有限公司 | Data acquisition method of slope monitoring internet of things equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107256225B (en) | Method and device for generating heat map based on video analysis | |
| CN110610150B (en) | Tracking method, device, computing equipment and medium of target moving object | |
| CN105574894B (en) | A kind of screening technique and system of moving object feature point tracking result | |
| CN108564579B (en) | Concrete crack detection method and detection device based on time-space correlation | |
| CN104657945A (en) | Infrared small target detection method for multi-scale spatio-temporal union filtering under complex background | |
| CN102073851A (en) | Method and system for automatically identifying urban traffic accident | |
| KR101051459B1 (en) | Apparatus and method for extracting edges of an image | |
| CN101286239A (en) | Aerial shooting traffic video frequency vehicle rapid checking method | |
| CN111340749B (en) | Image quality detection method, device, equipment and storage medium | |
| JP4999794B2 (en) | Still region detection method and apparatus, program and recording medium | |
| CN110415208A (en) | A kind of adaptive targets detection method and its device, equipment, storage medium | |
| CN115294035B (en) | Bright spot positioning method, bright spot positioning device, electronic equipment and storage medium | |
| CN116343436A (en) | Landslide detection method, landslide detection device, landslide detection equipment and landslide detection medium | |
| CN116228712A (en) | Multi-scale slope disaster monitoring method, system and device | |
| CN115311623A (en) | Equipment oil leakage detection method and system based on infrared thermal imaging | |
| KR101690050B1 (en) | Intelligent video security system | |
| CN103489201A (en) | Method for tracking target based on motion blur information | |
| CN106778822B (en) | Image straight line detection method based on funnel transformation | |
| CN112330618A (en) | Image offset detection method, device and storage medium | |
| CN113592801B (en) | Stripe interference detection method and device for video image | |
| CN103473753A (en) | Target detection method based on multi-scale wavelet threshold denoising | |
| CN111860326B (en) | Building site article movement detection method, device, equipment and storage medium | |
| CN112581489A (en) | Video compression method, device and storage medium | |
| US20240221181A1 (en) | Image processing system | |
| CN114359183A (en) | Image quality evaluation method and device, and lens occlusion determination method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |