CN116453278A - Intrusion monitoring method combining deep learning intelligent detection and optical fiber vibration sensing - Google Patents

Abstract

The invention discloses an intrusion monitoring method combining deep learning intelligent detection and optical fiber vibration sensing, comprising the following steps: (1) receiving early warning information reported by an optical fiber vibration sensing device, and simultaneously acquiring an image of an optical fiber vibration; (2) Distinguish the environmental scene and application scene of the intrusion site, and construct a deep learning model library based on the differentiated multi-scenario; (3) synthesize the current environmental scene and application scene of the optical fiber vibration sensing device, according to wind force, illumination and For viewing angle information, match the corresponding deep learning model from the deep learning model library to perform target detection on the image to determine the final intrusion alarm. The trained deep learning model library in the present invention can intelligently monitor and identify intrusion behaviors by scene more accurately.

Description

An Intrusion Monitoring Method Combining Deep Learning Intelligent Detection and Optical Fiber Vibration Sensing

technical field

The invention belongs to the field of security protection, and in particular relates to an intrusion monitoring method combining deep learning intelligent detection and optical fiber vibration sensing.

Background technique

In recent years, distributed optical fiber sensing technology has developed rapidly, which has the advantages of high precision, fast response speed and wide dynamic range. Optical cable is used as the sensor and signal transmission medium, no on-site power supply is required, and it has the characteristics of intrinsic safety, anti-corrosion and explosion-proof, etc. It can effectively monitor high-voltage cable anti-theft, submarine cable disturbance monitoring, petrochemical pipeline anti-theft monitoring, and distributed optical fiber intrusion security along the railway. Other distributed optical fiber perimeter security, etc., eliminate security accidents in the bud and prevent problems before they happen.

Sensing technology based on distributed optical fiber has been widely used in perimeter security, seismic exploration, rail transit and pipeline transportation and other fields. Distributed optical fiber sensors mainly have two deployment methods: buried and fence. The fence deployment method is mainly used for perimeter security monitoring of factories, airports, military bases and prisons. The buried deployment method is mainly used to prevent the underground wires, oil pipelines and gas pipelines from being damaged by manual or mechanical excavation.

At the same time, smart camera technology is playing an increasingly important role in national defense and major livelihood projects. For example, in the field of electric power, the external force of excavators will damage underground transmission lines, seriously threatening the safety of power supply for industrial production and people's lives. At present, many transmission corridor projects have been monitored by remote cameras, but operation and maintenance personnel are required to monitor on the job for a long time, and the efficiency is very low. In the field of oil and gas pipelines, pipeline accidents caused by excavator construction occur from time to time.

Using optical fiber sensing technology alone for intrusion monitoring has a high false alarm rate. For example, the shaking of the optical cable fence in a windy day will be mistaken for intrusion, and the optical cable buried near the road will vibrate when large vehicles pass by, causing false alarms. Using camera technology alone for intrusion monitoring also has a high rate of false alarms. For example, camera shake in severe weather conditions such as strong winds, and camera blur in insufficient light conditions will all generate false alarms. How to efficiently and accurately monitor destructive intrusions around the clock has become an urgent problem to be solved.

Contents of the invention

The purpose of the present invention is to overcome the deficiencies in the above-mentioned prior art, to distinguish the environmental scene and the application scene on the spot, to consider multiple environmental scenes and multi-dimensional constructions to cover the deep learning model library of each destructive intrusion behavior in different application scenes, and to adopt distributed The intrusion detection signal of the optical fiber cable triggers the camera to collect images of the intrusion site, establishes an image sample library as a training data set, and inputs the training data set into the deep learning model library for training. The trained deep learning model library can more accurately classify the scene Intelligent monitoring and identification of intrusion behavior.

In order to achieve the purpose of the above invention, the present invention provides an intrusion monitoring method combining deep learning intelligent detection and optical fiber vibration sensing, including the following steps: (1) receiving the intrusion warning information reported by the optical fiber vibration sensing device, and simultaneously obtaining the optical fiber vibration location The image of the intrusion warning information includes at least position, wind factor, width, duration and intensity; (2) distinguish the environmental scene and the application scene of the intrusion site, and build a deep learning model library based on the differentiated multi-scenario; ( 3) Integrating the current environmental scene and application scene of the optical fiber vibration sensing device, according to the wind force, illumination and viewing angle information, matching the corresponding deep learning model from the deep learning model library to perform target detection on the image, to Determine the final intrusion alert.

Further, the environmental scene in step (2) includes weather conditions, lighting conditions, and image sampling angle positions; the application scene includes fence application defense areas and buried application defense areas.

Further, in the step (1), the way to acquire images is to regularly collect and cache the monitoring images taken by the fixed camera device preset in the optical fiber vibration monitoring area, and call when the optical fiber vibration sensing device alarms The latest cached image at the reported intrusion alarm location.

Further, in the step (1), the way of acquiring images is to schedule a movable camera device preset in the optical fiber vibration monitoring area to shoot images at the intrusion alarm position reported by the optical fiber vibration sensing device.

Further, the movable camera device includes an aerial patrol drone, a patrol robot and a rotating camera.

Further, before performing target detection through the deep learning model, first use the background difference method to detect whether there is an active target in the image, if there is an active target, then perform target detection through the deep learning model; if there is no active target, there is no need to continue Target Detection.

Further, the defense zone is applied to the fence, and at least deep learning model libraries for anti-climbing, anti-shearing and anti-knocking are respectively constructed.

Further, for the buried application defense zone, at least deep learning model libraries for anti-excavation and anti-pipe jacking are respectively constructed.

Further, the deep learning model library is constructed from three dimensions of weather conditions, lighting conditions, and image sampling perspective positions, wherein the weather conditions include normal weather and windy weather, the lighting conditions include scenes including normal lighting and poor lighting, and the image sampling perspective The location includes a high-altitude perspective and a ground perspective; the deep learning model library includes multiple deep learning models, and different value combinations of the three dimensions correspond to a deep learning model.

Further, the training method of the deep learning model library includes the following steps: (a) according to the intrusion warning information detected by the optical fiber vibration sensing device, the collected intrusion site images are automatically marked, and an image sample library is established accordingly for training Data set; (b) training the training data set based on the deep learning target detection algorithm; (c) updating the corresponding deep learning model library according to the application scene, wind force, illumination, and viewing angle information.

Further, it is characterized in that the image sample library in step (a) includes a positive sample library and a negative sample library, the positive sample library is an image set with destructive intrusion behavior, and the negative sample library is an image set without damage Collection of images of sexual assault.

Further, it is judged whether there is an active target in the intrusion scene image, if yes, the intrusion scene image is included in the positive sample library, and if not, the intrusion scene image is included in the negative sample library.

Further, a background difference method is used to judge whether there is an active target in the intrusion scene image.

Further, the intrusion scene images with active targets must also meet the optical fiber vibration alarm threshold before being included in the positive sample library, otherwise included in the negative sample library.

Further, adding labeling information to the active target in the intrusion scene image, the labeling information includes a labeling frame and a label, the labeling frame is the largest bounding rectangle of the largest connected area of the active target, and the label is used to mark the active target The belonging intrusion scene image is a positive sample or a negative sample.

Further, the intrusion site image and its annotation information are stored by scene classification, so as to be used for different deep learning models in the deep learning model library.

Further, in the step (b), the YOLOv5 target detection algorithm is used for training, and the trained deep learning model is suitable for screening the annotation frame of the input intrusion scene image in a weighted non-maximum manner, and finally outputs the activity Classification conclusion and bounding box regression of whether the target has destructive intrusion behavior.

Further, the vibration detection sensitivity of the optical fiber vibration sensing device in step (1) is adjustable.

Compared with prior art, the beneficial effect of the present invention is:

1. The distributed vibration optical cable is used as the trigger device to reduce the false alarm of the camera at night or when the line of sight is unclear. No alarm will be issued for normal construction and excavation of farmland and roads near the pipeline, which reduces false alarms in the application of buried pipelines. Intelligent camera technology is used to further identify and confirm the alarms caused by the shaking of the fence caused by strong winds, so as to reduce false alarms of perimeter security.

2. Construct a deep learning model library for each sabotage behavior from the three dimensions of illumination, viewing angle, and weather, which can identify intrusion behaviors in a more precise and sub-scene. Differentiate the training data of the scene, the training model is more targeted, and the recognition effect is better.

3. Using fiber optic sensor vibration detection technology, the camera is triggered to collect real intrusion images in the real environment, and a sample library is established. The camera angle and image quality are more in line with the on-site environment. The trained model is more targeted and has better performance.

4. The background difference technology is used to predetermine the moving target for automatic labeling, which improves the efficiency and accuracy of labeling.

Description of drawings

Fig. 1 is a flowchart of an embodiment of the present invention.

Fig. 2 is a schematic diagram of a deep learning model library in an embodiment of the present invention.

Fig. 3 is a schematic diagram of detecting an excavator's digging action (moving target) in an image by background subtraction method in an embodiment of the present invention.

Fig. 4 is a schematic diagram of a comparison between real intrusion behavior and disturbance in an embodiment of the present invention.

Detailed ways

The technical solutions of the present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

As shown in Figure 1, an embodiment of the intrusion monitoring method combined with deep learning intelligent detection and optical fiber vibration sensing of the present invention includes the following steps: (1) receiving the intrusion warning information reported by the optical fiber vibration sensing device, and simultaneously obtaining the optical fiber vibration The image at the location; the intrusion warning information includes at least position, wind factor, width, duration and intensity; (2) distinguish the environmental scene and the application scene of the intrusion site, and build a deep learning model library based on the differentiated multi-scenario; (3) Synthesizing the current environmental scene and application scene of the optical fiber vibration sensing device, according to the wind force, illumination and viewing angle information, matching the corresponding deep learning model from the deep learning model library to perform target detection on the image, To determine the final intrusion alarm.

In one embodiment, the environmental scene in step (2) includes weather conditions, lighting conditions, and image sampling angle positions; the application scene includes fence application defense areas and buried application defense areas.

In one embodiment, in the step (1), the way of acquiring images is to regularly collect and cache the monitoring images taken by the fixed camera device preset in the optical fiber vibration monitoring area. Call the latest cached image at the reported intrusion alarm position when an alarm is issued.

In one embodiment, in the step (1), the way of acquiring images is to schedule shooting at the intrusion alarm position reported by the optical fiber vibration sensing device with the movable camera device preset in the optical fiber vibration monitoring area image.

In one embodiment, the movable camera device includes an aerial patrol drone, a patrol robot and a rotating camera.

In one embodiment, before performing target detection by the deep learning model, first use the background difference method to detect whether there is an active target in the image, if there is an active target, then perform target detection by the deep learning model; if there is no active target, then No further object detection is required.

In one embodiment, a defense zone is applied to the fence, and at least deep learning model libraries for anti-climbing, anti-shearing and anti-knocking are respectively constructed.

In one embodiment, for the buried application defense zone, at least deep learning model libraries for anti-excavation and anti-pipe jacking are respectively constructed.

In one embodiment, the deep learning model library is constructed from three dimensions of weather conditions, lighting conditions, and image sampling perspective positions, wherein the weather conditions include normal weather and windy weather, and the lighting conditions include scenes with normal lighting and poor lighting. The sampling perspective positions include high-altitude perspectives and ground perspectives; the deep learning model library includes multiple deep learning models, and different value combinations of the three dimensions correspond to one deep learning model.

In one embodiment, the training method of the deep learning model library includes the following steps: (a) according to the intrusion warning information detected by the optical fiber vibration sensing device, automatically label the images of the intrusion site collected, and establish an image sample library accordingly Used as a training data set; (b) training the training data set based on a deep learning target detection algorithm; (c) updating the corresponding deep learning model library according to the application scene and wind, illumination, and viewing angle information.

In one embodiment, it is characterized in that the image sample library in step (a) includes a positive sample library and a negative sample library, the positive sample library is an image set with destructive intrusion behavior, and the negative sample library is A collection of images in which destructive intrusions do not exist.

In one embodiment, it is judged whether there is an active target in the image of the intrusion scene, if yes, the image of the intrusion scene is included in the positive sample library, if not, the image of the intrusion scene is included in the negative sample library .

In one embodiment, a background difference method is used to judge whether there is an active target in the intrusion scene image.

In one embodiment, the intrusion scene images with active targets must also meet the optical fiber vibration alarm threshold before being included in the positive sample library, otherwise included in the negative sample library.

In one embodiment, annotation information is added to the active target in the intrusion scene image, the annotation information includes an annotation frame and a label, the annotation frame is the largest bounding rectangle of the largest connected area of the active object, and the label is used to mark The intrusion scene image to which the active target belongs is a positive sample or a negative sample.

In one embodiment, the intrusion scene image and its annotation information are stored by scene classification, so as to be used for different deep learning models in the deep learning model library.

In one embodiment, in the step (b), the YOLOv5 target detection algorithm is used for training, and the trained deep learning model is suitable for screening the label frame of the input intrusion scene image in a weighted non-maximum manner, The final output is the classification conclusion and border regression of whether there is destructive intrusion behavior in the active target.

In one embodiment, the vibration detection sensitivity of the optical fiber vibration sensing device in step (1) is adjustable.

The hardware modules involved in the present invention include an optical fiber sensing vibration detection device, a camera unit, and an intrusion target detection and comprehensive judgment module. The alarm signal is transmitted to the intrusion target detection and comprehensive judgment module. The intrusion target detection and comprehensive judgment module is responsible for dispatching the camera unit to capture intrusion images on the scene, and for intrusion recognition on the intrusion images, and finally integrates the optical fiber intrusion alarm and image intrusion target detection results to determine the final intrusion alarm and report the alarm to the user .

The basic process of intrusion detection is as follows:

Step 1: Detect intrusion signals based on optical fiber vibration

A fiber optic perimeter intrusion monitoring method based on image recognition provided by patent CN201610476700.7 can be used. The monitoring method is completed by a fiber optic vibration sensing system. The system includes a detection cable connected together, a monitoring host and a host computer; The monitoring method includes: the vibration caused by external intrusion is detected by the detection optical cable and the optical signal is transmitted to the monitoring host; the monitoring host first converts the received optical signal into an electrical signal, and then samples the electrical signal and converts it to analog to digital to obtain a discrete digital signal , the digital signal is transmitted to the host computer; the host computer processes the collected digital signal to obtain the characteristic quantity of the processed signal to form one or more waterfall diagrams, and perform image pattern recognition according to the shape of the waterfall diagram. If it is judged as an intrusion event, an intrusion alarm will be triggered.

Step 2: Carry out intrusion recognition on the intrusion image, and finally integrate the intrusion alarm of the optical fiber and the image intrusion target detection result to determine the final intrusion alarm and report the alarm to the user.

From the perspective of camera, the angle of view of high-altitude inspection is different from that of ordinary ground camera, so there will be a big difference between the training result and the actual deployment effect.

The coverage of high-altitude drone inspections is wider, and there is no need to lay a large number of cameras in a wide range. Therefore, from an engineering point of view, there are application scenarios for both high-altitude and ordinary ground perspectives.

High-altitude inspection Since the camera shoots from high altitude, the higher the height of the camera, the smaller the images of ground objects and target objects. The complex image background also has a certain impact on the recognition of excavators. Therefore, it is more difficult to accurately detect excavator objects under high-altitude inspections.

From the perspective of image quality, images during the day and night have a greater impact on target recognition. During the day, the sun is sufficient and the line of sight is clear, so target detection is relatively easy. At night, especially in the perimeter and remote pipeline corridor areas, there are often only partial lights, which increases the difficulty of target identification.

When the camera device is installed on a slightly higher gimbal and the wind force is relatively strong, such as above level 5-6, the gimbal will vibrate to a certain extent, and road inspection robots or drones will also vibrate, which increases the The difficulty of intrusion target detection.

The present invention constructs a deep learning model library from the three dimensions of illumination, viewing angle, and weather, as shown in Figure 2, which can identify intrusion behaviors in a more refined and sub-scene. The deep learning model library includes at least 8 deep learning models, which are the 8 small cubes in Figure 2, which are A. Normal lighting + high-altitude viewing angle + normal weather; B. Poor lighting + high-altitude viewing angle + normal weather; C. .Normal lighting + ground viewing angle + normal weather; D. Poor lighting + ground viewing angle + normal weather; E. Normal lighting + high-altitude viewing angle + windy weather; F. Poor lighting + high-altitude viewing angle + windy weather; G. Normal lighting + ground viewing angle + windy weather; H. Poor lighting + ground perspective + windy weather.

Step 1: Receive alarm information such as location, wind factor, width, duration, and intensity reported by the optical fiber vibration sensing device.

The calculation method of the wind force factor reported by the optical fiber vibration sensing device is as follows:

The waterfall image disclosed in the patent CN201610476700.7 can be used as a grayscale image between 0 and 255, and the average value of image grayscale values in a large windy wide area can be calculated as a representation of the wind force factor.

f _n is the gray value, d0 is the starting calculation position of the optical fiber in the windy area, and d1 is the end calculation position of the optical fiber in the windy area.

The waterfall diagram adopts the technical scheme of the patent CN201610476700.7. Each pixel can be the direct signal difference, signal variance, correlation value, power or energy characteristics of a certain frequency band after FFT transformation, and detailed energy of each scale after wavelet decomposition. feature.

Step 2: Synthesize the scene information of the current environment. include:

According to the position information of the defense area during the laying stage of the optical cable construction, whether the position is a fence or buried application,

Judging whether it is a windy scene according to the windy factor reported by the fiber optic vibration sensor,

According to the time of local sunlight intensity configuration in the camera debugging stage, determine the scenes with normal lighting and poor lighting. For example, from 7:00 a.m. to 5:00 p.m., it is a scene with normal lighting during the day, and the rest of the time is a scene with poor lighting at night;

According to the configuration of the camera installation and construction stage, whether the camera at this position is a high-altitude view or a normal view.

The judgment method of the windy scene is that if the factor _wind of the strong wind exceeds the threshold Thr _{wind of} the strong wind, it is judged as a windy scene, and if the threshold condition is not met, it is normal weather

Step 3: Select different deep learning models according to application scenarios

For the area where the fence is applied, you can choose from three deep learning model libraries of anti-climbing, anti-shear net and anti-knock impact, and each deep learning model library has the structure shown in Figure 2.

For the buried area, you can choose from two deep learning model libraries of anti-excavation and anti-pipe jacking, and each deep learning model library has the structure shown in Figure 2.

Step 4: Select the corresponding model from the deep learning model library for intelligent detection according to the wind, light, and viewing angle information. Determine the final intrusion alert.

At present, there are many types of devices that use cameras to monitor, including traditional fixed or rotating cameras, cameras configured by patrol robots that move on the ground or at high altitudes, and cameras that specialize in aerial patrol drones. For these non-fixed camera devices, it is necessary to schedule the camera to aim at the intrusion site according to the position information of the alarm information detected by the optical fiber vibration.

Deploying cameras to intrusion areas involves:

1) Schedule the rotatable camera to aim at the intruded location area

2) Dispatch the UAV to the site location area.

3) Dispatch inspection robots to the on-site location area

For a fixed camera or a camera that is already at a preset position, images can be cached for a period of time. If an alarm from the vibration sensor is triggered, these cached images can be directly called for deep learning target detection, which greatly saves the cost of scheduling cameras. time.

Using the background subtraction method, the active target is detected first in the captured image, so that the subsequent intrusion target detection is more accurate. This method can avoid false alarms when the excavator does not excavate.

The schematic diagram of detecting excavator digging action by background difference technique is shown in Fig.3.

The background subtraction method is commonly used to detect moving objects in video images, and is one of the mainstream methods for moving object detection at present. The basic principle is to subtract the current frame in the image sequence from the background reference model (background image) that has been determined or acquired in real time, and calculate the area where the pixel difference from the background image exceeds a certain threshold as the motion area, so as to determine the motion. Features such as object position, contour, size, etc. The difficulty of the background subtraction method is how to determine the background model through a certain number of images.

Step 1: Obtain a certain number of images for background modeling to obtain background image frame B.

The background modeling methods adopted in the present invention include median method background modeling, mean value method background modeling, single Gaussian distribution model, mixed Gaussian distribution model, Kalman filter model, advanced background model and the like.

Step 2: Subtract the gray value of the corresponding pixel of the current frame image and the background frame image, and take its absolute value to obtain the difference image

D _n (x,y)=|f _n (x,y)-B(x,y)|

The current video image frame is fn, and the gray values of the corresponding pixels in the background frame and the current frame are respectively recorded as B(x, y) and fn(x, y),

Step 3: Set the threshold T, and perform binarization processing on the pixels one by one to obtain a binarized image R. Among them, the point with a gray value of 255 is the foreground (moving target) point, and the point with a gray value of 0 is the background point;

Step 4: Perform connectivity analysis on the image R. If the connected contour area is greater than the set threshold, it is considered that there is a large-scale moving target, and an intrusion detection judgment is required.

training dataset

The invention adopts the optical fiber sensor vibration detection technology, triggers the camera to collect real intrusion images in the real environment, and establishes a sample library. The camera angle and image quality are more in line with the on-site environment, the training model is more targeted, and the performance is more excellent.

The invention adopts the background difference technology to predetermine the moving target and perform automatic labeling, thereby improving the efficiency and accuracy of labeling.

Step 1: Detect intrusion alarm based on optical fiber vibration

In the early stage of deep learning model training, when the amount of image data is small, consider setting the fiber optic vibration sensor as a relatively sensitive parameter to collect on-site data as much as possible.

Step 2: Use the background subtraction method to judge whether there is an active object.

Step 3: Label the active object, including:

Step 3.1: Annotate the callout box of the object

According to the maximum connected region of the active target obtained by the background difference method in step 2, determine the largest circumscribing rectangle of the region, and mark the largest circumscribing rectangle as a label box. Carry out connectivity analysis on the image R, if the connected outline area is greater than the set threshold, it is considered that there is a large-scale moving target, and an intrusion detection judgment is required. "

Step 3.2: Determine whether it is a positive sample or a negative sample according to the width and intensity of the vibrating optical fiber alarm in step 1 and the largest circumscribing rectangle in step 3.1.

In step 1, if the width of the alarm is greater than the width threshold Thr _Width and the intensity is greater than the threshold Thr _intensity , and in step 3.1 the largest circumscribed rectangle is greater than the area threshold, the intrusion image captured by the camera is considered as a positive sample, otherwise it is judged as a negative sample.

As shown in FIG. 4 , the width of the waterfall diagram of the real tap damage alarm is wider, while the disturbance of the vehicle on the road is a discontinuous and relatively narrow signal. The image captured by the camera triggered by a wide signal is a positive intrusion image, while the image captured by a camera triggered by a narrow signal is a negative sample (vehicle) image.

Step 4: According to different scenes, save the image and label information.

Step 4.1: Determine the scene information of the current environment. include:

According to the position information of the defense area during the laying stage of the optical cable construction, whether the position is a fence or buried application,

Judging whether it is a windy scene according to the windy factor reported by the fiber optic vibration sensor,

According to the time of local sunlight intensity configuration in the camera debugging stage, determine the scenes with normal lighting and poor lighting. For example, from 7:00 a.m. to 5:00 p.m., it is a scene with normal lighting during the day, and the rest of the time is a scene with poor lighting at night;

According to the configuration of the camera installation and construction stage, whether the camera at this position is a high-altitude view or a normal view.

The judging method of the windy scene is that if the windy factor Factor _wind exceeds the windy threshold Thr _wind , it is judged as a windy scene.

Step 4.2: Save the image and annotation information in step 3 according to the scene classification in 4.1.

The annotation information includes: the label frame of the target object, and the label information of whether the target object is a positive sample or a negative sample.

train

The training data set established by the aforementioned automatic labeling is manually reviewed and corrected for the image labeling information to improve the quality of the training data.

The mainstream target detection algorithms are roughly divided into two types: one-stage and two-stage. The two-stage algorithm represents the R-CNN series, and the one-stage algorithm represents the YOLO series. The two-stage algorithm first passes the input image through the candidate frame generation network (such as the RPN network in faster rcnn), and then passes through the classification network to classify the content of the candidate frame; the one-stage algorithm passes the input image through only one network, and the generated Results include both location and category information. Compared with one-stage, two-stage has high precision, but the calculation amount is larger, so the operation is slower.

Step 2: Train with deep learning techniques

The present invention uses the YOLOV5 target detection algorithm for model training.

The YOLO algorithm has been continuously updated and improved, gradually evolving from the initial YOLOV1 to YOLOV5.

The YOLOv5 model was released publicly by Ultralytics on June 9, 2020. The YOLOv5 model is improved based on the YOLOv3 model. There are four models: YOLOv5s, YOLOv5m, YOLOv5l, and YOLOv5x. The YOLOv5s network is the network with the smallest depth and the smallest width of the feature map in the YOLOv5 series, with the fastest running speed and the lowest AP accuracy. The other three networks, on this basis, continue to deepen and widen the network, and the AP accuracy is also continuously improved, but the amount of calculation is also increasing.

The YOLOv5 model is mainly composed of a backbone network and a head network module.

The input of the YOLOv5 model uses Mosaic data enhancement, adaptive anchor frame calculation, and adaptive image scaling technology to improve performance.

The backbone network of the YOLOv5 model mainly includes modules such as Conv convolution block, C3 structure and SPPF. The C3 structure is applied to the Backbone backbone network and is also used in the Head network. The C3 structure used by YOLOv5 can improve the network speed while ensuring accuracy.

The head of YOLOv5m uses multi-scale feature maps for detection, using large images to detect small targets, and small images to detect large targets. For the three different scale feature maps of the neck, through Conv, C3, Upsample and Concat operations, three feature maps with sizes of 80*80*255, 40*40*255, and 20*20*255 are finally obtained.

In the embodiment of the present invention, a 640*640 picture and a YOLOv5m model are used. In the embodiment of the present invention, there are two types of detection results for intrusion and destruction, for example, with mining behavior and without mining behavior, and three types of candidate frames with different sizes are used to predict the three scale feature maps. Finally, the target box is screened by weighted non-maximum value, and the target classification and frame regression are output.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that the present invention can still be Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention shall fall within the protection scope of the claims of the present invention.

Claims (18)

1. The intrusion monitoring method combining deep learning intelligent detection and optical fiber vibration sensing is characterized by comprising the following steps of:

(1) Receiving intrusion early warning information reported by an optical fiber vibration sensing device, and simultaneously acquiring an image of an optical fiber vibration position;

(2) Distinguishing environmental scenes and application scenes of an intrusion site, and constructing a deep learning model library based on the distinguished multiple scenes;

(3) And integrating the current environment scene and application scene of the optical fiber vibration sensing device, matching corresponding deep learning models from the deep learning model library according to wind power, illumination and visual angle information, and performing target detection on the image so as to determine the final intrusion alarm.

2. The method of intrusion monitoring combined with fiber vibration sensing for deep learning intelligent detection of claim 1, wherein the environmental scene in step (2) includes weather conditions, lighting conditions, and image sampling perspective positions; the application scene comprises a fence application defense area and a buried application defense area.

3. The method for monitoring intrusion by combining deep learning intelligent detection and optical fiber vibration sensing according to claim 1, wherein in the step (1), the image acquisition mode is to acquire and cache monitoring images shot by a fixed camera device preset in the optical fiber vibration monitoring area at fixed time, and the latest cache image at the reported intrusion alarm position is called when the optical fiber vibration sensing device alarms.

4. The method for intrusion detection by combining deep learning intelligent detection with optical fiber vibration sensing according to claim 1, wherein in the step (1), the image is acquired by scheduling a movable camera device preset in the optical fiber vibration monitoring area to be aligned with an intrusion alarm position reported by the optical fiber vibration sensing device.

5. The method for intrusion detection combined with fiber vibration sensing according to claim 4, wherein the movable camera device comprises an unmanned aerial vehicle for patrol, a patrol robot and a rotary camera.

6. The method for intrusion detection by combining deep learning intelligent detection with optical fiber vibration sensing according to claim 3 or 4, wherein before object detection by a deep learning model, a background difference method is used to detect whether a moving object exists in an image, if so, object detection is performed by the deep learning model; if there is no active target, then no target detection needs to be continued.

7. The method for intrusion detection by combination of deep learning intelligent detection and optical fiber vibration sensing according to claim 2, wherein a defense area is applied to the fence, and at least a deep learning model library of anti-climbing, anti-shearing and anti-knock impact is constructed respectively.

8. The method for intrusion detection by combination of deep learning intelligent detection and optical fiber vibration sensing according to claim 2, wherein a deep learning model library for preventing excavation and jacking pipe is constructed for the buried application defense area at least respectively.

9. The method for intrusion monitoring combined with fiber vibration sensing according to claim 2, wherein the deep learning model library is constructed from three dimensions of weather conditions including normal weather and strong wind weather, lighting conditions including light normal and light poor scenes, and image sampling view angle positions including high altitude view angle and ground view angle; the deep learning model library comprises a plurality of deep learning models, and different value combinations of three dimensions correspond to one deep learning model.

10. The method for intrusion monitoring combined with fiber vibration sensing for deep learning intelligent detection according to any one of claims 1-9, wherein the training method for the deep learning model library comprises the following steps:

(a) According to intrusion early warning information detected by the optical fiber vibration sensing device, automatically labeling the acquired intrusion site image, and accordingly establishing an image sample library to be used as a training data set;

(b) Training the training data set based on a deep learning target detection algorithm;

(c) And updating the corresponding deep learning model library according to the application scene, wind power, illumination and visual angle information.

11. The method of claim 10, wherein the image sample library in step (a) comprises a positive sample library and a negative sample library, the positive sample library being an image set with destructive intrusion behavior and the negative sample library being an image set without destructive intrusion behavior.

12. The method of claim 10, wherein determining whether a moving object exists in the intrusion field image, if so, includes the intrusion field image into the positive sample library, and if not, includes the intrusion field image into the negative sample library.

13. The method for intrusion detection by combining deep learning intelligent detection with fiber vibration sensing according to claim 12, wherein a background difference method is used to determine whether a moving target exists in the intrusion scene image.

14. The method of claim 12, wherein the intrusion field image with moving objects is incorporated into the positive sample library only if the intrusion field image meets a fiber vibration alert threshold, otherwise the intrusion field image is incorporated into the negative sample library.

15. The method for intrusion monitoring by combining deep learning intelligent detection with optical fiber vibration sensing according to any one of claims 10 to 14, wherein labeling information is added to a moving object in an intrusion scene image, the labeling information comprises a labeling frame and a label, the labeling frame is a maximum circumscribed rectangle of a maximum communication area of the moving object, and the label is used for marking the intrusion scene image to which the moving object belongs as a positive sample or a negative sample.

16. The method of any one of claims 10-14, wherein the scene images and their annotation information are stored in scene categories for use in different deep learning models in the library of deep learning models.

17. The method for intrusion detection by combination of deep learning intelligent detection and optical fiber vibration sensing according to any one of claims 10 to 14, wherein the training is performed in the step (b) by using a YOLOv5 target detection algorithm, the trained deep learning model is suitable for screening the label frame of the input intrusion field image in a weighted non-maximum manner, and finally, the classification conclusion and the frame regression of whether the destructive intrusion behavior exists in the moving target are output.

18. The method for intrusion detection combined with optical fiber vibration sensing according to claim 1, wherein the sensitivity of the optical fiber vibration sensing device for detecting vibration in step (1) is adjustable.