CN111898563A - Comprehensive safety monitoring equipment and method for protected area - Google Patents
Comprehensive safety monitoring equipment and method for protected area Download PDFInfo
- Publication number
- CN111898563A CN111898563A CN202010770673.0A CN202010770673A CN111898563A CN 111898563 A CN111898563 A CN 111898563A CN 202010770673 A CN202010770673 A CN 202010770673A CN 111898563 A CN111898563 A CN 111898563A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- unmanned aerial
- aerial vehicle
- image
- monitoring
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
Abstract
The invention relates to a comprehensive safety monitoring device and a comprehensive safety monitoring method for a protected area, and mainly solves the technical problems that the existing construction monitoring cannot realize all-dimensional monitoring and needs manual intervention and the like. The equipment comprises a fixed-point monitoring system, a daily patrol vehicle, an unmanned aerial vehicle monitoring system and a cloud processing system; the fixed-point monitoring system, the daily inspection vehicle and the unmanned aerial vehicle monitoring system jointly carry out multi-dimensional data acquisition and front-end data processing on the protected area, the cloud processing system receives data of the acquisition system, intelligently identifies construction behaviors by using a deep learning model, and simultaneously sends violation operation types and related information to the data management platform and the mobile phone terminal. Compared with the prior art, the method has the advantages of integrating multiple systems to acquire data at different visual angles, combining front and rear ends to identify illegal operation, acquiring all-around data, processing multi-level data, being high in identification precision, having no dead angle in monitoring and the like.
Description
Technical Field
The invention relates to a comprehensive safety monitoring device and a monitoring method for a protection area, in particular to a comprehensive safety monitoring device and a monitoring method for a ground protection area of underground engineering such as tunnels, pipe galleries and the like, which are applied to the field of safety monitoring of the protection area.
Background
The traditional on-site safety monitoring measures mainly include: firstly, the field inspection by workers is relied on; secondly, a monitoring camera is installed at a fixed position to collect video image information, and then background workers carry out on-site monitoring through a screen. Because the construction field range is large, a small amount of monitoring personnel flow to patrol and inspect and are difficult to meet the requirements of all-dimensional real-time monitoring, the video monitoring is difficult to monitor the construction operation of the field personnel at a fixed position, and background watching personnel are easy to have the problems of fatigue and missed detection, the problems are particularly prominent at night and in early morning hours, meanwhile, different operation types are not effectively managed, and the early warning mode is single and lagged.
The utility model relates to a movable road construction monitoring system (CN201721547496. X), which provides a movable construction monitoring method consisting of a wind power generation system, a photovoltaic power generation system, a wireless video monitoring system, a dust noise wind power information detection system and an energy storage system; the utility model discloses a building site construction monitored control system (CN 201820161951.0) proposes a modularization construction monitored control system based on host computer unit, power supply unit and customer end. However, the above patents have the following problems: 1. the construction part can be monitored only from a single visual angle and height, and the construction site cannot be monitored in all directions in real time; 2. the judgment of the site violation needs later-stage manual participation, so that the judgment accuracy is difficult to ensure while the monitoring cost is increased; 3. and the automatic early warning function is not provided, and the violation early warning still needs manual participation.
In summary, there is an urgent need for a comprehensive safety monitoring device and method for a protected area with global monitoring, high automation, intelligence and automatic early warning.
Disclosure of Invention
The invention aims to provide a comprehensive monitoring device and a monitoring method for safety of a protected area. The main technical problem that the existing construction monitoring cannot realize all-round monitoring and needs manual intervention and the like is solved.
The purpose of the invention can be realized by the following technical scheme:
the utility model provides a guardianship equipment is synthesized to protection zone safety, includes fixed point monitored control system, daily car, unmanned aerial vehicle monitored control system and high in the clouds processing system that patrols and examines, fixed point monitored control system, daily car, unmanned aerial vehicle monitored control system and high in the clouds processing system pass through internet access, fixed point monitored control system, daily car and unmanned aerial vehicle monitored control system jointly carry out video, sound, vibration multisource signal acquisition, high in the clouds processing system deploys at high in the clouds server, stores fixed point monitored control system, daily car and the unmanned aerial vehicle monitored control system that patrols and examines to based on the degree of deep learning model of training in advance and carry out automatic identification and location to the interior construction safety action of protection zone, send the illegal operation action of discernment to data management platform and cell-phone terminal simultaneously.
A comprehensive safety monitoring method for a protected area comprises the following steps:
s1, aiming at the fixed-point monitoring system, screening a visible light image containing the small construction machinery based on a segmentation algorithm;
s2, screening visible light images containing constructors by using infrared thermal imaging images of the daily inspection vehicle;
s3, seamlessly splicing the single-time inspection images of the unmanned aerial vehicle to generate a panoramic image of the protected area;
s4 model-based
Identifying constructors and small-sized construction machinery on the screened visible light image;
s5 model-based
Carrying out large-scale construction machinery identification and positioning on the spliced panoramic image;
and S6, correcting the identification result of the large construction machine by combining the vibration signal and the sound signal intensity change.
The invention is further improved in that the fixed-point monitoring system comprises a video module, a sound module, a vibration module, a micro-processing module, an audible and visual alarm module and a collecting upright rod, and is fixedly arranged in a protection area to carry out 24-hour all-around monitoring on small construction machines such as a small bulldozer, a small drilling machine and the like in the protection area.
Furthermore, the video module comprises a monitoring cloud platform and a high-definition camera which are arranged at the tail end of the cross arm of the collecting vertical rod and connected with the micro-processing module, the horizontal rotation angle of the monitoring cloud platform is not less than 180 degrees, and the monitoring picture of the high-definition camera covers the whole protection area range; the height of the acquisition vertical rod is H, and the installed video module monitors that the small construction machine is not shielded.
Furthermore, the sound module adopts outdoor adapter, and farthest pickup range is 30m, and the adapter is installed in collection pole setting height 0.5H department.
Furthermore, the vibration module comprises an acceleration sensor and a time recorder, the acceleration sensor is installed in the center of the protection area, detects vibration signals in the range of the whole protection area, is connected with the time recorder, collects and records data, and the time recorder is connected with the micro-processing module.
Furthermore, the micro-processing module comprises an FPGA image processing unit, a 4G/5G communication unit and an ARM controller, the FPGA image processing unit is connected with the ARM controller, visible light images containing the small construction machinery are screened based on a segmentation algorithm, the 4G/5G communication unit is connected with the ARM controller and used for sending screened data to a cloud processing system, and the specific steps of screening the visible light images containing the small construction machinery based on the segmentation algorithm are as follows:
(1) the visible light image is segmented by the Vibe algorithm to obtain a binary image
；
(2) Statistics of
The image area occupied by the middle foreground;
(3) root of herbaceous plantAccording to threshold value
Screening for the presence of visible light images of small construction machines.
Furthermore, the acousto-optic alarm module comprises an LED warning lamp and a loudspeaker, wherein the LED warning lamp can display red, yellow and green colors, and the LED warning lamp and the loudspeaker are installed at the middle position of the cross arm of the collecting vertical rod together.
The daily inspection vehicle comprises a vehicle carrier, an image acquisition module and a vehicle-mounted micro-processing module, and is used for carrying out periodic or irregular inspection in a protection area and monitoring the behaviors of a constructor such as whether the constructor wears a safety helmet or not, whether the constructor wears reflective clothes or not and the like.
Furthermore, the vehicle carrier is a car and carries image acquisition equipment to carry out multi-line mobile inspection in a protection area.
Furthermore, the image acquisition module comprises a vehicle-mounted holder, a vehicle-mounted high-definition camera and a vehicle-mounted infrared thermal imaging camera which are fixed to the top of a vehicle carrier, the vehicle-mounted holder is installed on the left side and the right side of the advancing direction of a vehicle, the bottom of each vehicle-mounted holder is fixed to the vehicle carrier through a shock absorber, the vehicle-mounted high-definition camera and the vehicle-mounted infrared thermal imaging camera are installed on the vehicle-mounted holder side by side, and each vehicle-mounted high-definition camera and each vehicle-mounted infrared thermal imaging camera are responsible for a monitoring range of 180 degrees through rotation of.
Further, on-vehicle little processing module include on-vehicle FPGA image processing unit, on-vehicle 4G 5G communication unit and on-vehicle ARM controller, on-vehicle FPGA image processing unit is connected with on-vehicle ARM controller, utilizes the daily infrared thermal imaging image who patrols the car, screens the visible light image that contains the constructor, on-vehicle 4G 5G communication unit is connected with on-vehicle ARM controller for data transmission to the high in the clouds processing system after the screening, the daily infrared thermal imaging image who patrols the car of utilization, the specific step of the visible light image that contains the constructor of screening is as follows:
(1) will time of day
Conversion of a captured color infrared thermographic image to a grayscale image
；
(2) Setting a threshold according to the gray value of the human body temperature in the infrared thermal imaging image
；
(3) Segmenting a grayscale image according to the following formula
Obtaining a binary image of the high temperature region
：
Wherein
Representing the positions of the pixel points;
(4) for binary image
Performing morphological operation of corrosion and expansion, eliminating noise interference to obtain binary image
；
(5) To pair
Carrying out rectangle fitting after extracting the contour information, removing the rectangle with the aspect ratio smaller than 2, eliminating the heating interference of the construction machinery engine, and considering the moment if the rectangle with the aspect ratio larger than 2 exists
The constructors enter the monitoring area and will make the moment
Visible light image of
And (4) screening.
The invention is further improved in that the unmanned aerial vehicle monitoring system comprises an unmanned aerial vehicle, an image acquisition module, an unmanned aerial vehicle micro-processing module and a ground position calibration pile, and is used for monitoring and accurately positioning a drilling machine, a pile driver, a crane, an excavator, a soil piling machine, a climbing vehicle, foundation pit excavation and large soil piling.
Further, unmanned aerial vehicle is many rotor unmanned aerial vehicle, adopts the mode of periodic inspection to carry out aerial shooting to the protected area along fixed route.
Further, the image acquisition module is including installing the unmanned aerial vehicle high definition camera that carries on at the triaxial machinery cloud platform and the cloud platform of unmanned aerial vehicle bottom, and unmanned aerial vehicle high definition camera shoots downwards.
Further, unmanned aerial vehicle micro-processing module include unmanned aerial vehicle FPGA image processing unit, unmanned aerial vehicle 4G 5G communication unit and unmanned aerial vehicle ARM controller, unmanned aerial vehicle FPGA image processing unit is connected with unmanned aerial vehicle ARM controller, patrols and examines the image to the unmanned aerial vehicle single and carries out seamless concatenation, generates guard area panoramic image, unmanned aerial vehicle 4G 5G communication unit is connected with unmanned aerial vehicle ARM controller for data transmission to the high in the clouds processing system after the concatenation, patrolling and examining the image to the unmanned aerial vehicle single and carry out seamless concatenation, generate the guard area panoramic image concrete step as follows:
(1) extracting FAST angular points from all images in a single inspection process of the unmanned aerial vehicle;
(2) calculating a transformation matrix of two adjacent images based on FAST corners of the two adjacent images, and transforming the adjacent images to the same coordinate system by using the transformation matrix;
(3) and sequentially fusing overlapping areas of adjacent images by adopting a weighted average algorithm to generate a panoramic image of the protected area.
Further, ground position marks stake surface and scribbles different striking colours, disperses in a certain order at the protection zone internal fixation, provides the position guide at unmanned aerial vehicle patrols and examines the in-process.
The invention is further improved in that the cloud processing system stores front-end data processing results of the fixed-point monitoring system, the daily patrol vehicle and the unmanned aerial vehicle monitoring system and is based on the model
Identifying constructors and small-sized construction machines on the screened visible light images, if the constructors do not wear safety helmets or wear reflective clothes, and the small-sized construction machines are constructed in non-construction periods or non-construction areas, feeding back alarm signals to an acousto-optic alarm module of a fixed-point monitoring system, and based on the model
Carrying out large-scale construction machinery identification and positioning on the spliced panoramic image, correcting the identification result of the large-scale construction machinery by combining the intensity change of the vibration signal and the sound signal, feeding an alarm signal back to an acousto-optic alarm module of a fixed-point monitoring system if the large-scale construction machinery is constructed in a non-construction period or a non-construction position, and sending the identification result to a data management platform and a mobile phone terminal at the same time, wherein the model is a model
And
the training process of (2) is as follows:
(1) carrying out data annotation on constructors 'construction behaviors and small-sized construction machinery in the visible light image, wherein the constructors' construction behaviors are marked with two types of safety caps and reflective clothes, and the small-sized construction machinery is marked with two types of small-sized excavators and small-sized drilling machines;
(2) carrying out data annotation on the panoramic image, and labeling an excavator, a soil piling car, a drilling machine and a stirring machine;
(3) performing data training on the labeling results of the visible light image and the panoramic image by adopting a target detection algorithm YOLO to obtain a model
And
。
the invention is further improved in that the concrete steps of combining the vibration signal and the sound signal intensity change and correcting the large construction machine identification result are as follows:
(1) if the moment of time
Detecting a large construction machine in the panoramic image;
(2) detecting the time of day
If both the sound signal and the vibration signal have local peaks, it is determined that the large construction machine exists at that time.
The invention is further improved in that the LED warning lamps with three colors of red, yellow and green of the sound-light warning module respectively correspond to constructors, small construction machinery and large construction machinery, and are matched with the loudspeaker to adopt different warning sounds to carry out violation early warning.
The invention has the beneficial effects that:
(1) the invention is that a plurality of acquisition systems comprehensively carry out multi-dimensional and multi-source signal acquisition in the air, on the ground and underground, and carry out multi-level and multi-hand comprehensive monitoring on a protection area;
(2) the comprehensive micro-processing unit and the cloud server realize front-end data processing and rear-end target identification, reduce data storage and transmission cost and improve identification accuracy;
(3) the construction safety behavior is automatically identified and tracked in real time and the position is tracked in real time by combining deep learning and computer vision, so that intelligent comprehensive monitoring is realized.
Drawings
FIG. 1 is an overall block diagram of the present invention;
FIG. 2 is a flow chart of a method of the present invention;
FIG. 3 is a diagram of a fixed point monitoring system;
FIG. 4 is a diagram of a microprocessor module of the fixed point monitoring system;
FIG. 5 is a diagram of a daily inspection vehicle;
FIG. 6 is a block diagram of the vehicle microprocessor module;
FIG. 7 is a diagram of a UAV surveillance system;
fig. 8 is a diagram of the structure of the micro-processing module of the unmanned aerial vehicle.
In the figure: 1-fixed point monitoring system, 11-monitoring tripod head, 12-high definition camera, 13-acquisition upright rod, 14-loudspeaker, 15-LED warning light, 16-sound pick-up, 17-acceleration sensor, 18-time recorder, 19-microprocessing module, 191-FPGA image processing unit, 192-4G/5G communication unit, 193-ARM controller, 2-daily inspection vehicle, 21-vehicle carrier, 22-vehicle tripod head, 23-vehicle high definition camera, 24-vehicle infrared thermal imaging camera, 25-vehicle microprocessing module, 251-vehicle FPGA image processing unit, 252-vehicle 4G/5G communication unit, 253-vehicle controller, 3-unmanned aerial vehicle monitoring system, 31-unmanned aerial vehicle, 32-ground position calibration pile, 33-three-axis mechanical holder, 34-unmanned aerial vehicle high-definition camera, 35-unmanned aerial vehicle microprocessing module, 351-unmanned aerial vehicle FPGA image processing unit, 352-unmanned aerial vehicle 4G/5G communication unit, 353-unmanned aerial vehicle ARM controller and 4-cloud server.
Detailed Description
The invention is described in detail below with reference to the figures and specific embodiments. The present embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation manner and a specific operation process are given, but the scope of the present invention is not limited to the following embodiments.
As shown in figure 1, the comprehensive safety monitoring equipment for the protected area comprises a fixed-point monitoring system 1, a daily patrol car 2, an unmanned aerial vehicle monitoring system 3 and a cloud processing system, wherein the fixed-point monitoring system 1, the daily patrol car 2, the unmanned aerial vehicle monitoring system 3 and the cloud processing system are connected through a network, the fixed-point monitoring system 1, the daily patrol car 2 and the unmanned aerial vehicle monitoring system 3 are combined to acquire multi-source signals such as video, sound and vibration, the cloud processing system is deployed on a cloud server 4 and stores front-end data processing results of the fixed-point monitoring system 1, the daily patrol car 2 and the unmanned aerial vehicle monitoring system 3, automatic identification and positioning are carried out on construction safety behaviors in the protected area based on a pre-trained deep learning model, and meanwhile, identified illegal operation behaviors are sent to a data management platform and a mobile phone terminal.
As shown in fig. 2, the invention discloses a comprehensive safety monitoring method for a protected area, which comprises the following steps:
s1, aiming at the fixed-point monitoring system, screening a visible light image containing the small construction machinery based on a segmentation algorithm;
s2, screening visible light images containing constructors by using infrared thermal imaging images of the daily inspection vehicle;
s3, seamlessly splicing the single-time inspection images of the unmanned aerial vehicle to generate a panoramic image of the protected area;
s4 model-based
Identifying constructors and small-sized construction machinery on the screened visible light image;
s5 model-based
Carrying out large-scale construction machinery identification and positioning on the spliced panoramic image;
and S6, correcting the identification result of the large construction machine by combining the vibration signal and the sound signal intensity change.
As shown in fig. 3, the fixed-point monitoring system 1 includes six parts, namely a video module, a sound module, a vibration module, a micro-processing module 19, an audible and visual alarm module and a collecting upright 13.
The video module comprises a monitoring cloud platform 11 and a high-definition camera 12 which are arranged at the tail end of a cross arm of a collecting vertical rod 13, the horizontal rotation angle of the monitoring cloud platform 11 is not less than 180 degrees, the monitoring picture of the high-definition camera 12 covers the whole protection area range, a USB interface is connected with a microprocessing module 19, the height of the collecting vertical rod 13 is 6m, and the installed high-definition camera 12 can monitor small construction machinery such as a small bulldozer, a small drilling machine and the like in a protection area of 50m x 60m without being shielded by an object under the horizontal rotation of the monitoring cloud platform 11.
The sound module adopts outdoor rifle bolt formula adapter 16, and adapter 16 installs in the 3m department of gathering pole setting 13 height, fastens on gathering pole setting 13 through the bolt, and directional protection zone direction links to each other with microprocessor module 19 through gathering the inside line of pole setting 13.
The vibration module includes acceleration sensor 17 and time recorder 18, acceleration sensor 17 adopts triaxial force balance formula acceleration sensor, there are special horizontal adjusting bolt and horizontal bubble in the corner, level control for during the installation, the center design has the mounting hole, can fix whole acceleration sensor 17 with a bolt, install the central point in the protected area, protect through stainless steel seal shell, time recorder 18 is connected with acceleration sensor 17, provide data acquisition, record and transmission function, link to each other with microprocessor module 19.
As shown in FIG. 4, the micro-processing module 19 comprises an FPGA image processing unit 191, a 4G/5G communication unit 192 and an ARM controller 193, wherein the model of the FPGA image processing unit 191 is Altera-EP4CE10, the model of the 4G/5G communication unit 192 is WH-LTE-7S 4V 2, the model of the ARM controller 193 is STM32F103VCT6, the FPGA image processing unit 191 is connected with the ARM controller 193, visible light images containing small construction machines are screened based on a segmentation algorithm, and the 4G/5G communication unit 192 is connected with the ARM controller 193.
The method comprises the following specific steps of screening the visible light image containing the small construction machine based on a segmentation algorithm:
(1) vibe algorithm segmentation visible light image, binary image
The class of Vibe algorithms used is as follows:
Class Vibe
{
void init (const Mat background image)// parameter initialization
void processFirstFrame (const Mat background image)// background modeling
VOID TEStAndUpdata (const Mat RECETIMage)// segmenting the visible light image
MatgetMask (void);/return binarized image
}
(2) Statistics of
The image area occupied by the middle foreground;
(3) according to a threshold value
Screening for the presence of visible light images of small construction machines.
The acousto-optic warning module contains LED warning light 15 and megaphone 14, links to each other with ARM controller 193, receives the alarm signal that high in the clouds server 4 sent through 4G 5G communication unit 192, reacts the alarm signal who comes from high in the clouds server 4, LED warning light 15 and megaphone 14 are together installed at the intermediate position of gathering pole setting 13 xarm, LED warning light 15 can show red-yellow-green three kinds of colours, possesses the flash of light and slowly dodges the function.
As shown in fig. 5, the daily polling car 2 includes a car carrier 21, an image acquisition module and a car-mounted micro-processing module 25
The vehicle carrier 21 is a car and carries image acquisition equipment to carry out multi-line mobile inspection in a protection area.
The image acquisition module comprises vehicle-mounted cloud platforms 22, vehicle-mounted high-definition cameras 23 and vehicle-mounted infrared thermal imaging cameras 24 which are fixed on the top of a vehicle carrier 21, the vehicle-mounted cloud platforms 22 are installed on the left side and the right side of the advancing direction of the vehicle carrier 21, the bottom of each vehicle-mounted cloud platform 22 is fixed on the vehicle carrier through a shock absorber, the vehicle-mounted high-definition cameras 23 and the vehicle-mounted infrared thermal imaging cameras 24 are installed on the vehicle-mounted cloud platforms 22 side by side and are connected with a vehicle-mounted micro-processing module 25, each vehicle-mounted high-definition camera 23 and the vehicle-mounted infrared thermal imaging camera 24 are in charge of a monitoring range of 180 degrees through rotation of the vehicle-mounted cloud platforms 22, the vehicle-mounted high-definition cameras 23 are used for monitoring whether constructors wear safety helmets, whether reflective clothes and the like, the vehicle-mounted infrared thermal imaging, The vehicle-mounted infrared thermal imaging camera 24 is connected with the vehicle-mounted micro-processing module 25.
As shown in FIG. 6, the vehicle-mounted micro-processing module 25 comprises a vehicle-mounted FPGA image processing unit 251, a vehicle-mounted 4G/5G communication unit 252 and a vehicle-mounted ARM controller 253, the model of the vehicle-mounted FPGA image processing unit 251 is Altera-EP4CE10, the model of the vehicle-mounted 4G/5G communication unit 252 is WH-LTE-7S 4V 2, the model of the vehicle-mounted ARM controller 253 is STM32F103VCT6, the vehicle-mounted FPGA image processing unit 251 is connected with the vehicle-mounted ARM controller 253, the vehicle-mounted 4G/5G communication unit 252 is connected with the vehicle-mounted ARM controller 253, the vehicle-mounted ARM controller 253 performs data transmission with the cloud server 4 through the vehicle-mounted 4G/5G communication unit 252, when data collected by the vehicle-mounted high-definition camera 23 and the vehicle-mounted infrared thermal imaging camera 24 are transmitted to the vehicle-mounted FPGA image processing unit 251, the vehicle-mounted FPGA image processing unit 251 utilizes daily thermal imaging images, the screening includes constructor's visible light image, sends data to the high in the clouds processing system through on-vehicle 4G 5G communication unit 252, utilizes the infrared thermal imaging image of daily inspection car, and the screening includes constructor's visible light image concrete step as follows:
(1) will time of day
Conversion of a captured color infrared thermographic image to a grayscale image
The conversion function used is cvtColor (InputAlysrc, OutputAlydst, int code, int dstCn = 0);
(2) setting a threshold according to the gray value of the human body temperature in the infrared thermal imaging image
；
(3) For gray scale images according to the following formula
Segmenting the high-temperature region to obtain a binary image
：
Wherein
Representing the positions of the pixel points;
(4) for binary image
Performing morphological operation of corrosion and expansion, eliminating noise interference to obtain binary image
The corrosion and expansion functions used are respectively as follows:
void erode(InputArraysrc，OutputArraydst，InputArray kernel)，
void dilate(InputArraysrc，OutputArraydst，InputArray kernel)；
(5) to pair
Carrying out rectangle fitting after extracting the contour information, removing the rectangle with the aspect ratio smaller than 2, eliminating the heating interference of the construction machinery engine, and considering the moment if the rectangle with the aspect ratio larger than 2 exists
The constructors enter the monitoring area and will make the moment
Visible light image of
The profile extraction function used was the following, filtered out:
findContours (InputOutputOutputAlrray image, OutputAlyOfArraysconsistent, inmode, int method, Point offset = Point ()), the rectangle fitting function used is as follows:
RectboundingRect( InputArray points)。
as shown in fig. 7, the drone monitoring system 3 includes a drone 31, an image acquisition module, a drone microprocessor module 35, and a ground position calibration pile 32.
The unmanned aerial vehicle adopts many rotor unmanned aerial vehicle 31, adopts the mode of periodic inspection, follows fixed route and carries out aerial control to the protected area with reference to ground position calibration stake 32.
The image acquisition module is including installing unmanned aerial vehicle high definition camera 34 at the triaxial machinery cloud platform 33 and the cloud platform of unmanned aerial vehicle bottom, and unmanned aerial vehicle high definition camera 34 is shot downwards, is connected with unmanned aerial vehicle micro-processing module 35, shoots protection zone rig, pile driver, hoist, excavator, mound machine, the car of ascending a height, foundation ditch excavation and large-scale mound.
As shown in fig. 8, the drone microprocessor module 35 includes a drone FPGA image processing unit 351, a drone 4G/5G communication unit 352 and a drone ARM controller 353, the model of the drone FPGA image processing unit 351 is Altera-EP4CE10, the model of the drone 4G/5G communication unit 352 is WH-LTE-7S 4V 2, the model of the drone ARM controller 353 is STM32F103VCT6, the drone FPGA image processing unit 351 is connected with the drone ARM controller 353, the drone single inspection images are seamlessly spliced to generate a panoramic image of the protected area, the drone 4G/5G communication unit 352 is connected with the drone ARM controller 353, the drone ARM controller 353 performs data transmission with the cloud server 4 through the drone 4G/5G communication unit 352 to perform seamless splicing on the drone single inspection images, the specific steps of generating the panoramic image of the protected area are as follows:
(1) extracting FAST angular points from all images in a single inspection process of the unmanned aerial vehicle;
(2) calculating a transformation matrix of two adjacent images based on FAST corners of the two adjacent images, transforming the adjacent images to the same coordinate system by using the transformation matrix, and calculating the transformation matrix by adopting the following functions:
Mat cv::findHomography( InputArraysrcPoints, InputArraydstPoints, intmethod = 0,double ransacReprojThreshold = 3, OutputArray mask = noArray(),const int maxIters = 2000, const double confidence = 0.995)；
(3) and sequentially fusing overlapping areas of adjacent images by adopting a weighted average algorithm to generate a panoramic image of the protected area.
Ground position calibration stake 32 is the plastics material, and the surface scribbles different striking colours, according to certain order dispersion in the protected area with the help of the bolt fastening, patrols and examines the in-process at unmanned aerial vehicle and provides the position guide.
The cloud processing system is deployed in the cloud server 4, stores front-end data processing results of the fixed-point monitoring system 1, the daily patrol vehicle 2 and the unmanned aerial vehicle monitoring system 3, and is based on a model
Identifying constructors and small-sized construction machines on the screened visible light images, if the constructors do not wear safety helmets or wear reflective clothes, and the small-sized construction machines are constructed in non-construction periods or non-construction areas, feeding back alarm signals to an acousto-optic alarm module of a fixed-point monitoring system, and based on the model
Carrying out large-scale construction machinery identification and positioning on the spliced panoramic image, correcting the identification result of the large-scale construction machinery by combining the intensity change of the vibration signal and the sound signal, feeding an alarm signal back to an acousto-optic alarm module of the fixed-point monitoring system 1 if the large-scale construction machinery is constructed in a non-construction period or a non-construction position, and sending the identification result to a data management platform and a mobile phone terminal at the same time, wherein the model is a model
And
the training process of (2) is as follows:
(1) carrying out data annotation on constructors 'construction behaviors and small-sized construction machinery in the visible light image, wherein the constructors' construction behaviors are marked with two types of safety caps and reflective clothes, and the small-sized construction machinery is marked with two types of small-sized excavators and small-sized drilling machines;
(2) carrying out data annotation on the panoramic image, and labeling an excavator, a soil piling car, a drilling machine and a stirring machine;
(3) performing data training on the labeling results of the visible light image and the panoramic image by adopting a target detection algorithm YOLO to obtain a model
And
。
the invention is further improved in that the concrete steps of combining the vibration signal and the sound signal intensity change and correcting the large construction machine identification result are as follows:
(1) if the moment of time
Detecting a large construction machine in the panoramic image;
(2) detecting the time of day
If both the sound signal and the vibration signal have local peaks, it is determined that the large construction machine exists at that time.
LED warning lights in three colors of red, yellow and green of the sound-light alarm module respectively correspond to constructors, small-sized construction machinery and large-sized construction machinery, and are matched with the loudspeaker to adopt different alarm sounds to perform violation early warning.
The foregoing detailed description of the preferred embodiments of the invention has been presented. It should be understood that numerous modifications and variations could be devised by those skilled in the art in light of the present teachings without departing from the inventive concepts. Therefore, the technical solutions available to those skilled in the art through logic analysis, reasoning and limited experiments based on the prior art according to the concept of the present invention should be within the scope of protection defined by the claims.
Claims (11)
1. A comprehensive safety monitoring device for a protected area is characterized in that: including fixed point monitored control system, daily car, unmanned aerial vehicle monitored control system and high in the clouds processing system that patrols and examines, fixed point monitored control system, daily car, unmanned aerial vehicle monitored control system and high in the clouds processing system pass through network connection, fixed point monitored control system, daily car and the unmanned aerial vehicle monitored control system of patrolling and examining jointly carry out video, sound, vibration multisource signal acquisition, high in the clouds processing system deploys at the high in the clouds server, stores fixed point monitored control system, daily car and the unmanned aerial vehicle monitored control system of patrolling and examining to carry out automatic identification and location to construction safety action in the protection zone based on the degree of deep learning model of training in advance, send the illegal operation action of discernment to data management platform and cell-phone terminal simultaneously.
2. The comprehensive safety monitoring device for the protected area according to claim 1, wherein: the fixed-point monitoring system comprises a video module, a sound module, a vibration module, a micro-processing module, an acousto-optic alarm module and a collecting vertical rod, and is fixedly installed in a protection area, wherein the video module comprises a monitoring holder and a high-definition camera which are installed at the tail end of a cross arm of the collecting vertical rod and is connected with the micro-processing module, the height of the collecting vertical rod is H, the sound module adopts an outdoor pickup and is installed at the position of 0.5H of the height of the collecting vertical rod and is connected with the micro-processing module; the vibration module comprises an acceleration sensor and a time-course recorder, the acceleration sensor is arranged in the center of the protection area, detects vibration signals in the whole protection area range, is connected with the time-course recorder and collects and records data, and the time-course recorder is connected with the micro-processing module; the micro-processing module comprises an FPGA image processing unit, a 4G/5G communication unit and an ARM controller, the FPGA image processing unit screens visible light images containing small construction machinery based on a segmentation algorithm and is connected with the ARM controller, and the 4G/5G communication unit is connected with the ARM controller; the sound and light alarm module comprises an LED warning lamp and a loudspeaker, and the LED warning lamp and the loudspeaker are mounted at the middle position of the cross arm of the collecting vertical rod together.
3. The comprehensive safety monitoring device for the protected area according to claim 1, wherein: the daily inspection vehicle comprises a vehicle carrier, an image acquisition module and a vehicle-mounted micro-processing module; the vehicle carrier is a car and carries image acquisition equipment to carry out multi-line mobile inspection in a protection area; the image acquisition module comprises a vehicle-mounted holder, a vehicle-mounted high-definition camera and a vehicle-mounted infrared thermal imaging camera which are fixed at the top of a vehicle carrier, wherein the vehicle-mounted holder is arranged on the left side and the right side of the advancing direction of a vehicle, and the vehicle-mounted high-definition camera and the vehicle-mounted infrared thermal imaging camera are arranged on the vehicle-mounted holder side by side; the vehicle-mounted micro-processing module comprises a vehicle-mounted FPGA image processing unit, a vehicle-mounted 4G/5G communication unit and a vehicle-mounted ARM controller, the vehicle-mounted FPGA image processing unit utilizes infrared thermal imaging images of a daily patrol vehicle, visible light images containing constructors are screened and connected with the vehicle-mounted ARM controller, and the vehicle-mounted 4G/5G communication unit is connected with the vehicle-mounted ARM controller.
4. The comprehensive safety monitoring device for the protected area according to claim 1, wherein: the unmanned aerial vehicle monitoring system comprises an unmanned aerial vehicle, an image acquisition module, an unmanned aerial vehicle micro-processing module and a ground position calibration pile, and is used for monitoring and accurately positioning a drilling machine, a pile driver, a crane, an excavator, a stacker, a climbing vehicle, foundation pit excavation and large-scale stacking; the unmanned aerial vehicle is a multi-rotor unmanned aerial vehicle, and the protected area is shot in the air along a fixed route in a periodic inspection mode; the image acquisition module comprises a three-axis mechanical holder and an unmanned aerial vehicle high-definition camera which are arranged at the bottom of the unmanned aerial vehicle, and the unmanned aerial vehicle high-definition camera shoots downwards; unmanned aerial vehicle micro-processing module include unmanned aerial vehicle FPGA image processing unit, unmanned aerial vehicle 4G 5G communication unit and unmanned aerial vehicle ARM controller, unmanned aerial vehicle FPGA image processing unit patrols and examines the image to the unmanned aerial vehicle single and carries out seamless concatenation, generates guard area panoramic image, is connected with unmanned aerial vehicle ARM controller, unmanned aerial vehicle 4G 5G communication unit is connected with unmanned aerial vehicle ARM controller.
5. The monitoring method of the comprehensive monitoring equipment for the safety of the protected area as claimed in claim 1, which is characterized in that: the method comprises the following steps:
s1, aiming at the fixed-point monitoring system, screening a visible light image containing the small construction machinery based on a segmentation algorithm;
s2, screening visible light images containing constructors by using infrared thermal imaging images of the daily inspection vehicle;
s3, seamlessly splicing the single-time inspection images of the unmanned aerial vehicle to generate a panoramic image of the protected area;
s4 model-based
Identifying constructors and small-sized construction machinery on the screened visible light image;
s5 model-based
Carrying out large-scale construction machinery identification and positioning on the spliced panoramic image;
and S6, correcting the identification result of the large construction machine by combining the vibration signal and the sound signal intensity change.
6. The monitoring method as set forth in claim 5, wherein: the specific steps of step S1 are as follows:
(1) the visible light image is segmented by the Vibe algorithm to obtain a binary image
；
(2) Statistics of
The image area occupied by the middle foreground;
(3) according to a threshold value
Screening for the presence of visible light images of small construction machines.
7. The monitoring method as set forth in claim 5, wherein: the specific steps of step S2 are as follows:
(1) will time of day
Conversion of a captured color infrared thermographic image to a grayscale image
；
(2) Setting a threshold according to the gray value of the human body temperature in the infrared thermal imaging image
；
(3) Segmenting a grayscale image according to the following formula
Obtaining a binary image of the high temperature region
：
Wherein
Representing the positions of the pixel points;
(4) for binary image
Performing morphological operation of corrosion and expansion, eliminating noise interference to obtain binary image
；
(5) To pair
Carrying out rectangle fitting after extracting the contour information, removing the rectangle with the aspect ratio smaller than 2, eliminating the heating interference of the construction machinery engine, and considering the moment if the rectangle with the aspect ratio larger than 2 exists
The constructors enter the monitoring area and will make the moment
Visible light image of
And (4) screening.
8. The monitoring method as set forth in claim 5, wherein: the specific steps of step S3 are as follows:
(1) extracting FAST angular points from all images in a single inspection process of the unmanned aerial vehicle;
(2) calculating a transformation matrix of two adjacent images based on FAST corners of the two adjacent images, and transforming the adjacent images to the same coordinate system by using the transformation matrix;
(3) and sequentially fusing overlapping areas of adjacent images by adopting a weighted average algorithm to generate a panoramic image of the protected area.
9. The monitoring method as set forth in claim 5, wherein:the specific steps of step S4 are as follows: based on a model
And identifying constructors and small-sized construction machines on the screened visible light images, if the constructors do not wear safety helmets, do not wear reflective clothes and construct the small-sized construction machines in non-construction periods or non-construction areas, feeding back alarm signals to an acousto-optic alarm module of the fixed-point monitoring system, and sending identification results to a data management platform and a mobile phone terminal.
10. The monitoring method as set forth in claim 5, wherein: the specific steps of step S5 are as follows: based on a model
And (3) identifying and positioning the spliced panoramic image of the large construction machine, correcting the identification result of the large construction machine by combining the intensity change of the vibration signal and the sound signal, feeding back an alarm signal to an acousto-optic alarm module of a fixed-point monitoring system if the large construction machine is constructed in a non-construction period or a non-construction position, and sending the identification result to a data management platform and a mobile phone terminal.
11. The monitoring method as set forth in claim 5, wherein: the specific steps of step S6 are as follows:
(1) if the moment of time
Detecting a large construction machine in the panoramic image;
(2) detecting the time of day
If the sound signal and the vibration signal of (2) have local peaks, it is considered that the large construction machine exists at that time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010770673.0A CN111898563A (en) | 2020-08-04 | 2020-08-04 | Comprehensive safety monitoring equipment and method for protected area |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010770673.0A CN111898563A (en) | 2020-08-04 | 2020-08-04 | Comprehensive safety monitoring equipment and method for protected area |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111898563A true CN111898563A (en) | 2020-11-06 |
Family
ID=73184086
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010770673.0A Pending CN111898563A (en) | 2020-08-04 | 2020-08-04 | Comprehensive safety monitoring equipment and method for protected area |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111898563A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112487894A (en) * | 2020-11-17 | 2021-03-12 | 厦门路桥信息股份有限公司 | Automatic inspection method and device for rail transit protection area based on artificial intelligence |
| CN112799394A (en) * | 2020-12-15 | 2021-05-14 | 广州极飞科技股份有限公司 | Unmanned operation equipment control method, device, equipment and storage medium |
| CN112866639A (en) * | 2021-01-07 | 2021-05-28 | 北京家人智能科技有限公司 | Patrol warning method and device |
| CN114973684A (en) * | 2022-07-25 | 2022-08-30 | 深圳联和智慧科技有限公司 | Construction site fixed-point monitoring method and system |
| CN115355952A (en) * | 2022-10-20 | 2022-11-18 | 山东联合能源管道输送有限公司 | Intelligent inspection system for crude oil storage tank |
| CN115439988A (en) * | 2022-07-20 | 2022-12-06 | 杭州无限环境设计工程有限公司 | City cooperative safety monitoring system based on street lamp networking |
| CN115793093A (en) * | 2023-02-02 | 2023-03-14 | 水利部交通运输部国家能源局南京水利科学研究院 | Empty ground integrated equipment for diagnosing hidden danger of dam |
| CN116580717A (en) * | 2023-07-12 | 2023-08-11 | 南方科技大学 | On-line correction method and system for noise background interference of construction site field boundary |
-
2020
- 2020-08-04 CN CN202010770673.0A patent/CN111898563A/en active Pending
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112487894A (en) * | 2020-11-17 | 2021-03-12 | 厦门路桥信息股份有限公司 | Automatic inspection method and device for rail transit protection area based on artificial intelligence |
| CN112487894B (en) * | 2020-11-17 | 2024-04-23 | 厦门路桥信息股份有限公司 | Automatic track traffic protection area inspection method and device based on artificial intelligence |
| CN112799394A (en) * | 2020-12-15 | 2021-05-14 | 广州极飞科技股份有限公司 | Unmanned operation equipment control method, device, equipment and storage medium |
| CN112799394B (en) * | 2020-12-15 | 2022-09-13 | 广州极飞科技股份有限公司 | Unmanned operation equipment control method, device, equipment and storage medium |
| CN112866639B (en) * | 2021-01-07 | 2023-04-28 | 珠海市横琴盈实科技研发有限公司 | Patrol warning method and equipment |
| CN112866639A (en) * | 2021-01-07 | 2021-05-28 | 北京家人智能科技有限公司 | Patrol warning method and device |
| CN115439988A (en) * | 2022-07-20 | 2022-12-06 | 杭州无限环境设计工程有限公司 | City cooperative safety monitoring system based on street lamp networking |
| CN115439988B (en) * | 2022-07-20 | 2024-04-09 | 杭州无限环境设计工程有限公司 | Urban collaborative safety monitoring system based on street lamp networking |
| CN114973684A (en) * | 2022-07-25 | 2022-08-30 | 深圳联和智慧科技有限公司 | Construction site fixed-point monitoring method and system |
| CN114973684B (en) * | 2022-07-25 | 2022-10-14 | 深圳联和智慧科技有限公司 | Fixed-point monitoring method and system for construction site |
| CN115355952A (en) * | 2022-10-20 | 2022-11-18 | 山东联合能源管道输送有限公司 | Intelligent inspection system for crude oil storage tank |
| CN115355952B (en) * | 2022-10-20 | 2023-01-20 | 山东联合能源管道输送有限公司 | Intelligent inspection system for crude oil storage tank |
| CN115793093B (en) * | 2023-02-02 | 2023-05-16 | 水利部交通运输部国家能源局南京水利科学研究院 | Dam hidden disease diagnosis air-ground equipment |
| CN115793093A (en) * | 2023-02-02 | 2023-03-14 | 水利部交通运输部国家能源局南京水利科学研究院 | Empty ground integrated equipment for diagnosing hidden danger of dam |
| CN116580717A (en) * | 2023-07-12 | 2023-08-11 | 南方科技大学 | On-line correction method and system for noise background interference of construction site field boundary |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111898563A (en) | Comprehensive safety monitoring equipment and method for protected area | |
| CN109164443A (en) | Rail track foreign matter detecting method and system based on radar and image analysis | |
| CN101465033B (en) | Automatic tracking recognition system and method | |
| CN111341068B (en) | Construction method of drilling site dangerous area early warning system based on deep learning | |
| CN109872483B (en) | Intrusion alert photoelectric monitoring system and method | |
| CN112149522A (en) | Intelligent visual external-damage-prevention monitoring system and method for cable channel | |
| CN108109385A (en) | A kind of vehicle identification of power transmission line external force damage prevention and hazardous act judgement system and method | |
| CN106780284A (en) | tunnel wall rock integrity information collecting device and evaluation method | |
| CN104766086A (en) | Supervising method and system of way mark | |
| CN102542806A (en) | Real-time monitoring system for license plate shielding of vehicles running on speedway and control method thereof | |
| CN103456024B (en) | A kind of moving target gets over line determination methods | |
| CN107977018A (en) | Crop straw burning monitoring method based on binocular stereo vision and unmanned plane | |
| CN204155450U (en) | High definition bayonet apparatus on a kind of road | |
| CN102867417A (en) | Taxi anti-forgery system and taxi anti-forgery method | |
| CN105118301B (en) | The illegal monitoring system of vehicular | |
| CN107749055A (en) | A kind of fault detection method, system and the device of LED traffic guidances screen | |
| CN105163014A (en) | Road monitoring device and method | |
| CN202018736U (en) | Vehicle-mounted monitoring system for illegal bus lane occupation | |
| CN110885019A (en) | Tower crane monitoring and real-time alarm system based on BIM model | |
| CN107200274A (en) | A kind of anti-container truck based on machine vision is lifted method | |
| CN115171054A (en) | Fishing administration patrol method and system | |
| CN116846059A (en) | Edge detection system for power grid inspection and monitoring | |
| CN110349172B (en) | Power transmission line external damage prevention early warning method based on image processing and binocular stereo ranging | |
| CN116453278A (en) | Intrusion monitoring method combining deep learning intelligent detection and optical fiber vibration sensing | |
| CN205383993U (en) | Identification system is measured to vehicle license plate and profile |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |