CN114219999A - Anti-drop machine vision automatic monitoring method and system for structural externally hung decorative plate - Google Patents
Anti-drop machine vision automatic monitoring method and system for structural externally hung decorative plate Download PDFInfo
- Publication number
- CN114219999A CN114219999A CN202111308943.7A CN202111308943A CN114219999A CN 114219999 A CN114219999 A CN 114219999A CN 202111308943 A CN202111308943 A CN 202111308943A CN 114219999 A CN114219999 A CN 114219999A
- Authority
- CN
- China
- Prior art keywords
- decorative plate
- abnormal
- structural
- externally hung
- color
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 106
- 238000000034 method Methods 0.000 title claims abstract description 31
- 230000002159 abnormal effect Effects 0.000 claims abstract description 90
- 238000001514 detection method Methods 0.000 claims abstract description 49
- 238000012423 maintenance Methods 0.000 claims abstract description 11
- 230000000007 visual effect Effects 0.000 claims abstract description 8
- 238000007781 pre-processing Methods 0.000 claims abstract description 6
- 238000012790 confirmation Methods 0.000 claims abstract description 5
- 238000012360 testing method Methods 0.000 claims description 27
- 238000012549 training Methods 0.000 claims description 21
- 238000005253 cladding Methods 0.000 claims description 15
- 238000007689 inspection Methods 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000009826 distribution Methods 0.000 claims description 4
- 238000005286 illumination Methods 0.000 claims description 4
- 230000008569 process Effects 0.000 claims description 4
- 238000002955 isolation Methods 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 230000007613 environmental effect Effects 0.000 abstract description 5
- 230000005856 abnormality Effects 0.000 description 5
- 230000036541 health Effects 0.000 description 4
- 239000000463 material Substances 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 229910052751 metal Inorganic materials 0.000 description 2
- 239000002184 metal Substances 0.000 description 2
- 230000035882 stress Effects 0.000 description 2
- 229910000838 Al alloy Inorganic materials 0.000 description 1
- 230000032683 aging Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000004566 building material Substances 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000009430 construction management Methods 0.000 description 1
- 230000007797 corrosion Effects 0.000 description 1
- 238000005260 corrosion Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 229920003023 plastic Polymers 0.000 description 1
- 239000004033 plastic Substances 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/243—Classification techniques relating to the number of classes
- G06F18/24323—Tree-organised classifiers
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a visual automatic monitoring method and system for an anti-dropping machine of a structural externally hung decorative plate. The method comprises the steps of distributing monitoring cameras around a target structure, collecting images of a monitored area in real time, preprocessing the collected images, extracting color moment characteristics, detecting a first isolated forest abnormity detection algorithm based on the color moment characteristics and positioning abnormal images; isolating the abnormal image, extracting the spatial feature of the abnormal image, and performing secondary isolated forest algorithm detection on the spatial feature of the abnormal image; and early warning is carried out on the structural externally hung decorative plate in the dangerous stage, and monitoring and maintenance are carried out on the structural externally hung decorative plate by combining a manual on-site confirmation mode. The invention automatically learns the characteristics of the normal sample, does not need human intervention, has strong robustness, is not influenced by environmental factors, and can monitor the state of the externally hung decorative plate of the structure in real time, thereby ensuring the personal safety of pedestrians and working personnel around the structure.
Description
Technical Field
The invention belongs to the technical field of structural safety monitoring, and relates to a visual automatic monitoring method and system for an anti-falling machine of a structural externally hung decorative plate.
Background
The structural outer decorative plate serving as a building material has the advantages of light weight, attractiveness, flexible design and the like, is widely applied to general civil buildings and industrial buildings, and mainly comprises libraries, production plants, industrial warehouses, logistics buildings, stadiums, airport terminal buildings, train stations and the like. The externally hung decorative board belongs to a light building envelope system, and each component is fixed with a main structure through mechanical connection. Common external decorative plate materials include glass, aluminum alloy, lightweight concrete and novel plastics. Due to environmental and time factors such as corrosion and aging of the connecting materials, failure of components or connections is likely to occur when subjected to large external loads or temperatures. At the moment, the external decorative plate can fall off and hurt people, and the danger caused by the falling off is larger especially for places with dense people flows, such as commercial districts, airports, high-speed rails and the like. Therefore, it is necessary to automatically monitor the dropping state of the external decorative plate of the building structure.
At present, the monitoring means of the external decorative plate is relatively lacked. On the one hand, the mainstream monitoring method of the structural externally hung decorative plate at the present stage depends on that personnel regularly go to the house for inspection, the workload is very large, and the operation safety risk exists. On the other hand, the appearance decorative plate has large use area and complex house structure, is influenced by factors such as insufficient night illumination and bad weather, is difficult to be monitored comprehensively by monitoring equipment, and is easy to have monitoring dead angles so as not to check the potential risk source of the externally hung decorative plate in time.
At present, few literature patents related to the state monitoring of external decorative boards are provided, and an inspection and maintenance technology for a metal roof system of a canopy of a high-speed railway station is provided in research on inspection and maintenance technologies of the metal roof system of the canopy of the high-speed railway station (2016-01-12. in the Shanghai city, the Shanghai railway administration), and a high-definition video inspection and monitoring system is constructed by utilizing a video monitoring platform, so that managers can inspect and monitor target sites at any time through the monitoring system. The monitoring point of the system realizes remote monitoring based on an IP network, has the limitations of network delay, instability, high traffic cost and the like, realizes the purpose of anomaly detection by manually checking monitoring videos and manually judging experience, and is greatly influenced by environmental factors and manually subjective experience. And the system still needs a large-size display for convenient manual monitoring and high cost.
Disclosure of Invention
The invention discloses an anti-drop machine vision automatic monitoring method and system for a structural externally hung decorative plate, which comprises the steps of distributing monitoring cameras around a target structure, collecting images of a monitoring area in real time, preprocessing the collected images, extracting color moment characteristics, detecting a first isolated forest abnormity detection algorithm based on the color moment characteristics and positioning abnormal images; isolating the abnormal image, extracting the spatial feature of the abnormal image, and performing secondary isolated forest algorithm detection on the spatial feature of the abnormal image; and early warning is carried out on the structural externally hung decorative plate in the dangerous stage, and monitoring and maintenance are carried out on the structural externally hung decorative plate by combining a manual on-site confirmation mode.
In order to realize the aim of the invention, the automatic visual monitoring method for the anti-falling machine of the structural externally hung decorative plate, provided by the invention, comprises the following steps:
s1: a group of cameras are arranged on the periphery of the structure or a stress component of the structure main body;
s2: carrying out image acquisition on the structural externally hung decorative plate in the preset monitoring area through the camera;
s3: taking a contour of an external decorative plate object to be monitored, dividing m multiplied by n cells by taking the image as a background, numbering the cells, and taking each vertex coordinate of each cell as a corresponding coordinate range;
s4: converting the collected images of the structural externally hung decorative plates in the monitoring area from an RGB space to an HSV space to obtain an image sample set;
s5: extracting color moment characteristics of all image samples in the image sample set;
s6: training and testing the color moment characteristics by a first isolated forest anomaly detection algorithm, calculating a first anomaly score of each image sample, and isolating the abnormal image samples according to the first anomaly score;
s7: extracting spatial features of the abnormal image sample isolated in the S6;
s8: and training and testing the spatial features extracted in the S7 by using a second isolated forest anomaly detection algorithm, calculating a second anomaly score of each image sample, determining an abnormal image sample based on the second anomaly score, and positioning coordinates of an anomaly point for the abnormal image sample.
Preferably, the time interval for acquiring the images at S2 is 30S/time.
Preferably, the position and the number of the cameras require that the visual field combination can cover all the external decorative plate areas of the target structure, and the monitoring areas of the cameras are enclosed to form a closed monitoring network.
Preferably, in S3, the divided cell size is determined according to the area size of the monitored target structure external decorative plate and the size of the structure external decorative plate.
Preferably, in S5, when extracting the color moment feature, the color distribution of the image is expressed by using the first moment, the second moment and the third moment of the color, the color moment of the image includes 3 color components, and each component has 3 lower-order moments:
(1) in the formulae (1) to (3), EiRepresenting the first order color moments on the ith channel, N representing the total number of pixels in the picture, pijRepresenting the colour value, σ, of the jth pixel on the ith colour channeliRepresenting the second order color moment, s, on the ith color channeliRepresenting the third order moment of color on the ith color channel.
Preferably, in S6, when performing first isolated forest anomaly detection algorithm training and testing based on color moment features, the color moment features include normal and abnormal, the trained model accuracy is above 95%, the first isolated forest anomaly detection algorithm is divided into a daytime mode and a nighttime mode under the influence of brightness, and when the illumination brightness is not less than 400 lumens, the daytime mode is selected; otherwise, it is in night mode.
Preferably, in S8, when the abnormal detection algorithm of the isolated forest of the second time is trained and tested based on the spatial characteristics, the spatial characteristics include the edge warping of the cladding panel, the slippage and dislocation of the cladding panel, the tearing of the cladding panel, the bulging of the cladding panel and the falling of the cladding panel.
Preferably, the anomaly detection process of the isolated forest anomaly detection algorithm in S6 and S8 is:
s01: training, uniformly sampling and constructing an iTree tree and an iForest forest;
s02: testing, performing a binary division test on each iTree tree in the iForest forest according to the characteristics of the extracted image sample, and calculating the abnormal score of each piece of test data, wherein when the abnormal detection of the first isolated forest abnormal detection algorithm in S6 is performed, the characteristics are color moment characteristics, when the abnormal detection of the second isolated forest abnormal detection algorithm in S8 is performed, the characteristics are space characteristics, and the abnormal score calculation formula is as follows:
wherein h (x) is the path length, E (h (x)) is the average path length of all iTree trees in the forest, and c (n) is the average path length of the binary search tree, which is used to normalize the result:
c(n)＝2H(n-1)-2(n-1)/n (5)
h (n-1) is the sum of the sums.
Preferably, the following steps are further included after step S9:
s9: feeding back the monitoring abnormity early warning information to relevant departments in a grading manner;
s10: technicians check the monitoring video, verify the actual condition of the externally hung decorative plate on site, and make maintenance decisions according to the actual manual inspection and confirmation condition.
The monitoring system for realizing the monitoring comprises the following modules:
the image acquisition module is used for acquiring images of the structural externally hung decorative plate in the preset monitoring area;
the contour acquisition module is used for acquiring a contour of the external decorative plate object to be monitored, dividing m multiplied by n unit cells by taking the image as a background and numbering the unit cells, wherein each unit cell takes each vertex coordinate as a corresponding coordinate range;
the image preprocessing module is used for converting the acquired images of the structural externally hung decorative plates in the monitoring area from an RGB space to an HSV space to obtain an image sample set;
the color moment extraction module is used for extracting color moment characteristics of all image samples in the image sample set;
the abnormal image sample isolation module is used for training and testing the color moment characteristics by a first isolated forest abnormal detection algorithm, calculating a first abnormal score of each image sample, and isolating the abnormal image samples according to the first abnormal score;
the spatial feature extraction module is used for extracting spatial features from the isolated abnormal image samples;
and the determining module is used for training and testing the extracted spatial features by a second isolated forest anomaly detection algorithm, calculating a second anomaly score of each image sample, determining the abnormal image samples based on the second anomaly scores, and positioning the coordinates of the anomaly points of the abnormal image samples.
Compared with the existing monitoring means, the invention can realize the following beneficial effects:
(1) the health monitoring of the external decorative board of the existing structure mainly takes manual regular inspection as a main part, and needs maintainers to use instruments for detection or diagnose according to experience, so that the error is large. The invention can realize the real-time monitoring of the outer hanging decorative plate only by a set of complete machine vision monitoring system, thereby saving a great deal of manpower and material resources and greatly improving the monitoring efficiency and accuracy.
(2) Compared with the conventional timing maintenance, the monitoring method can realize the real-time monitoring of the state of the external decorative plate all day long, and timely discover and feed back the problems. The safety of the building can be fully guaranteed in some severe weather such as wind, rain, snow and the like, so that the personal safety of personnel is guaranteed.
(3) The traditional monitoring means is to select a plurality of representative measuring points for monitoring, the monitoring range is limited, although the safety condition of the structural externally hung decorative plate can be approximately reflected, the traditional monitoring means is difficult to deal with the occurrence of accidental events due to the fact that factors such as weather, load and the like change and are measured. The monitoring method of the invention is that the monitoring camera forms a closed monitoring network for the target external decorative plate, and can monitor the falling state information of the external decorative plate of the whole structure in an all-round way.
(4) In recent years, some large buildings also implement a health detection system for remote real-time monitoring and early warning, and such health monitoring systems are also a development trend of building health management. However, the current monitoring method still needs a large amount of sensor equipment and is high in cost. The monitoring means of the invention is realized mainly by a set of machine vision monitoring equipment, and compared with the prior art, the invention has lower price and is easier to realize.
(5) The invention automatically learns the characteristics of the normal sample, does not need human intervention, has strong robustness, is not influenced by environmental factors, and can monitor the state of the externally hung decorative plate of the structure in real time, thereby ensuring the personal safety of pedestrians and working personnel around the structure.
(6) The invention relates to secondary isolated forest anomaly detection based on color moment characteristics combined with spatial characteristics, which has the functions of automatic system training, detection, automatic monitoring information storage and the like, is suitable for the condition that an abnormal sample is few or even none, is slightly influenced by environmental factors, and does not need manual intervention. The requirements can be met by using a common monitoring camera.
Drawings
FIG. 1 is a structural diagram of the anti-drop machine vision automatic monitoring method of the structural externally hung decorative plate of the invention.
Fig. 2 is a schematic view of the overlapping part of the monitoring areas of the two fixed cameras according to the present invention.
FIG. 3 is a schematic structural view of an anti-drop machine vision automatic monitoring system for the structural externally hung decorative plate of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, the automatic visual monitoring method for the anti-falling machine of the structural externally hung decorative plate provided by the invention comprises the following steps:
s1: a group of fixed cameras are arranged on the periphery of the structure or a stress component of the structure main body.
In some embodiments of the present invention, the camera device can be deployed by means of a pole, a surrounding building, a street lamp, etc. already built by a communication network operator, so as to reduce the construction cost. The position and the number of the cameras require that the visual field combination can cover all target external decorative plate areas, and the camera monitoring areas are enclosed to form a closed monitoring network.
In some embodiments of the present invention, the cameras are all fixed cameras, that is, the monitoring view angle of each camera is fixed. An 1/4 monitoring area overlapping region exists in every two cameras monitoring visual angles, so that whether each camera works normally or not can be conveniently checked, and the correctness and the validity of a monitoring result can be verified. And moreover, the monitoring overlapping area is selected as a dangerous place where the monitored external decorative plate is easy to damage, and double detection guarantee is provided for the weak link of the external decorative plate.
S2: collecting images by a monitoring camera: and carrying out image acquisition on the structural externally hung decorative plate in the preset monitoring area.
In some embodiments of the invention, the time interval for acquiring the images is 30 s/time to meet the requirement of real-time monitoring. Of course, in other embodiments, other time intervals may be set as desired.
S3: dividing a preset monitoring area: and (3) taking the outline of the external decorative plate object to be monitored to further accurately structure the monitoring area of the external decorative plate, dividing m multiplied by n quadrilateral cells by taking the image as a background and numbering the cells, wherein 4 vertex coordinates (X, Y) of each cell are taken as the corresponding coordinate range.
In some embodiments of the invention, the area division is carried out without accurately dividing the cell size, a user determines the divided cell size according to the area size of the monitored target structure external decorative plate and the size of the structure external decorative plate, and each monitored cell area comprises not less than 2 external decorative plates.
In some embodiments of the present invention, to be suitable for target monitoring objects of different shapes, the cells are divided into quadrilateral cells without limitation to the shape type.
S4: image preprocessing: and (4) converting the image color space, and converting the collected images of the structural externally hung decorative plates in the monitoring area from the RGB space to the HSV space to obtain an image sample set.
The RGB channel can not well reflect the specific color information of the structural external decorative plate, the color information is greatly influenced by the brightness, the HSV space consists of 3 matrixes of 'hue', 'saturation' and 'value', the color information and the image brightness are separated, the color information is suitable for being segmented, and the subsequent color moment characteristic can be conveniently extracted.
S5: and extracting color moment characteristics of each image sample in the image sample set.
In some embodiments of the present invention, when the color moment features are extracted, since the color distribution information is mainly concentrated in the low-order moments, it is sufficient to express the color distribution of the image using the first, second, and third moments of the color. The image color moments require 9 components (including 3 color components, each component having 3 lower order moments):
(1) in the formulae (1) to (3), EiRepresenting the first order color moments on the ith channel, N representing the total number of pixels in the picture, pijRepresenting the colour value, σ, of the jth pixel on the ith colour channeliRepresenting the second order color moment, s, on the ith color channeliRepresenting the third order moment of color on the ith color channel.
S6: training and testing a first isolated forest anomaly detection algorithm based on the color moment features extracted in the S5, calculating a first anomaly score of each test sample, and isolating abnormal image samples according to the anomaly scores.
In some embodiments of the invention, if the first anomaly score is not greater than 0.5, then the image sample is normal; if the first abnormality score is greater than 0.5, the image sample is abnormal, and the closer to 1, the higher the abnormality of the image sample is; if the first abnormal scores are all around 0.5, no obvious abnormal sample exists in the image samples of the batch.
In some embodiments of the invention, when the first training and testing of the isolated forest anomaly detection algorithm is performed, the color moment features comprise normal and abnormal features, and the accuracy of the trained model is ensured to be more than 95%. The first isolated forest anomaly detection is divided into a daytime mode and a nighttime mode under the influence of brightness, and the daytime mode is adopted when the illumination brightness is not less than 400 lumens; otherwise, it is in night mode.
S7: spatial features are extracted for the isolated abnormal image samples of S6.
S8: training and testing of a second isolated forest anomaly detection algorithm are carried out based on the spatial features extracted in S7, a second anomaly score of each image sample is calculated, an abnormal image sample is determined based on the second anomaly score, and coordinates of an anomaly point are located according to the abnormal image sample.
Also, in some embodiments of the invention, if the resulting anomaly score is not greater than 0.5, then the image sample is normal; if the abnormality score is greater than 0.5, the image sample is abnormal, and the closer to 1, the higher the abnormality of the image sample is; and if the abnormal scores are all around 0.5, no obvious abnormal sample exists in the image samples of the batch.
In some embodiments of the invention, during the second training and testing of the isolated forest, the spatial characteristics of the abnormal image sample isolated in S6 include the edge warpage of the cladding panel, slippage and dislocation of the cladding panel, tearing of the cladding panel, bulging of the cladding panel and dropping of the cladding panel, and the accuracy of the trained model is guaranteed to be above 95%.
And carrying out isolated forest anomaly detection for the first time based on the color moment characteristics, and isolating the image samples with the abnormal color moment characteristics, which is used for detecting whether abnormal conditions exist in the large-range monitored object. If the abnormal image sample is detected for the first time, the abnormal image sample is subjected to isolated forest abnormality detection for the second time based on the spatial features, and the specific abnormal condition of the abnormal image sample can be further judged according to the spatial features. And carrying out two times of isolated forest abnormity detection, and combining the color moment characteristic and the spatial characteristic, so that the detection accuracy and the detection efficiency can be further improved.
In some embodiments of the present invention, the specific process of using the isolated forest algorithm to detect the anomaly in S6 and S8 is as follows:
s01: training, uniformly sampling and constructing an iTree tree and an iForest forest;
s02: testing, performing a binary division test on each iTree tree in the iForest forest according to the extracted features of the image sample (the features in S6 are color moment features, and the features in S8 are spatial features), and calculating an abnormal score of each piece of test data, wherein the abnormal score calculation formula is as follows:
where h (x) is the path length of sample x, and E (h (x)) is the average path length of all iTree trees in the forest. c (n) is the average path length of the binary search tree when the given sample number is n, and is used for carrying out normalization processing on the result:
c(n)＝2H(n-1)-2(n-1)/n (5)
h (n-1) in the formula (5) can be determined by the following formula:
H(n-1)＝ln(n-1)+ξ (6)
in the formula (6), ξ is an Euler constant whose value is 0.5772156649 and H (n-1) is the sum of the tones.
In some embodiments of the invention, the system automatically saves the image monitoring information after completing the second isolated forest anomaly algorithm detection of S8. Wherein the normal monitoring information is stored for one week; and the abnormity monitoring information is permanently stored, so that the follow-up customized maintenance and repair plan is facilitated.
S9: the system carries out early warning on the monitoring abnormal information to relevant departments in a grading manner.
In some embodiments of the invention, the system early warning information is divided into three levels according to the abnormal area of the structural external decorative plate, and the yellow early warning, the orange early warning and the red early warning are ranked from low severity to high severity.
Wherein, the abnormal area is not more than 1 cell area, and the yellow early warning is carried out; the early warning is orange when the abnormal area is 2-4 cell areas; and the abnormal area is not less than 4 cell areas, and the early warning is red. The system sends the early warning level, the abnormal image and the positioning coordinate to a user side of a building related manager, the manager can directly judge the abnormal degree of the externally hung decorative plate according to the early warning level and the abnormal image, and can give an alarm in case of emergency and send early warning information to a related emergency maintenance department in a one-key mode.
The abnormal area is the area of the abnormal unit multiplied by the number of the abnormal units.
S10: technicians review the monitoring video and verify the actual falling condition of the externally hung decorative plate on site. And making a maintenance decision according to the actual manual inspection and confirmation condition.
Referring to fig. 3, an embodiment of the invention further provides a monitoring system for implementing the monitoring method. Anti-drop machine vision automatic monitoring system of external decorative board of structure includes following module:
the image acquisition module is used for acquiring images of the structural externally hung decorative plate in the preset monitoring area;
the contour acquisition module is used for acquiring a contour of the external decorative plate object to be monitored, dividing m multiplied by n unit cells by taking the image as a background and numbering the unit cells, wherein each unit cell takes each vertex coordinate as a corresponding coordinate range;
the image preprocessing module is used for converting the acquired images of the structural externally hung decorative plates in the monitoring area from an RGB space to an HSV space to obtain an image sample set;
the color moment extraction module is used for extracting color moment characteristics of all image samples in the image sample set;
the abnormal image sample isolation module is used for training and testing the color moment characteristics by a first isolated forest abnormal detection algorithm, calculating a first abnormal score of each image sample, and isolating the abnormal image samples according to the first abnormal score;
the spatial feature extraction module is used for extracting spatial features from the isolated abnormal image samples;
the determining module is used for training and testing the extracted spatial features by a second isolated forest anomaly detection algorithm, calculating second anomaly scores of all image samples, determining abnormal image samples based on the second anomaly scores, and positioning coordinates of anomaly points for the abnormal image samples;
and the early warning module is used for early warning the monitored abnormal information to relevant departments in a grading manner.
The machine vision-based monitoring method is a low-cost effective monitoring means. The invention provides a novel anti-falling machine vision automatic monitoring method and system for a structural externally hung decorative plate based on a secondary isolated forest algorithm. The isolated forest algorithm is a novel method suitable for continuous data and unsupervised anomaly detection. The basic idea is to recursively and randomly divide the data set until each sample is independent or the depth of the tree reaches a limit value. Thus, the average path length of outliers is also shorter than the average path length of other objects on the set of isolated trees. Different from other algorithms, the distance and the density need to be calculated to search for abnormal data, the isolated forest algorithm can well process high-dimensional data and big data, the calculation efficiency is high, and the real-time requirement can be met. Therefore, the isolated forest algorithm is combined with machine vision, so that real-time monitoring, real-time early warning and effective maintenance of the structural externally hung decorative plate in the use stage can be realized, the inspection efficiency and the management quality of a house construction management department are improved, the state of the structural externally hung decorative plate is accurately mastered, the potential safety hazard of the externally hung decorative plate is timely discovered and treated, and the falling phenomenon is fundamentally avoided.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. The anti-drop machine vision automatic monitoring method of the structure external decorative plate is characterized by comprising the following steps:
s1: a group of cameras are arranged on the periphery of the structure or a stress component of the structure main body;
s2: carrying out image acquisition on the structural externally hung decorative plate in the preset monitoring area through the camera;
s3: taking a contour of an external decorative plate object to be monitored, dividing m multiplied by n cells by taking the image as a background, numbering the cells, and taking each vertex coordinate of each cell as a corresponding coordinate range;
s4: converting the collected images of the structural externally hung decorative plates in the monitoring area from an RGB space to an HSV space to obtain an image sample set;
s5: extracting color moment characteristics of all image samples in the image sample set;
s6: training and testing the color moment characteristics by a first isolated forest anomaly detection algorithm, calculating a first anomaly score of each image sample, and isolating the abnormal image samples according to the first anomaly score;
s7: extracting spatial features of the abnormal image sample isolated in the S6;
s8: and training and testing the spatial features extracted in the S7 by using a second isolated forest anomaly detection algorithm, calculating a second anomaly score of each image sample, determining an abnormal image sample based on the second anomaly score, and positioning coordinates of an anomaly point for the abnormal image sample.
2. The automatic machine vision monitoring method for preventing the structural externally hung decorative plate from falling off is characterized by comprising the following steps of: s2 the time interval for acquiring the images is 30S/time.
3. The automatic machine vision monitoring method for preventing the structural externally hung decorative plate from falling off is characterized by comprising the following steps of: the position and the number of the cameras require that the visual field combination can cover all the external decorative plate areas of the target structure, and the monitoring areas of the cameras are enclosed to form a closed monitoring network.
4. The automatic machine vision monitoring method for preventing the structural externally hung decorative plate from falling off is characterized by comprising the following steps of: and S3, determining the size of the divided cells according to the area size of the monitored target structure external hanging decorative plate and the size of the structure external hanging decorative plate.
5. The automatic machine vision monitoring method for preventing the structural externally hung decorative plate from falling off is characterized by comprising the following steps of: in S5, when extracting color moment features, the color distribution of an image is expressed by using the first, second, and third moments of color, where the color moments of an image include 3 color components, and each component has 3 lower order moments:
(1) in the formulae (1) to (3), EiRepresenting the first order color moments on the ith channel, N representing the total number of pixels in the picture, pijIndicating that the jth pixel is on the ith color channelColor value on track, σiRepresenting the second order color moment, s, on the ith color channeliRepresenting the third order moment of color on the ith color channel.
6. The automatic machine vision monitoring method for preventing the structural externally hung decorative plate from falling off is characterized by comprising the following steps of: in S6, when training and testing a first isolated forest anomaly detection algorithm based on color moment features, the color moment features comprise a normal mode and an abnormal mode, the accuracy of the trained model is more than 95%, the first isolated forest anomaly detection algorithm is divided into a daytime mode and a nighttime mode under the influence of brightness, and when the illumination brightness is not less than 400 lumens, the daytime mode is adopted; otherwise, it is in night mode.
7. The automatic machine vision monitoring method for preventing the structural externally hung decorative plate from falling off is characterized by comprising the following steps of: in S8, when carrying out the training and testing of the second isolated forest anomaly detection algorithm based on the spatial characteristics, the spatial characteristics comprise the edge warping of the cladding panel, the slippage and dislocation of the cladding panel, the tearing of the cladding panel, the bulging of the cladding panel and the falling of the cladding panel.
8. The automatic machine vision monitoring method for preventing the structural externally hung decorative plate from falling off is characterized by comprising the following steps of: the anomaly detection process of the isolated forest anomaly detection algorithm in S6 and S8 is as follows:
s01: training, uniformly sampling and constructing an iTree tree and an iForest forest;
s02: testing, performing a binary division test on each iTree tree in the iForest forest according to the characteristics of the extracted image sample, and calculating the abnormal score of each piece of test data, wherein when the abnormal detection of the first isolated forest abnormal detection algorithm in S6 is performed, the characteristics are color moment characteristics, when the abnormal detection of the second isolated forest abnormal detection algorithm in S8 is performed, the characteristics are space characteristics, and the abnormal score calculation formula is as follows:
wherein h (x) is the path length, E (h (x)) is the average path length of all iTree trees in the forest, and c (n) is the average path length of the binary search tree, which is used to normalize the result:
c(n)＝2H(n-1)-2(n-1)/n (5)
h (n-1) is the sum of the sums.
9. The automatic machine vision monitoring method for preventing the structural external decorative plate from falling off is characterized by further comprising the following steps after the step S9:
s9: feeding back the monitoring abnormity early warning information to relevant departments in a grading manner;
s10: technicians check the monitoring video, verify the actual condition of the externally hung decorative plate on site, and make maintenance decisions according to the actual manual inspection and confirmation condition.
10. An anti-drop machine vision automatic monitoring system for a structural externally hung decorative plate, which is used for realizing the monitoring method of any one of claims 1 to 9, and the system comprises:
the image acquisition module is used for acquiring images of the structural externally hung decorative plate in the preset monitoring area;
the contour acquisition module is used for acquiring a contour of the external decorative plate object to be monitored, dividing m multiplied by n unit cells by taking the image as a background and numbering the unit cells, wherein each unit cell takes each vertex coordinate as a corresponding coordinate range;
the image preprocessing module is used for converting the acquired images of the structural externally hung decorative plates in the monitoring area from an RGB space to an HSV space to obtain an image sample set;
the color moment extraction module is used for extracting color moment characteristics of all image samples in the image sample set;
the abnormal image sample isolation module is used for training and testing the color moment characteristics by a first isolated forest abnormal detection algorithm, calculating a first abnormal score of each image sample, and isolating the abnormal image samples according to the first abnormal score;
the spatial feature extraction module is used for extracting spatial features from the isolated abnormal image samples;
the determining module is used for training and testing the extracted spatial features by a second isolated forest anomaly detection algorithm, calculating second anomaly scores of all image samples, determining abnormal image samples based on the second anomaly scores, and positioning coordinates of anomaly points for the abnormal image samples;
and the early warning module is used for early warning the monitored abnormal information to relevant departments in a grading manner.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111308943.7A CN114219999B (en) | 2021-11-05 | 2021-11-05 | Automatic machine vision monitoring method and system for preventing external decorative plate of structure from falling off |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111308943.7A CN114219999B (en) | 2021-11-05 | 2021-11-05 | Automatic machine vision monitoring method and system for preventing external decorative plate of structure from falling off |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114219999A true CN114219999A (en) | 2022-03-22 |
| CN114219999B CN114219999B (en) | 2024-03-19 |
Family
ID=80696576
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111308943.7A Active CN114219999B (en) | 2021-11-05 | 2021-11-05 | Automatic machine vision monitoring method and system for preventing external decorative plate of structure from falling off |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114219999B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111127448A (en) * | 2019-12-26 | 2020-05-08 | 西南交通大学 | Model and method for detecting air spring fault based on isolated forest |
| CN111340063A (en) * | 2020-02-10 | 2020-06-26 | 北京华电天仁电力控制技术有限公司 | Coal mill data anomaly detection method |
| CN112287602A (en) * | 2020-10-28 | 2021-01-29 | 北京国信会视科技有限公司 | Motor car axle temperature fault early warning method based on machine learning and isolated forest |
-
2021
- 2021-11-05 CN CN202111308943.7A patent/CN114219999B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111127448A (en) * | 2019-12-26 | 2020-05-08 | 西南交通大学 | Model and method for detecting air spring fault based on isolated forest |
| CN111340063A (en) * | 2020-02-10 | 2020-06-26 | 北京华电天仁电力控制技术有限公司 | Coal mill data anomaly detection method |
| CN112287602A (en) * | 2020-10-28 | 2021-01-29 | 北京国信会视科技有限公司 | Motor car axle temperature fault early warning method based on machine learning and isolated forest |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114219999B (en) | 2024-03-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112435207B (en) | Forest fire monitoring and early warning method based on sky-ground integration | |
| CN111259892B (en) | Inspection method, inspection device, inspection equipment and inspection medium for state of indicator lamp | |
| CN111342391B (en) | Power transmission line insulator and line fault inspection method and inspection system | |
| CN111047818A (en) | Forest fire early warning system based on video image | |
| CN111446920A (en) | Photovoltaic power station monitoring method, device and system | |
| CN117037406B (en) | Intelligent monitoring and early warning system for forest fire | |
| CN102446390A (en) | Method and system for safety detection and early warning of monitoring areas near power transmission lines | |
| CN109359858A (en) | Intelligent campus security-protection management system Internet-based | |
| CN109711348A (en) | Intelligent monitoring method and system based on the long-term real-time architecture against regulations in hollow panel | |
| CN112288692B (en) | Crack monitoring and management system and monitoring and management method | |
| KR20210115432A (en) | Operation and management system of photovoltaic power plant and method thereof | |
| CN108389359A (en) | A kind of Urban Fires alarm method based on deep learning | |
| CN108919681A (en) | A kind of analysis system of freeway bridge tunnel dynamical health information | |
| CN114973148B (en) | Municipal facilities on-line monitoring digital intelligent management platform based on smart city construction | |
| CN109001210B (en) | System and method for detecting aging and cracking of sealing rubber strip of civil air defense door | |
| CN114882410B (en) | Tunnel dome lamp fault detection method and system based on improved positioning loss function | |
| CN112861623A (en) | Monitoring system and method for high-rise building engineering safety | |
| CN116846059A (en) | Edge detection system for power grid inspection and monitoring | |
| CN110543809A (en) | Electric power operation site risk supervision intrusion video identification method | |
| CN114219999B (en) | Automatic machine vision monitoring method and system for preventing external decorative plate of structure from falling off | |
| Aliyari et al. | Drone-based bridge inspection in harsh operating environment: Risks and safeguards | |
| CN109712018A (en) | A kind of power transmission line comprehensive monitoring system | |
| CN112270234A (en) | Power transmission line insulation sub-target identification method based on aerial image | |
| CN115436384A (en) | Distribution box surface defect detection system and method based on unmanned aerial vehicle image | |
| CN112965433B (en) | Safety monitoring intelligent management system based on big data Internet of things |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |