CN117041502B - Dangerous scene analysis and monitoring system and method based on machine vision - Google Patents
Dangerous scene analysis and monitoring system and method based on machine vision Download PDFInfo
- Publication number
- CN117041502B CN117041502B CN202311302412.6A CN202311302412A CN117041502B CN 117041502 B CN117041502 B CN 117041502B CN 202311302412 A CN202311302412 A CN 202311302412A CN 117041502 B CN117041502 B CN 117041502B
- Authority
- CN
- China
- Prior art keywords
- dangerous
- partition
- monitoring
- goods
- inspection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 85
- 238000000034 method Methods 0.000 title claims abstract description 35
- 238000004458 analytical method Methods 0.000 title claims abstract description 16
- 238000007689 inspection Methods 0.000 claims abstract description 67
- 230000000007 visual effect Effects 0.000 claims abstract description 43
- 238000012806 monitoring device Methods 0.000 claims abstract description 40
- 230000002776 aggregation Effects 0.000 claims abstract description 29
- 238000004220 aggregation Methods 0.000 claims abstract description 29
- 238000004891 communication Methods 0.000 claims abstract description 11
- 230000001502 supplementing effect Effects 0.000 claims abstract description 11
- 238000000638 solvent extraction Methods 0.000 claims abstract description 7
- 238000005192 partition Methods 0.000 claims description 65
- 230000006399 behavior Effects 0.000 claims description 53
- 230000001133 acceleration Effects 0.000 claims description 14
- 238000007726 management method Methods 0.000 claims description 12
- 230000033001 locomotion Effects 0.000 claims description 9
- 238000005286 illumination Methods 0.000 claims description 7
- 238000006073 displacement reaction Methods 0.000 claims description 6
- 230000001105 regulatory effect Effects 0.000 claims description 3
- 238000001514 detection method Methods 0.000 description 9
- 238000003860 storage Methods 0.000 description 9
- 239000007787 solid Substances 0.000 description 6
- 238000010276 construction Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000009545 invasion Effects 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 231100001261 hazardous Toxicity 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01D—MEASURING NOT SPECIALLY ADAPTED FOR A SPECIFIC VARIABLE; ARRANGEMENTS FOR MEASURING TWO OR MORE VARIABLES NOT COVERED IN A SINGLE OTHER SUBCLASS; TARIFF METERING APPARATUS; MEASURING OR TESTING NOT OTHERWISE PROVIDED FOR
- G01D21/00—Measuring or testing not otherwise provided for
- G01D21/02—Measuring two or more variables by means not covered by a single other subclass
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/70—Circuitry for compensating brightness variation in the scene
- H04N23/74—Circuitry for compensating brightness variation in the scene by influencing the scene brightness using illuminating means
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/90—Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Library & Information Science (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Human Computer Interaction (AREA)
- Computer Graphics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a dangerous scene analysis monitoring system and method based on machine vision, comprising a first visual monitoring device, a patrol equipment, a monitoring host, a patrol control platform, a management server and a portable terminal; the first visual monitoring device is in communication connection with the monitoring host; the inspection equipment is in communication connection with the inspection control platform; the management server is respectively connected with the monitoring host, the inspection control platform and the portable terminal in a communication way; the inspection equipment comprises a second visual monitoring device, a light intensity sensor, a leakage sensor and a light supplementing lamp; the first visual monitoring device is used for collecting images of the monitoring area and identifying dangerous goods characteristic information in the images; the management server is used for partitioning the monitoring area according to the dangerous grade and the aggregation degree of dangerous goods. According to the invention, the dangerous levels of different subareas are adjusted in a targeted manner, and the dangerous levels and subareas are updated in real time by combining a plurality of monitoring means, so that the accuracy and the instantaneity of visual monitoring are improved.
Description
Technical Field
The invention relates to the technical field of machine vision, in particular to a dangerous scene analysis and monitoring system and method based on machine vision.
Background
At present, the machine vision technology is applied to various automatic control systems, and under the condition that artificial vision is difficult to work, such as dangerous scenes of a production line, a field workplace, nuclear power, a mine hole, dangerous goods processing, dangerous goods storage and the like with high test requirements, the machine vision is used for replacing manual measurement or detection, so that the manual limitation can be broken through, the safety and the accuracy of work are improved, and meanwhile, the work efficiency is also improved by utilizing a highly-automatic control system.
However, there are still challenges to visual monitoring in hazardous scenes. First, the environment of the dangerous scene is often complex, and the machine vision equipment is difficult to accurately identify dangerous objects and dangerous behaviors. Secondly, dangerous goods and dangerous behaviors in different areas in practical application are different, and in the prior art, when monitoring a dangerous scene, each area is not monitored and controlled in a targeted manner, so that hidden danger is difficult to discover in time. In addition, in the prior art, the identification means for dangerous scenes is single, the image identification is often carried out only through a camera, the accuracy is low, and the judgment for the dangerous level cannot be updated in real time.
The invention patent application CN112216049A in the prior art provides a construction warning area monitoring and early warning system and method based on image recognition, wherein the system comprises an image input module, an image splicing module, an interaction calibration module, a pedestrian detection module, a visual invasion module, a feature extraction module and a decision module, and by combining the visual invasion detection, target detection and re-recognition technologies, surrounding environment information and in-out person information are acquired in real time through arranging cameras around the construction warning area or around dangerous construction equipment, when a person invasion signal exists, an alarm is sent out to remind illegal personnel to prohibit entering the area, and the safety of the construction warning area is ensured. However, the patent application of the invention does not carry out partition monitoring, the identification means is single, the real-time updating is not possible, and the accuracy and the reliability are low.
In addition, the invention patent CN115099760B provides an intelligent detection and early warning method for dangerous goods based on computer machine vision, wherein the dangerous goods storage warehouse is subjected to regional detection analysis through a regional detection module, the dangerous goods storage warehouse is divided into a plurality of monitoring areas, temperature data, humidity data and leakage data of the monitoring areas are obtained, numerical calculation is carried out to obtain environmental coefficients, and the monitoring areas are marked as normal areas or abnormal areas through the numerical values of the environmental coefficients; the method can carry out regional detection analysis on the dangerous goods storage warehouse, improves the detection precision of the dangerous goods storage environment, and can also carry out source tracing when abnormality occurs, thereby improving the storage safety of dangerous goods. Although the invention performs partition monitoring, the identification means of the invention is single, the partition condition cannot be updated in real time, and potential safety hazards cannot be found in time.
Disclosure of Invention
The invention aims to: aiming at the problems, the invention provides a dangerous scene analysis and monitoring system and method based on machine vision.
The technical scheme is as follows: in a first aspect, the invention provides a dangerous scene analysis and monitoring system based on machine vision, which comprises a first visual monitoring device, a patrol equipment, a monitoring host, a patrol control platform, a management server and a portable terminal;
the first visual monitoring device is in communication connection with the monitoring host; the inspection equipment is in communication connection with the inspection control platform;
the management server is respectively in communication connection with the monitoring host, the inspection control platform and the portable terminal;
the inspection equipment comprises a second visual monitoring device, a light intensity sensor, a leakage sensor and a light supplementing lamp;
the first visual monitoring device is used for collecting images of the monitoring area and identifying dangerous goods characteristic information in the images;
the management server is used for partitioning the monitoring area according to the dangerous grade and the aggregation degree of dangerous goods, adjusting the dangerous grade of the first partition according to the dangerous grade of dangerous behaviors, and adjusting the inspection frequency of the inspection equipment according to the dangerous grade of the first partition;
the monitoring host is used for judging the types of dangerous goods according to the characteristic information of the dangerous goods, searching the database to obtain the dangerous grade corresponding to the types of dangerous goods, and identifying dangerous behaviors in the first partition.
Preferably, the management server partition is further used for acquiring coordinates of each dangerous object in the monitoring image in a horizontal plane coordinate system of the monitoring area;
acquiring the distance between dangerous objects according to the coordinates, and judging whether the distance is smaller than a first distance threshold value;
dividing the articles with the spacing less than a first distance threshold into an aggregation cluster;
the method comprises the steps of obtaining the number of dangerous goods in each aggregation cluster and the dangerous grade of the dangerous goods, and calculating the dangerous grade of each aggregation cluster;
and acquiring the central coordinates of the clustered clusters, wherein a circle formed by taking the central coordinates of the clustered clusters as a circle center and the first distance as a radius is the first partition.
Preferably, the monitoring host is further used for acquiring outlines of dangerous goods in the monitoring image;
analyzing an article center point of the dangerous article profile;
judging the displacement of the center point according to the images of the adjacent frames, so as to calculate the moving acceleration of the center point;
and analyzing whether the acceleration of the movement of the center point exceeds a threshold value, and if so, judging that dangerous behaviors occur.
Preferably, the inspection equipment judges the light intensity based on the illumination sensor, and starts the light supplementing lamp when the light intensity is too low, and the dangerous goods and dangerous behaviors are identified through the second visual monitoring device.
In a second aspect, the present invention further provides a machine vision-based dangerous scene analysis and monitoring method, which includes:
s1, identifying the dangerous grade of dangerous goods based on machine vision; comprising the following steps:
s11, acquiring an image of a monitoring area by a first visual monitoring device, and identifying dangerous goods characteristic information in the image;
s12, judging the types of dangerous goods according to the characteristic information of the dangerous goods in the image;
s13, searching a database, and inquiring a dangerous grade corresponding to the dangerous object type;
s2, partitioning the monitoring area based on the dangerous grade and the aggregation degree of dangerous goods; comprising the following steps:
s21, acquiring coordinates of each dangerous object in the monitoring image in a horizontal plane coordinate system of a monitoring area;
s22, acquiring the distance between dangerous objects according to the coordinates, and judging whether the distance is smaller than a first distance threshold value or not;
s23, dividing the articles with the spacing smaller than a first distance threshold into an aggregation cluster;
s24, acquiring the number of dangerous objects in each aggregation cluster and the dangerous grade of the dangerous objects, and calculating the dangerous degree G of each aggregation cluster;
；
wherein E is M To aggregate the highest risk level in a cluster, N M N is the number of highest risk levels in the cluster 0 Is a preset adjustment coefficient;
s25, acquiring the central coordinates of the clustered clusters, wherein a circle formed by taking the central coordinates of the clustered clusters as the circle center and the first distance R as the radius is a first partition;
；
wherein alpha is an adjustment coefficient, and N is the total number of dangerous articles in the subarea;
s3, identifying dangerous behaviors in the first subarea based on machine vision; comprising the following steps:
s31, acquiring outlines of dangerous goods in the monitoring image;
s32, analyzing an article center point of the dangerous article outline;
s33, judging the displacement of the center point according to the images of the adjacent frames, so as to calculate the acceleration of the movement of the center point;
s34, analyzing whether the acceleration of the movement of the center point exceeds a threshold value, and judging dangerous behavior if yes;
s4, adjusting the dangerous grade of the first partition according to the dangerous behavior;
s41, calculating a risk level E of the first partition:
；
s42, identifying the level E of dangerous behavior M The method comprises the steps of carrying out a first treatment on the surface of the Wherein the level of dangerous behavior is in positive correlation with the acceleration in step S3;
s43, judging the distance between dangerous behaviors and dangerous objects in the first partition, and obtaining a minimum distance d;
s44, updating the risk level of the first partition to obtain a second risk level E of the first partition R ；
；
Wherein d 0 The distance is preset, and beta is an adjusting coefficient;
s5, adjusting the frequency of the inspection equipment based on the second danger level of the first subarea;
s51, setting the inspection frequency F of each first partition in a preset time period according to the updated partition danger level:
；
wherein F is 0 For the preset initial frequency, gamma is the regulating coefficient, E 0 The method comprises the steps of setting a dangerous initial value;
s52, planning a routing inspection route of the routing inspection equipment in each first partition;
preferably, the first visual monitoring device is used for acquiring the image of the monitoring area, acquiring the obstacle area and the inspection area, and performing route planning in the inspection area so as to establish an inspection route of the inspection equipment.
S6, identifying dangerous goods and dangerous behaviors based on a second visual monitoring device on the inspection equipment;
s61, the inspection equipment judges the light intensity based on the illumination sensor, if the light intensity is too low, the step S62 is carried out, otherwise, the step S63 is carried out;
s62, starting a light supplementing lamp, and identifying dangerous goods and dangerous behaviors through a second visual monitoring device;
s63, identifying dangerous goods and dangerous behaviors through a second visual monitoring device;
s64, updating the dangerous grade of the dangerous goods, the dangerous grade of the dangerous behavior and the dividing result of the first subarea;
s7, the inspection equipment acquires leakage information update and updates the danger level of the first partition;
s71, collecting dangerous goods leakage information through a sensor on the inspection equipment;
s72, judging whether the leakage information exceeds a preset threshold value;
s73, if the threshold value is exceeded, updating the risk level of the first partition to be the third risk level E U ；
；
Wherein w is an adjustment coefficient, H is leakage information, and H 0 Is a leakage threshold;
s8, displaying the real-time state of each first partition in real time through a three-dimensional model;
comprising the following steps: s81, displaying the danger level, the danger level of dangerous behaviors and the danger level of dangerous goods of each first subarea in real time by the three-dimensional model;
s82, when the danger level exceeds a threshold value, sending alarm information to the portable terminal;
s83, in response to the control instruction on the portable terminal, the association factor generating the alarm information is sent to the portable terminal.
Compared with the prior art, the invention has the following beneficial effects:
1. according to the invention, based on the first visual monitoring device which is fixedly arranged and the second visual monitoring device on the movable inspection equipment, dangerous objects and dangerous behaviors in a dangerous scene are identified, and the dangerous grades of the dangerous objects and the dangerous behaviors are judged, so that the accuracy and the reliability of identifying the dangerous scene can be improved.
2. According to the method, the monitoring area is divided into the first partitions based on the dangerous grade and the aggregation degree of dangerous goods, different inspection frequencies are set for different first partitions, the inspection frequencies can be adjusted in a targeted mode for different dangerous grades, the method is more efficient, abnormal conditions can be found more timely, and potential safety hazards can be found timely.
3. The inspection equipment can also identify and update the dangerous level again through the second visual monitoring device during inspection, so that the identification accuracy is improved. And moreover, the inspection equipment is provided with the leakage sensor, so that leakage information of dangerous articles can be identified, and the dangerous grade of the first subarea is corrected, and the real state of each first subarea is judged more objectively and in real time.
4. The inspection equipment is provided with the light supplementing lamp and the light intensity sensor, so that illumination compensation can be performed when the optical fiber is darker, and the accuracy of machine vision monitoring is improved.
Drawings
Fig. 1 is a schematic structural diagram of a dangerous scene analysis and monitoring system based on machine vision according to an embodiment of the present invention;
FIG. 2 is a flow chart of a dangerous scene analysis and monitoring method based on machine vision according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a first partition of a monitoring area according to an embodiment of the present invention.
Detailed Description
It will be apparent that many modifications and variations are possible within the scope of the invention, as will be apparent to those skilled in the art based upon the teachings herein.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless expressly stated otherwise, as understood by those skilled in the art. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element or component is referred to as being "connected" to another element or component, it can be directly connected to the other element or component or intervening elements or components may also be present. The term "and/or" as used herein includes any and all combinations of one or more of the associated listed items.
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Example 1
Referring to fig. 1, fig. 1 is a schematic structural diagram of a dangerous scene analysis and monitoring system based on machine vision, where the system includes:
the system comprises a first visual monitoring device, a patrol equipment, a monitoring host, a patrol control platform, a management server and a portable terminal;
the first visual monitoring device is in communication connection with the monitoring host; the inspection equipment is in communication connection with the inspection control platform;
the management server is respectively in communication connection with the monitoring host, the inspection control platform and the portable terminal;
the inspection equipment comprises a second visual monitoring device, a light intensity sensor, a leakage sensor and a light supplementing lamp;
the first visual monitoring device is used for collecting images of the monitoring area and identifying dangerous goods characteristic information in the images;
the management server is used for partitioning the monitoring area according to the dangerous grade and the aggregation degree of dangerous goods, adjusting the dangerous grade of the first partition according to the dangerous grade of dangerous behaviors, and adjusting the inspection frequency of the inspection equipment according to the dangerous grade of the first partition;
the monitoring host is used for judging the types of dangerous goods according to the characteristic information of the dangerous goods, searching the database to obtain the dangerous grade corresponding to the types of dangerous goods, and identifying dangerous behaviors in the first partition.
Preferably, the management server partition is further used for acquiring coordinates of each dangerous object in the monitoring image in a horizontal plane coordinate system of the monitoring area;
acquiring the distance between dangerous objects according to the coordinates, and judging whether the distance is smaller than a first distance threshold value;
dividing the articles with the spacing less than a first distance threshold into an aggregation cluster;
the method comprises the steps of obtaining the number of dangerous goods in each aggregation cluster and the dangerous grade of the dangerous goods, and calculating the dangerous grade of each aggregation cluster;
and acquiring the central coordinates of the clustered clusters, wherein a circle formed by taking the central coordinates of the clustered clusters as a circle center and the first distance as a radius is the first partition.
Preferably, the monitoring host is further used for acquiring outlines of dangerous goods in the monitoring image;
analyzing an article center point of the dangerous article profile;
judging the displacement of the center point according to the images of the adjacent frames, so as to calculate the moving acceleration of the center point;
and analyzing whether the acceleration of the movement of the center point exceeds a threshold value, and if so, judging that dangerous behaviors occur.
Preferably, the inspection equipment judges the light intensity based on the illumination sensor, and starts the light supplementing lamp when the light intensity is too low, and the dangerous goods and dangerous behaviors are identified through the second visual monitoring device.
Example two
The embodiment of the invention also provides a dangerous scene analysis and monitoring method based on machine vision, referring specifically to fig. 2, fig. 2 is a flowchart of the dangerous scene analysis and monitoring method based on machine vision, which includes the steps of:
s1, identifying the dangerous grade of dangerous goods based on machine vision; comprising the following steps:
s11, acquiring an image of a monitoring area by a first visual monitoring device, and identifying dangerous goods characteristic information in the image;
s12, judging the types of dangerous goods according to the characteristic information of the dangerous goods in the image;
s13, searching a database, and inquiring a dangerous grade corresponding to the dangerous object type;
the monitoring area can be a high-risk environment such as a dangerous goods storage warehouse, a dangerous goods processing workshop and the like; the first visual monitoring device is a plurality of fixed visual monitoring devices, such as cameras, arranged in the monitoring area; dangerous goods characteristic information includes, but is not limited to, dangerous goods packaging pattern information, bar codes, two-dimensional codes, and/or goods shapes, etc.
S2, partitioning the monitoring area based on the dangerous grade and the aggregation degree of dangerous goods; comprising the following steps:
s21, acquiring coordinates of each dangerous object in the monitoring image in a horizontal plane coordinate system of a monitoring area;
s22, acquiring the distance between dangerous objects according to the coordinates, and judging whether the distance is smaller than a first distance threshold value or not;
s23, dividing the articles with the spacing smaller than a first distance threshold into an aggregation cluster;
s24, acquiring the number N of dangerous goods and the dangerous grade of the dangerous goods in each aggregation cluster, and calculating the dangerous degree G of each aggregation cluster;
；
wherein E is M To aggregate the highest risk level of dangerous objects in a cluster, N M N is the number of highest risk levels in the cluster 0 Is a preset adjustment coefficient;
s25, acquiring the central coordinates of the clustered clusters, wherein a circle formed by taking the central coordinates of the clustered clusters as the circle center and the first distance R as the radius is a first partition;
；
wherein alpha is an adjustment coefficient, and N is the total number of dangerous articles in the aggregation cluster;
the range of influence of the articles with high dangerous levels is larger when accidents occur, and the larger range is also influenced when the number of dangerous articles is larger, so that the dangerous levels of the aggregation clusters in the first partition and the number of dangerous articles in the first partition are considered when the first partition is arranged; further, the product of G and N also represents the risk level of the first partition;
specifically, fig. 3 is a schematic diagram of a first partition of a monitoring area provided by an embodiment of the present invention, in fig. 3, a solid rectangular frame is a monitoring area, and a black solid small rectangle and a black solid small triangle represent dangerous objects with different dangerous grades; other dangerous objects with the distance smaller than the threshold value do not exist nearby the black solid small rectangle, so that the dangerous objects independently form an aggregation cluster; taking a black solid small rectangle as a circle center and taking a dotted line circle with R1 as a radius as a first partition; the center positions of the 3 black solid small triangles are used as circle centers, and the dotted line circle with R2 as the radius is used as a first partition.
S3, identifying dangerous behaviors in the first subarea based on machine vision; comprising the following steps:
s31, acquiring outlines of dangerous goods in the monitoring image;
s32, analyzing an article center point of the dangerous article outline;
s33, judging the displacement of the center point according to the images of the adjacent frames, so as to calculate the acceleration of the movement of the center point;
s34, analyzing whether the acceleration of the movement of the center point exceeds a threshold value, and judging dangerous behavior if yes;
s4, adjusting the dangerous grade of the first partition according to the dangerous behavior;
s41, calculating a risk level E of the first partition:
；
s42, identifying the level E of dangerous behavior M The method comprises the steps of carrying out a first treatment on the surface of the Wherein the level of dangerous behavior is in positive correlation with the acceleration in step S3;
s43, judging the distance between dangerous behaviors and dangerous objects in the first partition, and obtaining a minimum distance d;
s44, updating the risk level of the first partition to obtain a second risk level E of the first partition R ；
；
Wherein d 0 The distance is preset, and beta is an adjusting coefficient;
in the summary of the storage and production processes of dangerous goods, the closer the distance between dangerous behaviors and dangerous goods is, the higher the dangerous grade of the dangerous behaviors is, the more serious the consequences are, and the higher the dangerous grade is, so the dangerous grade of the first partition is adjusted according to the dangerous behavior condition;
s5, adjusting the frequency of the inspection equipment based on the second danger level of the first subarea;
s51, setting the inspection frequency F of each first partition in a preset time period according to the updated partition danger level:
；
wherein F is 0 For the preset initial frequency, gamma is the regulating coefficient, E 0 The method comprises the steps of setting a dangerous initial value;
s52, planning a routing inspection route of the routing inspection equipment in each first partition;
preferably, the first visual monitoring device is used for acquiring the image of the monitoring area, acquiring the obstacle area and the inspection area, and performing route planning in the inspection area so as to establish an inspection route of the inspection equipment.
S6, identifying dangerous goods and dangerous behaviors based on a second visual monitoring device on the inspection equipment;
s61, the inspection equipment judges the light intensity based on the illumination sensor, if the light intensity is too low, the step S62 is carried out, otherwise, the step S63 is carried out;
s62, starting a light supplementing lamp, and identifying dangerous goods and dangerous behaviors through a second visual monitoring device;
s63, identifying dangerous goods and dangerous behaviors through a second visual monitoring device;
s64, updating the dangerous grade of the dangerous goods, the dangerous grade of the dangerous behavior and the dividing result of the first subarea;
because the condition of darker light is frequently generated in the storage environment of dangerous goods, and the accuracy of machine vision monitoring is affected by the darker light, and the deviation of image recognition analysis is caused, the dangerous behavior is secondarily recognized through the second vision monitoring device on the inspection equipment; based on the result of the secondary identification, updating the dangerous grade of the dangerous goods, the dangerous grade of the dangerous behavior and the dividing result of the first subarea;
s7, the inspection equipment acquires leakage information update and updates the danger level of the first partition;
s71, collecting dangerous goods leakage information through a sensor on the inspection equipment;
s72, judging whether the leakage information exceeds a preset threshold value;
s73, if the threshold value is exceeded, updating the risk level of the first partition to be the third risk level E U ；
；
Wherein w is an adjustment coefficient, H is leakage information, and H 0 Is a leakage threshold;
s8, displaying the real-time state of each first partition in real time through a three-dimensional model;
s81, displaying the danger level, the danger level of dangerous behaviors and the danger level of dangerous goods of each first subarea in real time by the three-dimensional model;
s82, when the danger level exceeds a threshold value, sending alarm information to the portable terminal;
s83, in response to the control instruction on the portable terminal, the association factor generating the alarm information is sent to the portable terminal.
In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, so that the same or similar parts between the embodiments are referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
Finally, it is further noted that, in this document, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
Claims (6)
1. The monitoring method applied to the dangerous scene analysis and monitoring system based on the machine vision comprises a first visual monitoring device, a patrol equipment, a monitoring host, a patrol control platform, a management server and a portable terminal; the first visual monitoring device is used for collecting images of a monitoring area and identifying dangerous goods characteristic information in the images; the management server is used for partitioning the monitoring area according to the dangerous grade and the aggregation degree of dangerous goods, adjusting the dangerous grade of the first partition according to the dangerous grade of dangerous behaviors, and adjusting the inspection frequency of the inspection equipment according to the dangerous grade of the first partition; the monitoring host is used for judging the types of dangerous goods according to the characteristic information of the dangerous goods, searching the database to obtain the dangerous grade corresponding to the types of the dangerous goods, and identifying dangerous behaviors in the first partition; the management server is also used for acquiring coordinates of each dangerous object in the monitoring image in a horizontal plane coordinate system of the monitoring area; acquiring the distance between dangerous objects according to the coordinates, and judging whether the distance is smaller than a first distance threshold value; dividing the articles with the spacing less than a first distance threshold into an aggregation cluster; the method comprises the steps of obtaining the number of dangerous goods in each aggregation cluster and the dangerous grade of the dangerous goods, and calculating the dangerous grade of each aggregation cluster; acquiring the central coordinates of the clustered clusters, wherein a circle formed by taking the central coordinates of the clustered clusters as a circle center and a first distance as a radius is a first partition;
the monitoring method is characterized by comprising the following steps:
s1, identifying the dangerous grade of dangerous goods based on machine vision; comprising the following steps: s11, acquiring an image of a monitoring area by a first visual monitoring device, and identifying dangerous goods characteristic information in the image; s12, judging the types of dangerous goods according to the characteristic information of the dangerous goods in the image; s13, searching a database, and inquiring a dangerous grade corresponding to the dangerous object type;
s2, partitioning the monitoring area based on the dangerous grade and the aggregation degree of dangerous goods; comprising the following steps:
s21, acquiring coordinates of each dangerous object in the monitoring image in a horizontal plane coordinate system of a monitoring area;
s22, acquiring the distance between dangerous objects according to the coordinates, and judging whether the distance is smaller than a first distance threshold value or not;
s23, dividing the articles with the spacing smaller than a first distance threshold into an aggregation cluster;
s24, acquiring the number of dangerous objects in each aggregation cluster and the dangerous grade of the dangerous objects, and calculating the dangerous degree G of each aggregation cluster;
；
wherein E is M For highest risk in clustered clusters, etcStage N M N is the number of highest risk levels in the cluster 0 Is a preset adjustment coefficient;
s25, acquiring the central coordinates of the clustered clusters, wherein a circle formed by taking the central coordinates of the clustered clusters as the circle center and the first distance R as the radius is a first partition;
；
wherein alpha is an adjustment coefficient, and N is the total number of dangerous articles in the subarea;
s3, identifying dangerous behaviors in the first subarea based on machine vision; comprising the following steps: s31, acquiring outlines of dangerous goods in the monitoring image; s32, analyzing an article center point of the dangerous article outline; s33, judging the displacement of the center point according to the images of the adjacent frames, so as to calculate the acceleration of the movement of the center point; s34, analyzing whether the acceleration of the movement of the center point exceeds a threshold value, and judging dangerous behavior if yes;
s4, adjusting the dangerous grade of the first partition according to the dangerous behavior; comprising the following steps:
s41, calculating a risk level E of the first partition:
；
s42, identifying the level E of dangerous behavior M ；
S43, judging the distance between dangerous behaviors and dangerous objects in the first partition, and obtaining a minimum distance d;
s44, updating the risk level of the first partition to obtain a second risk level E of the first partition R ；
；
Wherein d 0 The distance is preset, and beta is an adjusting coefficient;
s5, adjusting the frequency of the inspection equipment based on the second danger level of the first subarea;
s6, identifying dangerous goods and dangerous behaviors based on a second visual monitoring device on the inspection equipment;
s7, the inspection equipment acquires leakage information update and updates the danger level of the first partition; comprising the following steps:
s71, collecting dangerous goods leakage information through a sensor on the inspection equipment;
s72, judging whether the leakage information exceeds a preset threshold value;
s73, if the threshold value is exceeded, updating the risk level of the first partition to be the third risk level E U ；
；
Wherein w is an adjustment coefficient, H is leakage information, and H 0 Is a leakage threshold;
and S8, displaying the real-time state of each first partition in real time through the three-dimensional model.
2. The method of claim 1, wherein the first visual monitoring device is communicatively coupled to a monitoring host; the inspection equipment is in communication connection with the inspection control platform; the management server is respectively in communication connection with the monitoring host, the inspection control platform and the portable terminal; the inspection equipment comprises a second visual monitoring device, a light intensity sensor, a leakage sensor and a light supplementing lamp.
3. The method of claim 2, wherein the monitoring host is further configured to obtain outlines of dangerous goods in the monitored image; analyzing an article center point of the dangerous article profile; judging the displacement of the center point according to the images of the adjacent frames, so as to calculate the moving acceleration of the center point; and analyzing whether the acceleration of the movement of the center point exceeds a threshold value, and if so, judging that dangerous behaviors occur.
4. A monitoring method according to claim 3, wherein the inspection device determines the light intensity based on the illumination sensor, and when the light intensity is too low, the light supplementing lamp is turned on, and the dangerous goods and dangerous behaviors are identified by the second visual monitoring device.
5. The method according to claim 4, wherein the step S5 of adjusting the frequency of the inspection device based on the second risk level of the first partition includes:
s51, setting the inspection frequency F of each first partition in a preset time period according to the updated partition danger level:
；
wherein F is 0 For the preset initial frequency, gamma is the regulating coefficient, E 0 The method comprises the steps of setting a dangerous initial value;
s52, planning a routing inspection route of the routing inspection equipment in each first partition.
6. The method according to claim 5, wherein the step S6 of identifying dangerous goods and dangerous behavior based on the second visual monitoring device on the inspection apparatus includes:
s61, the inspection equipment judges the light intensity based on the illumination sensor, if the light intensity is too low, the step S62 is carried out, otherwise, the step S63 is carried out;
s62, starting a light supplementing lamp, and identifying dangerous goods and dangerous behaviors through a second visual monitoring device;
s63, identifying dangerous goods and dangerous behaviors through a second visual monitoring device;
and S64, updating the dangerous level of the dangerous goods, the dangerous level of the dangerous behavior and the dividing result of the first subarea.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311302412.6A CN117041502B (en) | 2023-10-10 | 2023-10-10 | Dangerous scene analysis and monitoring system and method based on machine vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311302412.6A CN117041502B (en) | 2023-10-10 | 2023-10-10 | Dangerous scene analysis and monitoring system and method based on machine vision |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117041502A CN117041502A (en) | 2023-11-10 |
| CN117041502B true CN117041502B (en) | 2023-12-08 |
Family
ID=88623122
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311302412.6A Active CN117041502B (en) | 2023-10-10 | 2023-10-10 | Dangerous scene analysis and monitoring system and method based on machine vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117041502B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117876930B (en) * | 2024-01-11 | 2024-08-13 | 北京北测数字技术有限公司 | Video anomaly recognition processing method and system based on artificial intelligence |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH11353565A (en) * | 1998-06-09 | 1999-12-24 | Yazaki Corp | Method and device for alarm of collision for vehicle |
| WO2001017838A1 (en) * | 1999-09-09 | 2001-03-15 | Tiefenbach Gmbh | Method for monitoring a danger area |
| KR101579275B1 (en) * | 2015-05-22 | 2015-12-21 | 주식회사 사라다 | Security system using real-time monitoring with location-trace for dangerous-object |
| DE102016110459A1 (en) * | 2015-06-07 | 2016-12-08 | BOS Connect GmbH | Method and system for determining the location and application documentation for dangerous goods use |
| CN106228332A (en) * | 2016-08-11 | 2016-12-14 | 电信科学技术第四研究所 | A kind of dangerous chemical industry feed stream WMS based on Internet of Things and method |
| CN110992683A (en) * | 2019-10-29 | 2020-04-10 | 山东科技大学 | Dynamic image perception-based intersection blind area early warning method and system |
| KR102126498B1 (en) * | 2019-11-15 | 2020-06-25 | 한국건설기술연구원 | Apparatus, system and method for detecting dangerous situation based on image recognition |
| CN112257494A (en) * | 2020-09-09 | 2021-01-22 | 贵州赋行智能科技有限公司 | Behavior recognition method based on intelligent video analysis technology and application |
| CN114023044A (en) * | 2021-11-08 | 2022-02-08 | 西安链科信息技术有限公司 | Dangerous gas detection early warning processing method, device, equipment and terminal |
| CN114782988A (en) * | 2022-03-29 | 2022-07-22 | 西安交通大学 | Construction environment-oriented multi-stage safety early warning method |
| CN115482507A (en) * | 2022-09-22 | 2022-12-16 | 武汉理工光科股份有限公司 | Crowd gathering fire-fighting early warning method and system based on artificial intelligence |
| US11676291B1 (en) * | 2020-04-20 | 2023-06-13 | Everguard, Inc. | Adaptive multimodal safety systems and methods |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7522066B2 (en) * | 2006-02-23 | 2009-04-21 | Rockwell Automation Technologies, Inc. | Systems and methods that evaluate distance to potential hazards utilizing overlapping sensing zones |
| US8330605B2 (en) * | 2009-08-14 | 2012-12-11 | Accenture Global Services Limited | System for providing real time locating and gas exposure monitoring |
| KR102013935B1 (en) * | 2017-05-25 | 2019-08-23 | 삼성전자주식회사 | Method and system for detecting a dangerous situation |
-
2023
- 2023-10-10 CN CN202311302412.6A patent/CN117041502B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH11353565A (en) * | 1998-06-09 | 1999-12-24 | Yazaki Corp | Method and device for alarm of collision for vehicle |
| WO2001017838A1 (en) * | 1999-09-09 | 2001-03-15 | Tiefenbach Gmbh | Method for monitoring a danger area |
| KR101579275B1 (en) * | 2015-05-22 | 2015-12-21 | 주식회사 사라다 | Security system using real-time monitoring with location-trace for dangerous-object |
| DE102016110459A1 (en) * | 2015-06-07 | 2016-12-08 | BOS Connect GmbH | Method and system for determining the location and application documentation for dangerous goods use |
| CN106228332A (en) * | 2016-08-11 | 2016-12-14 | 电信科学技术第四研究所 | A kind of dangerous chemical industry feed stream WMS based on Internet of Things and method |
| CN110992683A (en) * | 2019-10-29 | 2020-04-10 | 山东科技大学 | Dynamic image perception-based intersection blind area early warning method and system |
| KR102126498B1 (en) * | 2019-11-15 | 2020-06-25 | 한국건설기술연구원 | Apparatus, system and method for detecting dangerous situation based on image recognition |
| US11676291B1 (en) * | 2020-04-20 | 2023-06-13 | Everguard, Inc. | Adaptive multimodal safety systems and methods |
| CN112257494A (en) * | 2020-09-09 | 2021-01-22 | 贵州赋行智能科技有限公司 | Behavior recognition method based on intelligent video analysis technology and application |
| CN114023044A (en) * | 2021-11-08 | 2022-02-08 | 西安链科信息技术有限公司 | Dangerous gas detection early warning processing method, device, equipment and terminal |
| CN114782988A (en) * | 2022-03-29 | 2022-07-22 | 西安交通大学 | Construction environment-oriented multi-stage safety early warning method |
| CN115482507A (en) * | 2022-09-22 | 2022-12-16 | 武汉理工光科股份有限公司 | Crowd gathering fire-fighting early warning method and system based on artificial intelligence |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117041502A (en) | 2023-11-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN117041502B (en) | Dangerous scene analysis and monitoring system and method based on machine vision | |
| CN113726606B (en) | Abnormality detection method and apparatus, electronic device, and storage medium | |
| CN110458157B (en) | Intelligent monitoring system for power cable production process | |
| CN114155601A (en) | Vision-based method and system for detecting dangerous behaviors of operating personnel | |
| CN113452912B (en) | Pan-tilt camera control method, device, equipment and medium for inspection robot | |
| CN107194360B (en) | Reverse current object identifying method, apparatus and system | |
| JP2013131159A (en) | Area monitoring system | |
| CN113076899B (en) | High-voltage transmission line foreign matter detection method based on target tracking algorithm | |
| CN112532927A (en) | Intelligent safety management and control system for construction site | |
| CN115431266A (en) | Inspection method, inspection device and inspection robot | |
| CN114730192A (en) | Object moving system | |
| CN112906441B (en) | Image recognition system and method for exploration and maintenance in communication industry | |
| CN117711131A (en) | Constructor warning protection system based on gesture danger analysis | |
| CN112446543A (en) | Urban underground comprehensive pipe gallery risk management and control system and method based on neural network | |
| CN115880598A (en) | Ground image detection method based on unmanned aerial vehicle and related device | |
| CN115144399A (en) | Assembly quality detection method and device based on machine vision | |
| CN113965733A (en) | Binocular video monitoring method, system, computer equipment and storage medium | |
| CN112802058A (en) | Method and device for tracking illegal moving target | |
| CN112802100A (en) | Intrusion detection method, device, equipment and computer readable storage medium | |
| CN111461079A (en) | Binocular image-based method for detecting personnel under suspension arm | |
| CN113989335A (en) | Method for automatically positioning workers in factory building | |
| CN115797411A (en) | Method for online identifying deformation of cable bridge of hydropower station by using machine vision | |
| CN115802002A (en) | Safety monitoring method for electric power operation | |
| CN117897737A (en) | Unmanned aerial vehicle monitoring method and device, unmanned aerial vehicle and monitoring equipment | |
| CN114758239B (en) | Method and system for monitoring flying of article away from preset travelling line based on machine vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |