CN112216073A - Ladder violation operation warning method and device - Google Patents
Ladder violation operation warning method and device Download PDFInfo
- Publication number
- CN112216073A CN112216073A CN202011086081.3A CN202011086081A CN112216073A CN 112216073 A CN112216073 A CN 112216073A CN 202011086081 A CN202011086081 A CN 202011086081A CN 112216073 A CN112216073 A CN 112216073A
- Authority
- CN
- China
- Prior art keywords
- ladder
- human body
- detection frame
- detection
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/18—Status alarms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Multimedia (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Human Computer Interaction (AREA)
- Bioinformatics & Computational Biology (AREA)
- Business, Economics & Management (AREA)
- Emergency Management (AREA)
- Alarm Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a ladder violation operation warning method and device, wherein the method comprises the following steps: detecting a ladder and a human body in an image sequence of a monitoring video to obtain a detection result comprising a ladder detection frame and a human body detection frame; determining whether the using state of the ladder is an illegal operating state or not according to the ladder detection frame and the human body detection frame in the detection result; the using state of the ladder is that under the condition of the illegal operating state, an alarm is initiated, the problems that in the related art, the ladder is fixed by using an additional device, the ladder is difficult to stabilize by using a safety rope due to various shapes of objects depended on by the ladder, and whether the ladder is used in an illegal mode cannot be judged are solved, the using process of the ladder is monitored in real time, the using state of the ladder is determined, when the ladder is in the illegal operating state, the alarm is initiated, a user can be timely prompted not to operate the ladder in an illegal mode, and safety guarantee is provided for the user.
Description
Technical Field
The invention relates to the field of image processing, in particular to a ladder violation operation warning method and device.
Background
In the power industry, workers often need to use ladders for climbing operations. However, ladder violation situations often occur due to poor worker safety awareness or inadequate field supervision. The illegal use of the electric power ladder is a great hidden danger of the electric power production safety and the personal safety of workers. Therefore, the automatic alarming method for illegal use of the electric power ladder is provided, and guarantee is provided for electric power safety production.
The single ladder body and the tower are connected through the two sets of safety ropes, so that the ladder is stabilized, and the safety of the ladder is improved. However, the ladder and the equipment are fixed by using the additional device, the shape of an object on which the ladder depends is various, and the ladder is difficult to stabilize by using a safety rope due to the fact that the ladder is provided with a wall body, an electric wire bar, a transformer and the like.
The safety rope ladder is fixed by using an additional device in the related art, and the problem that the ladder is difficult to stabilize by using a safety rope due to various shapes of objects depended on by the ladder and whether the ladder is illegally used cannot be judged is solved.
Disclosure of Invention
The embodiment of the invention provides a ladder violation operation warning method and device, and aims to at least solve the problems that in the related art, due to the fact that the ladder is fixed by using an additional device, due to the fact that the ladder depends on various shapes of objects, a safety rope is difficult to stabilize the ladder, and whether the ladder is used in a violation mode cannot be judged.
According to one embodiment of the invention, a ladder violation operation warning method is provided, and comprises the following steps:
detecting a ladder and a human body in an image sequence of a monitoring video to obtain a detection result comprising a ladder detection frame and a human body detection frame;
determining whether the using state of the ladder is an illegal operating state or not according to the ladder detection frame and the human body detection frame in the detection result;
and initiating an alarm when the using state of the ladder is the illegal operating state.
Optionally, detecting a ladder and a human body in an image sequence of a surveillance video, and obtaining the detection result including the ladder detection frame and the human body detection frame includes:
and inputting the image sequence into a pre-trained target neural network model to obtain the detection result output by the target neural network model.
Optionally, determining whether the usage state of the ladder is the violation operation state according to the ladder detection frame and the human body detection frame in the detection result includes:
under the condition that the detection result at least comprises one ladder detection frame and a plurality of human body detection frames, judging whether the ladder is obliquely placed or not according to the length-width ratio of the ladder detection frames;
determining the position relation between the ladder detection frame and the human body detection frames under the condition that the judgment result is yes;
and determining whether the using state of the ladder is an illegal operating state or not according to the position relation.
Optionally, the determining the positional relationship between the ladder detection frame and the plurality of human body detection frames comprises:
determining the up-down relationship and the overlapping relationship between the ladder detection frame and the plurality of human body detection frames;
and determining the position relation between the ladder detection frame and the human body detection frames according to the up-down relation and the overlapping relation.
Optionally, the determining the up-down relationship and the overlapping relationship between the ladder detection frame and the plurality of human body detection frames comprises:
determining the up-down relation between the ladder detection frame and the human body detection frames through the central longitudinal coordinate of the ladder detection frame and the central longitudinal coordinates of the human body detection frames respectively;
and determining the overlapping relation between the ladder detection frame and the plurality of human body detection frames respectively through intersection ratios.
Optionally, determining whether the usage state of the ladder is the illegal operation state according to the position relationship comprises:
judging whether only one first human body detection frame meeting a first position relation exists in the plurality of human body detection frames, wherein the first position relation is that the central longitudinal coordinate of the first human body detection frame is smaller than or equal to the central longitudinal coordinate of the ladder detection frame, and the intersection ratio of the first human body detection frame and the ladder detection frame is larger than a preset value;
under the condition that only one first human body detection frame meeting a first position relation exists in the plurality of human body detection frames, judging whether at least one second human body detection frame meeting a second position relation exists in the plurality of human body detection frames, wherein the second position relation is that the central longitudinal coordinate of the second human body detection frame is larger than the central longitudinal coordinate of the ladder detection frame and the intersection ratio of the second human body detection frame and the ladder detection frame is larger than the preset value; determining that the using state of the ladder is the illegal operation state under the condition that the judgment result is negative;
and determining that the using state of the ladder is the illegal operating state under the condition that at least two first human body detection frames meeting the first position relation exist in the plurality of human body detection frames.
Optionally, the method further comprises:
and sending the monitoring video to a server, wherein the server is used for storing the monitoring video, and the image sequence of the monitoring video carries the detection result and the using state of the ladder.
There is also provided, in accordance with another embodiment of the present invention, a ladder violation operation warning device, including:
the detection module is used for detecting the ladder and the human body in the image sequence of the monitoring video to obtain a detection result comprising a ladder detection frame and a human body detection frame;
the determining module is used for determining whether the using state of the ladder is an illegal operating state according to the ladder detecting frame and the human body detecting frame in the detection result;
and the warning module is used for initiating warning under the condition that the using state of the ladder is the illegal operation state.
Optionally, the detection module is further used for
And inputting the image sequence into a pre-trained target neural network model to obtain the detection result output by the target neural network model.
Optionally, the determining module includes:
the judgment submodule is used for judging whether the ladder is obliquely placed or not according to the length-width ratio of the ladder detection frame under the condition that the detection result at least comprises one ladder detection frame and a plurality of human body detection frames;
the first determining submodule is used for determining the position relation between the ladder detection frame and the human body detection frames under the condition that the judgment result is yes;
and the second determining submodule is used for determining whether the using state of the ladder is an illegal operating state according to the position relation.
Optionally, the first determining sub-module includes:
a first determination unit configured to determine an upper-lower relationship and an overlapping relationship between the ladder detection frame and the plurality of human body detection frames;
and the second determining unit is used for determining the position relation between the ladder detection frame and the human body detection frames according to the up-down relation and the overlapping relation.
Optionally, the first determining unit is further configured to
Determining the up-down relation between the ladder detection frame and the human body detection frames through the central longitudinal coordinate of the ladder detection frame and the central longitudinal coordinates of the human body detection frames respectively;
and determining the overlapping relation between the ladder detection frame and the plurality of human body detection frames respectively through intersection ratios.
Optionally, the second determining sub-module includes:
the judgment unit is used for judging whether only one first human body detection frame meeting a first position relationship exists in the plurality of human body detection frames, wherein the first position relationship is that the central longitudinal coordinate of the first human body detection frame is less than or equal to the central longitudinal coordinate of the ladder detection frame and the intersection ratio of the first human body detection frame and the ladder detection frame is greater than a preset value;
a third determining unit, configured to determine whether at least one second human body detection frame satisfying a second positional relationship exists in the plurality of human body detection frames when only one first human body detection frame satisfying the first positional relationship exists in the plurality of human body detection frames, where the second positional relationship is that a central ordinate of the second human body detection frame is greater than a central ordinate of the ladder detection frame and an intersection ratio of the second human body detection frame to the ladder detection frame is greater than the preset value; determining that the using state of the ladder is the illegal operation state under the condition that the judgment result is negative;
a fourth determining unit, configured to determine that the usage state of the ladder is the illegal operation state when at least two first human detection frames satisfying the first positional relationship exist in the plurality of human detection frames.
Optionally, the apparatus further comprises:
and the sending module is used for sending the monitoring video to a server, and the server is used for storing the monitoring video, wherein the image sequence of the monitoring video carries the detection result and the using state of the ladder.
According to a further embodiment of the present invention, a computer-readable storage medium is also provided, in which a computer program is stored, wherein the computer program is configured to perform the steps of any of the above-described method embodiments when executed.
According to yet another embodiment of the present invention, there is also provided an electronic device, including a memory in which a computer program is stored and a processor configured to execute the computer program to perform the steps in any of the above method embodiments.
According to the invention, the detection result is obtained by detecting the ladder and the human body in the image sequence of the monitoring video collected in real time, the using state of the ladder is determined based on the ladder detection frame and the human body detection frame in the detection result, if the ladder is determined to be illegal operation, namely the using state of the ladder is determined to be illegal operation state, the warning is initiated, the problems that the ladder is difficult to stabilize by using a safety rope due to various shapes of objects depended on by the ladder and cannot be judged whether the ladder is illegally used in the related technology can be solved, when the ladder is detected to be illegal operation state, the user is prompted in a warning mode, the user can be prompted in time not to illegally operate the ladder, and the safety guarantee is provided for the user.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
FIG. 1 is a block diagram of a hardware configuration of a mobile terminal of a ladder violation operation warning method according to an embodiment of the present invention;
FIG. 2 is a flow diagram of a ladder violation operation warning method according to an embodiment of the present invention;
FIG. 3 is a flow diagram of a ladder violation use auto-alert method according to an embodiment of the present invention;
FIG. 4 is a flow chart of determining an illegal use of a ladder according to an embodiment of the present invention;
FIG. 5 is a block diagram of a ladder violation operation warning device according to an embodiment of the present invention.
Detailed Description
The invention will be described in detail hereinafter with reference to the accompanying drawings in conjunction with embodiments. It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.
Example 1
The method provided by the first embodiment of the present application may be executed in a mobile terminal, a computer terminal, or a similar computing device. Taking a mobile terminal as an example, fig. 1 is a block diagram of a hardware structure of a mobile terminal of a ladder violation operation warning method according to an embodiment of the present invention, and as shown in fig. 1, the mobile terminal may include one or more processors 102 (only one is shown in fig. 1) (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), and a memory 104 for storing data, and optionally, the mobile terminal may further include a transmission device 106 for a communication function and an input/output device 108. It will be understood by those skilled in the art that the structure shown in fig. 1 is only an illustration, and does not limit the structure of the mobile terminal. For example, the mobile terminal may also include more or fewer components than shown in FIG. 1, or have a different configuration than shown in FIG. 1.
The memory 104 may be used to store a computer program, for example, a software program and a module of application software, such as a computer program corresponding to the ladder violation operation warning method in the embodiment of the present invention, and the processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, so as to implement the method described above. The memory 104 may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory located remotely from the processor 102, which may be connected to the mobile terminal over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission device 106 is used for receiving or transmitting data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a Network adapter (NIC), which can be connected to other Network devices through a base station so as to communicate with the internet. In one example, the transmission device 106 may be a Radio Frequency (RF) module, which is used for communicating with the internet in a wireless manner.
In this embodiment, a ladder violation operation warning method operating in the mobile terminal or the network architecture is provided, and fig. 2 is a flowchart of the ladder violation operation warning method according to the embodiment of the present invention, as shown in fig. 2, the flowchart includes the following steps:
step S202, detecting a ladder and a human body in an image sequence of a monitoring video to obtain a detection result comprising a ladder detection frame and a human body detection frame;
further, the step S204 may specifically include: inputting the image sequence into a pre-trained target neural network model, and carrying out ladder and human body detection to obtain the detection result output by the target neural network model.
Step S204, determining whether the using state of the ladder is an illegal operating state according to the ladder detection frame and the human body detection frame in the detection result;
and step S206, initiating an alarm under the condition that the using state of the ladder is the illegal operating state.
Through the steps S202 to S206, detecting ladders and human bodies in the image sequence of the monitoring video to obtain a detection result; determining whether the using state of the ladder is an illegal operating state or not according to the detection result; the using state of the ladder is that under the condition of the illegal operating state, an alarm is initiated, the problems that in the related art, the ladder is fixed by using an additional device, the ladder is difficult to stabilize by using a safety rope due to various shapes of objects depended on by the ladder, and whether the ladder is used in an illegal mode cannot be judged are solved, the using process of the ladder is monitored in real time, the using state of the ladder is determined, when the ladder is in the illegal operating state, the alarm is initiated, a user can be timely prompted not to operate the ladder in an illegal mode, and safety guarantee is provided for the user.
In an embodiment of the present invention, the step S204 may specifically include: under the condition that the detection result at least comprises one ladder detection frame and a plurality of human body detection frames, namely at least one ladder and a plurality of people in the monitoring video are shown, only one person can be on the ladder in the regular operation process, and at least one person holds up the ladder below the ladder. Whether the operation is an illegal operation or not can be determined further according to the relation between the ladder and the person, namely whether the using state of the ladder is the illegal operation state or not is determined according to the ladder detection frame and the human body detection frames. And if the detection result only comprises one ladder detection frame and one human body detection frame, directly judging and locating the illegal operation, namely the using state of the ladder is the illegal operation state.
In an optional embodiment, the determining whether the usage status of the ladder is an illegal operation status according to the ladder detection box and the plurality of human body detection boxes may specifically include:
judging whether the ladder is obliquely placed or not according to the length-width ratio of the ladder detection frame;
if the determination result is yes, determining a positional relationship between the ladder detection frame and the plurality of human body detection frames, and further determining an upper-lower relationship and an overlapping relationship between the ladder detection frame and the plurality of human body detection frames, which may be specifically determined by: determining the up-down relation between the ladder detection frame and the human body detection frames through the central longitudinal coordinate of the ladder detection frame and the central longitudinal coordinates of the human body detection frames respectively; determining the overlapping relationship between the ladder detection frame and the human body detection frames through an intersection ratio, and determining the position relationship between the ladder detection frame and the human body detection frames according to the up-down relationship and the overlapping relationship;
determining whether the using state of the ladder is an illegal operating state or not according to the position relationship, and further judging whether only one first human body detection frame meeting a first position relationship exists in the plurality of human body detection frames or not, wherein the first position relationship is that the central longitudinal coordinate of the first human body detection frame is less than or equal to the central longitudinal coordinate of the ladder detection frame and the intersection ratio of the first human body detection frame and the ladder detection frame is greater than a preset value; under the condition that only one first human body detection frame meeting a first position relation exists in the plurality of human body detection frames, judging whether at least one second human body detection frame meeting a second position relation exists in the plurality of human body detection frames, wherein the second position relation is that the central longitudinal coordinate of the second human body detection frame is larger than the central longitudinal coordinate of the ladder detection frame and the intersection ratio of the second human body detection frame and the ladder detection frame is larger than the preset value; determining that the using state of the ladder is the illegal operation state under the condition that the judgment result is negative; if the judgment result is yes, determining that the using state of the ladder is a normal operating state; determining that the using state of the ladder is a normal operating state under the condition that a first human body detection frame meeting the first position relation does not exist in the plurality of human body detection frames; and determining that the using state of the ladder is the illegal operating state under the condition that at least two first human body detection frames meeting the first position relation exist in the plurality of human body detection frames.
In another optional embodiment, a monitoring video may be further stored by the server, so as to facilitate subsequent viewing, and specifically, the monitoring video is sent to the server, and the server stores the monitoring video, wherein an image sequence of the monitoring video carries the detection result and the use state of the ladder. The server can be a central server or a background server and the like, the detection result can be overlaid to the monitoring video in the background to be displayed, supervision personnel can conveniently supervise the site, and the central server or the background server records and archives the using state of the ladder.
According to the embodiment of the invention, a detection algorithm based on YOLOv3 is used for simultaneously detecting the electric power ladder and the person in the monitoring picture and analyzing whether the ladder is illegally used. And if the illegal use happens, an alarm is given. And transmitting the ladder use state and monitoring video information to a central server for recording the information. A ladder and a rectangular frame of a human body are added in a background monitoring picture so as to facilitate the supervision of an administrator on the site. FIG. 3 is a flow chart of a ladder violation use auto-alert method according to an embodiment of the present invention, as shown in FIG. 3, including:
in step S301, an image sequence of a video is obtained.
And installing a monitoring camera on the site using the ladder, and adjusting the camera to enable the construction area to occupy most of the monitoring picture. The surveillance video needs to be decimated to obtain image data and then detected on a frame-by-frame basis.
In step S302, a ladder and a person are detected using YOLOv 3.
YOLOv3 is an efficient target detection network. According to the scheme, the trained YOLOv3 model is used for detecting the ladder and the human body in the monitoring image to obtain the detection result.
And step S303, judging whether the ladder is illegally used, and executing step S304 if the judgment result is yes, or executing step S305 if the judgment result is not yes.
Detecting ladders and human bodies in a picture in real time, and judging whether the ladders are illegally used, wherein fig. 4 is a flow chart for determining illegal use of the ladders according to the embodiment of the invention, and as shown in fig. 4, the flow chart comprises the following steps:
step S401 is to determine whether or not at least 1 ladder is leaned against the ladder according to the aspect ratio of the ladder frame, and if yes, step S402 is executed.
Step S402, calculating rjy、tiyAnd IOUijThe vertical relationship between the leaning ladder and the human body frame can be judged by the vertical coordinate ty of the center of the ladder frame and the vertical coordinate ry of the center of the human body frame, and the overlapping relationship between the ladder frame and the human body frame can be judged by the Intersection Over Unit (IOU). The IOU calculation formula of the ladder frame and the human body frame is as follows:
wherein, A represents a ladder frame, and B represents a human body frame. The value of IOU is [0,1], and the bigger the IOU is, the bigger the coincidence degree of A and B is.
If the length-width ratio of the ladder frame is less than 0.5, the vertical edge of the ladder frame is more than 2 times of the horizontal edge, so that the ladder can be judged to be leaned.
Using the upper left corner of the image as the origin of coordinates, the picture is respectively the positive direction of abscissa and the positive direction of ordinate from left to right and from top to bottom, then the relation of ladder and people respectively includes:
Siju＝(rjy≤tiy)&(IOUij＞0.1)，
Sijd＝(rjy＞tiy)&(IOUij＞0.1)，
wherein S isijuIndicating that the jth individual is working on the ith leaning ladder, SijdIndicating that the jth individual is holding the ladder below the ith ladder, rjyRepresents the center ordinate, t, of the prediction box of the jth personiyIndicating the center ordinate, IOU, of the prediction box of the ith ladderijIndicating the intersection ratio of the jth individual to the ith ladder. And j is a value range of all human body detection frames in the picture, and i is a value range of all detection frames leaning against the ladder in the picture. If r isjy≤tiyShown above the ladder with a person; if r isjy＞tiyShown below the ladder; if the human body frame and the ladder frame IOUij>0.1, the ladder frame and the human body frame are considered to have an overlapped part.
Step S403, for the ith ladder, judging whether the jth individual satisfies SijuIf the judgment result is yes, executing step S404, otherwise executing step S407;
step S404, for satisfying SijuThe ith ladder of (1), and whether the kth individual is present or not is judged to satisfy SiguIf the determination result is yes, executing step S406, otherwise executing step S407;
step S405, for satisfying SijuThe ith ladder of (1), judging whether the g-th person is present or notikdIf the judgment result is yes, executing step S405, otherwise executing step S406;
step S406, alarming, namely sending an alarm;
in step S407, detection frame information of all persons and ladders is output.
Thus, not only can S passijuAnd SijdAnd judging whether a person holds the ladder or not under the ladder, and counting the number of people working on the same ladder.
And step S304, sending out an alarm.
If the ladder is illegally used, alarming can be carried out, and two illegal using conditions can be alarmed: people work on the ladder, but no people hold the ladder below the ladder; two people work on the same ladder.
Step S305, the monitoring video and the ladder use state are sent to a central server.
And transmitting the construction site video, the target frame position information and the alarm state to a central server. The background superimposes the detection result on the monitoring video to be displayed, so that supervision personnel can conveniently supervise the site. The central server archives ladder usage status records.
And the ladder illegal use alarm is realized by analyzing the monitoring video. There are two alarm scenarios: people work on the ladder, but no people hold the ladder under the ladder. Two people are on the same ladder. By comparing the longitudinal coordinates r of the center of the human body framejyAnd the vertical coordinate t of the center of the ladder frameiyAnd IOUijAnd judging the position relation between the ladder and the person. The detailed determination method is shown in step 3. And transmitting the construction site video, the target frame position information and the alarm state to a central server, and simultaneously overlapping the detection result to the monitoring video for displaying, thereby facilitating supervision of the site by supervisors. The central server archives ladder usage status records.
Through the mode of video monitoring, judge ladder illegal use automatically and send the warning, can judge the in service behavior of many ladders simultaneously in two kinds of different illegal use scenes. Additional devices such as safety ropes and the like do not need to be added on each ladder, so that material resources are saved; the field information of the ladder is transmitted back to the central server, the using state of the ladder is recorded, the intellectualization and the informatization of power grid management are enhanced, the method is strong in expansibility, and iterative upgrading is facilitated.
Example 2
There is also provided, in accordance with another embodiment of the present invention, a ladder violation operation warning device, fig. 5 is a block diagram of a ladder violation operation warning device in accordance with an embodiment of the present invention, as shown in fig. 5, including:
the detection module 52 is configured to detect a ladder and a human body in an image sequence of the monitoring video, and obtain a detection result including a ladder detection frame and a human body detection frame;
a determining module 54, configured to determine whether a usage state of the ladder is an illegal operation state according to the ladder detection frame and the human body detection frame in the detection result;
and the warning module 56 is used for initiating a warning when the using state of the ladder is the illegal operating state.
Optionally, the detection module 52 is further configured to
And inputting the image sequence into a pre-trained target neural network model to obtain the detection result output by the target neural network model.
Optionally, the determining module 54 includes:
the judgment submodule is used for judging whether the ladder is obliquely placed or not according to the length-width ratio of the ladder detection frame under the condition that the detection result at least comprises one ladder detection frame and a plurality of human body detection frames;
the first determining submodule is used for determining the position relation between the ladder detection frame and the human body detection frames under the condition that the judgment result is yes;
and the second determining submodule is used for determining whether the using state of the ladder is an illegal operating state according to the position relation.
Optionally, the first determining sub-module includes:
a first determination unit configured to determine an upper-lower relationship and an overlapping relationship between the ladder detection frame and the plurality of human body detection frames;
and the second determining unit is used for determining the position relation between the ladder detection frame and the human body detection frames according to the up-down relation and the overlapping relation.
Optionally, the first determining unit is further configured to
Determining the up-down relation between the ladder detection frame and the human body detection frames through the central longitudinal coordinate of the ladder detection frame and the central longitudinal coordinates of the human body detection frames respectively;
and determining the overlapping relation between the ladder detection frame and the plurality of human body detection frames respectively through intersection ratios.
Optionally, the second determining sub-module includes:
the judgment unit is used for judging whether only one first human body detection frame meeting a first position relationship exists in the plurality of human body detection frames, wherein the first position relationship is that the central longitudinal coordinate of the first human body detection frame is less than or equal to the central longitudinal coordinate of the ladder detection frame and the intersection ratio of the first human body detection frame and the ladder detection frame is greater than a preset value;
a third determining unit, configured to determine whether at least one second human body detection frame satisfying a second positional relationship exists in the plurality of human body detection frames when only one first human body detection frame satisfying the first positional relationship exists in the plurality of human body detection frames, where the second positional relationship is that a central ordinate of the second human body detection frame is greater than a central ordinate of the ladder detection frame and an intersection ratio of the second human body detection frame to the ladder detection frame is greater than the preset value; determining that the using state of the ladder is the illegal operation state under the condition that the judgment result is negative;
a fourth determining unit, configured to determine that the usage state of the ladder is the illegal operation state when at least two first human detection frames satisfying the first positional relationship exist in the plurality of human detection frames.
Optionally, the apparatus further comprises:
and the sending module is used for sending the monitoring video to a server, and the server is used for storing the monitoring video, wherein the image sequence of the monitoring video carries the detection result and the using state of the ladder.
It should be noted that, the above modules may be implemented by software or hardware, and for the latter, the following may be implemented, but not limited to: the modules are all positioned in the same processor; alternatively, the modules are respectively located in different processors in any combination.
Example 3
Embodiments of the present invention also provide a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to perform the steps of any of the above method embodiments when executed.
Alternatively, in the present embodiment, the storage medium may be configured to store a computer program for executing the steps of:
s1, detecting the ladder and the human body in the image sequence of the monitoring video to obtain a detection result comprising a ladder detection frame and a human body detection frame;
s2, determining whether the using state of the ladder is an illegal operating state or not according to the ladder detection frame and the human body detection frame in the detection result;
and S3, if the using state of the ladder is the illegal operating state, initiating an alarm.
Optionally, in this embodiment, the storage medium may include, but is not limited to: various media capable of storing computer programs, such as a usb disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic disk, or an optical disk.
Example 4
Embodiments of the present invention also provide an electronic device comprising a memory having a computer program stored therein and a processor arranged to run the computer program to perform the steps of any of the above method embodiments.
Optionally, the electronic apparatus may further include a transmission device and an input/output device, wherein the transmission device is connected to the processor, and the input/output device is connected to the processor.
Optionally, in this embodiment, the processor may be configured to execute the following steps by a computer program:
s1, detecting the ladder and the human body in the image sequence of the monitoring video to obtain a detection result comprising a ladder detection frame and a human body detection frame;
s2, determining whether the using state of the ladder is an illegal operating state or not according to the ladder detection frame and the human body detection frame in the detection result;
and S3, if the using state of the ladder is the illegal operating state, initiating an alarm.
Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation manners, and this embodiment is not described herein again.
It will be apparent to those skilled in the art that the modules or steps of the present invention described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of multiple computing devices, and alternatively, they may be implemented by program code executable by a computing device, such that they may be stored in a storage device and executed by a computing device, and in some cases, the steps shown or described may be performed in an order different than that described herein, or they may be separately fabricated into individual integrated circuit modules, or multiple ones of them may be fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A ladder violation operation warning method is characterized by comprising the following steps:
detecting a ladder and a human body in an image sequence of a monitoring video to obtain a detection result comprising a ladder detection frame and a human body detection frame;
determining whether the using state of the ladder is an illegal operating state or not according to the ladder detection frame and the human body detection frame in the detection result;
and initiating an alarm when the using state of the ladder is the illegal operating state.
2. The method of claim 1, wherein detecting a ladder and a human in an image sequence of a surveillance video, and obtaining the detection result comprising the ladder detection box and the human detection box comprises:
and inputting the image sequence into a pre-trained target neural network model to obtain the detection result output by the target neural network model.
3. The method of claim 1, wherein determining whether the usage status of the ladder is the violation operation status according to the ladder detection box and the human detection box in the detection result comprises:
under the condition that the detection result at least comprises one ladder detection frame and a plurality of human body detection frames, judging whether the ladder is obliquely placed or not according to the length-width ratio of the ladder detection frames;
determining the position relation between the ladder detection frame and the human body detection frames under the condition that the judgment result is yes;
and determining whether the using state of the ladder is the illegal operating state or not according to the position relation.
4. The method of claim 3, wherein determining the positional relationship of the ladder detection frame to the plurality of human detection frames comprises:
determining the up-down relationship and the overlapping relationship between the ladder detection frame and the plurality of human body detection frames;
and determining the position relation between the ladder detection frame and the human body detection frames according to the up-down relation and the overlapping relation.
5. The method of claim 4, wherein determining an up-down relationship and an overlapping relationship of the ladder detection frame and the plurality of human detection frames comprises:
determining the up-down relation between the ladder detection frame and the human body detection frames through the central longitudinal coordinate of the ladder detection frame and the central longitudinal coordinates of the human body detection frames respectively;
and determining the overlapping relation between the ladder detection frame and the plurality of human body detection frames respectively through intersection ratios.
6. The method of claim 5, wherein determining whether the usage status of the ladder is the violation status based on the positional relationship comprises:
judging whether only one first human body detection frame meeting a first position relation exists in the plurality of human body detection frames, wherein the first position relation is that the central longitudinal coordinate of the first human body detection frame is smaller than or equal to the central longitudinal coordinate of the ladder detection frame, and the intersection ratio of the first human body detection frame and the ladder detection frame is larger than a preset value;
under the condition that only one first human body detection frame meeting a first position relation exists in the plurality of human body detection frames, judging whether at least one second human body detection frame meeting a second position relation exists in the plurality of human body detection frames, wherein the second position relation is that the central longitudinal coordinate of the second human body detection frame is larger than the central longitudinal coordinate of the ladder detection frame and the intersection ratio of the second human body detection frame and the ladder detection frame is larger than the preset value; determining that the using state of the ladder is the illegal operation state under the condition that the judgment result is negative;
and determining that the using state of the ladder is the illegal operating state under the condition that at least two first human body detection frames meeting the first position relation exist in the plurality of human body detection frames.
7. The method according to any one of claims 1 to 6, further comprising:
and sending the monitoring video to a server, wherein the server is used for storing the monitoring video, and the image sequence of the monitoring video carries the detection result and the using state of the ladder.
8. A ladder violation operation warning device, comprising:
the detection module is used for detecting the ladder and the human body in the image sequence of the monitoring video to obtain a detection result comprising a ladder detection frame and a human body detection frame;
the determining module is used for determining whether the using state of the ladder is an illegal operating state according to the ladder detecting frame and the human body detecting frame in the detection result;
and the warning module is used for initiating warning under the condition that the using state of the ladder is the illegal operation state.
9. A computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to carry out the method of any one of claims 1 to 7 when executed.
10. An electronic device comprising a memory and a processor, wherein the memory has stored therein a computer program, and wherein the processor is arranged to execute the computer program to perform the method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011086081.3A CN112216073B (en) | 2020-10-12 | 2020-10-12 | Ladder violation operation warning method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011086081.3A CN112216073B (en) | 2020-10-12 | 2020-10-12 | Ladder violation operation warning method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112216073A true CN112216073A (en) | 2021-01-12 |
| CN112216073B CN112216073B (en) | 2022-05-03 |
Family
ID=74053742
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011086081.3A Active CN112216073B (en) | 2020-10-12 | 2020-10-12 | Ladder violation operation warning method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112216073B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113627302A (en) * | 2021-08-03 | 2021-11-09 | 云南大学 | Method and system for detecting compliance of ascending construction |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101459293B1 (en) * | 2013-09-06 | 2014-11-07 | 주식회사 신광테크놀러지 | Special Purpose Vehicle |
| CN110807429A (en) * | 2019-10-23 | 2020-02-18 | 西安科技大学 | Construction safety detection method and system based on tiny-YOLOv3 |
| CN111310665A (en) * | 2020-02-18 | 2020-06-19 | 深圳市商汤科技有限公司 | Violation event detection method and device, electronic equipment and storage medium |
| CN111325128A (en) * | 2020-02-13 | 2020-06-23 | 上海眼控科技股份有限公司 | Illegal operation detection method and device, computer equipment and storage medium |
| CN111669548A (en) * | 2020-06-04 | 2020-09-15 | 赛特斯信息科技股份有限公司 | Method for realizing safety supervision and treatment aiming at pole climbing operation of power distribution network |
| CN111726586A (en) * | 2020-06-29 | 2020-09-29 | 上海药明生物技术有限公司 | Production system operation standard monitoring and reminding system |
-
2020
- 2020-10-12 CN CN202011086081.3A patent/CN112216073B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101459293B1 (en) * | 2013-09-06 | 2014-11-07 | 주식회사 신광테크놀러지 | Special Purpose Vehicle |
| CN110807429A (en) * | 2019-10-23 | 2020-02-18 | 西安科技大学 | Construction safety detection method and system based on tiny-YOLOv3 |
| CN111325128A (en) * | 2020-02-13 | 2020-06-23 | 上海眼控科技股份有限公司 | Illegal operation detection method and device, computer equipment and storage medium |
| CN111310665A (en) * | 2020-02-18 | 2020-06-19 | 深圳市商汤科技有限公司 | Violation event detection method and device, electronic equipment and storage medium |
| CN111669548A (en) * | 2020-06-04 | 2020-09-15 | 赛特斯信息科技股份有限公司 | Method for realizing safety supervision and treatment aiming at pole climbing operation of power distribution network |
| CN111726586A (en) * | 2020-06-29 | 2020-09-29 | 上海药明生物技术有限公司 | Production system operation standard monitoring and reminding system |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113627302A (en) * | 2021-08-03 | 2021-11-09 | 云南大学 | Method and system for detecting compliance of ascending construction |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112216073B (en) | 2022-05-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106878670B (en) | A kind of method for processing video frequency and device | |
| CN109484935A (en) | A kind of lift car monitoring method, apparatus and system | |
| CN104038952B (en) | The Forecasting Methodology and forecasting system of accident | |
| CN106791655B (en) | A kind of method for processing video frequency and device | |
| CN109102677A (en) | Construction site safety monitoring system and method | |
| US20230005176A1 (en) | Throwing position acquisition method and apparatus, computer device and storage medium | |
| CN112363443A (en) | Automatic monitoring method and system for data center | |
| CN210691346U (en) | Automatic identification system based on construction site hidden danger | |
| CN114821406A (en) | Method and system for judging electric power operation field violation behaviors | |
| CN112112629A (en) | Safety business management system and method in drilling operation process | |
| CN110751809A (en) | Construction safety monitoring method and related product | |
| CN112216073B (en) | Ladder violation operation warning method and device | |
| CN112288320A (en) | Subway operation risk monitoring and management system | |
| CN210222962U (en) | Intelligent electronic fence system | |
| CN112381435A (en) | Gridding directional pushing management method for dynamic risk in hydropower station operation process | |
| CN111860187A (en) | High-precision worn mask identification method and system | |
| CN111539595A (en) | Method, apparatus, computer device and storage medium for handling alarms | |
| CN111465045B (en) | AP monitoring method, monitoring server and monitoring system | |
| CN111540020B (en) | Method and device for determining target behavior, storage medium and electronic device | |
| CN113506416A (en) | Engineering abnormity early warning method and system based on intelligent visual analysis | |
| CN107196815B (en) | Method and equipment for determining difference of flow analysis capacity | |
| CN109120896B (en) | Security video monitoring guard system | |
| CN115223091A (en) | AI-based construction site hidden danger monitoring method, device, equipment and storage medium | |
| CN114821957A (en) | AI video analysis system and method | |
| CN115937735A (en) | Risk monitoring method and system for monitoring safety of aerial work in real time |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |