CN115841497B - Boundary detection method and escalator area intrusion detection method and system - Google Patents
Boundary detection method and escalator area intrusion detection method and system Download PDFInfo
- Publication number
- CN115841497B CN115841497B CN202310143406.4A CN202310143406A CN115841497B CN 115841497 B CN115841497 B CN 115841497B CN 202310143406 A CN202310143406 A CN 202310143406A CN 115841497 B CN115841497 B CN 115841497B
- Authority
- CN
- China
- Prior art keywords
- boundary
- tripwire
- line
- connecting line
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02B—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO BUILDINGS, e.g. HOUSING, HOUSE APPLIANCES OR RELATED END-USER APPLICATIONS
- Y02B50/00—Energy efficient technologies in elevators, escalators and moving walkways, e.g. energy saving or recuperation technologies
Abstract
The invention belongs to the field of Internet of things, and discloses a method for detecting a boundary line between a floor plate and a footpath of an escalator based on image semantic segmentation, which comprises the following steps: step 1: acquiring an image; step 2: drawing a boundary line in the image; and step 3: drawing convex points facing to an x coordinate axis on the boundary line; and 4, step 4: calculating the slope of the first connecting line; and 5: calculating the slope of the second connecting line; step 6: judging whether the included angle between the slope of the first connecting line and the second connecting line is within a preset error range or not; and 7: and removing the two outermost protruded points on the boundary line from the boundary line, and performing the step 5. The method adopts two outermost points on the boundary line to judge the slope, and removes the outermost points with larger errors through calculation to obtain the slope of the boundary line with high relative precision. Meanwhile, the invention also discloses an escalator three-dimensional area intrusion detection method and system.
Description
Technical Field
The invention relates to the technical field of Internet of things, in particular to a boundary line detection method, an escalator region intrusion detection method and an escalator region intrusion detection system, and more particularly relates to a boundary line detection method between a floor plate and a footpath, and an escalator three-dimensional region intrusion detection method and an escalator three-dimensional region intrusion detection system based on human skeleton and image semantic segmentation.
Background
The applicant proposes an invention patent application CN114708552A in the following, which discloses a three-dimensional region intrusion detection method based on human bones, comprising the following steps: step 1: after a human body enters a shooting area, identifying human skeleton to obtain skeleton information, wherein a plurality of position points are marked in the skeleton information; step 2: when a human body enters the escalator, acquiring a first image, and calculating the height relation between the handrail and two position points of the human body at different heights; and step 3: in the running process of the escalator, a second image is obtained, and according to the height relation and the two position points, a point on the handrail, which keeps the same horizontal height with the human body, is determined; and 4, step 4: and (3) drawing a plumb line on the point determined in the step (3) on the handrail, judging whether the human skeleton is intersected with the plumb line, and giving an alarm if the human skeleton is intersected with the plumb line.
It has the following problems:
1. the scheme determines the wiring through the hip bone, but the heights of all people are different, so that large errors are easy to generate;
2. the scheme needs to draw a connecting line once when entering the elevator, and the calculation is complex;
3. the scheme needs to draw a connecting line once when entering the elevator, needs to track the human body, and is easy to lose the algorithm;
4. the camera is required to be installed to be opposite to the center of the elevator, and the installation requirement is high;
5. the camera can shoot the deformation of the object under the wide-angle lens. If the escalator is in the escalator area, the intrusion judgment is wrong due to deviation;
to solve this problem, we propose to draw a tripwire based on a vertical line drawn from a boundary between a floor panel and a walkway, and to detect whether or not a three-dimensional area intrusion is generated by the tripwire.
However, based on the 2 realistic cases mentioned above, the accuracy of the dividing line is not too high, specifically:
1. the camera is required to be installed to be opposite to the center of the elevator, and the installation requirement is high;
2. semantic segmentation aims at the boundary between a floor plate and a footpath to obtain a jagged line, and an accurate slope cannot be found when the boundary is drawn.
Finally, there is no way to obtain a boundary line with high precision, and if the boundary line with high precision is not obtained, a perpendicular line and a tripwire with high precision are not obtained.
Based on this, the technical problem that this case solved is: how accurately the slope of the boundary is known.
Disclosure of Invention
The invention aims to provide a method for detecting a boundary between a floor plate and a footpath.
In the actual research process, it is found that it is not enough to use only the boundary line between the floor plate and the footpath to draw the tripwire, and in the wide-angle camera, if the person is far away, the image distortion is generated, which causes the accuracy of the tripwire to be reduced.
Therefore, the invention also discloses an escalator three-dimensional area intrusion detection method based on human skeleton and image semantic segmentation, which adopts the shin bones of passengers for connecting lines, draws a first connecting line, searches an intersection point A and an intersection point B, sets two tripwire lines, selects one of the two tripwire lines as a standard tripwire line, and can reduce the error under the condition that a camera is not accurately installed and only adopts the first tripwire line as the standard tripwire line;
the method of the invention does not need to track the human body when the passenger enters the elevator, does not need to require the installation precision of the camera too rigorously, and has sensitive alarm.
Finally, the invention also provides an escalator three-dimensional area intrusion detection system based on human skeleton and image semantic segmentation.
In order to achieve the purpose, the invention provides the following technical scheme: a detection method for a boundary between a floor plate and a footpath of an escalator based on image semantic segmentation comprises the following steps:
step 1: acquiring images of a floor plate and a footpath comprising the escalator;
step 2: drawing a boundary between a floor plate and a footpath of the escalator in the image by adopting an image semantic segmentation algorithm; the boundary is zigzag;
and 3, step 3: placing the boundary in an x-y coordinate system, enabling one side close to the floor plate to face an x coordinate axis, and drawing convex points facing the x coordinate axis on the boundary;
and 4, step 4: selecting two salient points closest to the x axis for connecting to obtain a first connecting line, and calculating the slope of the first connecting line;
and 5: connecting the two outermost protruding points on the boundary to obtain a second connecting line, and calculating the slope of the second connecting line;
step 6: judging whether the included angle between the slope of the first connecting line and the second connecting line is within a preset error range, if so, correcting the boundary line by using the slope of the second connecting line, and determining the slope of the boundary line as the slope of the second connecting line; if not, performing step 7;
and 7: the two outermost raised points on the dividing line are removed from the dividing line and step 5 is performed.
In the above method for detecting the boundary between the floor slab and the walkway of the escalator based on image semantic segmentation, the preset error range is 1-10 °.
Meanwhile, the invention also discloses an escalator three-dimensional area intrusion detection method based on human skeleton and image semantic segmentation, which comprises the following steps of:
step 1: acquiring an image of the escalator through a camera;
step 2: drawing boundaries of the footpath and the baffles on two sides of the footpath in the image through an image semantic segmentation algorithm;
drawing a boundary between a floor plate and a footpath of the escalator by the detection method of the boundary between the floor plate and the footpath based on image semantic segmentation;
drawing human skeleton of the human body on the footpath in the image through a human skeleton algorithm, calculating the slope of the spine of the human body, and drawing a first connecting line at the lower parts or the lower ends of two shinbones of the human body;
and step 3: extending the first connecting line to the boundary line of the footpath and the baffles at the two sides of the footpath to obtain an intersection A and an intersection B;
and 4, step 4: making a vertical line along a boundary between a floor plate and a footpath, and translating the vertical line to an intersection point A and an intersection point B to obtain a first tripwire; drawing a second tripwire from an intersection point A and an intersection point B according to the slope of the spine of the human body;
and 5: calculating the included angle of the first tripwire and the second tripwire, and if the included angle is smaller than or equal to a first preset angle, using the second tripwire as a standard tripwire; if the included angle is larger than a first preset angle, the first tripwire is used as a standard tripwire;
step 6: and judging whether the specific part of the human skeleton exceeds a standard trip line, and if so, giving an alarm.
In the escalator three-dimensional area intrusion detection method based on human skeleton and image semantic segmentation, the first preset angle is 20-40 degrees.
In the escalator three-dimensional area intrusion detection method based on human skeleton and image semantic segmentation, the specific part is one or more of a wrist, an elbow, a shoulder joint, a cervical vertebra, a spine, a hip joint, a knee joint and a lumbar vertebra.
Finally, the invention also discloses an escalator three-dimensional area intrusion detection system based on human skeleton and image semantic segmentation, which comprises a camera and a server, wherein the image range covers the escalator;
the server comprises the following modules:
a communication module: the system is used for acquiring a head portrait from a camera;
the first image semantic segmentation calculating module: the system is used for drawing boundaries of the footpath and the baffles on two sides of the footpath in the image through an image semantic segmentation algorithm;
the second image semantic segmentation calculating module: the method is used for drawing a boundary between a floor plate and a footpath of the escalator in an image;
human skeleton calculation module: the human skeleton drawing device is used for drawing human skeletons of a human body on a footpath in an image through a human skeleton algorithm, calculating the slope of a human spine and drawing a first connecting line at the lower parts or the lower ends of two shinbones of the human body;
an intersection point drawing module: the system comprises a first connecting line, a second connecting line, a third connecting line, a fourth connecting line, a fifth connecting line and a sixth connecting line, wherein the first connecting line is used for extending the first connecting line on the image to a boundary line of a footpath and baffles on two sides of the footpath to obtain an intersection A and an intersection B;
a tripwire drawing module: the system comprises a first tripwire, a second tripwire and a third tripwire, wherein the first tripwire is used for drawing a vertical line on an image along a boundary between a floor board and a footpath and translating the vertical line to an intersection point A and an intersection point B to obtain a first tripwire; the tripwire system is used for drawing a second tripwire from an intersection point A and an intersection point B on the image according to the slope of the spine of the human body;
a standard tripwire selection module: the device is used for calculating the included angles of the first tripwire and the second tripwire, and if the included angle is smaller than or equal to a first preset angle, the second tripwire is used as a standard tripwire; if the included angle is larger than a first preset angle, the first tripwire is used as a standard tripwire;
a judging module: the tripwire is used for judging whether the specific part of the human skeleton exceeds a standard tripwire or not, and if so, alarming;
the second image semantic segmentation calculation module comprises the following sub-modules:
a boundary acquisition submodule: the image semantic segmentation algorithm is used for drawing a boundary between a floor plate and a footpath of the escalator in the image; the boundary is zigzag;
a salient point obtaining submodule: placing the boundary line obtained by the boundary line obtaining submodule calculation into an x-y coordinate system, enabling one side close to the floor plate to face an x coordinate axis, and drawing a convex point facing the x coordinate axis on the boundary line obtained by the boundary line obtaining submodule calculation;
a first connecting line drawing submodule: the method comprises the steps of selecting two salient points closest to an x axis for connecting to obtain a first connecting line, and calculating the slope of the first connecting line;
a second connecting line drawing submodule: the boundary acquisition submodule is used for calculating the boundary of the two convex points on the outermost side on the boundary of the two convex points on the outermost side;
a judgment submodule: the slope correction device is used for judging whether the difference value between the slope of the first connecting line and the slope of the second connecting line is within a preset error range, if so, correcting the boundary line by using the slope of the second connecting line, and determining the slope of the boundary line as the slope of the second connecting line; if not, informing the result to the salient point removing submodule;
removing a convex point submodule: and the second connecting line drawing submodule is used for removing the two outermost protruding points on the boundary line calculated by the boundary line obtaining submodule from the boundary line and sending the boundary line with the corresponding protruding points removed to the second connecting line drawing submodule, and the second connecting line drawing submodule adopts the boundary line to draw a second connecting line.
In the escalator three-dimensional area intrusion detection system based on human skeleton and image semantic segmentation, the first preset angle is 20-40 degrees.
In the escalator three-dimensional region intrusion detection system based on human skeleton and image semantic segmentation, the specific part is one or more of a wrist, an elbow, a shoulder joint, a cervical vertebra, a spine, a hip joint, a knee joint and a lumbar vertebra.
Compared with the prior art, the invention has the beneficial effects that:
the detection method of the boundary of the invention realizes the improvement of the precision of the first tripwire by carrying out slope calibration on the boundary between the floor plate and the footpath in the image after the semantic segmentation processing of the image; the slope calibration method comprises the steps of finding out a protruded point on a zigzag boundary, drawing a first connecting line and a second connecting line, and determining the slope of the boundary by finding out a proper second connecting line; by the method, the real slope of the boundary between the floor plate and the footpath can be calculated, and a more accurate data basis is provided for the subsequent calculation of the tripwire.
According to the three-dimensional area intrusion detection method, the drawing precision of the standard tripwire can be improved through the first tripwire and the second tripwire, the problem that the first tripwire fails due to distortion of a wide-angle lens long-range image is avoided, the alarm precision is improved, and the false alarm probability is reduced;
through the design, false alarms generated by calculation of the first tripwire can be reduced (not completely eradicated), the alarm precision can be improved, the calculation amount is reduced, and human body tracking is not needed.
Drawings
FIG. 1 is a schematic representation of semantically segmented images of examples 1 and 2;
FIG. 2 is a schematic view of the boundary line of example 1 before the outermost two protruded points on the boundary line are removed;
FIG. 3 is a schematic view of the boundary line of example 1 after removing the outermost two protruded points on the boundary line;
FIG. 4 is a flowchart of example 1;
FIG. 5 is a schematic illustration of the strand of FIG. 6;
FIG. 6 is a schematic view of a passenger on a walkway;
FIG. 7 is a flowchart of example 2;
FIG. 8 is a block diagram showing the structure of embodiment 3;
FIG. 9 is a structural diagram of a second image semantic segmentation computation module of embodiment 3;
fig. 10 is an image display state at the time of warning in embodiment 3.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Example 1
Referring to fig. 4, a method for detecting a boundary between a floor panel and a walkway of an escalator based on image semantic segmentation includes the following steps:
step 1: acquiring images of a floor plate and a footpath comprising the escalator;
the image semantic Segmentation algorithm is to group (Grouping)/segment (Segmentation) pixels according to different semantic meanings expressed in an image; the difference between the algorithm and a general object recognition algorithm is that after a specific object is recognized, all pixel point positions of the object on a picture need to be recognized, and the algorithm is generally used in an automobile automatic driving function at present. The invention introduces this algorithm in order to automatically identify the position of various zones of the escalator, such as floor slab H (cyan zone), left and right stops C, footpath G (red zone), etc. FIG. 1 as follows;
and 2, step: drawing a boundary between a floor plate and a footpath of the escalator in the image by adopting an image semantic segmentation algorithm; the boundary is zigzag;
and step 3: arranging a boundary in an x-y coordinate system, enabling one side close to a floor plate to face an x coordinate axis, and drawing a convex point Z facing the x coordinate axis on the boundary;
in fig. 2, a jagged boundary is illustrated, and the boundary is placed in a coordinate system, so that the number of salient points in fig. 2 is small, and the number of salient points is very large in the actual calculation process, and fig. 2 is only illustrated;
in fig. 2, two points Z1 and Z2 are actually large in error, and one of the reasons for this is that the baffles on the floor plate, the walkway and the two sides of the walkway of the escalator meet together, the image semantic segmentation algorithm cannot accurately distinguish the three with high precision, and the accuracy of these positions is not high;
and the intersection positions have the further characteristic that the positions of the convex points are basically not close to the X axis, so that the drawing of the later first connecting line cannot be influenced, namely the collection points of the first connecting line cannot be collected to the intersection positions.
And 4, step 4: selecting two convex points closest to the x axis for connecting to obtain a first connecting line, and calculating the slope of the first connecting line;
as can be seen by fig. 2, the first line is X; the two protruding points where the first connecting line is located are very close to real points, but because the distance between the two points cannot be estimated, in some cases, the two points are very close to each other, and in some cases, the two points are far away from each other; if the interval is far away, the difference between the slope of the first connecting line and the real slope of the dividing line is not large; if the distance is relatively close, the slope deviation of the first connecting line is large; but the computer program cannot distinguish how well the two points of the first link are spaced;
therefore, in the present embodiment, the first connection is taken as a reference connection only;
and 5: connecting two outermost protruding points on the boundary to obtain a second connecting line, and calculating the slope of the second connecting line;
as can be seen from fig. 2, Y1 is the second connection line in this step, and the slope deviation between Y1 and the first connection line is relatively large, for which reference may be made to step 3;
step 6: judging whether the included angle between the slope of the first connecting line and the second connecting line is within a preset error range, if so, correcting the boundary line by using the slope of the second connecting line, and determining the slope of the boundary line as the slope of the second connecting line; if not, performing step 7; in this embodiment, the preset error range is generally selected to be 10 °; of course, an angle of 5 °, 8 °, or the like may be selected.
And 7: the two outermost raised points on the dividing line are removed from the dividing line and step 5 is performed.
Referring to fig. 3, the erased image has Z1 and Z2 removed, and the two outermost protruded points on the boundary lines become Z3 and Z4; obtaining a second connecting line Y2 through connecting;
by the above method, the length of the second connecting line is maximized, and the included angle between the second connecting line and the first connecting line is minimized, so that the finally obtained slope of the second connecting line can represent the real slope of the boundary between the floor slab and the footpath.
Example 2
Referring to fig. 5, 6 and 7, a method for detecting intrusion of a three-dimensional region of an escalator based on human skeleton and image semantic segmentation comprises the following steps:
step 11: acquiring an image of the escalator through the camera 1;
the camera 1 can be most of commercially available cameras 1, and in the embodiment, a wide-angle camera 1 is adopted, and the wide-angle camera 1 has the advantage of large shooting range, but the image in the deep part of the lens is easy to distort;
step 12: drawing the boundary of a footpath G in the image and baffles C at two sides of the footpath G by an image semantic segmentation algorithm;
the boundary between the floor plate H and the footpath G of the escalator is drawn by the method as described in example 1;
the image semantic Segmentation algorithm is to group/segment (Segmentation) pixels according to different semantic meanings expressed in an image; the difference between the algorithm and a general object recognition algorithm is that after a specific object is recognized, all pixel point positions of the object on a picture need to be recognized, and the algorithm is generally used for an automatic driving function of an automobile at present. The invention introduces this algorithm in order to automatically identify the position of various zones of the escalator, such as floor slab H (cyan zone), left and right stops C, footpath G (red zone), etc. FIG. 1 as follows;
it should be noted that the image semantic segmentation algorithm is an algorithm mature in the field, is not an innovation of the present invention, and is not limited or emphasized too much.
Drawing human skeleton of a human body on a footpath G in the image through a human skeleton algorithm, calculating the slope of a human spine E, and drawing a first connecting line D of the lower ends (namely ankle positions) of two shinbones F of the human body;
the classic algorithm of the human Skeleton algorithm can use a human Skeleton key point detection model Skeleton or AlphaPose algorithm, a pyrrch-openpose algorithm and the like;
human posture estimation algorithms include alphase, opennase, mmnase, blazepose, openpifaf, and the like. There are 2 major classes in principle, top-down and bottom-up. Regardless of the algorithm, the final output is the coordinates and confidence of the human bone points.
The image semantic segmentation algorithm is commonly FCN-HRNetW, PP-Lite-Seg, segformer, deeplavv 3+ and the like. At present, the algorithm adopted by the region segmentation is FCN-HRNetW18, the backhaul thereof is HRNet (High Resolution Networks), and the head thereof is FCN (full connectivity Networks). HRNet is a multi-branch high-resolution network originally designed for the pose estimation task, but since the network maintains high resolution throughout, spatial information of the input picture can be maintained, spatial information loss due to downsampling is reduced, and the HRNet is naturally suitable for the pixel-level prediction task such as semantic segmentation. Besides maintaining high-resolution spatial information, the network also performs multi-scale information fusion, so that the network can extract stronger semantic information, and the expression capability of the network is further enhanced.
Step 13: extending the first connecting line D to the boundary line of the footpath G and the baffles C at two sides of the footpath G to obtain an intersection A and an intersection B;
it should be noted that the implementation of this step is premised on that the double tibiae F are side by side when a person stands, and if the double tibiae F are not side by side, the method of the present invention is not used; judging whether the shin bones F are parallel, judging whether the shin bones F are approximately equal in length by using a bone picture drawn by the human body bone algorithm in the step 2, and if the shin bones F are located at the front and back positions, judging that the shin bones F are not equal in length; at this time, other methods are needed for judgment and are not within the protection scope of the invention;
step 14: drawing a vertical line I along a boundary between the floor plate H and the footpath G, and translating the vertical line I to an intersection A and an intersection B to obtain a first tripwire J; drawing a second tripwire K from the intersection point A and the intersection point B according to the slope of a human spine E;
the first trip wire J and the second trip wire K can refer to fig. 2;
step 15: calculating included angles of the first tripwire J and the second tripwire K, and if the included angle is smaller than or equal to a first preset angle such as 30 degrees, using the second tripwire K as a standard tripwire; if the included angle is larger than a first preset angle, using a first tripwire J as a standard tripwire;
in practical application, the allowable included angle of the first trip wire J and the second trip wire K can be 20 degrees, 25 degrees, 30 degrees, 35 degrees or 40 degrees;
if the included angle between the first tripwire J and the second tripwire K is less than or equal to 30 degrees, the probability that a person is not askew or stretches out is high, and the spine of the person is taken as a vertical standard; if the included angle of the first tripwire J and the second tripwire K is more than 30 degrees, the spine of a person is seriously inclined, and the first tripwire J is taken as a standard tripwire at the moment;
step 16: judging whether the specific part of the human skeleton exceeds a standard trip line or not, and if so, giving an alarm;
the specific part is one or more of wrist, elbow, shoulder joint, cervical vertebra, spine E, hip joint, knee joint and lumbar vertebra, and in this embodiment, the specific part is the elbow.
Two standard trip lines form an area that is used only to determine if the passenger is beyond the handrail area, risking a fall; if the risks of other passengers are judged, the steps 1-6 are implemented again for judgment;
by introducing the semantic segmentation algorithm, no matter how many cameras are installed (horizontal or vertical), the boundary line between the floor plate and the footpath can be obtained by a semantic segmentation mode and used as a horizontal line. A true vertical line is taken from the horizontal line.
The method can be used for installing the field camera and is greatly helpful. Because of the actual installation angle, it is not possible to be all parallel to the actual escalator. Or the installation position of the camera is not in the center of the escalator, so that the horizontal line is imaged to be deviated. Plus distortion of the camera lens, can also be corrected by the vertical line.
The blending line is drawn through the intersection point of the human shin bone and the left baffle and the right baffle. The height of each person does not need to be concerned, and the introduction of a tracking algorithm does not exist naturally, so that the height information is lost because a person cannot be tracked.
Theoretically, intrusion detection can be realized only by adopting the first trip wire, but with the wide use of wide-angle lenses, obvious errors are brought if detection is carried out by only adopting the first trip wire at a far position in the lens, for example, most wide-angle lenses sold in the market have the condition that 50 pixels are 1 meter at the far position in an image, and the image distortion is easily generated at the far position, and at the moment, false alarm is easily generated by adopting the first trip wire;
in order to solve the problem, a second tripwire is drawn through the tibia of the human leg, and the tibia is close to the ground, so that the variable of height does not need to be introduced;
of course, there are several unexpected situations that need to be ignored or considered when using a second tripwire:
1. the person stands front and back, and the first connection is inaccurate when the person stands front and back, which is not a problem considered or solved by the invention, and the invention only solves the condition that the shines are side by side; note: if the condition is considered and still based on a bone judgment algorithm, whether the two tibias are arranged side by side or not is calculated, and if the two tibias are not arranged side by side, the method is not used;
2. in the present invention, since the second tripwire of the spine is inaccurate when the human body is inclined, it is necessary to determine whether to select the first tripwire or the second tripwire by determining an angle between the first tripwire and the second tripwire, and one of the gist of the present invention is: the second trip wire can be employed, with the first being minimized, and if the second trip wire is severely misaligned, the first trip wire is employed;
through the design, false alarms generated by calculation of the first tripwire can be reduced (not completely eradicated), the alarm precision can be improved, the calculation amount is reduced, and human body tracking is not needed.
Through the above method, compared with the prior application, the following advantages are provided:
1. human body tracking is not needed;
2. detecting a human body without the need of comparing the human body with a calculation result at the entering moment in real time when a passenger enters the footpath G;
3. the installation requirement of the camera 1 of an installer is reduced;
4. the lens is applicable to most wide-angle lenses;
5. and the alarm error is small.
Example 3
Referring to fig. 8, an escalator three-dimensional area intrusion detection system based on human skeleton and image semantic segmentation comprises a camera 1 and a server 2, wherein the image range covers the escalator;
the server 2 comprises the following modules:
the communication module 3: used for obtaining the head portrait from the camera 1;
the first image semantic segmentation computation module 40: the system is used for drawing boundaries of the footpath and the baffles on two sides of the footpath in the image through an image semantic segmentation algorithm;
the second image semantic segmentation calculating module 4: the method is used for drawing a boundary between a floor plate and a footpath of the escalator in an image;
human skeleton calculation module 5: the human skeleton drawing device is used for drawing human skeletons of a human body on a footpath G in an image through a human skeleton algorithm, calculating the slope of a human spine E and drawing a first connecting line D of the lower parts or the lower ends of two shinbones F of the human body;
the junction drawing module 6: the image acquisition device is used for extending a first connecting line D on the image to a boundary line of a footpath G and baffles C at two sides of the footpath G to obtain an intersection point A and an intersection point B;
tripwire drawing module 7: the method comprises the following steps of drawing a vertical line I along a boundary between a floor board H and a footpath G on an image, and translating the vertical line I to an intersection point A and an intersection point B to obtain a first tripwire J; the tripwire system is used for drawing a second tripwire K from an intersection point A and an intersection point B on the image according to the slope of the human spine E;
standard tripwire selection module 8: the included angle of the first tripwire J and the second tripwire K is calculated, and if the included angle is smaller than or equal to a first preset angle, the second tripwire K is used as a standard tripwire; if the included angle is larger than a first preset angle, using a first tripwire J as a standard tripwire;
a judging module 9: the method is used for judging whether the specific part of the human skeleton exceeds a standard trip line or not, and if so, giving an alarm.
The second image semantic segmentation calculating module 4 comprises the following sub-modules:
boundary acquisition submodule 41: the image semantic segmentation algorithm is used for drawing a boundary between a floor plate and a footpath of the escalator in the image; the boundary is zigzag;
salient point acquisition submodule 42: placing the boundary line obtained by the boundary line obtaining submodule calculation into an x-y coordinate system, enabling one side close to the floor plate to face an x coordinate axis, and drawing a convex point facing the x coordinate axis on the boundary line obtained by the boundary line obtaining submodule calculation;
first connecting line drawing submodule 43: the method comprises the steps of selecting two salient points closest to an x axis for connecting to obtain a first connecting line, and calculating the slope of the first connecting line;
second link drawing submodule 44: the boundary acquisition submodule is used for connecting two outermost protruding points on the boundary calculated by the boundary acquisition submodule to obtain a second connecting line and calculating the slope of the second connecting line;
the judgment sub-module 45: the device is used for judging whether the included angle between the slope of the first connecting line and the second connecting line is within a preset error range, if so, correcting the boundary line by using the slope of the second connecting line, and determining the slope of the boundary line as the slope of the second connecting line; if not, the result is notified to the salient point removing submodule;
salient point removal submodule 46: and the second connecting line drawing submodule is used for drawing a second connecting line by adopting the boundary line, and judging whether a new second connecting line meets the error requirement through the judgment of the judging submodule.
Referring to fig. 9, the second image semantic segmentation calculating module 4 operates as follows: the boundary acquisition submodule 41 draws a boundary between a floor plate and a footpath of the escalator in the image, and the salient point acquisition submodule 42 draws salient points; the first connecting line drawing submodule 43 draws a first connecting line; the second connection line drawing submodule 44 draws a second connection line; the judgment submodule 45 judges whether the second connecting line is qualified, if so, the slope of the second connecting line is determined as the slope of the boundary, if not, the protruding point removing submodule 46 is instructed to remove two protruding points where the second connecting line is located, then the second connecting line drawing submodule 44 redraws the second connecting line, the judgment submodule 45 judges again until a second connecting line meeting the requirement is obtained, and the slope of the second connecting line meeting the requirement is used as the slope of the boundary between the floor plate and the footpath.
In the operation process, the camera 1 continuously acquires images; and transmits the image to the communication module 3; the communication module 3 sends the image to a first image semantic segmentation calculation module 40, a second image semantic segmentation calculation module 4 and a human skeleton calculation module 5; the first image semantic segmentation calculation module 40 and the second image semantic segmentation calculation module 4 output semantic segmentation results, namely, boundaries between a floor plate H and a footpath G, boundaries between the footpath G and baffles C on two sides of the footpath G are marked on an image; the human body skeleton calculation module 5 marks a skeleton node diagram of passengers on the elevator, calculates the slope of a human body spine E and draws a first connecting line D at the lower parts or the lower ends of two shinbones F of a human body according to the skeleton node diagram; the junction drawing module 6 extends the first connecting line D on the image to reach the boundary line of the footpath G and the baffles C at the two sides of the footpath G to obtain a junction A and a junction B; the tripwire drawing module 7 draws a first tripwire J and a second tripwire K on the image; the standard tripwire selection module 8 selects which tripwire is selected as a standard tripwire according to a set standard; the judging module 9 judges whether the specific part of the human skeleton exceeds a standard trip line, if so, an alarm is given; the warning mode can output text or image warning on a display screen connected with the server 2, referring to fig. 10, when the hands of passengers on the walkway extend out of the guardrail in fig. 10, a red circle is formed near the elbows of the passengers to show that the image warning is performed, or a buzzer connected with the server 2 is controlled to generate the warning; the buzzer may be located adjacent the escalator.
Claims (4)
1. A detection method for a boundary between a floor plate and a footpath of an escalator based on image semantic segmentation is characterized by comprising the following steps:
step 1: acquiring images of a floor plate and a footpath comprising the escalator;
and 2, step: drawing a boundary between a floor plate and a footpath of the escalator in the image by adopting an image semantic segmentation algorithm; the boundary is zigzag;
and 3, step 3: arranging a boundary in an x-y coordinate system, enabling one side close to a floor plate to face an x coordinate axis, and drawing convex points facing the x coordinate axis on the boundary;
and 4, step 4: selecting two convex points closest to the x axis for connecting to obtain a first connecting line, and calculating the slope of the first connecting line;
and 5: connecting two outermost protruding points on the boundary to obtain a second connecting line, and calculating the slope of the second connecting line;
step 6: judging whether the included angle between the slope of the first connecting line and the second connecting line is within a preset error range, if so, correcting the boundary line by using the slope of the second connecting line, and determining the slope of the boundary line as the slope of the second connecting line; if not, performing step 7;
and 7: the two outermost raised points on the dividing line are removed from the dividing line and step 5 is performed.
2. The method for detecting the boundary between the floor slab and the walkway of the escalator based on image semantic segmentation as claimed in claim 1, wherein the preset error range is 1-10 °.
3. An escalator three-dimensional area intrusion detection method based on human skeleton and image semantic segmentation is characterized by comprising the following steps:
step 1: acquiring an image of the escalator through a camera;
and 2, step: drawing boundaries of the footpath and the baffles on two sides of the footpath in the image through an image semantic segmentation algorithm;
drawing a boundary between a floor panel and a walkway of an escalator by a method according to claim 1 or 2;
drawing human skeleton of the human body on the footpath in the image through a human skeleton algorithm, calculating the slope of the spine of the human body, and drawing a first connecting line at the lower parts or the lower ends of two shinbones of the human body;
and 3, step 3: extending the first connecting line to the boundary line of the footpath and the baffles at the two sides of the footpath to obtain an intersection A and an intersection B;
and 4, step 4: making a vertical line along a boundary between a floor plate and a footpath, and translating the vertical line to an intersection point A and an intersection point B to obtain a first tripwire; drawing a second tripwire from an intersection point A and an intersection point B according to the slope of the spine of the human body;
and 5: calculating the included angle of the first tripwire and the second tripwire, and if the included angle is smaller than or equal to a first preset angle, using the second tripwire as a standard tripwire; if the included angle is larger than a first preset angle, the first tripwire is used as a standard tripwire;
step 6: and judging whether the specific part of the human skeleton exceeds a standard trip line, and if so, giving an alarm.
4. A three-dimensional region intrusion detection system of an escalator based on human skeleton and image semantic segmentation is characterized by comprising a camera and a server, wherein the camera and the server cover the escalator in an image range;
the server comprises the following modules:
a communication module: the system is used for acquiring a head portrait from a camera;
the first image semantic segmentation calculating module: the system is used for drawing boundaries of the footpath in the image and the baffle plates on two sides of the footpath through an image semantic segmentation algorithm;
the second image semantic segmentation calculating module: the system is used for drawing a boundary between a floor plate and a footpath of the escalator in an image;
human skeleton calculation module: the human skeleton drawing device is used for drawing human skeletons of a human body on a footpath in an image through a human skeleton algorithm, calculating the slope of a human spine and drawing a first connecting line at the lower parts or the lower ends of two shinbones of the human body;
an intersection point drawing module: the image acquisition device is used for extending a first connecting line on an image to a boundary line of barriers at two sides of a footpath and the footpath to obtain an intersection point A and an intersection point B;
a tripwire drawing module: the system comprises a first tripwire, a second tripwire and a third tripwire, wherein the first tripwire is used for drawing a vertical line on an image along a boundary between a floor board and a footpath and translating the vertical line to an intersection point A and an intersection point B to obtain a first tripwire; the tripwire system is used for drawing a second tripwire from an intersection point A and an intersection point B on the image according to the slope of the spine of the human body;
a standard tripwire selection module: the device is used for calculating the included angles of the first tripwire and the second tripwire, and if the included angle is smaller than or equal to a first preset angle, the second tripwire is used as a standard tripwire; if the included angle is larger than a first preset angle, the first tripwire is used as a standard tripwire;
a judging module: the tripwire is used for judging whether the specific part of the human skeleton exceeds a standard tripwire or not, and if so, alarming;
the second image semantic segmentation calculation module comprises the following sub-modules:
a boundary acquisition submodule: the image semantic segmentation algorithm is used for drawing a boundary between a floor plate and a footpath of the escalator in the image; the boundary is zigzag;
a salient point obtaining submodule: placing the boundary line obtained by the boundary line obtaining submodule calculation into an x-y coordinate system, enabling one side close to the floor plate to face an x coordinate axis, and drawing a convex point facing the x coordinate axis on the boundary line obtained by the boundary line obtaining submodule calculation;
a first connecting line drawing submodule: the method comprises the steps of selecting two salient points closest to an x axis for connecting to obtain a first connecting line, and calculating the slope of the first connecting line;
a second connection line drawing submodule: the boundary acquisition submodule is used for calculating the boundary of the two convex points on the outermost side on the boundary of the two convex points on the outermost side;
a judgment submodule: the device is used for judging whether the difference value of the slope of the first connecting line and the slope of the second connecting line is within a preset error range, if so, correcting the boundary line by the slope of the second connecting line, and determining the slope of the boundary line as the slope of the second connecting line; if not, the result is notified to the salient point removing submodule;
removing a convex point submodule: and the second connecting line drawing submodule is used for removing the two outermost protruding points on the boundary line calculated by the boundary line obtaining submodule from the boundary line and sending the boundary line with the corresponding protruding points removed to the second connecting line drawing submodule, and the second connecting line drawing submodule adopts the boundary line to draw a second connecting line.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310143406.4A CN115841497B (en) | 2023-02-21 | 2023-02-21 | Boundary detection method and escalator area intrusion detection method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310143406.4A CN115841497B (en) | 2023-02-21 | 2023-02-21 | Boundary detection method and escalator area intrusion detection method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115841497A CN115841497A (en) | 2023-03-24 |
| CN115841497B true CN115841497B (en) | 2023-04-18 |
Family
ID=85579968
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310143406.4A Active CN115841497B (en) | 2023-02-21 | 2023-02-21 | Boundary detection method and escalator area intrusion detection method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115841497B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116434145B (en) * | 2023-04-21 | 2024-04-16 | 北京日立电梯工程有限公司 | Escalator passenger dangerous behavior analysis and monitoring system based on image recognition |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107218894A (en) * | 2017-04-28 | 2017-09-29 | 合肥雅视智能科技有限公司 | A kind of subpixel accuracy thickness of detector detection method of fast and stable |
| CN109492548A (en) * | 2018-10-24 | 2019-03-19 | 广东佳鸿达科技股份有限公司 | The preparation method of region mask picture based on video analysis |
| CN113256550A (en) * | 2020-12-18 | 2021-08-13 | 深圳市怡化时代科技有限公司 | Image defect detection method, device, equipment and storage medium |
| CN114266770A (en) * | 2022-03-02 | 2022-04-01 | 中铁电气化局集团有限公司 | Method for detecting dropper defect of high-speed rail contact net through neural network learning method |
| CN114708552A (en) * | 2022-04-07 | 2022-07-05 | 广州鲁邦通物联网科技股份有限公司 | Three-dimensional area intrusion detection method and system based on human skeleton |
-
2023
- 2023-02-21 CN CN202310143406.4A patent/CN115841497B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107218894A (en) * | 2017-04-28 | 2017-09-29 | 合肥雅视智能科技有限公司 | A kind of subpixel accuracy thickness of detector detection method of fast and stable |
| CN109492548A (en) * | 2018-10-24 | 2019-03-19 | 广东佳鸿达科技股份有限公司 | The preparation method of region mask picture based on video analysis |
| CN113256550A (en) * | 2020-12-18 | 2021-08-13 | 深圳市怡化时代科技有限公司 | Image defect detection method, device, equipment and storage medium |
| CN114266770A (en) * | 2022-03-02 | 2022-04-01 | 中铁电气化局集团有限公司 | Method for detecting dropper defect of high-speed rail contact net through neural network learning method |
| CN114708552A (en) * | 2022-04-07 | 2022-07-05 | 广州鲁邦通物联网科技股份有限公司 | Three-dimensional area intrusion detection method and system based on human skeleton |
Non-Patent Citations (1)
| Title |
|---|
| 王国庆 等.基于增量张量子空间分析的电扶梯背景建模.《中国高新科技》.2022,(第4期),第14-15页. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115841497A (en) | 2023-03-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11232326B2 (en) | System and process for detecting, tracking and counting human objects of interest | |
| US8810653B2 (en) | Vehicle surroundings monitoring apparatus | |
| JP3279479B2 (en) | Video monitoring method and device | |
| EP2458553A1 (en) | Surveillance camera terminal | |
| CN101167086A (en) | Human detection and tracking for security applications | |
| CN111241913A (en) | Method, device and system for detecting falling of personnel | |
| JP2003216937A (en) | Night vision system | |
| CN115841497B (en) | Boundary detection method and escalator area intrusion detection method and system | |
| JPH1166319A (en) | Method and device for detecting traveling object, method and device for recognizing traveling object, and method and device for detecting person | |
| CN104966062A (en) | Video monitoring method and device | |
| CN114005167A (en) | Remote sight estimation method and device based on human skeleton key points | |
| CN110147748A (en) | A kind of mobile robot obstacle recognition method based on road-edge detection | |
| US20220366570A1 (en) | Object tracking device and object tracking method | |
| JP4067340B2 (en) | Object recognition device and object recognition method | |
| CN112347817B (en) | Video target detection and tracking method and device | |
| JP4150218B2 (en) | Terrain recognition device and terrain recognition method | |
| JP2002367077A (en) | Device and method for deciding traffic congestion | |
| CN115731563A (en) | Method for identifying falling of remote monitoring personnel | |
| CN112767452B (en) | Active sensing method and system for camera | |
| CN116129471A (en) | Escalator three-dimensional area intrusion detection method and system based on human skeleton and image semantic segmentation | |
| JP2004042737A (en) | Crossing monitoring device and crossing monitoring method | |
| CN113762164A (en) | Fire fighting access barrier identification method and system | |
| JPH0991432A (en) | Method for extracting doubtful person | |
| CN117058767B (en) | Training field monitoring method, training field monitoring equipment, storage medium and training field monitoring device | |
| CN114550074B (en) | Image recognition method and system based on computer vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |