CN115909135A - Escalator drive chain extension early warning method and system - Google Patents

Abstract

The invention discloses an escalator drive chain elongation early warning method and system, the method divides a first video stream into a plurality of first images; performing image processing on the plurality of first images to synthesize a first key frame image; extracting a first area feature and a first perimeter feature of a contour in a first key frame image, and constructing an abnormal jitter prediction model with the area and the perimeter as features; processing the plurality of second images to synthesize a second key frame image; extracting a second area characteristic and a second perimeter characteristic of the contour in the second key frame image, inputting the second area characteristic and the second perimeter characteristic into an abnormal shaking prediction model, and judging whether the escalator shakes abnormally or not; acquiring disparity map data of a driving chain image when the escalator stops running by adopting a stereo matching method; obtaining the length of a driving chain; and acquiring early warning information and sending the early warning information to the client. The invention can prevent the chain breakage of the driving chain in advance and improve the safety of the escalator.

Description

Escalator drive chain elongation early warning method and system

Technical Field

The invention relates to the technical field of escalator safety detection, in particular to an escalator drive chain elongation early warning method and system.

Background

Escalators are typical devices used in public places to transport passengers. The escalator drive chain is a connecting piece between an escalator main machine and an escalator main drive wheel. If the drive chain stretches or even breaks in the normal operation of the escalator, the escalator can be reversed or accelerated to descend, the safety of passengers is seriously affected, and the result that the passengers are injured probably occurs.

In the prior art, whether the driving chain is broken or not is monitored through a series of phenomena generated after the driving chain is broken, and only protective measures are taken after the driving chain is broken.

Disclosure of Invention

The present invention is directed to solving at least one of the problems of the prior art. Therefore, the invention provides an escalator drive chain extension early warning method and system, which can reduce the incidence of drive chain breakage, can predict and position an escalator with easy drive chain breakage risk in advance, and improve the safety of the escalator.

In a first aspect, an embodiment of the present invention provides an escalator drive chain elongation early-warning method, where the escalator drive chain elongation early-warning method includes:

the method comprises the steps of obtaining a first video stream when the escalator runs, and dividing the first video stream into a plurality of first images;

carrying out image processing on the plurality of first images to obtain first contour images of a plurality of driving chains, and synthesizing the first contour images of the plurality of driving chains into a first key frame image;

extracting a first area feature and a first perimeter feature of a contour in the first key frame image according to the first key frame image, inputting the first area feature and the first perimeter feature into a deep learning model for training, and constructing an abnormal jitter prediction model with the area and the perimeter as features;

acquiring a second video stream to be predicted, and dividing the second video stream into a plurality of second images;

carrying out image processing on the plurality of second images to obtain second contour images of the plurality of driving chains, and synthesizing the second contour images of the plurality of driving chains into a second key frame image;

extracting a second area characteristic and a second perimeter characteristic of the contour in the second key frame image according to the second key frame image, inputting the second area characteristic and the second perimeter characteristic into the abnormal shaking prediction model, and judging whether the escalator is abnormally shaken or not; if the abnormal jitter continuously occurs within the preset time, controlling the escalator to stop running;

acquiring a driving chain image when the escalator stops running, and acquiring disparity map data of the driving chain image when the escalator stops running by adopting a stereo matching method;

obtaining the length of the driving chain according to the disparity map data;

and comparing the length of the driving chain with the initial length of the driving chain, if the length of the driving chain is longer than the initial length, acquiring early warning information, and sending the early warning information to a client.

Compared with the prior art, the first aspect of the invention has the following beneficial effects:

the method comprises the steps of constructing an abnormal jitter prediction model with area and perimeter as characteristics; inputting the second area characteristic and the second perimeter characteristic into an abnormal shaking prediction model, and judging whether the escalator shakes abnormally or not; if the abnormal shaking continuously occurs within the preset time, the escalator is stopped, the occurrence rate of chain breakage of the driving chain can be reduced, the chain breakage of the driving chain is prevented in advance, and the safety of the escalator is improved; by comparing the length of the drive chain with the initial length of the drive chain, if the length of the drive chain is longer than the initial length, the early warning information is obtained and sent to the client, the escalator with the risk of chain breakage of the drive chain easily occurring can be predicted and positioned in advance, and escalator maintenance work is carried out in advance.

According to some embodiments of the invention, the image processing the plurality of first images to obtain a plurality of first contour images of the driving chains, and synthesizing the plurality of first contour images of the driving chains into a first key frame image includes:

carrying out graying processing on the first images to obtain first grayed images;

performing binarization processing on the first gray images by adopting a threshold segmentation method, segmenting the foreground and the background of the first gray images, and obtaining first contour images of the driving chains;

and performing multi-frame synthesis processing on the first contour images of the plurality of driving chains to obtain a first key frame image.

According to some embodiments of the invention, the extracting, from the first key frame image, a first area feature and a first perimeter feature of a contour in the first key frame image comprises:

extracting a first contour of the driving chain in motion from the first key frame image;

and performing contour drawing on the first contour, and calculating a first area characteristic and a first perimeter characteristic corresponding to a contour region after contour drawing by using a calculation obtaining function in the OPENCV.

According to some embodiments of the invention, the image processing the plurality of second images to obtain a plurality of second contour images of the driving chains, and synthesizing the plurality of second contour images of the driving chains into a second key frame image includes:

graying the plurality of second images to obtain a plurality of second grayed images;

performing binarization processing on the plurality of second grayscale images by adopting a threshold segmentation method, and segmenting the foreground and the background of the second grayscale images to obtain second contour images of the plurality of driving chains;

and performing multi-frame synthesis processing on the second contour images of the plurality of driving chains to obtain a second key frame image.

According to some embodiments of the invention, the extracting, from the second key frame image, a second area feature and a second perimeter feature of the outline in the second key frame image comprises:

extracting a second contour when the driving chain moves from the second key frame image;

and performing contour drawing on the second contour, and calculating by adopting a calculation function in the OPENCV to obtain a second area characteristic and a second perimeter characteristic corresponding to the contour region after contour drawing.

According to some embodiments of the invention, the second area characteristic and the second perimeter characteristic are input to the abnormal shaking prediction model to judge whether the escalator shakes abnormally; if the abnormal shaking continuously occurs within the preset time, controlling the escalator to stop running, comprising the following steps of:

inputting the second area feature and the second perimeter feature into the abnormal shake prediction model, judging, by the abnormal shake prediction model, whether the second area feature exceeds the range of the first area feature, and judging whether the second perimeter feature exceeds the range of the first perimeter feature;

if the second area characteristic does not exceed the range of the first area characteristic and the second perimeter characteristic does not exceed the range of the first perimeter characteristic, judging that the escalator does not have abnormal shaking; if the second area characteristic exceeds the range of the first area characteristic or the second perimeter characteristic exceeds the range of the first perimeter characteristic, judging that the escalator shakes abnormally;

continuously monitoring preset time, and controlling the escalator to stop running if the escalator continuously generates abnormal jitter when the preset time is over; and if the abnormal shaking of the escalator stops within the preset time, canceling the abnormal shaking signal of the escalator and controlling the escalator to normally run.

According to some embodiments of the present invention, the acquiring a driving chain image when the escalator stops operating and acquiring disparity map data of the driving chain image when the escalator stops operating by using a stereo matching method include:

shooting by using a binocular camera to obtain a driving chain image when the escalator stops running, wherein the driving chain image comprises a chain left view and a chain right view;

calibrating the binocular camera, obtaining internal parameters of each camera through calibration, and measuring the relative position between the two cameras of the binocular camera;

according to the internal parameters and the relative positions, respectively carrying out distortion elimination and line alignment on the chain left view and the chain right view so as to enable the imaging coordinates of the chain left view and the chain right view to be consistent, the visual axes of two cameras of the binocular camera to be parallel, and a left imaging plane and a right imaging plane to be coplanar;

and matching corresponding pixel points on the chain left view and the chain right view in the same scene by adopting the stereo matching method to obtain the parallax image data of the driving chain image when the escalator stops running.

According to some embodiments of the invention, the obtaining the length of the drive chain from the disparity map data comprises:

calculating the offset among the pixels according to the disparity map data by a triangulation principle to acquire three-dimensional information of the driving chain;

calculating to obtain the distance between adjacent chain links in the driving chain and the key length of a chain plate by adopting a calculation function in the OPENCV according to the three-dimensional information of the driving chain;

and calculating to obtain the length of the driving chain according to the distance between the adjacent chain links and the key length of the chain plates.

In a second aspect, an embodiment of the present invention further provides an escalator drive chain elongation early warning system, where the escalator drive chain elongation early warning system includes a vision acquisition device, a vision calculation device, an escalator main control subsystem, and an escalator early warning cloud platform:

the visual acquisition equipment is in communication connection with the visual computing equipment and is used for acquiring a first video stream when the escalator runs, a second video stream to be predicted and a driving chain image when the escalator stops running, and transmitting the first video stream when the escalator runs, the second video stream to be predicted and the driving chain image when the escalator stops running to the visual computing equipment;

the visual computing device is in communication connection with the visual acquisition device, and is used for receiving a first video stream when the escalator runs, a second video stream to be predicted and a driving chain image when the escalator stops running, and executing the escalator driving chain elongation early warning method to obtain early warning information and abnormal jitter information;

the escalator main control subsystem is in communication connection with the vision computing equipment and is used for receiving the abnormal jitter information and controlling the escalator to stop running;

the escalator early warning cloud platform is in communication connection with the visual computing equipment and is used for receiving the early warning information and sending the early warning information to a client.

Compared with the prior art, the second aspect of the invention has the following beneficial effects:

the system can obtain early warning information and abnormal jitter information through the visual computing equipment; receiving abnormal shaking information through the escalator main control subsystem to stop the escalator or normally operate the escalator; and receiving the early warning information through the escalator early warning cloud platform, and sending the early warning information to the client. Therefore, the system can reduce the incidence of chain breakage of the drive chain, prevent the chain breakage of the drive chain in advance, improve the safety of the escalator, predict and position the escalator with the risk of chain breakage of the drive chain in advance, and carry out escalator maintenance work in advance.

According to some embodiments of the invention, the vision acquisition equipment comprises one or more of a surveillance camera, a binocular camera, a depth camera, a structured light camera.

Drawings

The above and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:

fig. 1 is a flow chart of a method for warning elongation of an escalator drive chain according to an embodiment of the present invention;

FIG. 2 is a schematic view of the distance between adjacent links and the link plate key length according to an embodiment of the present invention;

fig. 3 is a diagram of a station-level driving chain early warning structure according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of a drive train state according to an embodiment of the present invention;

fig. 5 is a block diagram of an escalator drive chain elongation warning system in accordance with an embodiment of the present invention;

fig. 6 is a top view of an escalator drive chain elongation warning system in accordance with an embodiment of the present invention.

Detailed Description

Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention.

In the description of the present invention, if there are first, second, etc. described, it is only for the purpose of distinguishing technical features, and it is not understood that relative importance is indicated or implied or that the number of indicated technical features is implicitly indicated or that the precedence of the indicated technical features is implicitly indicated.

In the description of the present invention, it should be understood that the orientation or positional relationship referred to, for example, the upper, lower, etc., is indicated based on the orientation or positional relationship shown in the drawings, and is only for convenience of description and simplification of description, but does not indicate or imply that the device or element referred to must have a specific orientation, be constructed in a specific orientation, and be operated, and thus should not be construed as limiting the present invention.

In the description of the present invention, it should be noted that unless otherwise explicitly defined, terms such as arrangement, installation, connection and the like should be broadly understood, and those skilled in the art can reasonably determine the specific meanings of the above terms in the present invention in combination with the specific contents of the technical solutions.

Escalators are typical devices used in public places to transport passengers. The escalator drive chain is a connecting piece between an escalator main machine and an escalator main drive wheel. If the drive chain stretches or even breaks in the normal operation of the escalator, the escalator can be reversed or accelerated to descend, the safety of passengers is seriously affected, and the result that the passengers are injured probably occurs.

In the prior art, whether the driving chain is broken or not is monitored through a series of phenomena generated after the driving chain is broken, and only protective measures are taken after the driving chain is broken.

In order to solve the problems, the invention constructs an abnormal jitter prediction model which is characterized by area and perimeter; inputting the second area characteristic and the second perimeter characteristic into an abnormal shaking prediction model, and judging whether the escalator shakes abnormally; if the abnormal shaking continuously occurs within the preset time, the escalator is stopped, the occurrence rate of chain breakage of the driving chain can be reduced, the chain breakage of the driving chain is prevented in advance, and the safety of the escalator is improved; by comparing the length of the drive chain with the initial length of the drive chain, if the length of the drive chain is longer than the initial length, the early warning information is obtained and sent to the client, the escalator with the risk of chain breakage of the drive chain easily occurring can be predicted and positioned in advance, and escalator maintenance work is carried out in advance.

Referring to fig. 1, an embodiment of the present invention provides an escalator drive chain elongation early warning method, including:

and S100, acquiring a first video stream when the escalator runs, and dividing the first video stream into a plurality of first images.

Specifically, a first video stream containing the running state of the escalator drive chain is acquired by using a vision acquisition device and sent to a vision computing device, and the first video stream is divided into a plurality of single-frame pictures by using OpenCV frame advance and subtraction processing, that is, the first video stream is divided into a plurality of first images, for example, a 4-second video shot at 25 frames per second can be divided into 100 first images.

It should be noted that the visual acquisition device in this embodiment includes one or more of a monitoring camera, a binocular camera, a depth camera, and a structured light camera.

Step S200, image processing is carried out on the multiple first images to obtain first contour images of the multiple driving chains, and the first contour images of the multiple driving chains are combined into a first key frame image.

Specifically, the visual computing equipment acquires a plurality of first images, and grays the plurality of first images to acquire a plurality of first grayed images;

performing binarization processing on the first gray images by adopting a threshold segmentation method, segmenting the foreground and the background of the first gray images, and obtaining first contour images of the driving chains;

and carrying out multi-frame synthesis processing on the first contour images of the plurality of driving chains to obtain a first key frame image.

In the embodiment, images within a period of time t are synthesized to obtain all the superposed track images of the driving chain in the period of time t. The more pictures that are provided for synthesis, the higher the accuracy of the calculation, and the more accurate the synthesized superimposed trajectory image. If a divided picture of a certain frame is directly used, the jitter generated during the operation of the driving chain is only the track of the frame, and whether the driving chain continuously jitters or not cannot be judged, so that the accuracy of the chain breakage prediction of the driving chain is low.

Step S300, according to the first key frame image, extracting a first area feature and a first perimeter feature of the outline in the first key frame image, inputting the first area feature and the first perimeter feature into a deep learning model for training, and constructing an abnormal jitter prediction model with the area and the perimeter as features.

Specifically, a first contour when the driving chain moves is extracted from the first key frame image; and performing contour drawing on the first contour, and calculating a first area characteristic and a first perimeter characteristic corresponding to the contour region after contour drawing by using a calculation obtaining function in the OPENCV. The method specifically comprises the following steps:

extracting the driving chain contour from the first key frame image by using a contour detection function findContours (), wherein a picture can have a plurality of contours, such as the contour of a driving chain wheel, the contour of an escalator motor, the contour generated by the movement of a driving chain, and only the contour generated by the movement of the driving chain is detected in the embodiment;

drawing the outline of the driving chain by using an outline drawing function drawContours ();

calculating a first area characteristic of a contour region of a contour generated by driving chain motion by using a contour region area calculating function ContourAREA ();

the first perimeter feature of the contour resulting from driving the link motion is calculated using the calculate contour length function arcLength ().

Step S400, a second video stream to be predicted is obtained, and the second video stream is divided into a plurality of second images.

Specifically, a visual acquisition device is used for acquiring a second video stream to be predicted, which contains the running state of the escalator drive chain, the second video stream is sent to a visual computing device, and the second video stream is divided into a plurality of single-frame pictures by using OpenCV frame advance and retreat processing.

It should be noted that the visual acquisition equipment in this embodiment includes one or more of a monitoring camera, a binocular camera, a depth camera, and a structured light camera.

Step S500, image processing is carried out on the plurality of second images to obtain second contour images of the plurality of driving chains, and the second contour images of the plurality of driving chains are combined into a second key frame image.

Specifically, graying is carried out on the plurality of second images to obtain a plurality of second grayed images;

performing binarization processing on the plurality of second grayscale images by adopting a threshold segmentation method, segmenting the foreground and the background of the second grayscale images, and obtaining second contour images of the plurality of driving chains;

and performing multi-frame synthesis processing on the second contour images of the plurality of driving chains to obtain a second key frame image. The second keyframe image contains the superimposed outline shapes that the drive chain runs at different times.

S600, extracting a second area characteristic and a second perimeter characteristic of the contour in the second key frame image according to the second key frame image, inputting the second area characteristic and the second perimeter characteristic into an abnormal shaking prediction model, and judging whether the escalator is abnormally shaken or not; and if the abnormal shaking continuously occurs within the preset time, controlling the escalator to stop running.

Specifically, a second contour when the driving chain moves is extracted from the second key frame image; and performing contour drawing on the second contour, and calculating by using a calculation function in the OPENCV to obtain a second area characteristic and a second perimeter characteristic corresponding to the contour region after contour drawing. The method specifically comprises the following steps:

extracting the driving chain contour from the second key frame image by using a contour detection function findContours (), wherein a picture can have a plurality of contours, such as the contour of a driving chain wheel, the contour of an escalator motor, the contour generated by the movement of a driving chain, and only the contour generated by the movement of the driving chain is detected in the embodiment;

drawing the outline of the driving chain by using an outline drawing function drawContours ();

calculating a second area characteristic of the contour region of the contour generated by the drive chain motion by using a calculate contour region area function ContourAREA ();

a second perimeter characteristic of the contour resulting from the driving chain motion is calculated using the calculate contour length function arcLength ().

Inputting the second area characteristic and the second perimeter characteristic into an abnormal jitter prediction model, judging whether the second area characteristic exceeds the range of the first area characteristic or not through the abnormal jitter prediction model, and judging whether the second perimeter characteristic exceeds the range of the first perimeter characteristic or not;

if the second area characteristic does not exceed the range of the first area characteristic and the second perimeter characteristic does not exceed the range of the first perimeter characteristic, judging that the escalator does not shake abnormally; if the second area characteristic exceeds the range of the first area characteristic or the second perimeter characteristic exceeds the range of the first perimeter characteristic, judging that the escalator shakes abnormally, and sending abnormal shaking information to the escalator early warning cloud platform to remind personnel to check;

continuously monitoring the preset time, and controlling the escalator to stop running if the escalator continuously shakes abnormally after the preset time is over; and if the abnormal shaking of the escalator stops within the preset time, canceling the abnormal shaking signal of the escalator, and controlling the escalator to recover to a normal running state by the vision computing equipment.

It should be noted that the preset time in this embodiment may be changed according to actual needs, and this embodiment is not limited in particular.

And S700, acquiring a driving chain image when the escalator stops running, and acquiring disparity map data of the driving chain image when the escalator stops running by adopting a stereo matching method.

Specifically, a binocular camera is adopted to shoot and obtain a driving chain image when the escalator stops running, wherein the driving chain image comprises a chain left view and a chain right view; calibrating the binocular cameras, obtaining internal parameters of each camera through calibration, and measuring the relative position between the two cameras of the binocular cameras; according to the internal parameters and the relative positions, respectively carrying out distortion elimination and line alignment on the chain left view and the chain right view so as to enable the imaging coordinates of the chain left view and the chain right view to be consistent, and the visual axes of two cameras of a binocular camera to be parallel, wherein the left imaging plane and the right imaging plane are coplanar; and matching corresponding pixel points on the left view and the right view of the chain in the same scene by adopting a stereo matching method to obtain the parallax image data of the driving chain image when the escalator stops running. The method comprises the following specific steps:

in the present embodiment, the vision acquisition apparatus employs a binocular camera. Binocular vision is a method for passively sensing distance and object length by using a computer, which simulates the human visual principle. The binocular camera shoots the driving chain from different 2 observation points, and images of the driving chain under different visual angles are obtained, wherein the images comprise a chain left view and a chain right view. The camera has distortion of the image due to the characteristics of the optical lens, and due to errors in assembly, the driving chain and the binocular camera are not completely parallel, which also causes distortion of the picture. Therefore, the binocular camera calibration is performed first, and internal parameters of each camera, such as focal length, radial distortion, tangential distortion, imaging origin coordinates, and the like, are obtained through calibration. And the relative position between the two cameras, for example, the rotation matrix amount and the translation amount of the camera 1 relative to the camera 2, needs to be measured through calibration. And then carrying out binocular correction, respectively carrying out distortion elimination and line alignment on the chain left view and the chain right view according to the camera internal reference data obtained after the cameras are calibrated and the relative position between the two cameras, so that the imaging coordinates of the chain left view and the chain right view are consistent, the visual axes of the two cameras of the binocular camera are parallel, and the left imaging plane and the right imaging plane are coplanar. After the operation, any pixel point on the left view of the chain and the corresponding point on the right view of the chain have the same line number necessarily, and the pixel point of the corresponding chain can be matched and found only by searching the line. And then carrying out binocular matching work, matching corresponding pixel points on the left view and the right view of the chain in the same scene by adopting a stereo matching method, and obtaining disparity map data of the driving chain image when the escalator stops running.

And step S800, obtaining the length of the driving chain according to the disparity map data.

Specifically, according to the disparity map data, the offset between pixels is calculated through a triangulation principle to obtain three-dimensional information of a driving chain;

calculating to obtain the distance between adjacent chain links and the key length of a chain plate in the driving chain by adopting a calculation function in the OPENCV according to the three-dimensional information of the driving chain;

and calculating to obtain the length of the driving chain according to the distance between the adjacent chain links and the key length of the chain plates. The method specifically comprises the following steps:

referring to fig. 2, L1 in fig. 2 represents the distance between adjacent links, L2 represents the key length of the link plate, and the distance between each adjacent link and the key length of the link plate are added to obtain the length of the driving chain.

And S900, comparing the length of the drive chain with the initial length of the drive chain, if the length of the drive chain is longer than the initial length, acquiring early warning information, and sending the early warning information to the client.

Specifically, the length of the drive chain is compared with the initial length of the drive chain, and if the length of the drive chain is longer than the initial length, the early warning information is obtained and sent to the client. The method specifically comprises the following steps:

referring to fig. 3 to 4, for example, the length of a drive chain of an escalator at a station is obtained, a plurality of escalators are provided at a general station, and the length of the drive chain of the same escalator at different times is compared for multiple times, and the elongation of the drive chain is calculated according to the length of the drive chain at different times and the initial length of the drive chain (i.e., the length of the drive chain in fig. 4 when the drive chain is normal), for example, the length of the drive chain when the drive chain is elongated is subtracted from the length of the drive chain when the drive chain is normal, and if the length of the drive chain is longer than the initial length, the warning information is obtained; and the extension amount of the drive chain is provided for the escalator early warning cloud platform, the escalator early warning cloud platform can analyze a drive chain extension trend change curve report of each escalator, and the early warning information and the drive chain extension trend change curve report are sent to a client side to provide accurate chain early warning information for maintenance personnel.

In the embodiment, an abnormal jitter prediction model which is characterized by the area and the perimeter is constructed; inputting the second area characteristic and the second perimeter characteristic into an abnormal shaking prediction model, and judging whether the escalator shakes abnormally or not; if the abnormal shaking continuously occurs within the preset time, the escalator is stopped, the occurrence rate of chain breakage of the driving chain can be reduced, the chain breakage of the driving chain is prevented in advance, and the safety of the escalator is improved; by comparing the length of the drive chain with the initial length of the drive chain, if the length of the drive chain is longer than the initial length, the early warning information is obtained and sent to the client, the escalator with the risk of chain breakage of the drive chain easily occurring can be predicted and positioned in advance, and escalator maintenance work is carried out in advance. Therefore, the embodiment can remind maintenance personnel to intervene in advance before the broken chain of the drive chain occurs, so that the prevention effect is achieved, the remedial measure after the fault occurs is not taken, the broken chain occurrence rate of the drive chain can be reduced to the maximum extent, and the busy accident caused by the reversion of the escalator due to the broken chain is avoided. The embodiment can calculate the elongation of the escalator drive chain and upload the elongation to the escalator early warning cloud platform. The length and the extension amount of the driving chain when all the escalators stop last time can be checked at the rear end of the escalator early warning cloud platform. A change curve can be drawn through elongation change trend analysis, so that the escalator with the risk of driving chain breakage easily occurring is predicted and positioned in advance, and escalator maintenance work is carried out in advance.

Referring to fig. 5, an embodiment of the present invention further provides an escalator drive chain elongation early warning system, where the escalator drive chain elongation early warning system includes a vision acquisition device, a vision calculation device, an escalator main control subsystem, and an escalator early warning cloud platform:

the visual acquisition equipment is in communication connection with the visual computing equipment and is used for acquiring a first video stream when the escalator operates, a second video stream to be predicted and a driving chain image when the escalator stops operating, and transmitting the first video stream when the escalator operates, the second video stream to be predicted and the driving chain image when the escalator stops operating to the visual computing equipment;

the visual computing device is in communication connection with the visual acquisition device, is used for receiving a first video stream when the escalator runs, a second video stream to be predicted and a driving chain image when the escalator stops running, and is also used for executing the escalator driving chain elongation early warning method in the embodiment so as to obtain early warning information and abnormal jitter information;

the escalator main control subsystem is in communication connection with the vision computing equipment and is used for receiving abnormal jitter information and controlling the escalator to stop running;

staircase early warning cloud platform communication connection vision computing equipment, staircase early warning cloud platform are used for receiving early warning information to send early warning information to the customer end.

Referring to fig. 6, the escalator control system comprises a vision acquisition device, a vision computing device, an escalator main control subsystem and an escalator early warning cloud platform in fig. 6, wherein the vision acquisition device is fixed in a truss of an escalator machine room through a support, and a vision acquisition lens looks at a drive chain of the escalator.

The embodiment can obtain the early warning information and the abnormal jitter information through the visual computing equipment; receiving abnormal shaking information through the escalator main control subsystem to stop the escalator or normally operate the escalator; and receiving the early warning information through the escalator early warning cloud platform, and sending the early warning information to the client. Therefore, the system in the embodiment can reduce the incidence of the broken chain of the driving chain, prevent the broken chain of the driving chain in advance, improve the safety of the escalator, predict and position the escalator with the risk of the broken chain of the driving chain in advance, and carry out the maintenance work of the escalator in advance.

In some embodiments, the vision acquisition device comprises one or more of a surveillance camera, a binocular camera, a depth camera, a structured light camera.

The embodiments of the present invention have been described in detail with reference to the drawings, but the present invention is not limited to the embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the gist of the present invention.

Claims (10)

1. The escalator drive chain elongation early warning method is characterized by comprising the following steps of:

the method comprises the steps of obtaining a first video stream when the escalator runs, and dividing the first video stream into a plurality of first images;

performing image processing on the plurality of first images to obtain first contour images of a plurality of driving chains, and synthesizing the first contour images of the plurality of driving chains into a first key frame image;

extracting a first area feature and a first perimeter feature of a contour in the first key frame image according to the first key frame image, inputting the first area feature and the first perimeter feature into a deep learning model for training, and constructing an abnormal jitter prediction model with the area and the perimeter as features;

acquiring a second video stream to be predicted, and dividing the second video stream into a plurality of second images;

performing image processing on the plurality of second images to obtain a plurality of second contour images of the driving chains, and synthesizing the plurality of second contour images of the driving chains into a second key frame image;

extracting a second area characteristic and a second perimeter characteristic of the contour in the second key frame image according to the second key frame image, inputting the second area characteristic and the second perimeter characteristic into the abnormal shaking prediction model, and judging whether the escalator is abnormally shaken or not; if the abnormal jitter continuously occurs within the preset time, controlling the escalator to stop running;

acquiring a driving chain image when the escalator stops running, and acquiring disparity map data of the driving chain image when the escalator stops running by adopting a stereo matching method;

obtaining the length of the driving chain according to the disparity map data;

and comparing the length of the driving chain with the initial length of the driving chain, if the length of the driving chain is longer than the initial length, acquiring early warning information, and sending the early warning information to a client.

2. The escalator drive chain extension warning method according to claim 1, wherein the image processing the first images to obtain first contour images of the drive chains and combining the first contour images of the drive chains into a first keyframe image comprises:

carrying out graying processing on the first images to obtain first grayed images;

performing binarization processing on the first gray images by adopting a threshold segmentation method, segmenting the foreground and the background of the first gray images, and obtaining first contour images of the driving chains;

and performing multi-frame synthesis processing on the first contour images of the plurality of driving chains to obtain a first key frame image.

3. The escalator drive chain elongation warning method according to claim 1, wherein the extracting a first area feature and a first perimeter feature of a contour in the first key frame image according to the first key frame image comprises:

extracting a first contour of the driving chain in motion from the first key frame image;

and performing contour drawing on the first contour, and calculating a first area characteristic and a first perimeter characteristic corresponding to a contour region after contour drawing by using a calculation obtaining function in the OPENCV.

4. The escalator drive chain extension warning method according to claim 1, wherein the image processing the plurality of second images to obtain a plurality of second contour images of the drive chain, and combining the plurality of second contour images of the drive chain into a second keyframe image comprises:

carrying out graying processing on the plurality of second images to obtain a plurality of second grayed images;

performing binarization processing on the second gray images by adopting a threshold segmentation method, segmenting the foreground and the background of the second gray images, and obtaining second contour images of the driving chains;

and performing multi-frame synthesis processing on the second contour images of the plurality of driving chains to obtain a second key frame image.

5. The escalator drive chain extension warning method according to claim 1, wherein the extracting second area features and second perimeter features of contours in the second key frame image according to the second key frame image comprises:

extracting a second contour when the driving chain moves from the second key frame image;

and performing contour drawing on the second contour, and calculating by adopting a calculation function in the OPENCV to obtain a second area characteristic and a second perimeter characteristic corresponding to the contour region after contour drawing.

6. The escalator drive chain extension warning method according to claim 1, wherein the second area characteristic and the second perimeter characteristic are input to the abnormal vibration prediction model to determine whether the escalator is abnormally vibrated; if the abnormal shaking continuously occurs within the preset time, controlling the escalator to stop running, comprising the following steps:

inputting the second area feature and the second perimeter feature into the abnormal shake prediction model, judging, by the abnormal shake prediction model, whether the second area feature exceeds the range of the first area feature, and judging whether the second perimeter feature exceeds the range of the first perimeter feature;

if the second area characteristic does not exceed the range of the first area characteristic and the second perimeter characteristic does not exceed the range of the first perimeter characteristic, judging that the escalator does not shake abnormally; if the second area characteristic exceeds the range of the first area characteristic or the second perimeter characteristic exceeds the range of the first perimeter characteristic, judging that the escalator shakes abnormally;

continuously monitoring preset time, and controlling the escalator to stop running if the escalator continuously shakes abnormally at the end of the preset time; and if the abnormal shaking of the escalator stops within the preset time, canceling the abnormal shaking signal of the escalator and controlling the escalator to normally run.

7. The escalator drive chain elongation early warning method according to claim 1, wherein the acquiring of the drive chain image when the escalator stops operating and the acquiring of the disparity map data of the drive chain image when the escalator stops operating by using a stereo matching method comprise:

shooting by using a binocular camera to obtain a driving chain image when the escalator stops running, wherein the driving chain image comprises a chain left view and a chain right view;

calibrating the binocular cameras, obtaining internal parameters of each camera through calibration, and measuring the relative position between the two cameras of the binocular cameras;

according to the internal parameters and the relative positions, respectively carrying out distortion elimination and line alignment on the chain left view and the chain right view so as to enable the imaging coordinates of the chain left view and the chain right view to be consistent, the visual axes of two cameras of the binocular camera to be parallel, and a left imaging plane and a right imaging plane to be coplanar;

and matching corresponding pixel points on the chain left view and the chain right view in the same scene by adopting the stereo matching method to obtain the parallax image data of the driving chain image when the escalator stops running.

8. The escalator drive chain extension warning method according to claim 7, wherein the obtaining the length of the drive chain from the disparity map data comprises:

calculating the offset between the pixels by a triangulation principle according to the disparity map data to acquire three-dimensional information of the driving chain;

according to the three-dimensional information of the driving chain, calculating by adopting a calculation function in the OPENCV to obtain the distance between adjacent chain links in the driving chain and the key length of a chain plate;

and calculating to obtain the length of the driving chain according to the distance between the adjacent chain links and the key length of the chain plates.

9. The utility model provides an automatic staircase driving chain extension early warning system, its characterized in that, automatic staircase driving chain extension early warning system includes vision collection equipment, vision computational equipment, staircase master control subsystem and staircase early warning cloud platform:

the visual acquisition equipment is in communication connection with the visual computing equipment and is used for acquiring a first video stream when the escalator runs, a second video stream to be predicted and a driving chain image when the escalator stops running and transmitting the first video stream when the escalator runs, the second video stream to be predicted and the driving chain image when the escalator stops running to the visual computing equipment;

the vision computing device is in communication connection with the vision acquisition device, and is used for receiving a first video stream when the escalator runs, a second video stream to be predicted and a driving chain image when the escalator stops running, and further used for executing the escalator driving chain elongation early warning method according to any one of claims 1 to 8 to obtain early warning information and abnormal jitter information;

the escalator main control subsystem is in communication connection with the vision computing equipment and is used for receiving the abnormal jitter information and controlling the escalator to stop running;

the escalator early warning cloud platform is in communication connection with the visual computing equipment and is used for receiving the early warning information and sending the early warning information to a client.

10. The escalator drive chain extension warning system of claim 9, wherein the vision acquisition equipment includes one or more of a surveillance camera, a binocular camera, a depth camera, and a structured light camera.

Images