CN117826273A - Automatic detection method, system and equipment for foreign matters of subway shielding door - Google Patents

Abstract

The invention discloses a method, a system and equipment for automatically detecting foreign matters of a subway shielding door, wherein the method respectively comprises the following steps: s1, closing a subway shielding door, starting an infrared light curtain, and acquiring infrared light curtain state data by a control unit; s2, in the process of infrared light curtain correlation, an abnormal state occurs, and the control unit acquires camera video data before and after the abnormal state of the infrared light curtain; s3, dividing the video frame by frame, and carrying out frame difference operation on the frame by frame and the template image; s4, comparing the differential result with a threshold value, and utilizing foreign object model reasoning to acquire a foreign object type; s5, displaying the judging result and the image of the detected foreign body type, and manually checking. According to the invention, when the shielding doors are closed, the infrared light curtains at the two ends of the shielding doors are opposite to each other, whether foreign matters exist between the shielding doors is judged, the foreign matters are judged through the visual recognition of the camera at the top of the shielding doors, and the state and the judging result are sent to the server through the control unit for personnel to judge and operate, so that the detection means are reliable, the labor investment is less, and the risks of clamping people and objects are reduced.

Description

Automatic detection method, system and equipment for foreign matters of subway shielding door

Technical Field

The invention relates to the technical field of automatic foreign matter detection, in particular to an automatic detection method, an automatic detection system and automatic detection equipment for foreign matter of a subway shielding door.

Background

The automatic detection technology for the foreign matters of the shielding door is a high-tech means such as a server, a camera, an infrared light curtain, a visual recognition algorithm and the like, and is based on artificial intelligence deep learning. The subway shielding door is an important component of a subway platform safety protection facility, and can effectively prevent passengers, luggage and other articles from accidentally entering a tunnel. However, during the operation of the subway, a case where foreign matters enter the shielding door sometimes occurs, which may cause the operation of the subway to malfunction, even to cause a safety accident.

Under actual conditions, the existing subway shielding door protection device products are overused, and the following two remarkable problems mainly exist:

by adopting the laser correlation principle, the laser detection equipment has large volume, is easy to cause infringement when installed at a platform and has high price; the light spots are small, deviation is caused by wind pressure and vibration of the external environment, and the alarm is easily triggered by mistake; the two ends are installed, the precision requirement is high, and the maintenance is difficult.

The laser radar scanning principle is adopted, so that the installation position is high, and the maintenance is difficult; the power failure is required to be applied for, and the frame elevator can be maintained; the mirror surface is easily interfered by tunnel dust, and false alarm occurs.

Therefore, the two significant problems are effectively solved or avoided, the infrared light curtain and the camera are utilized to measure and judge based on visual recognition, the landing of the foreign matter recognition technology in the field of shielding doors is facilitated, and the automatic detection and safety protection of the foreign matter on the subway shielding doors are of great significance.

Disclosure of Invention

The invention provides a method, equipment and a storage medium for automatically detecting foreign matters of a subway shielding door, which can at least solve one of the technical problems in the background technology.

In order to achieve the above purpose, the present invention adopts the following technical scheme:

an automatic detection method for foreign matters of a subway shielding door comprises the following steps,

the infrared curtain is detected, namely an infrared beam is used as a detection light source, and infrared protection light waves are formed at the sliding door through infrared emission and receiving devices arranged at two sides of each door body or each carriage; the subway shielding door infrared light curtain comprises a light curtain body, a control module, an infrared emitter and an infrared receiver; the light curtain body comprises a light curtain area surrounded by an infrared emitter and an infrared receiver, the light curtain body is arranged on the subway shielding door through a fixing device, and a plurality of infrared emitters and infrared receivers are distributed on the light curtain body; the control module is arranged on the subway shielding door and connected with the infrared transmitter and the infrared receiver, receives the subway train running state signal, controls the infrared transmitter to transmit infrared light according to the signal instruction, and controls the infrared receiver to receive the infrared light;

adopting camera visual recognition, namely adopting a camera to take images, analyzing the digital images through a server based on a recognition technology of machine vision, and thus realizing recognition and detection of the characteristics of the detected object; comprises a camera, an image analysis system and a display unit; the camera is arranged at the top of the subway shielding door and is used for collecting images of the shielding door; the image analysis system analyzes the image acquired by the camera to acquire visual information of the image after the shielding door is closed; the display unit is used for displaying the visual information acquired by the image analysis system;

the method comprises the steps of linking an infrared light curtain with camera vision, adjusting a transmitting device and a receiving device to the same transmission channel, calibrating the infrared light curtain, transmitting light curtain signals to a platform control indoor control unit through a signal cable when the light curtain signals are not abnormal, transmitting information to a platform door system control cabinet and an on-site control panel by the control unit, controlling a light belt at a station of a train driver head to fully light a green light, indicating that the train normally closes a shielding door and can leave a station, and enabling an intelligent security system to enter a standby mode;

when an obstacle appears in the operation process of the intelligent security system, the light curtain signal suddenly changes into a low-level signal, the control unit receives the abnormal low-level signal, collects video data collected by the field camera at the current time, takes the time of receiving the low-level signal as a node, extracts video data from 1s before the node to 1s after the shielding door is completely closed in real time, and uploads the video data to the control unit for the image processing unit to visually identify and detect and manually confirm with the station control center; meanwhile, the control unit sends information to the platform door system control cabinet and the on-site control panel, the control lamp is displayed in color, the abnormal corresponding carriage indicator lamp is red, the normal corresponding carriage indicator lamp is green, and a driver is prompted not to start the train; after the visual identification detection is finished, the information is transmitted back to the control unit, and is confirmed by combining the video information manually; the checking result shows that no foreign matter exists, the shielding door is normally closed, and the train can leave the station; the detection result is human body or foreign matter, the human body or foreign matter waits for manual confirmation and carries out field treatment, after the field situation is safe, the field is controlled by the field control panel to control the lamp strip to be full green, the shielding door is normally closed, and the train drives away from the station.

Further, the method for controlling the infrared light curtain comprises the following steps:

when receiving that a subway train enters a subway shielding door area and passengers get on or off, the shielding door starts a closing signal state signal, and the control module controls the infrared emitter to emit infrared light;

an infrared receiver receives infrared light;

the infrared receiver judges whether the subway shielding door is in a safe state according to the received infrared light condition, and sends a judging result to the control module;

the control module controls the infrared emitter to stop emitting infrared light or controls the infrared emitter to continuously emit infrared light according to the judging result sent by the infrared receiver.

Further, the recognition algorithm of the infrared curtain and camera visual linkage comprises 4 steps:

Step 3.1：

image extraction, storing collected video data by a camera, collecting video data VID (x, y, n) of an abnormal time period after the light curtain signal is abnormal, and analyzing the video data VID (x, y, n) frame by frame; establishing a shielding door crack template image IMG under normal condition of a target detection area in advance _Template (x, y), i.e. the area between the rail transit platform screen door and the train door, contains the screen door image;

dividing VID (x, y, n) into multiple frames of images, determining a suitable frame rate interval Freq _frame Dividing the video and extracting multi-frame images; IMG of nth frame image _n (x, y) cutting out the image IMG of the area with the designated size of the crack of the shielding door _CROPn (x,y)；

Step 3.2：

Differential image, IMG _CROPn (x, y) and screen door crack template image IMG _Template (x, y) performing differential operation to obtain a differential image D _pix-n (x, y); the differential result calculation formula is as follows:

D _pix-n (x，y)＝IMG _CROPn (x，y)-IMG _Template (x，y)

Step 3.3：

gaussian filtering, namely carrying out Gaussian filtering operation after differentiating the image according to Step 3.2, and carrying out smoothing treatment on noise points of the image caused by environmental state change; the Gaussian filter takes a weighted average of the image, and the two-dimensional Gaussian function formula is:

taking the radius of the field as r, and the discretized Gaussian kernel formula is as follows:

wherein G (x, y) is the filtered result, H _x,y A weight matrix of each point in the Gaussian kernel, wherein sigma is the standard deviation;

Step 3.4：

enhancing contrast, namely enhancing the contrast of the filtered image IMGgauss (x, y) to ensure that the target object area is easier to distinguish from the background area; setting the value of a pixel point Lower than a threshold Lower to 0 and setting the value of a pixel point higher than a threshold Upper to 1 based on a piecewise linear enhancement algorithm, and carrying out local amplification on the value of the pixel point between the two thresholds; the enhanced contrast formula is:

wherein g (x, y) is the result of piecewise linear enhancement;

Step 3.5：

image difference statistics, comparing the piecewise linear enhanced image pixel value with a threshold value Thre, and calculating whether each pixel point of each frame of image is a target object region attribute value or not _n (x, y) and counting the difference result D of each frame image _value-n The calculation formula is as follows:

wherein m is the image pixel width; n is the image pixel height;

storing the multi-frame image difference result in an array D _value-list In (c), as follows:

D _value-list ＝[D _value-1 ，D _value-2 ，...，D _value-x ，...，D _value-k ]，(n＝1，2，...，k)

comparison D _value-n And set a threshold THR _diff If D _value-n Array of arraysD _value-list Are all smaller than THR _diff The method comprises the steps of (1) stopping a visual detection process without alarming operation, normally closing a shielding door, and normally driving a train away from a station; if D _value-list Exists not less than THR _diff In the case of (1), the post-door image is extracted as a key frame image IMG _ABN (x, y), waiting for visual recognition of the object to detect the clamped object.

Further, the recognition algorithm of the visual linkage of the infrared light curtain and the camera further comprises:

Step 3.6：

model reasoning, creating a mask door foreign object detection model to abnormal key frame image IMG by using a machine learning-based target detection method _ABN (x, y) reasoning, obtaining and reporting a foreign matter detection result, and determining whether driving is influenced or not by a station control worker; if no influence exists, the corresponding carriage number indicator lamp is controlled to be changed into a green light for displaying, and a driver is informed of normally driving away from the station; if the influence exists, the TGPS sends an alarm signal to the PSL, prompts a site platform operator to treat the abnormality, sends a safety signal through the PSL after the abnormality is released, and the shielding door is normally closed, so that the train can normally leave the station, and the visual detection is finished and the next abnormal light curtain triggering is waited.

On the other hand, the invention also discloses an automatic detection system for the foreign matters of the subway shielding door, which comprises the following units,

the infrared light curtain detection unit comprises an infrared light curtain of a subway shielding door, and comprises a light curtain body, a control module, an infrared emitter and an infrared receiver; the light curtain body comprises a light curtain area surrounded by an infrared emitter and an infrared receiver, the light curtain body is arranged on the subway shielding door through a fixing device, and a plurality of infrared emitters and infrared receivers are distributed on the light curtain body; the control module is arranged on the subway shielding door and connected with the infrared transmitter and the infrared receiver, receives the subway train running state signal, controls the infrared transmitter to transmit infrared light according to the signal instruction, and controls the infrared receiver to receive the infrared light;

the camera visual recognition unit comprises a camera, an image analysis system and a display unit; the camera is arranged at the top of the subway shielding door and is used for collecting images of the shielding door; the image analysis system analyzes the image acquired by the camera to acquire visual information of the image after the shielding door is closed; the display unit is used for displaying the visual information acquired by the image analysis system;

the infrared light curtain and the camera vision unit are used for adjusting the transmitting device and the receiving device to the same transmission channel and calibrating the same, when the light curtain signal is not abnormal, the light curtain signal is transmitted to the platform control indoor control unit through the signal cable, and the control unit sends information to the platform door system control cabinet and the on-site control panel to control the light belt at the head site of a train driver to fully light a green light, so that the shielding door is normally closed and the train can leave the station, and the intelligent security system enters a standby mode.

Further, the intelligent security system also comprises a control unit, wherein when an obstacle appears in the operation process of the intelligent security system, the light curtain signal suddenly changes into a low-level signal, the control unit receives the abnormal low-level signal, video data collected by the field camera at the current time is collected, the time when the low-level signal is received is taken as a node, the video data in 1s before the node and 1s after the shielding door is completely closed are extracted in real time and uploaded to the control unit, and the video data are visually recognized and detected by the image processing unit and manually confirmed by the station control center;

meanwhile, the control unit sends information to the platform door system control cabinet and the on-site control panel, the control lamp is displayed in color, the abnormal corresponding carriage indicator lamp is red, the normal corresponding carriage indicator lamp is green, and a driver is prompted not to start the train; after the visual identification detection is finished, the information is transmitted back to the control unit, and is confirmed by combining the video information manually; the checking result shows that no foreign matter exists, the shielding door is normally closed, and the train can leave the station; the detection result is human body or foreign matter, the human body or foreign matter waits for manual confirmation and carries out field treatment, after the field situation is safe, the field is controlled by the field control panel to control the lamp strip to be full green, the shielding door is normally closed, and the train drives away from the station.

The automatic detection method of the foreign matter of the subway shielding door is also included, the automatic detection system of the foreign matter of the subway shielding door is adopted, the method comprises the following steps,

s1, closing a subway shielding door, starting an infrared light curtain, and acquiring infrared light curtain state data by a control unit;

s2, in the process of infrared light curtain correlation, an abnormal state occurs, and the control unit acquires camera video data before and after the abnormal state of the infrared light curtain;

s3, dividing the video frame by frame, and carrying out frame difference operation on the frame by frame and the template image;

s4, comparing the differential result with a threshold value, and utilizing foreign object model reasoning to acquire a foreign object type;

s5, displaying the judging result and the image of the detected foreign body type, and manually checking.

In yet another aspect, the invention also discloses a computer readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of the method as described above.

In yet another aspect, the invention also discloses a computer device comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to perform the steps of the method as above.

According to the technical scheme, when the shielding doors are closed, the infrared light curtains at the two ends of the shielding doors are used for correlation, whether the shielding doors are provided with the foreign matters or not is judged, the foreign matters are visually identified through the camera at the top of the shielding doors, the foreign matters are judged, and the state and the judging result are sent to the server through the control unit for personnel to judge and operate, so that the detection means is reliable, the labor input is less, and the risks of clamping people and objects are reduced.

Compared with the prior art, the invention has the beneficial effects that:

1. the invention adopts the infrared light curtain and machine vision recognition combined technology to avoid the problem that the type of the foreign matters clamped in the abnormal state of the shielding door is difficult to obtain under the actual working condition. Meanwhile, the problem of high cost of a platform operator at the fixed point of a shielding door arranged on a conventional subway platform is solved by a machine vision recognition technology.

2. The invention fuses the image difference operation, the Gaussian filtering algorithm and the piecewise linear image enhancement algorithm into the totally new improved shielding door foreign matter image detection algorithm, is more suitable for the shielding door image of the rail transit platform with easily changeable environment state, and reduces certain environmental influence.

3. The invention adopts the target detection algorithm based on machine learning to treat the problem of judging the foreign matters of the shielding door, judges whether the foreign matters possibly exist or not in advance through the size of the region of interest (ROI) in the flow of the preamble algorithm, improves the comprehensiveness and the reliability of the algorithm detection, and is more in line with the application scene of actual rail transit.

Drawings

FIG. 1 is a general frame diagram of the system of the present invention;

FIG. 2 is a flow chart of a method according to an embodiment of the present invention;

FIG. 3 is a flow chart of visual recognition according to an embodiment of the present invention;

FIG. 4 is a frame-by-frame acquisition diagram of a camera according to an embodiment of the present invention;

FIG. 5 is a flowchart illustrating the difference determination according to an embodiment of the present invention;

FIG. 6 is a differential image filtering enhancement diagram of an embodiment of the present invention;

FIG. 7 is a flow chart of reporting an inference result in an embodiment of the present invention;

fig. 8 is an exemplary interface diagram of an automatic detection system for foreign matters in a subway shield door according to an embodiment of the present invention.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention.

In order to solve the problems in the prior art, the technical scheme adopted by the embodiment of the invention mainly comprises three steps, including: infrared light curtain detection, camera visual identification, infrared light curtain and camera visual linkage. The method comprises the following specific steps:

Step 1：

the infrared light curtain is detected, an infrared beam is used as a detection light source, and infrared protection light waves are formed at the sliding door through infrared emission and receiving devices arranged at two sides of each door body or each carriage. The subway shielding door infrared light curtain comprises a light curtain body, a control module, an infrared emitter and an infrared receiver; the light curtain body comprises a light curtain area surrounded by the infrared transmitters and the infrared receivers, the light curtain body is arranged on the subway shielding door through a fixing device, and a plurality of infrared transmitters and the infrared receivers are distributed on the light curtain body; the control module is arranged on the subway shielding door, is connected with the infrared transmitter and the infrared receiver, receives the subway train running state signal, controls the infrared transmitter to transmit infrared light according to the signal instruction, and controls the infrared receiver to receive the infrared light.

The infrared light curtain control method mainly comprises 4 steps:

Step 1.1：

when receiving that a subway train enters a subway shielding door area and passengers get on or off, the shielding door starts a closing signal state signal, and the control module controls the infrared emitter to emit infrared light;

Step 1.2：

an infrared receiver receives infrared light;

Step 1.3：

the infrared receiver judges whether the subway shielding door is in a safe state according to the received infrared light condition, and sends a judging result to the control module;

Step 1.4：

the control module controls the infrared emitter to stop emitting infrared light or controls the infrared emitter to continuously emit infrared light according to the judging result sent by the infrared receiver.

Step 2：

The camera visual recognition adopts a camera to take images, and a recognition technology based on machine vision is adopted, and the digital images are analyzed through a server, so that the recognition and detection of the characteristics of the detected object are realized. The subway shielding door camera identification system comprises a camera, an image analysis system and a display unit. The camera is arranged at the top of the subway shielding door and is used for collecting images of the shielding door; the image analysis system analyzes the image acquired by the camera to acquire visual information of the image after the shielding door is closed; the display unit is used for displaying the visual information acquired by the image analysis system.

Step 3：

The infrared light curtain and the camera are in visual linkage, the transmitting device and the receiving device are adjusted to the same transmission channel and are calibrated, when the light curtain signal is abnormal, the light curtain signal is transmitted to the control unit (TGPB) in the platform control room through the signal cable, and the control unit (TGPB) sends information to the platform door system control cabinet (PSC) and the on-site control Panel (PSL), so that the light belt at the head site of a train driver is controlled to fully light a green light, the shielding door is indicated to be normally closed, the train can leave the station, and the intelligent security system (TGPS) enters a standby mode.

In the operation process of the TGPS, when an obstacle appears, the light curtain signal is suddenly changed into a low-level signal, the TGPB receives the abnormal low-level signal, video data collected by the field camera at the current time is collected, the time when the low-level signal is received is taken as a node, the video data in 1s before the node and 1s after the shielding door is completely closed are extracted in real time and uploaded to the TGPB, and the video data are visually recognized and detected by an image processing unit and manually confirmed by a station control center. Meanwhile, TGPB sends information to PSC and PSL, the control lamp is colored to display, the abnormal corresponding carriage indicator lamp is red, the normal corresponding carriage indicator lamp is green, and a driver is prompted not to start the train. And after the visual identification detection is finished, the information is transmitted back to the TGPB, and is confirmed by combining the video information manually. The checking result shows that no foreign matter exists, the shielding door is normally closed, and the train can leave the station; the detection result is human body or foreign matter, the human body or foreign matter waits for manual confirmation and carries out field treatment, after the field situation is safe, the field is displayed as full green through the PSL control lamp strip, the shielding door is normally closed, and the train drives away from the station.

The main recognition algorithm of the infrared curtain and camera vision linkage comprises 4 steps:

Step 3.1：

and (3) extracting images, storing collected video data by a camera, wherein an abnormality exists in a light curtain signal, and analyzing the video data VID (x, y, n) in an abnormal time period frame by frame. Establishing target detection area normal in advanceUnder the condition, shielding door crack template image IMG _Template (x, y), i.e. the area between the rail transit platform screen door and the train door, contains the screen door image.

Dividing VID (x, y, n) into multiple frames of images, determining a suitable frame rate interval Freq _frame To divide the video and extract multi-frame images. IMG of nth frame image _n (x, y) cutting out the image IMG of the area with the designated size of the crack of the shielding door _CROPn (x, y) to avoid the influence of environmental changes.

Step 3.2：

Differential image, IMG _CROPn (x, y) and screen door crack template image IMG _Template (x, y) performing differential operation to obtain a differential image D _pix-n (x, y). The differential result calculation formula is as follows:

D _pix-n (x，y)＝IMG _CROPn (x，y)-IMG _Template (x，y)

Step 3.3：

and carrying out Gaussian filtering, namely carrying out Gaussian filtering operation after differentiating the image according to Step 3.2, and carrying out smoothing treatment on noise points of the image caused by environmental state change. The Gaussian filter takes a weighted average of the image, and the two-dimensional Gaussian function formula is:

taking the radius of the field as r, and the discretized Gaussian kernel formula is as follows:

wherein G (x, y) is the filtered result, H _x,y The weight matrix for each point in the gaussian kernel, σ is the standard deviation.

Step 3.4：

And the contrast is enhanced, and the contrast of the filtered image IMGgauss (x, y) is enhanced, so that the target object area is more easily distinguished from the background area. Based on a piecewise linear enhancement algorithm, the value of a pixel point Lower than a threshold Lower is set to 0, the value of a pixel point higher than a threshold Upper is set to 1, and the value of a pixel point between two thresholds is locally amplified. The enhanced contrast formula is:

where g (x, y) is the result of piecewise linear enhancement.

Step 3.5：

Image difference statistics, comparing the piecewise linear enhanced image pixel value with a threshold value Thre, and calculating whether each pixel point of each frame of image is a target object region attribute value or not _n (x, y) and counting the difference result D of each frame image _value-n The calculation formula is as follows:

wherein m is the image pixel width; n is the image pixel height.

Storing the multi-frame image difference result in an array D _value-list In (c), as follows:

D _value-list ＝[D _value-1 ，D _value-2 ，...，D _value-n ，…，D _value-k ]，(n＝1，2，...，k)

comparison D _value-n And set a threshold THR _diff If D _value-n Array D _value-list Are all smaller than THR _diff The method comprises the steps of (1) stopping a visual detection process without alarming operation, normally closing a shielding door, and normally driving a train away from a station; if D _value-list Exists not less than THR _diff In the case of (1), the post-door image is extracted as a key frame image IMG _ABN (x, y), waiting for visual recognition of the object to detect the clamped object.

Step 3.6：

Model reasoning, creating a mask door foreign object detection model to abnormal key frame image IMG by using a machine learning-based target detection method _ABN And (x, y) reasoning, obtaining and reporting a foreign matter detection result, and determining whether driving is influenced or not by a station control worker. If no influence exists, the corresponding carriage number indicator lamp is controlled to be changed into a green light for displaying, and a driver is informed of normally driving away from the station; if the influence exists, the TGPS sends an alarm signal to the PSL, prompts the site platform staff to treat the abnormality, sends a safety signal through the PSL after waiting for the abnormality to be relieved, and the shielding door is normally closed, so that the train can normally leave the station. And (5) finishing the visual detection and waiting for the abnormal triggering of the next light curtain.

The following is a detailed description with reference to the accompanying drawings:

fig. 1 is a general architecture diagram of an automatic detection system for foreign matters in a subway shielding door according to an embodiment of the present invention, as shown in fig. 1, the automatic detection system for foreign matters in a subway shielding door includes the following functional modules: the device comprises an infrared light curtain, a camera, a control unit, a server and a display unit.

Specifically, the infrared light curtain and the camera acquire data to the control unit, the control unit transmits the data to the server, the server is internally provided with a neural network visual recognition algorithm to perform recognition and discrimination, and finally the display unit displays results.

Fig. 2 is a flow chart of a scheme of an automatic detection system for foreign matters in a subway shielding door according to an embodiment of the invention, as shown in fig. 2, the automatic detection system for foreign matters in the subway shielding door comprises the following steps:

s1, closing a subway shielding door, starting an infrared light curtain, and acquiring infrared light curtain state data by a control unit;

s2, in the process of infrared light curtain correlation, an abnormal state occurs, and the control unit acquires camera video data before and after the abnormal state of the infrared light curtain;

s3, dividing the video frame by frame, and carrying out frame difference operation on the frame by frame and the template image;

s4, comparing the differential result with a threshold value, and utilizing foreign object model reasoning to acquire a foreign object type;

s5, displaying the judging result and the image of the detected foreign body type, and manually checking.

A specific visual recognition flowchart is shown in fig. 3, and includes the following steps:

firstly, starting visual detection, acquiring video data of an abnormal time period, and taking video data in 1s before and after a time point of closing a shielding door.

Dividing the video frame by frame, performing frame difference operation on the frame by frame and the template image, filtering and enhancing the difference image, and calculating a difference result D _value-n 。

The specific camera acquires the images shown in fig. 4 frame by frame, and obtains the IMG through a difference solving flow chart shown in fig. 5 _CROPn (x, y) and screen door crack template image IMG _Template (x, y) performing differential operation to obtain a differential image D _pix-n (x, y). The differential result calculation formula is as follows:

D _pix-n (x,y)＝IMG _CROPn (x,y)-IMG _Template (x,y)

smoothing the differentiated image through Gaussian filtering operation, wherein the Gaussian filtering is used for taking a weighted average value of the image, and a two-dimensional Gaussian function formula is as follows:

where G (x, y) is the filtered result and σ is the standard deviation.

For filtered image IMG _gauss (x, y) enhances contrast, distinguishing the target region from the background region. And using a piecewise linear enhancement algorithm to set the value of the pixel point below the threshold Lower to 0, and set the value of the pixel point above the threshold Upper to 1, and locally amplifying the value of the pixel point between the two thresholds. The formula for enhancing contrast is:

where g (x, y) is the result of piecewise linear enhancement, and the enhanced differential image filtering enhancement graph is shown in fig. 6.

Comparing the piecewise linear enhanced image pixel value with a threshold value Thre, and calculating whether each pixel point of each frame of image is an attribute value of a target object area _n (x, y), and counting the difference result D of each frame image _value-n The calculation formula is as follows:

wherein m is the image pixel width; n is the image pixel height.

And step three, comparing the differential result obtained in the step two with a threshold THRdiff, if Dvalue-n is smaller than the threshold THRdiff, enabling the shielding door to normally close the train and leave the station, if Dvalue-n is larger than the threshold THRdiff, taking the door-closing image to perform foreign object model reasoning, obtaining the foreign object type, and reporting the reasoning result.

A specific reasoning result reporting procedure is shown in fig. 7. An interface of the automatic detection system for the foreign matters of the subway shielding door is shown in fig. 8.

And step four, after the station control staff judges and processes according to the reasoning result obtained in the step three, the shielding door is normally closed, the train drives away from the station, the visual detection is finished, the next light curtain triggering is waited, and the step one to the step four are repeated.

According to the method and the system for automatically detecting the foreign matters in the subway shielding door, provided by the embodiment of the invention, when the shielding door is closed, the infrared light curtains at the two ends of the shielding door are used for correlation, whether the foreign matters exist between the shielding doors or not is judged, the foreign matters are visually identified through the camera at the top of the shielding door, so that the foreign matters are judged, the state and the judging result are sent to the server through the control unit for personnel to judge and operate, the detection means is reliable, the labor investment is less, and the risks of clamping people and objects are reduced.

In yet another aspect, the invention also discloses a computer readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of the method as described above.

In yet another aspect, the invention also discloses a computer device comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to perform the steps of the method as above.

In yet another embodiment provided herein, there is also provided a computer program product containing instructions that, when run on a computer, cause the computer to perform any of the subway shield door foreign object automatic detection methods of the above embodiments.

It may be understood that the system provided by the embodiment of the present invention corresponds to the method provided by the embodiment of the present invention, and explanation, examples and beneficial effects of the related content may refer to corresponding parts in the above method.

The embodiment of the application also provides an electronic device, which comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus,

a memory for storing a computer program;

and the processor is used for realizing the automatic detection method of the foreign matters of the subway shielding door when executing the program stored in the memory.

The communication bus mentioned by the above electronic device may be a peripheral component interconnect standard (english: peripheral Component Interconnect, abbreviated: PCI) bus or an extended industry standard architecture (english: extended Industry Standard Architecture, abbreviated: EISA) bus, or the like. The communication bus may be classified as an address bus, a data bus, a control bus, or the like.

The communication interface is used for communication between the electronic device and other devices.

The Memory may include random access Memory (Random Access Memory, abbreviated as RAM) or nonvolatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.

The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), etc.; it may also be a digital signal processor (English: digital Signal Processing; DSP; for short), an application specific integrated circuit (English: application Specific Integrated Circuit; ASIC; for short), a Field programmable gate array (English: field-Programmable Gate Array; FPGA; for short), or other programmable logic device, discrete gate or transistor logic device, discrete hardware components.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present application, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, by wired (e.g., coaxial cable, optical fiber, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), etc.

It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.

The above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (8)

1. A method for automatically detecting foreign matters of a subway shielding door is characterized by comprising the following steps,

the infrared curtain is detected, namely an infrared beam is used as a detection light source, and infrared protection light waves are formed at the sliding door through infrared emission and receiving devices arranged at two sides of each door body or each carriage; the subway shielding door infrared light curtain comprises a light curtain body, a control module, an infrared emitter and an infrared receiver; the light curtain body comprises a light curtain area surrounded by an infrared emitter and an infrared receiver, the light curtain body is arranged on the subway shielding door through a fixing device, and a plurality of infrared emitters and infrared receivers are distributed on the light curtain body; the control module is arranged on the subway shielding door and connected with the infrared transmitter and the infrared receiver, receives the subway train running state signal, controls the infrared transmitter to transmit infrared light according to the signal instruction, and controls the infrared receiver to receive the infrared light;

adopting camera visual recognition, namely adopting a camera to take images, analyzing the digital images through a server based on a recognition technology of machine vision, and thus realizing recognition and detection of the characteristics of the detected object; comprises a camera, an image analysis system and a display unit; the camera is arranged at the top of the subway shielding door and is used for collecting images of the shielding door; the image analysis system analyzes the image acquired by the camera to acquire visual information of the image after the shielding door is closed; the display unit is used for displaying the visual information acquired by the image analysis system;

the method comprises the steps of linking an infrared light curtain with camera vision, adjusting a transmitting device and a receiving device to the same transmission channel, calibrating the infrared light curtain, transmitting light curtain signals to a platform control indoor control unit through a signal cable when the light curtain signals are not abnormal, transmitting information to a platform door system control cabinet and an on-site control panel by the control unit, controlling a light belt at a station of a train driver head to fully light a green light, indicating that the train normally closes a shielding door and can leave a station, and enabling an intelligent security system to enter a standby mode;

when an obstacle appears in the operation process of the intelligent security system, the light curtain signal suddenly changes into a low-level signal, the control unit receives the abnormal low-level signal, collects video data collected by the field camera at the current time, takes the time of receiving the low-level signal as a node, extracts video data from 1s before the node to 1s after the shielding door is completely closed in real time, and uploads the video data to the control unit for the image processing unit to visually identify and detect and manually confirm with the station control center; meanwhile, the control unit sends information to the platform door system control cabinet and the on-site control panel, the control lamp is displayed in color, the abnormal corresponding carriage indicator lamp is red, the normal corresponding carriage indicator lamp is green, and a driver is prompted not to start the train; after the visual identification detection is finished, the information is transmitted back to the control unit, and is confirmed by combining the video information manually; the checking result shows that no foreign matter exists, the shielding door is normally closed, and the train can leave the station; the detection result is human body or foreign matter, the human body or foreign matter waits for manual confirmation and carries out field treatment, after the field situation is safe, the field is controlled by the field control panel to control the lamp strip to be full green, the shielding door is normally closed, and the train drives away from the station.

2. The automatic detection method for foreign matter in a subway shield door according to claim 1, characterized in that: the control method of the infrared light curtain is as follows:

when receiving that a subway train enters a subway shielding door area and passengers get on or off, the shielding door starts a closing signal state signal, and the control module controls the infrared emitter to emit infrared light;

an infrared receiver receives infrared light;

the infrared receiver judges whether the subway shielding door is in a safe state according to the received infrared light condition, and sends a judging result to the control module;

the control module controls the infrared emitter to stop emitting infrared light or controls the infrared emitter to continuously emit infrared light according to the judging result sent by the infrared receiver.

3. The automatic detection method for foreign matter in a subway shield door according to claim 1, characterized in that: the recognition algorithm of the infrared light curtain and camera vision linkage comprises 4 steps:

Step 3.1：

image extraction, storing collected video data by a camera, collecting video data VID (x, y, n) of an abnormal time period after the light curtain signal is abnormal, and analyzing the video data VID (x, y, n) frame by frame; establishing a shielding door crack template image IMG under normal condition of a target detection area in advance _Template (x, y), i.e. the area between the rail transit platform screen door and the train door, contains the screen door image;

dividing VID (x, y, n) into multiple frames of images, determining a suitable frame rate interval Freq _frame Dividing the video and extracting multi-frame images; IMG of nth frame image _n (x, y) cutting out the image IMG of the area with the designated size of the crack of the shielding door _CROPn (x,y)；

Step 3.2：

Differential image, IMG _CROPn (x, y) and screen door crack template image IMG _Template (x, y) performing differential operation to obtain a differential image D _pix-n (x, y); the differential result calculation formula is as follows:

D _pix-n (x,y)＝IMG _CROPn (x,y)-IMG _Template (x,y)

Step 3.3：

gaussian filtering, namely carrying out Gaussian filtering operation after differentiating the image according to Step 3.2, and carrying out smoothing treatment on noise points of the image caused by environmental state change; the Gaussian filter takes a weighted average of the image, and the two-dimensional Gaussian function formula is:

taking the radius of the field as r, and the discretized Gaussian kernel formula is as follows:

wherein G (x, y) is the filtered result, H _x,y A weight matrix of each point in the Gaussian kernel, wherein sigma is the standard deviation;

Step 3.4：

enhancing contrast, namely enhancing the contrast of the filtered image IMGgauss (x, y) to ensure that the target object area is easier to distinguish from the background area; setting the value of a pixel point Lower than a threshold Lower to 0 and setting the value of a pixel point higher than a threshold Upper to 1 based on a piecewise linear enhancement algorithm, and carrying out local amplification on the value of the pixel point between the two thresholds; the enhanced contrast formula is:

wherein g (x, y) is the result of piecewise linear enhancement;

Step 3.5：

image difference statistics, comparing the piecewise linear enhanced image pixel value with a threshold value Thre, and calculating whether each pixel point of each frame of image is a target object region attribute value or not _n (x, y) and counting the difference result D of each frame image _value-n The calculation formula is as follows:

wherein m is the image pixel width; n is the image pixel height;

storing the multi-frame image difference result in an array D _value-list In (c), as follows:

D _value-list ＝[D _value-list-1 ，D _value-list-2 ，D _value-list-n ，...，D _value-k ]，(n＝1，2，...,k)

comparison D _value-n And set a threshold THR _diff If D _value-n Array D _value-list Are all smaller than THR _diff The method comprises the steps of (1) stopping a visual detection process without alarming operation, normally closing a shielding door, and normally driving a train away from a station; if D _value-list Exists not less than THR _diff In the case of (1), the post-door image is extracted as a key frame image IMG _ABN (x, y), waiting for visual recognition of the object to detect the clamped object.

4. The automatic detection method for foreign matter of subway shield door according to claim 3, characterized in that: the recognition algorithm of the infrared curtain and camera vision linkage further comprises:

Step 3.6：

model reasoning, creating a mask door foreign object detection model to abnormal key frame image IMG by using a machine learning-based target detection method _ABN (x, y) reasoning, obtaining and reporting a foreign matter detection result, and determining whether driving is influenced or not by a station control worker; if no influence exists, the corresponding carriage number indicator lamp is controlled to be changed into a green light for displaying, and a driver is informed of normally driving away from the station; if the influence exists, the intelligent security system sends an alarm signal to the on-site control panel, prompts the on-site platform staff to treat the abnormality, sends a safety signal through the PSL after waiting for the abnormality to be relieved, and the shielding door is normally closed, so that the train can normally leave the station, and the visual inspection is finished and the next abnormal light curtain triggering is waited.

5. A subway shield door foreign matter automated inspection system which characterized in that: comprising the following units of the device,

the infrared light curtain detection unit comprises an infrared light curtain of a subway shielding door, and comprises a light curtain body, a control module, an infrared emitter and an infrared receiver; the light curtain body comprises a light curtain area surrounded by an infrared emitter and an infrared receiver, the light curtain body is arranged on the subway shielding door through a fixing device, and a plurality of infrared emitters and infrared receivers are distributed on the light curtain body; the control module is arranged on the subway shielding door and connected with the infrared transmitter and the infrared receiver, receives the subway train running state signal, controls the infrared transmitter to transmit infrared light according to the signal instruction, and controls the infrared receiver to receive the infrared light;

the camera visual recognition unit comprises a camera, an image analysis system and a display unit; the camera is arranged at the top of the subway shielding door and is used for collecting images of the shielding door; the image analysis system analyzes the image acquired by the camera to acquire visual information of the image after the shielding door is closed; the display unit is used for displaying the visual information acquired by the image analysis system;

the infrared light curtain and the camera vision unit are used for adjusting the transmitting device and the receiving device to the same transmission channel and calibrating the same, when the light curtain signal is not abnormal, the light curtain signal is transmitted to the platform control indoor control unit through the signal cable, and the control unit sends information to the platform door system control cabinet and the on-site control panel to control the light belt at the head site of a train driver to fully light a green light, so that the shielding door is normally closed and the train can leave the station, and the intelligent security system enters a standby mode.

6. The automatic detection system for foreign matter of subway shield door according to claim 5, wherein: the intelligent security system also comprises a control unit, wherein when an obstacle appears in the operation process of the intelligent security system, the light curtain signal is suddenly changed into a low-level signal, the control unit receives the abnormal low-level signal, video data collected by the field camera at the current time is collected, the time of receiving the low-level signal is taken as a node, the video data in the time from 1s before the node to 1s after the shielding door is completely closed are extracted in real time and uploaded to the control unit, and the video data are used for visual identification and detection of an image processing unit and manual confirmation of a station control center;

meanwhile, the control unit sends information to the platform door system control cabinet and the on-site control panel, the control lamp is displayed in color, the abnormal corresponding carriage indicator lamp is red, the normal corresponding carriage indicator lamp is green, and a driver is prompted not to start the train; after the visual identification detection is finished, the information is transmitted back to the control unit, and is confirmed by combining the video information manually; the checking result shows that no foreign matter exists, the shielding door is normally closed, and the train can leave the station; the detection result is human body or foreign matter, the human body or foreign matter waits for manual confirmation and carries out field treatment, after the field situation is safe, the field is controlled by the field control panel to control the lamp strip to be full green, the shielding door is normally closed, and the train drives away from the station.

7. A method for automatically detecting foreign matters in a subway shielding door, which adopts the automatic detection system for foreign matters in a subway shielding door according to claim 5 or 6, and is characterized in that: comprises the steps of,

s1, closing a subway shielding door, starting an infrared light curtain, and acquiring infrared light curtain state data by a control unit;

s2, in the process of infrared light curtain correlation, an abnormal state occurs, and the control unit acquires camera video data before and after the abnormal state of the infrared light curtain;

s3, dividing the video frame by frame, and carrying out frame difference operation on the frame by frame and the template image;

s4, comparing the differential result with a threshold value, and utilizing foreign object model reasoning to acquire a foreign object type;

s5, displaying the judging result and the image of the detected foreign body type, and manually checking.

8. A computer device comprising a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method of any of claims 1 to 4.