CN116597394A - Railway foreign matter intrusion detection system and method based on deep learning - Google Patents

Railway foreign matter intrusion detection system and method based on deep learning Download PDF

Info

Publication number
CN116597394A
CN116597394A CN202310558280.7A CN202310558280A CN116597394A CN 116597394 A CN116597394 A CN 116597394A CN 202310558280 A CN202310558280 A CN 202310558280A CN 116597394 A CN116597394 A CN 116597394A
Authority
CN
China
Prior art keywords
target
module
alarm
foreign matter
railway
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310558280.7A
Other languages
Chinese (zh)
Inventor
范国海
徐绍伟
何洪伟
陈郑淏
冉思
石玲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chengdu National Railways Electrical Equipment Co ltd
Original Assignee
Chengdu National Railways Electrical Equipment Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chengdu National Railways Electrical Equipment Co ltd filed Critical Chengdu National Railways Electrical Equipment Co ltd
Priority to CN202310558280.7A priority Critical patent/CN116597394A/en
Publication of CN116597394A publication Critical patent/CN116597394A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00—Burglar, theft or intruder alarms
    • G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
    • G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19602—Image analysis to detect motion of the intruder, e.g. by frame subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00—Burglar, theft or intruder alarms
    • G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
    • G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19602—Image analysis to detect motion of the intruder, e.g. by frame subtraction
    • G08B13/19608—Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Image Analysis (AREA)
  • Burglar Alarm Systems (AREA)

Abstract

The application belongs to the technical field of railway foreign matter intrusion detection, and particularly relates to a railway foreign matter intrusion detection system based on deep learning, which comprises a video acquisition module, a limit marking module, a foreign matter intrusion detection module, a data processing module, an early warning module and an intrusion target tracing module, wherein the video acquisition module acquires railway limit video images, the limit marking module marks a limit area, the foreign matter intrusion detection module detects unknown type targets and known type targets, the detection result is transmitted to the data processing module, the data processing module performs target tracking and alarm filtering processing, filtered alarm information is transmitted to the early warning module, and the intrusion target tracing module traces the intrusion target according to the tracking ID of the alarm target. On one hand, the change detection range can cover all kinds of foreign object targets, the detection rate is higher, and the false alarm rate is greatly reduced by alarm filtering; on the other hand, tracing the invasion target, foreign matter invasion improvement can be carried out from the invasion source.

Description

Railway foreign matter intrusion detection system and method based on deep learning
Technical Field
The application belongs to the technical field of railway foreign matter intrusion detection, and particularly relates to a railway foreign matter intrusion detection system and method based on deep learning.
Background
With the development of railway construction in China, the operating mileage of China reaches 13.1 ten thousand kilometers by 2018, wherein the operating mileage of high-speed rail reaches 2.9 ten thousand kilometers. With the large-scale speed increasing of railways, the railway perimeter safety problem is increasingly prominent. The invasion of foreign matters on the periphery of the railway can cause large-area late railway transportation, even serious casualties and economic losses. Aiming at the occurrence of destructive activities along the railway and related facilities, serious hidden danger and threat are brought to the railway operation safety, and the railway perimeter safety problem becomes the important issue of railway safety management work. At present, railway management departments adopt various technical protection means aiming at railway perimeter safety, wherein perimeter intrusion behavior detection based on video monitoring is a main detection means adopted at present. With the explosive growth of video monitoring data and higher security requirements, current video analysis methods have difficulty in meeting the requirements of remote or small target detection.
In the running process of the railway train, the problem of foreign matter invasion on the railway line, such as personnel, falling rocks, debris flows and the like, can occur due to artificial or natural reasons. These foreign bodies, if not found and removed in time, can pose a serious threat to railway safety. The traditional railway foreign matter intrusion detection method mainly depends on manual inspection and observation of locomotive drivers, and is low in efficiency and has blind areas.
For a railway disaster prevention system, the foreign matters refer to barriers on a railway, which influence driving safety, and are foreign matters which influence driving around a railway line and fall off or are brought by disaster events. Such foreign objects, which may have an impact and threat to railway traffic safety, may be static, dynamic, fluid, solid, of unknown types of foreign objects, of unknown morphology, of unknown motion state. At present, foreign matter intrusion detection techniques are mainly classified into two types: moving object detection and object detection. However, moving object detection cannot detect a stationary object, and there are many false positives; target detection cannot identify unknown targets, relies on sample training, and is high in maintenance cost.
The railway foreign matter intrusion detection system at the present stage has the technical problems of more false alarms, weak real-time performance and incapability of covering all kinds of foreign matters.
Disclosure of Invention
Based on the technical problems in the prior art, the application provides a railway foreign matter intrusion detection system and a railway foreign matter intrusion detection method based on deep learning, which are used for solving the technical problems that the existing railway foreign matter intrusion detection system has a plurality of false alarms, is not strong in real-time performance and cannot cover all types of foreign matters.
In order to solve the technical problems, the application adopts the following technical scheme:
in one aspect, there is provided a railway foreign matter intrusion detection system based on deep learning, comprising:
the video acquisition module is used for acquiring railway limit video images;
the limit marking module is used for marking a limit area on the railway limit video image;
the foreign matter intrusion detection module is used for detecting an unknown class target and a known class target;
the data processing module is used for carrying out target tracking and alarm filtering processing according to the detection result of the foreign matter intrusion detection module and transmitting the filtered alarm information to the early warning module;
the early warning module is used for obtaining an alarm target according to the alarm information acquired from the data processing module and giving an alarm;
and the intrusion target tracing module is used for tracing the intrusion target according to the tracking ID of the alarm target of the early warning module, wherein the intrusion target is the alarm target.
Preferably, the video acquisition module adopts a camera erected along the periphery of the track;
the boundary marking module, the foreign matter intrusion detection module, the data processing module, the early warning module and the intrusion target tracing module are arranged in the edge equipment.
Preferably, the foreign object intrusion detection module detects an unknown class target based on a change detection algorithm of deep learning, and detects a known class target by using a target detection algorithm;
the system also comprises a matching fusion module, wherein the matching change detection algorithm and the target detection algorithm detect the same target.
Preferably, the foreign object invasion detection module uniformly samples 20 frames of images in the first 10 seconds and calculates pixel averages to obtain recent history frames, uniformly samples 60 frames of images in the first 1 hour and calculates pixel averages to obtain early history frames.
Preferably, the foreign object intrusion detection module calculates the change characteristics of the current frame, the recent history frame and the early history frame, and integrates the change characteristics of different time periods to detect a moving foreign object which changes in a short period and a stationary foreign object which changes in a long period.
Preferably, in the specific process of calculating the change characteristics of the current frame, the recent historical frame and the early historical frame, the foreign object intrusion detection module is based on a codec as a backbone network, the encoder is responsible for extracting useful characteristic representations from the input image, and the decoder is responsible for mapping the characteristics extracted by the encoder back to the original image space; a non-local feature pyramid network is used as an enhanced feature extraction and fusion module, and a feature fusion module based on dense connection is used for fusing the features of the double temporal images.
Preferably, the data processing module is provided with a filtering rule, the target is continuously tracked by a target tracking algorithm, and repeated alarms of the same target are filtered by the filtering rule.
On the other hand, the railway foreign matter intrusion detection method based on deep learning is provided, and is applied to the railway foreign matter intrusion detection system based on deep learning, and comprises the following steps:
s10: acquiring railway limit video images;
s11: marking a railway limit video image as a limit area;
s12: detecting an unknown class target and a known class target;
s13: performing target tracking and alarm filtering according to the detection result of the foreign matter intrusion detection module, and transmitting the filtered alarm information to the early warning module;
s14: obtaining an alarm target according to the processing result obtained from the data processing module, and giving an alarm;
s15: and the intrusion target tracing module is used for tracing the intrusion target according to the tracking ID of the alarm target of the early warning module.
Preferably, in step S12, the following specific steps are included:
s120: detecting known class targets such as falling rocks, personnel, animals, vehicles and the like by using a yolov5 target detection algorithm;
s121: detecting dynamic change targets, including known category targets and unknown category targets, by a change detection algorithm based on deep learning;
wherein S120 and S121 are performed simultaneously.
Preferably, step S13 comprises the following specific steps:
s130: the IoU size of the change detection and target detection alarm frame is calculated, the same target is matched when the size is larger than a threshold value, the coordinate and the category of the target detection alarm frame are output, and otherwise, the coordinate and the category of the change detection alarm frame are output;
s131: continuously tracking the target by using a target tracking algorithm, and marking the same tracking ID for an alarm frame of the same target;
s132: analyzing and judging whether the alarm type is a train, and if so, filtering the current alarm;
s133: in the 10-frame window, if the target alarm of the same tracking ID is less than 5 frames, filtering the current alarm;
s134: and if the same tracking ID target outputs an alarm before, filtering the current alarm.
The beneficial effects of the application include:
1. the dynamic target is detected by the foreign object intrusion detection module based on a change detection algorithm of deep learning, the dynamic target comprises an unknown class target, the known class target is detected by using the target detection algorithm, the dynamic target further comprises a matching fusion module, the same target detected by the change detection algorithm and the target detection algorithm is matched, the detection range can cover the whole class of foreign object targets, and compared with the target detection method, the detection rate is higher, and the universality is stronger.
2. And the data processing module is used for carrying out target tracking and alarm filtering processing, so that false alarm can be greatly reduced, and the practicability of the detection system is improved.
3. And outputting the track of the historical frame of the same target according to the tracking ID of the alarm target through an intrusion target tracing module, tracing the intrusion target, and improving foreign matter intrusion from an intrusion source.
Drawings
Fig. 1 is an overall algorithm flow chart of the deep learning-based railway foreign matter intrusion detection system.
Fig. 2 is a schematic diagram of an original FCCDN network structure in the prior art according to the present application.
Fig. 3 is a schematic diagram of an improved FCCDN network structure of the deep learning-based railroad foreign object intrusion detection system of the present application.
Fig. 4 is an overall flowchart of the deep learning-based railway foreign matter intrusion detection method of the present application.
Fig. 5 is a specific flowchart in S12 of the method for detecting railway foreign matter intrusion based on deep learning according to the present application.
FIG. 6 is a flowchart of the method for detecting railway foreign matter intrusion based on deep learning in S13
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. The components of the embodiments of the present application generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the application, as presented in the figures, is not intended to limit the scope of the application, as claimed, but is merely representative of selected embodiments of the application. All other embodiments, which can be made by a person skilled in the art without making any inventive effort, are intended to be within the scope of the present application.
The present application will present various aspects, embodiments, or features about a system that may include a plurality of devices, components, modules, etc. It is to be understood and appreciated that the various systems may include additional devices, components, modules, etc. and/or may not include all of the devices, components, modules etc. discussed in connection with the figures. Furthermore, combinations of these schemes may also be used.
In addition, in the embodiments of the present application, words such as "exemplary," "for example," and the like are used to indicate an example, instance, or illustration. Any embodiment or design described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments or designs. Rather, the term use of an example is intended to present concepts in a concrete fashion.
In the embodiment of the present application, "information", "signal", "message", "channel", and "signaling" may be used in a mixed manner, and it should be noted that the meaning of the expression is consistent when the distinction is not emphasized. "of", "corresponding" and "corresponding" are sometimes used in combination, and it should be noted that the meaning of the expression is consistent when the distinction is not emphasized.
The application is further described in detail below with reference to fig. 1 to 6:
example 1
A deep learning based railroad foreign object intrusion detection system comprising:
the video acquisition module comprises cameras which are erected along the periphery of the railway track, the cameras acquire video images and transmit the acquired video images to the edge equipment in real time; the effective distance range of the cameras and the edge devices arranged along the periphery of the railway track is 130-165 m, and as the effective distance is 165m at the longest distance, one set of the cameras and the edge devices are arranged every 165 meters.
The limit marking module is used for marking the limit area by dividing the track by using the semantic division model, and detecting and alarming foreign matter invasion only in the limit area, because only foreign matter invasion in the limit area can influence railway safety.
The foreign object intrusion detection module detects dynamic change targets based on a change detection algorithm of deep learning, wherein the dynamic change targets comprise unknown category targets, the target detection algorithm is used for detecting known category targets, the same targets detected by the change detection algorithm and the target detection algorithm are matched, the detection range can cover all category foreign object targets, and compared with the target detection method, the detection rate is higher, and the universality is stronger.
And the data processing module is used for carrying out target tracking and alarm filtering according to the detection result of the foreign matter intrusion detection module, transmitting the filtered alarm information to the early warning module, and greatly reducing false alarm and improving the practicability of the detection system.
The early warning module is used for alarming according to the processing result obtained from the data processing module, and allocating the staff closest to the alarm ID to arrive at the site for processing the invasion foreign matters according to the alarm condition and the alarm ID provided by the alarm module.
And the intrusion target tracing module is used for tracing the intrusion target according to the tracking ID of the alarm target of the early warning module, so as to realize the improvement of foreign matter intrusion from the intrusion source.
Referring to fig. 2, the original change detection FCCDN model compares two remote sensing images and detects a change between them. Referring to fig. 3, the technical scheme of the present application improves the network structure of the original change detection FCCDN model, adds one input branch, changes double-flow input into three-flow input, and can calculate the change characteristics of the current frame, the recent history frame and the early history frame, and finally synthesizes the change characteristics of different time periods for detecting the moving foreign object of short-term change and the stationary foreign object of long-term change.
The foreign matter intrusion detection module is matched with a change detection algorithm and a target detection algorithm, the change detection is used for detecting dynamic targets in the limited area, including unknown class targets, and the target detection is used for detecting known class targets, such as personnel, falling rocks, debris flows and the like. Therefore, the foreign object intrusion detection module can detect unknown type targets and known type targets, the two algorithms complement each other, and the detection range can cover all types of foreign object targets. The system also comprises a matching fusion module, wherein the matching change detection algorithm and the target detection algorithm detect the same target. The method is suitable for scene changes such as railway shooting environment, illumination, bad weather and the like, a model is retrained without accumulating samples after scene switching, a stable detection effect can be still maintained, and the universality is strong.
The foreign matter intrusion detection module uniformly samples 20 frames of images in the first 10 seconds and calculates the average of pixels to obtain a recent historical frame, uniformly samples 60 frames of images in the first 1 hour and calculates the average of pixels to obtain an early historical frame, the data processing module respectively calculates the change characteristics of the current frame, the recent historical frame and the early historical frame, finally synthesizes the change characteristics of different time periods, detects a short-term change moving foreign matter target and a long-term change stationary foreign matter target.
The foreign matter intrusion detection module is mainly based on a codec (dual encoder-decoder) as a backbone network in the specific process of calculating the change characteristics of the current frame, the recent history frame and the early history frame, the encoder is responsible for extracting useful characteristic representations from an input image, and the decoder is responsible for mapping the characteristics extracted by the encoder back to an original image space; a non-local feature pyramid network (nonlocal feature pyramid network) serves as an enhanced feature extraction and fusion module, and a dense connection-based feature fusion module (dense connection-based feature fusion module) is used for fusing bi-temporal image features.
And detecting unknown foreign matter invasion by integrating characteristic changes of the current frame, the recent historical frame and the early historical frame. Compared with the moving object detection method, the change detection method can detect a static or slow moving object, and greatly reduces false alarm under the scenes of camera shake, light and shadow change, dynamic background, bad weather and the like. The change detection algorithm can avoid sample accumulation and frequent model update aiming at specific scenes, and the maintenance cost is low.
The data processing module calculates the IoU size of the change detection and target detection alarm frame, and the size is larger than the threshold value, namely the size is matched with the same target, the size is output by the coordinates and the types of the target detection alarm frame, and otherwise, the size is output by the coordinates and the types of the change detection alarm frame.
The data processing module continuously tracks the target by using a target tracking algorithm, marks the same tracking ID for the alarm frame of the same target, judges whether the alarm type is a train or not, and if the alarm type is the train, filters the current alarm. Intuitively, the real foreign matter invasion is avoided before and after the train passes.
And in a 10-frame window, if the target alarm of the same tracking ID is less than 5 frames, filtering the current alarm. Intuitively, the limit-intrusion target finally stays in the limit area, and multi-frame alarm is carried out within a certain time range. The single frame alarm may be a false alarm such as rain and snow, winged insect, light and shadow change, etc., and may be filtered.
And if the same tracking ID target outputs an alarm before, filtering the current alarm. Intuitively, one target only alarms once, and the continuous false alarm of the same target can be reduced.
And outputting the track of the historical frame of the same target according to the tracking ID of the alarm target, tracing the intrusion target, and facilitating improvement from the intrusion source (illegal intruder tracing and the like).
Example 2
Based on embodiment 1, referring to fig. 1 and 4, the present embodiment provides a railway foreign matter intrusion detection method based on deep learning, including the steps of:
s10: acquiring railway limit video images;
s11: marking a railway limit video image as a limit area;
s12: detecting an unknown class target and a known class target;
s13: performing target tracking and alarm filtering according to the detection result of the foreign matter intrusion detection module, and transmitting the filtered alarm information to the early warning module;
s14: obtaining an alarm target according to the processing result obtained from the data processing module, and giving an alarm;
s15: and the intrusion target tracing module is used for tracing the intrusion target according to the tracking ID of the alarm target of the early warning module.
Referring to fig. 1 and 5, step S12 includes the following specific steps:
s120: detecting known class targets such as falling rocks, personnel, animals, vehicles and the like by using a yolov5 target detection algorithm;
s121: the change detection algorithm based on deep learning detects dynamic change targets, including known category targets and unknown category targets.
Referring to fig. 1 and 6, step S13 includes the following specific steps:
s130: the IoU size of the change detection and target detection alarm frame is calculated, the same target is matched when the size is larger than a threshold value, the coordinate and the category of the target detection alarm frame are output, and otherwise, the coordinate and the category of the change detection alarm frame are output;
s131: continuously tracking the target by using a target tracking algorithm, and marking the same tracking ID for an alarm frame of the same target;
s132: analyzing and judging whether the alarm type is a train, and if so, filtering the current alarm;
s133: in the 10-frame window, if the target alarm of the same tracking ID is less than 5 frames, filtering the current alarm;
s134: and if the same tracking ID target outputs an alarm before, filtering the current alarm.
Wherein, aiming at false alarm of the detection system, the application adds several filtering rules according to the prior criterion: filtering repeated alarms of the same target through target tracking; filtering the alarm of the train before and after passing; and filtering the alarm of quick crossing or short stay. Through the above-mentioned a series of filtering rules that set up, can reduce the false alarm rate by a wide margin, provide the practical level of whole foreign matter intrusion detection system.
In summary, according to the method for detecting the railway foreign matter intrusion based on deep learning provided by the application, on one hand, the limit marking module uses the semantic segmentation model to segment the track and mark the limit area, the foreign matter intrusion detection module uses the change detection algorithm and the target detection algorithm to detect and carry out foreign matter intrusion detection, the detection range can cover all types of foreign matter targets, and compared with the target detection method, the detection rate is higher, and the universality is stronger; the data processing module continuously tracks the target and carries out alarm filtering processing through a target tracking algorithm, so that the false alarm rate is greatly reduced; in still another aspect, the intrusion target tracing module outputs the track of the same target history frame according to the tracking ID of the alarm target, and tracing the intrusion target, so that foreign matter intrusion improvement can be performed from the intrusion source.
The above examples merely illustrate specific embodiments of the application, which are described in more detail and are not to be construed as limiting the scope of the application. It should be noted that it is possible for a person skilled in the art to make several variants and modifications without departing from the technical idea of the application, which fall within the scope of protection of the application.

Claims (10)

1. Railway foreign matter intrusion detection system based on deep learning, characterized by comprising:
the video acquisition module is used for acquiring railway limit video images;
the limit marking module is used for marking a limit area on the railway limit video image;
the foreign matter intrusion detection module is used for detecting an unknown class target and a known class target;
the data processing module is used for carrying out target tracking and alarm filtering processing according to the detection result of the foreign matter intrusion detection module and transmitting the filtered alarm information to the early warning module;
the early warning module is used for obtaining an alarm target according to the alarm information acquired from the data processing module and giving an alarm;
and the intrusion target tracing module is used for tracing the intrusion target according to the tracking ID of the alarm target of the early warning module, wherein the intrusion target is the alarm target.
2. The deep learning based railroad foreign object intrusion detection system of claim 1, wherein the video acquisition module employs cameras mounted along the perimeter of the track;
the boundary marking module, the foreign matter intrusion detection module, the data processing module, the early warning module and the intrusion target tracing module are arranged in the edge equipment.
3. The deep learning-based railway foreign object intrusion detection system according to claim 1, wherein the foreign object intrusion detection module detects an unknown class of targets based on a deep learning variation detection algorithm, and detects a known class of targets using a target detection algorithm;
the system also comprises a matching fusion module, wherein the matching change detection algorithm and the target detection algorithm detect the same target.
4. The deep learning based railroad foreign object intrusion detection system of claim 1, wherein the foreign object intrusion detection module uniformly samples 20 frames of images and calculates a pixel average for the first 10 seconds to obtain a recent history frame, uniformly samples 60 frames of images and calculates a pixel average for the first 1 hour to obtain an early history frame.
5. The deep learning based railway foreign object intrusion detection system according to claim 4, wherein the foreign object intrusion detection module calculates variation characteristics of a current frame and a recent history frame and an early history frame, respectively, and integrates the variation characteristics of different periods, detects a moving foreign object subject to short-term variation, and a stationary foreign object subject to long-term variation.
6. The deep learning based railroad foreign object intrusion detection system according to claim 5, wherein the foreign object intrusion detection module is based on a codec as a backbone network in a specific process of calculating the change characteristics of the current frame and the recent history frame and the early history frame, the encoder is responsible for extracting useful characteristic representations from the input image, and the decoder is responsible for mapping the characteristics extracted by the encoder back to the original image space; a non-local feature pyramid network is used as an enhanced feature extraction and fusion module, and a feature fusion module based on dense connection is used for fusing the features of the double temporal images.
7. The railway foreign matter intrusion detection system based on deep learning according to claim 1, wherein the data processing module is provided with a filtering rule, continuously tracks the target through a target tracking algorithm, and filters repeated alarms of the same target through the filtering rule.
8. A railway foreign matter intrusion detection method based on deep learning, which is applied to the railway foreign matter intrusion detection system based on deep learning as claimed in any one of claims 1 to 7, and is characterized by comprising the following steps:
s10: acquiring railway limit video images;
s11: marking a railway limit video image as a limit area;
s12: detecting an unknown class target and a known class target;
s13: performing target tracking and alarm filtering according to the detection result of the foreign matter intrusion detection module, and transmitting the filtered alarm information to the early warning module;
s14: obtaining an alarm target according to the processing result obtained from the data processing module, and giving an alarm;
s15: and outputting the track of the historical frame of the same target according to the tracking ID of the alarm target, and tracing the intrusion target.
9. The method for detecting railway foreign matter intrusion based on deep learning according to claim 8, wherein in step S12, the method comprises the following specific steps:
s120: detecting known class targets such as falling rocks, personnel, animals, vehicles and the like by using a yolov5 target detection algorithm;
s121: detecting dynamic change targets, including known category targets and unknown category targets, by a change detection algorithm based on deep learning;
wherein S120 and S121 are performed simultaneously.
10. The deep learning-based railway foreign matter intrusion detection system and method according to claim 8, wherein step S13 includes the following specific steps:
s130: the IoU size of the change detection and target detection alarm frame is calculated, the same target is matched when the size is larger than a threshold value, the coordinate and the category of the target detection alarm frame are output, and otherwise, the coordinate and the category of the change detection alarm frame are output;
s131: continuously tracking the target by using a target tracking algorithm, and marking the same tracking ID for an alarm frame of the same target;
s132: analyzing and judging whether the alarm type is a train, and if so, filtering the current alarm;
s133: in the 10-frame window, if the target alarm of the same tracking ID is less than 5 frames, filtering the current alarm;
s134: and if the same tracking ID target outputs an alarm before, filtering the current alarm.
CN202310558280.7A 2023-05-17 2023-05-17 Railway foreign matter intrusion detection system and method based on deep learning Pending CN116597394A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310558280.7A CN116597394A (en) 2023-05-17 2023-05-17 Railway foreign matter intrusion detection system and method based on deep learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310558280.7A CN116597394A (en) 2023-05-17 2023-05-17 Railway foreign matter intrusion detection system and method based on deep learning

Publications (1)

Publication Number Publication Date
CN116597394A true CN116597394A (en) 2023-08-15

Family

ID=87598643

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310558280.7A Pending CN116597394A (en) 2023-05-17 2023-05-17 Railway foreign matter intrusion detection system and method based on deep learning

Country Status (1)

Country Link
CN (1) CN116597394A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117011322A (en) * 2023-09-26 2023-11-07 天津七一二移动通信有限公司 Railway intrusion object identification method based on image

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117011322A (en) * 2023-09-26 2023-11-07 天津七一二移动通信有限公司 Railway intrusion object identification method based on image
CN117011322B (en) * 2023-09-26 2024-02-23 天津七一二移动通信有限公司 Railway intrusion object identification method based on image

Similar Documents

Publication Publication Date Title
KR101095528B1 (en) An outomatic sensing system for traffic accident and method thereof
CN105809679B (en) Mountain railway side slope rockfall detection method based on visual analysis
CN111626203B (en) Railway foreign matter identification method and system based on machine learning
CN103106766A (en) Forest fire identification method and forest fire identification system
CN111507235B (en) Railway perimeter foreign matter intrusion detection method based on video
KR20060031832A (en) A smart visual security system based on real-time behavior analysis and situation cognizance
CN104866827A (en) Method for detecting people crossing behavior based on video monitoring platform
CN106845325B (en) A kind of information detecting method and device
CN106355162A (en) Method for detecting intrusion on basis of video monitoring
CN108986472A (en) One kind turns around vehicle monitoring method and device
CN112776856A (en) Track foreign matter intrusion monitoring method, device and system and monitoring host equipment
CN116597394A (en) Railway foreign matter intrusion detection system and method based on deep learning
CN113484858A (en) Intrusion detection method and system
CN104881643A (en) Method and system for rapidly detecting remains
CN112287823A (en) Facial mask identification method based on video monitoring
CN103152558B (en) Based on the intrusion detection method of scene Recognition
CN112257683A (en) Cross-mirror tracking method for vehicle running track monitoring
CN116311727A (en) Intrusion response method, device, equipment and readable storage medium
Wang et al. Vision-based highway traffic accident detection
CN113963373A (en) Video image dynamic detection and tracking algorithm based system and method
CN116311071A (en) Substation perimeter foreign matter identification method and system integrating frame difference and CA
CN105338304A (en) Tunnel operation safety incident detection system based on video recognition
CN104408942A (en) Intelligent vehicle speed measuring device and method
CN114299413A (en) Image acquisition early warning system for constructional engineering
CN116485799B (en) Method and system for detecting foreign matter coverage of railway track

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination