CN112686107A - Tunnel invading object detection method and device - Google Patents
Tunnel invading object detection method and device Download PDFInfo
- Publication number
- CN112686107A CN112686107A CN202011522501.8A CN202011522501A CN112686107A CN 112686107 A CN112686107 A CN 112686107A CN 202011522501 A CN202011522501 A CN 202011522501A CN 112686107 A CN112686107 A CN 112686107A
- Authority
- CN
- China
- Prior art keywords
- tunnel
- scale feature
- information
- feature map
- position information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 52
- 238000000605 extraction Methods 0.000 claims abstract description 42
- 230000004927 fusion Effects 0.000 claims abstract description 41
- 238000012545 processing Methods 0.000 claims abstract description 35
- 238000012544 monitoring process Methods 0.000 claims abstract description 19
- 238000000034 method Methods 0.000 claims abstract description 15
- 238000004590 computer program Methods 0.000 claims description 11
- 238000004891 communication Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 5
- 230000007547 defect Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000001537 neural effect Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 241001465754 Metazoa Species 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000013138 pruning Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Images
Landscapes
- Image Analysis (AREA)
Abstract
The invention provides a tunnel intrusion object detection method and a tunnel intrusion object detection device. The method comprises the following steps: processing key frames of the tunnel monitoring video based on a lightweight feature extraction network to extract a multi-scale feature map of the tunnel monitoring video; processing the multi-scale feature map based on a multi-scale feature fusion network to obtain information of rail vehicles and pedestrians in the multi-scale feature map; processing the key frame based on a background difference algorithm to acquire position information of a moving object; and determining whether an invading object exists in the tunnel according to the information of the rail vehicles and the pedestrians and the position information of the moving object. The tunnel invading object detection method and the device provided by the invention can accurately detect the tunnel invading object, greatly reduce the misjudgment probability of the invading object in the tunnel, and provide guarantee for rapidly eliminating the potential safety hazard in the tunnel.
Description
Technical Field
The invention relates to the technical field of rail transit safety, in particular to a tunnel invading object detection method and a tunnel invading object detection device.
Background
For railway tunnels built between mountain lands with steep terrain, the railway tunnels are easily affected by geological disasters, and once safety accidents occur, the rescue difficulty is extremely high, so that the safety of driving and passengers in the tunnels is seriously threatened.
Due to the outbreak and unpredictability of tunnel disasters, the regular manual overhaul and safety inspection measures cannot meet the current safety requirements. With the rapid development of computer vision and artificial intelligence, researches for detecting and judging invading foreign matters based on an intelligent video analysis technology instead of manpower have been provided.
At present, a background difference mode is commonly used for analyzing a monitoring video of a tunnel. But because track vehicle frequently passes through in the tunnel, the indeterminate construction of maintainer etc. all can cause the change of tunnel environment, consequently a large amount of wrong reports can be aroused to current background difference mode, seriously influences early warning system normal work.
Therefore, how to provide a method can accurately detect the invading object in the tunnel, thereby eliminating the potential safety hazard and having very important significance.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides a tunnel intrusion object detection method and a tunnel intrusion object detection device.
The invention provides a tunnel intrusion object detection method, which comprises the following steps:
processing key frames of the tunnel monitoring video based on a lightweight feature extraction network to extract a multi-scale feature map of the tunnel monitoring video;
processing the multi-scale feature map based on a multi-scale feature fusion network to obtain information of rail vehicles and pedestrians in the multi-scale feature map;
processing the key frame based on a background difference algorithm to acquire position information of a moving object;
and determining whether an invading object exists in the tunnel according to the information of the rail vehicles and the pedestrians and the position information of the moving object.
In one embodiment, the processing, by the lightweight feature extraction-based network, the key frames of the tunnel surveillance video to extract the multi-scale feature map of the tunnel surveillance video includes:
cutting a target area in the key frame based on the lightweight feature extraction network;
performing convolution operation on the cut image to obtain the multi-scale feature map;
the multi-scale feature map comprises multi-layer images with different resolutions, and each layer of image comprises a spatial feature representing the position of an object and/or a semantic feature representing the category of the object.
In one embodiment, the processing the multi-scale feature map based on the multi-scale feature fusion network to obtain the rail vehicle and pedestrian information in the multi-scale feature map comprises:
fusing the spatial features of the images of each layer based on the multi-scale feature fusion network to determine the position information of the object in the target area;
fusing the semantic features of each layer of image to determine object category information corresponding to the position information;
and determining the rail vehicle and pedestrian information in the multi-scale feature map according to the position information and the object category information.
In one embodiment, processing the keyframes based on a background subtraction algorithm to obtain moving object information comprises:
determining a foreground and a background of the target region based on the background difference algorithm;
the foreground is filtered to determine moving object position information indicative of a position of the moving object in the target region.
In one embodiment, the determining whether there is an intruding object in the tunnel according to the rail vehicle and pedestrian information and the moving object position information comprises:
matching the positions corresponding to the rail vehicles and the pedestrians with the positions of the moving objects;
and if the moving object position which is not matched with the positions corresponding to the railway vehicle and the pedestrian exists, determining the moving object corresponding to the non-matched moving object position as the intrusion object.
The present invention also provides a tunnel intrusion object detection device, including:
the extraction module is used for processing key frames of the tunnel monitoring video based on the lightweight feature extraction network so as to extract a multi-scale feature map of the tunnel monitoring video;
the fusion module is used for processing the multi-scale feature map based on a multi-scale feature fusion network so as to obtain the information of the rail vehicles and the pedestrians in the multi-scale feature map;
the difference module is used for processing the key frame based on a background difference algorithm to acquire the position information of the moving object;
and the determining module is used for determining whether an invading object exists in the tunnel according to the information of the rail vehicles and the pedestrians and the position information of the moving object.
In one embodiment, the extraction module is specifically configured to:
cutting a target area in the key frame based on the lightweight feature extraction network;
performing convolution operation on the cut image to obtain the multi-scale feature map;
the multi-scale feature map comprises multi-layer images with different resolutions, and each layer of image comprises a spatial feature representing the position of an object and/or a semantic feature representing the category of the object.
In one embodiment, the fusion module is specifically configured to:
fusing the spatial features of the images of each layer based on the multi-scale feature fusion network to determine the position information of the object in the target area;
fusing the semantic features of each layer of image to determine object category information corresponding to the position information;
determining rail vehicle and pedestrian information in the multi-scale feature map according to the position information and the object category information
The invention further provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the computer program to realize the steps of any one of the above-mentioned tunnel intrusion object detection methods.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of any of the above-described tunnel intrusion object detection methods.
According to the tunnel invading object detection method and device provided by the invention, the information of the rail vehicles and the pedestrians is determined by using the lightweight characteristic extraction network and the multi-scale characteristic fusion network, and the invading object in the tunnel is determined by combining the moving object position information determined by the differential algorithm, so that the tunnel invading object can be accurately detected, the misjudgment probability of the invading object in the tunnel is greatly reduced, and the guarantee is provided for quickly eliminating the potential safety hazard in the tunnel.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
Fig. 1 is a schematic flow chart of a tunnel intrusion object detection method according to the present invention;
fig. 2 is a schematic structural diagram of a tunnel intrusion object detection device provided by the present invention;
fig. 3 is a schematic structural diagram of an electronic device provided in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is a schematic flow chart of a tunnel intrusion object detection method provided by the present invention. Referring to fig. 1, the method for detecting an object invaded in a tunnel according to the present invention may include:
s110, processing key frames of the tunnel monitoring video based on the lightweight feature extraction network to extract a multi-scale feature map of the tunnel monitoring video;
s120, processing the multi-scale feature map based on the multi-scale feature fusion network to obtain information of the rail vehicles and the pedestrians in the multi-scale feature map;
s130, processing the key frame based on a background difference algorithm to acquire position information of the moving object;
and S140, determining whether an invading object exists in the tunnel according to the information of the rail vehicles and the pedestrians and the position information of the moving object.
The execution subject of the tunnel intrusion object detection method provided by the invention can be electronic equipment, a component in the electronic equipment, an integrated circuit or a chip. The electronic device may be a mobile electronic device or a non-mobile electronic device. By way of example, the mobile electronic device may be a mobile phone, a tablet computer, a notebook computer, a palm top computer, a vehicle-mounted electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a Personal Digital Assistant (PDA), and the like, and the non-mobile electronic device may be a server, a Network Attached Storage (NAS), a personal computer (personal computer, PC), a Television (TV), a teller machine, a self-service machine, and the like, and the present invention is not limited in particular.
Specifically, all the tunnel surveillance video data of the surveillance area may be acquired first, and the RSTP stream is decoded using FFMPEG, for example, a 25-frame image may be processed in one second.
In order to reduce the redundancy degree of processing and improve the detection efficiency, the video can be subjected to frame extraction processing: extracting key frames at 2 frames/second, wherein the key Frame set is Frame ═ X1,X2,X3,……，Xi,Xn}. Then, the extracted keyframes may be preprocessed, for example, image denoising and image enhancement are performed.
After the key frame is obtained, the key frame can be processed according to the lightweight feature extraction network so as to extract the multi-scale feature map. The neural network parameters of the lightweight feature extraction network are less, so that the light-weight feature extraction network is beneficial to engineering deployment.
Meanwhile, the keyframes may be processed to determine the location information of the moving object based on a background difference algorithm.
The multi-scale feature map may then be processed using a multi-scale feature fusion network to determine rail vehicle and pedestrian information in the multi-scale feature map. The multi-scale feature fusion network has the advantage of efficiently detecting objects in a complex tunnel environment.
And finally, determining whether an invasive object exists in the tunnel by comparing the position information of the moving object determined by the background difference algorithm with the information of the rail vehicles and pedestrians determined by the multi-scale feature fusion network.
According to the tunnel invading object detection method, the information of the rail vehicles and the pedestrians is determined by using the lightweight feature extraction network and the multi-scale feature fusion network, and the moving object position information determined by the differential algorithm is combined to determine the invading object in the tunnel, so that the tunnel invading object can be accurately detected, the misjudgment probability of the invading object in the tunnel is greatly reduced, and the safety hazard in the tunnel is rapidly eliminated.
Further, in one embodiment, step S110 may include:
cutting a target area in the key frame based on the lightweight feature extraction network;
performing convolution operation on the cut image to obtain a multi-scale characteristic diagram;
the multi-scale feature map comprises multi-layer images with different resolutions, and each layer of image comprises a spatial feature representing the position of an object and/or a semantic feature representing the category of the object.
The target area may be an area in which the rail is located, for example, an area within about 1m of the rail; but also other areas that have a significant impact on the safe operation of the rail vehicle. The present invention is not particularly limited in this regard.
In this embodiment, the lightweight feature extraction network adopted by the present invention uses separable convolution instead of the conventional convolution kernel in the normal feature extraction network, thereby achieving an accurate feature extraction effect with a small number of parameters.
For example, a separable convolution is used instead of the original conventional convolution kernel, and a similar feature extraction effect is achieved with a smaller parameter number.
For example, replacing the k × k convolution kernel with a 1 × k and k × 1 distributed convolution effectively reduces k2-a parameter quantity of 2 k.
The lightweight feature extraction network adopted by the invention also combines an attention mechanism to learn the weight distribution features of the channel layer and compress the feature channels.
For example, through training, the network learns that the probability of the rail vehicles and pedestrians appearing in a certain scene or a certain position is high, and the weight of the corresponding space coordinate is gained. The probability of the object of one type appearing is extremely low, and simultaneously, the object of another type appears, and the channel represented by the object can be merged with the object of another type.
And aiming at each feature extraction layer, obtaining an optimal unit structure under the above strategy by using neural architecture search.
After the target area is cut, the lightweight feature extraction network can perform convolution operation on the cut image to determine the multi-scale feature map.
For example, after performing convolution operation on the clipped image, a feature map set P with gradually expanding top-down scale can be generatedi(i ═ 1,2,3,4, 5). Wherein P is1Layer comprisesThe space characteristics are more, so that the space position of a certain object can be calibrated conveniently; and P is5The layer contains richer semantic features for determining the specific category of objects at a certain spatial location.
According to the tunnel invading object detection method, the multi-scale feature map is obtained through the lightweight feature extraction network, so that the information of rail vehicles and pedestrians can be conveniently determined according to the multi-scale feature fusion network, and the efficiency of finally detecting the tunnel invading object is improved.
Further, in one embodiment, step S120 may include:
fusing the spatial features of each layer of image based on a multi-scale feature fusion network to determine the position information of the object in the target area;
fusing semantic features of each layer of image to determine object category information corresponding to the position information;
and determining the information of the rail vehicles and the pedestrians in the multi-scale feature map according to the position information and the object category information.
It should be noted that, under a fixed camera angle, objects in the tunnel have different dimensions, and an object in a region far from the camera has a small image, so that the detection effect is poor.
Therefore, the invention adopts the multi-scale feature fusion network to effectively fuse the multi-resolution feature map generated by the feature extraction network, and improves the detection performance of small-scale objects, specifically:
after feature graphs under multiple resolutions generated by the lightweight feature extraction network are input into the multi-scale feature fusion network, the multi-scale feature fusion network learns the locally optimal feature fusion unit in a neural architecture search mode, and spatial features of a positioned object and semantic features for judging object attributes are fully expressed.
For example, for a rail vehicle with distinct pixel characteristics, P3Or P4The layer is sufficient to give accurate position information, and the semantic features of the rail vehicle are P2Under the condition that the confidence of the layer is high enough, the relevance among other layers can be abandoned, and the multi-scale features can be effectively reducedAnd fusing the parameters of the network.
And sparsifying the learned fusion unit, pruning the connection with smaller contribution weight, and further reducing the parameter number of the fusion network.
And repeating the iterative fusion unit to improve the multi-scale expression capability of the features. The position and the type of the object are input into one convolution layer respectively, and the detection result of the object is output.
For example, convolution before outputting the position of a pedestrian or a rail vehicle is responsible for learning the expression of spatial position information in the feature map, and convolution before outputting the attribute of the pedestrian or the category is responsible for learning the expression of category attribute information in the feature map.
According to the tunnel invading object detection method, the information of the rail vehicles and the pedestrians in the multi-scale feature map is obtained by adopting the multi-scale feature fusion network, the defect that the object detection effect of the region far away from the camera is poor in the prior art is overcome, and the detection precision of the tunnel invading object is obviously improved.
Further, in one embodiment, step S130 may include:
determining the foreground and the background of the target area based on a background difference algorithm;
the foreground is filtered to determine moving object position information indicative of a position of the moving object in the target region.
The background difference algorithm takes a key frame set as input and outputs the moving foreground regions of the front frame and the rear frame of the image. For example: moving trains, passing pedestrians or animals, and areas in tunnels where foreign objects fall.
Specifically, the input key Frame information set is Frame ═ { X ═ X1,X2,X3,……，Xi,XnDifferentiating adjacent frames to form a subframe ═ S1,S2,S3,……，Si-1,Sn-1And (4) interframe information.
Modeling the background using a mixed gaussian model:
wherein, N (x | mu)k,Σk) Represents the mean value of μkVariance is ΣkA gaussian distribution of (a).
Setting a threshold value according to the Gaussian mixture distribution, setting the area with the difference value between frames at the time t larger than the threshold value as a foreground, and updating the parameter pi if the area is not used as the backgroundj (t)，μj (t)，Σj (t){j＝1，2，……，K}。
And filtering the foreground area according to the target detection result to obtain the detection result of the object without the attribute. And further carrying out denoising treatment on the filtered foreground region to finally obtain a region of the tunnel invading foreign matters.
According to the tunnel invading object detection method provided by the invention, the background difference algorithm is adopted to extract the motion foreground information among the video key frames, and the motion foreground is combined with the target detection result, so that the invading object in the tunnel environment can be effectively positioned.
In one embodiment, step S140 may include:
matching the positions corresponding to the rail vehicles and the pedestrians with the positions of the moving objects;
and if the position of the moving object which is not matched with the positions corresponding to the railway vehicle and the pedestrian exists, determining the moving object corresponding to the position of the moving object which is not matched as the invading object.
Specifically, the positions of the rail vehicle and the pedestrian in the target area are acquired in step S120, and the position of the moving object in the target area is acquired in step S130.
In the case where there is no tunnel intruding object, the positions of the rail vehicles and pedestrians should be in one-to-one correspondence with the positions of the moving objects.
In the case of an intruding object in a tunnel, the position of the moving object may include the position of the intruding object in addition to the positions of the rail vehicles and pedestrians.
Therefore, by matching the positions corresponding to the rail vehicle and the pedestrian with the position of the moving object, whether the invading object exists or not and the position of the invading object can be determined according to the matching result.
According to the tunnel invading object detection method, the positions corresponding to the rail vehicles and the pedestrians are matched with the position of the moving object, whether the invading object exists or not and the position of the invading object can be rapidly and accurately judged, and the tunnel invading object detection efficiency is improved.
The invention also provides a tunnel invading object detection device, which can be correspondingly referred to the tunnel invading object detection method.
Fig. 2 is a schematic structural diagram of a tunnel intrusion object detection apparatus provided in the present invention, and as shown in fig. 2, the apparatus includes:
the extraction module 210 is configured to process the key frames of the tunnel surveillance video based on the lightweight feature extraction network to extract a multi-scale feature map of the tunnel surveillance video;
the fusion module 220 is configured to process the multi-scale feature map based on a multi-scale feature fusion network to obtain information of rail vehicles and pedestrians in the multi-scale feature map;
a difference module 230, configured to process the key frame based on a background difference algorithm to obtain position information of the moving object;
and the determining module 240 is configured to determine whether an intruding object exists in the tunnel according to the information of the rail vehicles and the pedestrians and the position information of the moving object.
According to the tunnel invading object detection device, the information of the rail vehicles and the pedestrians is determined by using the lightweight feature extraction network and the multi-scale feature fusion network, and the invading object in the tunnel is determined by combining the moving object position information determined by the differential algorithm, so that the tunnel invading object can be accurately detected, the misjudgment probability of the invading object in the tunnel is greatly reduced, and the safety hazard in the tunnel is rapidly eliminated.
In one embodiment, the extraction module 210 is specifically configured to:
cutting a target area in the key frame based on the lightweight feature extraction network;
performing convolution operation on the cut image to obtain a multi-scale characteristic diagram;
the multi-scale feature map comprises multi-layer images with different resolutions, and each layer of image comprises a spatial feature representing the position of an object and/or a semantic feature representing the category of the object.
According to the tunnel invading object detection device, the multi-scale feature map is obtained through the lightweight feature extraction network, so that the information of rail vehicles and pedestrians can be conveniently determined according to the multi-scale feature fusion network, and the efficiency of finally detecting the tunnel invading object is improved.
In one embodiment, the fusion module 220 is specifically configured to:
fusing the spatial features of each layer of image based on a multi-scale feature fusion network to determine the position information of the object in the target area;
fusing semantic features of each layer of image to determine object category information corresponding to the position information;
and determining the information of the rail vehicles and the pedestrians in the multi-scale feature map according to the position information and the object category information.
According to the tunnel invading object detection device, the information of the rail vehicles and the pedestrians in the multi-scale feature map is obtained by adopting the multi-scale feature fusion network, the defect that the object detection effect of the region far away from the camera is poor in the prior art is overcome, and the detection precision of the tunnel invading object is obviously improved.
In one embodiment, the difference module 230 is specifically configured to:
determining the foreground and the background of the target area based on a background difference algorithm;
the foreground is filtered to determine moving object position information indicative of a position of the moving object in the target region.
According to the tunnel invading object detection device provided by the invention, the background difference algorithm is adopted to extract the motion foreground information among the video key frames, and the motion foreground is combined with the target detection result, so that the invading object in the tunnel environment can be effectively positioned.
In one embodiment, the determining module 240 is specifically configured to:
matching the positions corresponding to the rail vehicles and the pedestrians with the positions of the moving objects;
and if the position of the moving object which is not matched with the positions corresponding to the railway vehicle and the pedestrian exists, determining the moving object corresponding to the position of the moving object which is not matched as the invading object.
According to the tunnel invading object detection device, the positions corresponding to the rail vehicles and the pedestrians are matched with the position of the moving object, whether the invading object exists or not and the position of the invading object can be rapidly and accurately judged, and the tunnel invading object detection efficiency is improved.
The present invention also provides an electronic device, as shown in fig. 3, the electronic device may include: a processor (processor)310, a communication interface (communication interface)320, a memory (memory)330 and a communication bus (bus)340, wherein the processor 310, the communication interface 320 and the memory 330 complete communication with each other through the communication bus 340. The processor 310 may invoke logic instructions in the memory 330 to perform the steps of the tunnel intrusion object detection method provided by the above-described method embodiments, including, for example:
processing key frames of the tunnel monitoring video based on the lightweight feature extraction network to extract a multi-scale feature map of the tunnel monitoring video;
processing the multi-scale feature map based on the multi-scale feature fusion network to obtain information of rail vehicles and pedestrians in the multi-scale feature map;
processing the key frame based on a background difference algorithm to acquire position information of the moving object;
and determining the tunnel intrusion object according to the information of the rail vehicles and the pedestrians and the position information of the moving object.
In addition, the logic instructions in the memory 330 may be implemented in the form of software functional units and stored in a computer readable storage medium when the software functional units are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer readable storage medium, the computer program includes program instructions, and when the program instructions are executed by a computer, the computer can execute the steps of the tunnel intrusion object detection method provided by the above method embodiments, for example, the steps include:
processing key frames of the tunnel monitoring video based on the lightweight feature extraction network to extract a multi-scale feature map of the tunnel monitoring video;
processing the multi-scale feature map based on the multi-scale feature fusion network to obtain information of rail vehicles and pedestrians in the multi-scale feature map;
processing the key frame based on a background difference algorithm to acquire position information of the moving object;
and determining the tunnel intrusion object according to the information of the rail vehicles and the pedestrians and the position information of the moving object.
In still another aspect, the present invention further provides a non-transitory computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of the tunnel intrusion object detection method provided by the above method embodiments, for example, including:
processing key frames of the tunnel monitoring video based on the lightweight feature extraction network to extract a multi-scale feature map of the tunnel monitoring video;
processing the multi-scale feature map based on the multi-scale feature fusion network to obtain information of rail vehicles and pedestrians in the multi-scale feature map;
processing the key frame based on a background difference algorithm to acquire position information of the moving object;
and determining the tunnel intrusion object according to the information of the rail vehicles and the pedestrians and the position information of the moving object.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A tunnel intrusion object detection method is characterized by comprising the following steps:
processing key frames of the tunnel monitoring video based on a lightweight feature extraction network to extract a multi-scale feature map of the tunnel monitoring video;
processing the multi-scale feature map based on a multi-scale feature fusion network to obtain information of rail vehicles and pedestrians in the multi-scale feature map;
processing the key frame based on a background difference algorithm to acquire position information of a moving object;
and determining whether an invading object exists in the tunnel according to the information of the rail vehicles and the pedestrians and the position information of the moving object.
2. The method according to claim 1, wherein the processing key frames of the tunnel surveillance video by the network based on lightweight feature extraction to extract the multi-scale feature map of the tunnel surveillance video comprises:
cutting a target area in the key frame based on the lightweight feature extraction network;
performing convolution operation on the cut image to obtain the multi-scale feature map;
the multi-scale feature map comprises multi-layer images with different resolutions, and each layer of image comprises a spatial feature representing the position of an object and/or a semantic feature representing the category of the object.
3. The method for detecting the tunnel intrusion object according to claim 2, wherein the processing the multi-scale feature map based on the multi-scale feature fusion network to obtain the information of the rail vehicles and pedestrians in the multi-scale feature map comprises:
fusing the spatial features of the images of each layer based on the multi-scale feature fusion network to determine the position information of the object in the target area;
fusing the semantic features of each layer of image to determine object category information corresponding to the position information;
and determining the rail vehicle and pedestrian information in the multi-scale feature map according to the position information and the object category information.
4. The method of claim 3, wherein processing the keyframe based on a background difference algorithm to obtain moving object information comprises:
determining a foreground and a background of the target region based on the background difference algorithm;
the foreground is filtered to determine moving object position information indicative of a position of the moving object in the target region.
5. The method according to claim 4, wherein the determining whether there is an intruding object in the tunnel according to the rail vehicle and pedestrian information and the moving object position information comprises:
matching the positions corresponding to the rail vehicles and the pedestrians with the positions of the moving objects;
and if the moving object position which is not matched with the positions corresponding to the railway vehicle and the pedestrian exists, determining the moving object corresponding to the non-matched moving object position as the intrusion object.
6. A tunnel intrusion object detection device, comprising:
the extraction module is used for processing key frames of the tunnel monitoring video based on the lightweight feature extraction network so as to extract a multi-scale feature map of the tunnel monitoring video;
the fusion module is used for processing the multi-scale feature map based on a multi-scale feature fusion network so as to obtain the information of the rail vehicles and the pedestrians in the multi-scale feature map;
the difference module is used for processing the key frame based on a background difference algorithm to acquire the position information of the moving object;
and the determining module is used for determining whether an invading object exists in the tunnel according to the information of the rail vehicles and the pedestrians and the position information of the moving object.
7. The device according to claim 6, wherein the extraction module is specifically configured to:
cutting a target area in the key frame based on the lightweight feature extraction network;
performing convolution operation on the cut image to obtain the multi-scale feature map;
the multi-scale feature map comprises multi-layer images with different resolutions, and each layer of image comprises a spatial feature representing the position of an object and/or a semantic feature representing the category of the object.
8. The device according to claim 7, wherein the fusion module is specifically configured to:
fusing the spatial features of the images of each layer based on the multi-scale feature fusion network to determine the position information of the object in the target area;
fusing the semantic features of each layer of image to determine object category information corresponding to the position information;
and determining the rail vehicle and pedestrian information in the multi-scale feature map according to the position information and the object category information.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the tunnel intrusion object detection method according to any one of claims 1 to 5 when executing the computer program.
10. A non-transitory computer readable storage medium having stored thereon a computer program, which when executed by a processor implements the steps of the tunnel intrusion object detection method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011522501.8A CN112686107B (en) | 2020-12-21 | 2020-12-21 | Tunnel intrusion object detection method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011522501.8A CN112686107B (en) | 2020-12-21 | 2020-12-21 | Tunnel intrusion object detection method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112686107A true CN112686107A (en) | 2021-04-20 |
| CN112686107B CN112686107B (en) | 2024-10-15 |
Family
ID=75450112
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011522501.8A Active CN112686107B (en) | 2020-12-21 | 2020-12-21 | Tunnel intrusion object detection method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112686107B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114932927A (en) * | 2022-07-20 | 2022-08-23 | 北京大成国测智能科技有限公司 | Railway foreign matter identification method and system based on lightweight deep learning |
| CN117274762A (en) * | 2023-11-20 | 2023-12-22 | 西南交通大学 | Real-time track extraction method based on vision under subway tunnel low-illumination scene |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7227893B1 (en) * | 2002-08-22 | 2007-06-05 | Xlabs Holdings, Llc | Application-specific object-based segmentation and recognition system |
| CN107253485A (en) * | 2017-05-16 | 2017-10-17 | 北京交通大学 | Foreign matter invades detection method and foreign matter intrusion detection means |
| CN108510012A (en) * | 2018-05-04 | 2018-09-07 | 四川大学 | A kind of target rapid detection method based on Analysis On Multi-scale Features figure |
| CN110263732A (en) * | 2019-06-24 | 2019-09-20 | 京东方科技集团股份有限公司 | Multiscale target detection method and device |
| WO2019232836A1 (en) * | 2018-06-04 | 2019-12-12 | 江南大学 | Multi-scale sensing pedestrian detection method based on improved full convolutional network |
| CN111016932A (en) * | 2019-12-02 | 2020-04-17 | 江西理工大学 | Track inspection vehicle and detection method of energy-saving air rail system |
| CN111145475A (en) * | 2019-12-12 | 2020-05-12 | 上海微波技术研究所（中国电子科技集团公司第五十研究所） | Intrusion alarm system, method and medium based on vibration optical fiber and deep learning |
| CN111160125A (en) * | 2019-12-11 | 2020-05-15 | 北京交通大学 | Railway foreign matter intrusion detection method based on railway monitoring |
| CN111179318A (en) * | 2019-12-31 | 2020-05-19 | 浙江大学 | Double-flow method-based complex background motion small target detection method |
| CN111667655A (en) * | 2020-07-10 | 2020-09-15 | 上海工程技术大学 | Infrared image-based high-speed railway safety area intrusion alarm device and method |
| CN111709291A (en) * | 2020-05-18 | 2020-09-25 | 杭州电子科技大学 | Takeaway personnel identity identification method based on fusion information |
| CN111754477A (en) * | 2020-06-19 | 2020-10-09 | 北京交通大学 | Railway perimeter foreign matter intrusion detection method based on dynamic candidate area multi-scale images |
-
2020
- 2020-12-21 CN CN202011522501.8A patent/CN112686107B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7227893B1 (en) * | 2002-08-22 | 2007-06-05 | Xlabs Holdings, Llc | Application-specific object-based segmentation and recognition system |
| CN107253485A (en) * | 2017-05-16 | 2017-10-17 | 北京交通大学 | Foreign matter invades detection method and foreign matter intrusion detection means |
| CN108510012A (en) * | 2018-05-04 | 2018-09-07 | 四川大学 | A kind of target rapid detection method based on Analysis On Multi-scale Features figure |
| WO2019232836A1 (en) * | 2018-06-04 | 2019-12-12 | 江南大学 | Multi-scale sensing pedestrian detection method based on improved full convolutional network |
| CN110263732A (en) * | 2019-06-24 | 2019-09-20 | 京东方科技集团股份有限公司 | Multiscale target detection method and device |
| CN111016932A (en) * | 2019-12-02 | 2020-04-17 | 江西理工大学 | Track inspection vehicle and detection method of energy-saving air rail system |
| CN111160125A (en) * | 2019-12-11 | 2020-05-15 | 北京交通大学 | Railway foreign matter intrusion detection method based on railway monitoring |
| CN111145475A (en) * | 2019-12-12 | 2020-05-12 | 上海微波技术研究所（中国电子科技集团公司第五十研究所） | Intrusion alarm system, method and medium based on vibration optical fiber and deep learning |
| CN111179318A (en) * | 2019-12-31 | 2020-05-19 | 浙江大学 | Double-flow method-based complex background motion small target detection method |
| CN111709291A (en) * | 2020-05-18 | 2020-09-25 | 杭州电子科技大学 | Takeaway personnel identity identification method based on fusion information |
| CN111754477A (en) * | 2020-06-19 | 2020-10-09 | 北京交通大学 | Railway perimeter foreign matter intrusion detection method based on dynamic candidate area multi-scale images |
| CN111667655A (en) * | 2020-07-10 | 2020-09-15 | 上海工程技术大学 | Infrared image-based high-speed railway safety area intrusion alarm device and method |
Non-Patent Citations (5)
| Title |
|---|
| ZHU LIU等: "Indoor intrusion detection based on fuzzy membership-aided Dempster-Shaper theory", 《JOURNAL OF INTELLIGENT & FUZZY SYSTEMS》, vol. 38, no. 4, 30 April 2020 (2020-04-30), pages 3687 - 3696 * |
| 张芳杰等: "基于视频分析的周界入侵监测系统", 《数据通信》, 30 April 2020 (2020-04-30), pages 21 - 26 * |
| 王瑞等: "基于视频深度学习的铁路周界入侵检测算法研究", 《交通运输系统工程与信息》, vol. 20, no. 2, 30 April 2020 (2020-04-30), pages 61 - 68 * |
| 秦浩: "智能视频监控平台中区域目标检测和联合跟踪的设计与实现", 《中国优秀硕士学位论文全文数据库 信息科技辑》, no. 2, 15 February 2017 (2017-02-15), pages 136 - 1826 * |
| 谭笑等: "基于混合高斯和帧间差分的机场跑道入侵检测", 《计算机仿真》, vol. 31, no. 11, 30 November 2014 (2014-11-30), pages 38 - 41 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114932927A (en) * | 2022-07-20 | 2022-08-23 | 北京大成国测智能科技有限公司 | Railway foreign matter identification method and system based on lightweight deep learning |
| CN117274762A (en) * | 2023-11-20 | 2023-12-22 | 西南交通大学 | Real-time track extraction method based on vision under subway tunnel low-illumination scene |
| CN117274762B (en) * | 2023-11-20 | 2024-02-06 | 西南交通大学 | Real-time track extraction method based on vision under subway tunnel low-illumination scene |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112686107B (en) | 2024-10-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Bolte et al. | Towards corner case detection for autonomous driving | |
| CN110232380B (en) | Fire night scene restoration method based on Mask R-CNN neural network | |
| Breitenstein et al. | Systematization of corner cases for visual perception in automated driving | |
| CN111260666B (en) | Image processing method and device, electronic equipment and computer readable storage medium | |
| CN108416780B (en) | Object detection and matching method based on twin-region-of-interest pooling model | |
| CN110557521B (en) | Method, device and equipment for removing rain from video and computer readable storage medium | |
| CN111652181B (en) | Target tracking method and device and electronic equipment | |
| CN112686107B (en) | Tunnel intrusion object detection method and device | |
| CN114332473B (en) | Object detection method, device, computer apparatus, storage medium, and program product | |
| CN111738218A (en) | Human body abnormal behavior recognition system and method | |
| Han et al. | A method based on multi-convolution layers joint and generative adversarial networks for vehicle detection | |
| CN116935361A (en) | Deep learning-based driver distraction behavior detection method | |
| Fang et al. | Vision-based traffic accident detection and anticipation: A survey | |
| Mijić et al. | Traffic sign detection using YOLOv3 | |
| CN115861981A (en) | Driver fatigue behavior detection method and system based on video attitude invariance | |
| CN117611994A (en) | Remote sensing image target detection method based on attention mechanism weighting feature fusion | |
| Adewopo et al. | Smart city transportation: Deep learning ensemble approach for traffic accident detection | |
| CN114373162A (en) | Dangerous area personnel intrusion detection method and system for transformer substation video monitoring | |
| Yang et al. | Video anomaly detection for surveillance based on effective frame area | |
| Rocky et al. | Review of Accident Detection Methods Using Dashcam Videos for Autonomous Driving Vehicles | |
| Zhang et al. | Small target detection based on squared cross entropy and dense feature pyramid networks | |
| CN112597996A (en) | Task-driven natural scene-based traffic sign significance detection method | |
| Zhang et al. | Pre-locate net for object detection in high-resolution images | |
| Savakis et al. | Semantic background estimation in video sequences | |
| CN116310748A (en) | Automatic driving scene recovery and automatic driving prototype testing method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |