CN115272849A - Urban rail accumulated water identification method and system - Google Patents
Urban rail accumulated water identification method and system Download PDFInfo
- Publication number
- CN115272849A CN115272849A CN202210856178.0A CN202210856178A CN115272849A CN 115272849 A CN115272849 A CN 115272849A CN 202210856178 A CN202210856178 A CN 202210856178A CN 115272849 A CN115272849 A CN 115272849A
- Authority
- CN
- China
- Prior art keywords
- ponding
- identification
- model
- image
- unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 32
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 title claims description 54
- 238000013528 artificial neural network Methods 0.000 claims abstract description 28
- 238000012549 training Methods 0.000 claims abstract description 26
- 238000001514 detection method Methods 0.000 claims abstract description 21
- 238000009825 accumulation Methods 0.000 claims abstract description 20
- 238000012545 processing Methods 0.000 claims abstract description 17
- 230000000007 visual effect Effects 0.000 claims abstract description 14
- 238000013139 quantization Methods 0.000 claims abstract description 13
- 230000011218 segmentation Effects 0.000 claims abstract description 6
- 238000012795 verification Methods 0.000 claims abstract description 3
- 241001669679 Eleotris Species 0.000 claims description 19
- 229910000831 Steel Inorganic materials 0.000 claims description 19
- 239000010959 steel Substances 0.000 claims description 19
- 230000007246 mechanism Effects 0.000 claims description 15
- 238000003062 neural network model Methods 0.000 claims description 9
- 238000012544 monitoring process Methods 0.000 claims description 8
- 238000012360 testing method Methods 0.000 claims description 8
- 238000000605 extraction Methods 0.000 claims description 7
- 238000004458 analytical method Methods 0.000 claims description 6
- 238000010276 construction Methods 0.000 claims description 3
- 230000002708 enhancing effect Effects 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 3
- 238000009432 framing Methods 0.000 claims description 3
- 238000002372 labelling Methods 0.000 claims description 3
- 238000011176 pooling Methods 0.000 claims description 3
- 238000011002 quantification Methods 0.000 claims description 3
- 230000011514 reflex Effects 0.000 claims description 3
- IHQKEDIOMGYHEB-UHFFFAOYSA-M sodium dimethylarsinate Chemical class [Na+].C[As](C)([O-])=O IHQKEDIOMGYHEB-UHFFFAOYSA-M 0.000 claims description 2
- 238000011835 investigation Methods 0.000 abstract description 6
- 238000005516 engineering process Methods 0.000 abstract description 4
- 238000006243 chemical reaction Methods 0.000 abstract description 2
- 230000035508 accumulation Effects 0.000 description 14
- 238000010586 diagram Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 6
- 230000001788 irregular Effects 0.000 description 3
- 230000002265 prevention Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- RTAQQCXQSZGOHL-UHFFFAOYSA-N Titanium Chemical compound [Ti] RTAQQCXQSZGOHL-UHFFFAOYSA-N 0.000 description 1
- 230000003213 activating effect Effects 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000003912 environmental pollution Methods 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 239000002352 surface water Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B61—RAILWAYS
- B61L—GUIDING RAILWAY TRAFFIC; ENSURING THE SAFETY OF RAILWAY TRAFFIC
- B61L23/00—Control, warning or like safety means along the route or between vehicles or trains
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/28—Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V10/457—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by analysing connectivity, e.g. edge linking, connected component analysis or slices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
- G06V10/765—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects using rules for classification or partitioning the feature space
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Mechanical Engineering (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Processing (AREA)
Abstract
The invention discloses an urban rail ponding identification method and an identification system, wherein the urban rail ponding identification method mainly comprises the steps of collecting image data of surface ponding of each urban rail scene, establishing a semantic segmentation model for ponding identification, carrying out neural network training and verification, processing an image by using the trained model, and outputting result information to an interface service program, and the identification system comprises an image acquisition unit, a model training unit, a visual identification unit, a linear segment detection unit, an ROI quantization processing unit, the interface service program and an intelligent alarm unit. The method adopts the original visual detection technology and the method based on the slope search and the length accumulation of the multi-feature ROI, can tolerate the poor conditions of camera shake, light ray conversion and the like, can accurately identify shallow ponding and deep ponding in the urban rail area and can give an alarm in time, and improves the flood situation investigation capability of relevant departments and the subsequent emergency treatment work efficiency.
Description
Technical Field
The invention relates to a method and a system for identifying ponding, in particular to a method and a system for identifying urban rail ponding based on a neural network visual detection technology.
Background
Urban rail transit is a backbone of a large-city public transport system, is an important infrastructure for building modern cities, plays an increasingly important role in guiding and supporting city development, meeting people's trip, relieving traffic congestion, reducing environmental pollution and the like, and becomes an important guarantee for daily trip of people in large cities and the normal running of cities.
The operation safety of urban rail transit has important significance for guaranteeing the life and property safety of people, maintaining social stability and improving the acquisition feeling of people. In recent years, with the increasing of track lines, the rapid increase of operation mileage and scale and the increasing of passenger capacity, the difficulty of safety guarantee of urban track traffic is increasing, and higher requirements are put forward on operation safety management. However, in the aspect of flood prevention and flood control, workers are still relied on to perform periodic manual inspection and investigation at present, the investigation point positions are complex and comprise a plurality of areas such as tracks, tunnels, bridges and the like, the investigation process not only consumes a lot of manpower, but also risks such as missing inspection, untimely investigation and the like easily occur, and hidden dangers are brought to the safe operation of urban rail transit.
Disclosure of Invention
The purpose of the invention is as follows: the invention aims to provide an urban rail accumulated water identification method based on a neural network visual detection technology; it is another object of the invention to provide a system using the method.
The technical scheme is as follows: the urban rail ponding identification method comprises the following steps:
(1) Collecting surface ponding image data under each urban rail scene based on a target use environment, labeling ponding areas by using a Labelme tool in combination with random surface ponding data on a network so as to form a ponding recognition deep neural network data set, and dividing a ponding recognition neural network training set and a neural network testing set based on the data set;
(2) Establishing a semantic segmentation model for ponding recognition based on a standard FCN-8s full convolution neural network;
(3) Aiming at the irregular characteristics of the shape and the boundary of the ponding and the water surface image information changeability caused by the reflection of the ponding water surface, the reflection-based attention mechanism unit is adopted to compare and identify the related image properties of the water surface reflection, and the specific operation is as follows: adding a attention mechanism unit based on reflection in the step (2) to generate a novel FCN-8s full convolution neural network model, namely an FCN-8s-FL model, and further adding a loss function in the novel FCN-8s full convolution neural network model to generate an FCN-8s-FL-5RAU model;
(4) Performing neural network training based on the FCN-8s-FL model and the FCN-8s-FL-5RAU model proposed in the step (3) and the neural network training set completed in the step (1);
(5) Verifying the model trained in the step (4);
(6) Acquiring camera code stream data, and regularly framing the pictures to form a time sequence picture frame G0(x，y)、G1(x，y)……Gn(x, y) sequence picture frame G is trained by the model trained in step (4)0(x，y)、G1(x，y)……Gn(x, y) identifying, and outputting a binary identification result graph G0(x，y)′、G1(x，y)′……Gn(x，y)′；
(7) For the binarization result graph G0(x，y)′、G1(x，y)′……Gn(x, y)' performing pixel scanning statistics, calculating the number of accumulated water pixels, giving an accumulated water conclusion according to a threshold value, completing analysis and identification of water pits and shallow accumulated water in each frame of picture, and outputting an identification result, position information and time information of the identification point to an interface service program;
(8) For time series frame G0(x，y)、G1(x，y)……GnFiltering, ray processing, enhancing and binary identification processing are carried out on the (x, y) image to obtain an image H0(x，y)、H1(x，y)……Hn(x，y)；
(9) For image H0(x，y)、H1(x，y)......Hn(x, y) performing connected domain analysis, traversing all linear rectangular support regions R meeting the conditions that the Width is less than or equal to alpha and the Length is greater than or equal to beta in the image according to the contour pixel points of the connected domain0(x，y)、R1(x，y)......Rm(x, y) for all connected domain profiles R0(x，y)、R1(x，y)……RmTraversing the pixel points of (x, y), finding the linear rectangular support area where each pixel point is located, preferentially extracting the linear rectangular support area to the linear rectangular support area with a larger Length value, and updating the area to be R0(x，y)′、R1(x，y)′……Rm(x，y)′；
(10) For update region R0(x，y)′、R1(x，y)′......Rm(x, y), performing neighborhood detection on black point pixels in all rectangles, and performing solitary point detection on neighborhood homonymous pixelsRemoving lines, traversing all the rectangular support areas again, removing the rectangular support areas with Width more than alpha or Length less than beta to obtain R0(x，y)″、R1(x，y)″......Rm(x, y) ', and then all R's are combined0(x，y)″、R1(x，y)″......RmThe line segments in the wide sides of the rectangle (x, y)' are connected and marked back to the original image G0(x，y)、G1(x，y)......Gn(x, y);
(11) All time series frames G0(x，y)、G1(x，y)……GnThe steel rail, the sleeper, the track bed and the turnout in (x, y) are locally reversely rotated according to the angle of the linear line segment, so that the horizontal setting of the key linear line segment is realized, ROI extraction can be conveniently carried out on the local linear line segment region of the steel rail, the sleeper, the track bed and the turnout through a rectangular frame after the horizontal setting, therefore, ROI extraction of different characteristics is realized, the identification and detection of depth accumulated water are completed according to a quantification method based on slope search and length accumulation of a multi-characteristic ROI region, and the identification result, the position information and the time information of the identification point are output to an interface service program;
(12) And (3) matching the recognition results of the puddle and the shallow water output in the steps (1) to (7) and the deep water output in the steps (8) to (11) according to a space-time relationship by the interface service program, outputting and storing the matched recognition result, position information and time information of the current recognition point into a database, and simultaneously sending the matching recognition result, the position information and the time information to an intelligent warning unit for pushing through different warning levels.
Further, the model construction method of the attention mechanism unit based on reflection in the step (4) comprises the following steps: given a size of [ h, w, c]Is averaged and pooled in the horizontal direction to reduce to [ h, w/2, c](ii) a Then the vertical average pooling is performed to reduce the size to n, w/2]X of (1); thereafter, each row X of XiHas a size of [1,w/2 ]]Tiled or self-replicated to size [ n, w/2 ]]The feature maps from all rows are concatenated along feature axes to form a new feature map with dimensions [ n, w/2, c x n](ii) a Finally, this feature map is upsampled to [ h, w, c × n [ ]]Size and representsFor X', concatenating I n times along the characteristic axis to obtain the size of [ h, w, c X n]Subtracting I 'from X' yields the size of [ h, w, c n]And D, connecting the subtracted feature map with I' again in series, sending the feature map into the convolutional layer, activating the convolutional layer by the ReLU function, and generating a final output feature map with the same size as I.
Further, the step (3) of identifying the related image properties of the water surface reflection based on the reflected attention mechanism unit specifically includes searching an upper image reflected by the ponding water surface, searching for a reflection pixel point by matching an image area along a pixel column of the image, and traversing and searching with multiple resolutions in vertical matching.
Further, the step (5) of verifying that the trained model specifically operates as: the model after the verification training in the step (5) specifically operates as follows: the FCN-8s-FL model, the FCN-8s-FL-5RAU model in step (3) were applied through the test set completed in step (1) and compared to the FCN-8s neural network model that did not use a reflex-based attention mechanism unit.
Further, the quantization method based on the slope search and length accumulation of the multi-feature ROI region in step (11) is: and calculating the number and length accumulation sums of all line segments with the slope values of the line segments of the steel rail, the sleeper, the track bed and the turnout being less than k in the ROI area, wherein the number and length accumulation sums of ROI linear line segments of the steel rail, the sleeper, the track bed and the turnout are respectively Counts-g, counts-d, counts-c1, counts-c2, lenth-g, lenth-d, lenth-c1 and Lenth-c2, when deep ponding occurs, the steel rail, the sleeper, the track bed and the turnout are covered by ponding, the linear line segments in the ROI area corresponding to the accumulated ponding disappear, and the Counts-g, counts-d, counts-c1, counts-c2, lenth-g, lenth-d, lenth-c1 and Lenth-c2 are suddenly reduced, and when the number n frames are continuously smaller than a threshold value, the deep ponding state is judged.
The invention relates to an urban rail ponding recognition system which comprises an image acquisition unit, a model training unit, a visual recognition unit, a linear segment detection unit, a ROI quantization processing unit, an interface service program and an intelligent alarm unit, wherein the image acquisition unit is a camera which accords with a standard code stream output protocol; the model training unit is used for constructing an improved full convolution integral neural network, completing the training of a ponding recognition model and providing a basic model for visual recognition; the visual recognition unit carries out image recognition on a water pit and shallow ponding under the monitoring area through the model trained by the model training unit; the linear line segment detection unit is internally provided with an original linear line segment identification algorithm to realize the identification of the linear line segment characteristics of a fixed scene in an urban rail scene monitoring image; the ROI quantization processing unit is used for processing the image, extracting ROI from local linear line segment regions of a steel rail, a sleeper, a track bed and a turnout, and completing the identification and detection of depth accumulated water by a quantization method based on multi-feature ROI slope search and length accumulation; the interface service program is used for providing data flow service for each subsystem, matching according to the time-space relation of result data and outputting matched information to the intelligent alarm unit; and the intelligent alarm unit is used for sending the identification result to an urban rail emergency management center platform and pushing the identification result through different alarm levels.
Has the advantages that: compared with the prior art, the invention has the following remarkable advantages:
(1) The method adopts an original visual detection technology to accurately identify shallow water accumulation and deep water accumulation in urban rail areas and give an alarm in time, is suitable for all-weather scenes such as bridges, tunnels and outdoors, and improves the flood situation investigation capability and the subsequent emergency disposal work efficiency of relevant departments;
(2) The method integrates the water surface mirror image characteristic search and improves the full convolution neural network, enhances the identification performance of the model, solves the identification difficulty caused by irregular ponding shape and boundary, and improves the accuracy of identifying the water puddle and the shallow ponding;
(3) By adopting an original quantification method based on multi-feature ROI (region of interest) slope search and length accumulation, the characteristic differences between different parts such as a steel rail, a sleeper, a track bed, a turnout and the like and the deep ponding are rapidly distinguished, the deep ponding state of the railway covered by the ponding in a large area is effectively identified, the poor conditions such as camera shake, light ray conversion and the like can be tolerated, and the scene universality is strong;
(4) Through different algorithms, the shallow ponding and the deep ponding in the area are identified and result output at the same time, powerful data support is provided for risk identification and rapid decision of the flood prevention emergency system, and emergency disposal work of relevant departments is rapidly, efficiently and orderly done;
(5) The system adopts a standard protocol to acquire camera code stream data, and can utilize the existing cameras of the urban rail system to access, thereby greatly reducing the construction cost and the deployment difficulty and being convenient for related departments to rapidly popularize and use.
Drawings
FIG. 1 is a flow chart of the present invention;
FIG. 2 is a diagram of a semantic segmentation model network architecture;
FIG. 3 is a schematic diagram of a reflex attention mechanism unit;
FIG. 4 is a diagram of a novel FCN-8s full convolution neural network architecture;
FIG. 5 is a diagram of a semantic segmentation ponding recognition binarization effect;
FIG. 6 is a diagram illustrating the effect of image preprocessing;
FIG. 7 is a linear rectangular support area;
FIG. 8 is a diagram of a linear segment rotation setting process;
FIG. 9 is a ROI extraction and quantization map;
FIG. 10 is a graph of linear segment detection results for different water accumulations.
Detailed Description
The technical scheme of the invention is further explained by combining the attached drawings.
The urban rail ponding recognition system comprises: the system comprises an image acquisition unit, a model training unit, a visual identification unit, a linear line segment detection unit, an ROI quantization processing unit, an interface service program and an intelligent alarm unit, wherein as shown in figure 1:
the image acquisition unit is a camera conforming to a standard code stream output protocol and is used for acquiring real-time image data. The camera can be equipment which meets the requirements of a monitoring area in the original urban rail system; the model training unit is used for constructing an improved full convolution integral neural network and finishing the training of the ponding recognition model so as to provide a basic model for visual recognition; the visual recognition unit is used for carrying out image recognition on a water pit and shallow water accumulation under the monitoring area through the model trained by the model training unit and outputting the result to the interface service program; the linear segment detection unit is internally provided with an original linear segment identification algorithm to realize the identification of the linear segment characteristics of a fixed scene in an urban rail scene monitoring image and output the identification results of all the linear segment characteristics; the ROI quantization processing unit is used for processing the image, extracting ROI from local linear line segment regions of a steel rail, a sleeper, a track bed and a turnout, completing the identification and detection of the depth accumulated water through an original quantization method based on multi-feature ROI region slope search and length accumulation, and outputting the result to an interface service program; the interface service program is used for providing data flow service for each subsystem, matching according to the space-time relation of result data, and outputting the matched identification result, position information and time information of the current identification point to the intelligent alarm unit; the intelligent alarm unit is used for sending the identification result to an urban rail emergency management center platform, provides decision basis for a superior management system through pushing of different alarm levels, and helps relevant departments to rapidly, efficiently and orderly perform emergency disposal work.
The urban rail ponding identification method comprises the following steps:
(1) Collecting surface ponding image data under each urban rail scene based on a target using environment, labeling ponding areas by using a Labelme tool in combination with random surface ponding data on a network so as to form a ponding recognition deep neural network data set, and dividing a ponding recognition neural network training set and a neural network testing set based on the data set.
(2) A semantic segmentation model for water accumulation recognition is established based on a standard FCN-8s full convolution neural network. The network structure is shown in fig. 2, a standard full convolution neural network is adopted, and compared with a common convolution neural network, the network structure is characterized in that a last full connection layer is replaced by a convolution layer to obtain a two-dimensional characteristic diagram, and then a classification result of a corresponding pixel is obtained through a softmax function.
(3) Aiming at the characteristics of irregular shapes and boundaries of the ponding and the characteristics of water surface image information changeability caused by ponding water surface reflection, a reflection-based attention machine mechanism unit is adopted to compare and recognize the properties of related images of the water surface reflection, the reflection-based attention machine mechanism unit is added in the step (2), a novel FCN-8s full convolution neural network (FCN-8 s-FL model) is generated, the network structure of the model is shown in figure 4, a loss function (FCN-8 s-FL-5RAU model) is added in the novel FCN-8s full convolution neural network model, and the problem of unbalanced samples is solved by adopting focus loss.
The method comprises the steps of searching an upper image reflected by the water surface of the ponding water by adopting a reflection-based attention mechanism unit, searching reflection pixel points by matching an image area along a pixel column of the image, and traversing and searching by using multiple resolutions in vertical matching, so that tolerance of errors caused by problems such as perspective distortion, slight camera rotation (angle with the horizon), fuzzy reflection and the like is improved.
A schematic diagram of the structure of the attention mechanism unit based on reflection is shown in FIG. 3, which specifically includes a given size of [ h, w, c ]]Is averaged in the horizontal direction and pooled to reduce to [ h, w/2, c]Performing vertical average pooling to reduce it to a size of [ n, w/2]X of (1); then, each row X of XiHas a size of [1, w/2 ]]Tiled or self-replicated to size [ n, w/2 ]]The feature maps from all rows are concatenated along feature axes to form a new feature map with dimensions [ n, w/2, c x n](ii) a This signature is then upsampled to [ h, w, c n [ ]]Size, and denoted X', and concatenating I n times along the characteristic axis, giving a size [ h, w, c n [ ]]Subtracting I 'from X' yields the size of [ h, w, c n]D, which encodes the reflection relationship, the subtracted signature is again concatenated with I', fed into the convolutional layer and activated by the ReLU function, generating a final output map of the same size as I.
(4) Performing neural network training based on the model provided in the step (3) and the training set completed in the step (1), taking training on NVIDIA TITAN XP GPU with 12GB memory as an example, setting BatchSize to be 1, and setting learning rate to be initial 10-6And decreases by a factor of 0.2 at every 5 thousand iterations.
(5) Verifying the model trained in the step (4), applying the test set completed in the step (1) to the novel FCN-8s full convolution neural network model in the step (3), and comparing the model with the FCN-8s neural network model without a reflection-based attention mechanism unit to obtain that the model obtained in the step (4) has a better surface water identification function, wherein the related data are as follows:
table 1 comparative data sheet of model test results
| F value | Rate of accuracy | Recall rate | Rate of accuracy | |
| FCN-8s | 65.21％ | 69.81％ | 61.18％ | 99.05％ |
| FCN-8s-FL | 70.62％ | 74.38％ | 67.22％ | 99.19％ |
| FCN-8s-FL-5RAU | 76.91％ | 78.03％ | 75.81％ | 99.34％ |
(6) Acquiring camera code stream data and converting the data into a video file, and regularly framing the picture to form a time sequence picture frame G0(x，y)、G1(x，y)......Gn(x, y) sequence picture frame G is trained by the model trained in step (4)0(x，y)、G1(x，y)......Gn(x, y) and outputs a binarized recognition result graph G as shown in FIG. 50(x，y)′、G1(x，y)′......Gn(x，y)′。
(7) For the binarization result graph G0(x，y)′、G1(x，y)′......Gn(x, y)', performing pixel scanning statistics, calculating the number of accumulated water pixel points, giving an accumulated water conclusion according to a threshold value, completing analysis and identification of water pits and shallow accumulated water in each frame of picture, and outputting an identification result, position information and time information of the identification point to an interface service program.
(8) For time series frame G0(x，y)、G1(x，y)......Gn(x, y) filtering, ray processing, enhancing and binarization processing are carried out to obtain an image H0(x，y)、H1(x，y)......Hn(x, y) as shown in FIG. 6.
(9) For image H0(x，y)、H1(x，y)……Hn(x, y) carrying out connected domain analysis, traversing all linear rectangular support regions R with Width not more than alpha and Length not less than beta in the image according to connected domain contour pixel points0(x，y)、R1(x，y)......Rm(x, y), as shown in FIG. 7, for all connected components, the contour R0(x，y)、R1(x，y)......RmTraversing the (x, y) pixel points, finding out the linear rectangular support area where each pixel point is located, preferentially extracting the linear rectangular support area to the linear rectangular support area with a larger Length value, and updating the linear rectangular support area to be the linear rectangular support area with the larger Length valueR0(x，y)′、R1(x，y)′......Rm(x，y)′。
(10) For update region R0(x，y)′、R1(x，y)′......RmPerforming neighborhood detection on black point pixels in all rectangles in (x, y)', removing isolated points without the same pixels in the neighborhood, traversing all rectangular support areas again, and removing rectangular support areas with Width more than alpha or Length less than beta to obtain R0(x，y)″、R1(x，y)″......Rm(x, y) ", and then adding all R0(x，y)″、R1(x，y)″......RmThe line segments in the broad sides of the rectangle (x, y)' are connected and marked back to the original image G0(x，y)、G1(x，y)......Gn(x, y) as shown in FIG. 8.
(11) All time series frames G0(x，y)、G1(x，y)......GnThe steel rail, the sleeper, the track bed and the turnout in (x, y) are locally reversely rotated according to the angle of the linear line segment to realize the horizontal setting of the key linear line segment, after the horizontal setting, the ROI (region of interest) extraction can be conveniently carried out on the local linear line segment regions of the steel rail, the sleeper, the track bed and the turnout through the rectangular frame, thereby realizing the ROI extraction of different characteristics,
and calculating the number and length accumulation sums of all line segments with gradient values smaller than k in the ROI area, wherein the number and length accumulation sums of ROI linear line segments of the steel rail, the sleeper, the track bed and the turnout are respectively count-g, count-d, count-c 1, count-c 2, lenth-g, lenth-d, lenth-c1 and Lenth-c2, when deep ponding occurs, the steel rail, the sleeper, the track bed and the turnout are covered by ponding, the linear line segments in the corresponding ROI area disappear, count-g, count-d, count-c 1, count-c 2, lenth-g, lenth-d, lenth-c1 and Lenth-c2 are suddenly reduced, when the continuous n frames above the numerical value are judged to be smaller than the threshold value, judging that the depth state is the service point identification result, and outputting the ponding time information and the service point identification result to a program.
The algorithm in the steps (8) to (11) is adopted to detect the characteristics of the random accumulated water test picture, as shown in fig. 10, when light shadow and ripple exist in accumulated water, a small amount of linear line segment characteristics can be detected with small probability, however, the comparison between fig. 9 and fig. 10 can be obtained, after the rail is covered by the accumulated water deeply, although the linear line segment characteristics of the accumulated water area are completely different from the linear line segment characteristics of fixed slope generated by the steel rail, the sleeper, the track bed and the turnout in the rail, and the stability of the linear line segment characteristics is poor along with the movement of water waves, the linear line segment characteristics can be accurately distinguished through continuous identification of multi-frame pictures, and in the practical application process, the urban rail accumulated water can be effectively identified through the algorithm in the steps.
(12) And (3) matching the pit and shallow water identification results output in the steps (1) to (7) and the deep water identification results output in the steps (8) to (11) according to a space-time relationship by the interface service program, outputting the identification result, the position information and the time information of the matched current identification point to an alarm unit, storing the identification result, the position information and the time information into a database, and simultaneously sending the identification result, the position information and the time information to an intelligent alarm unit, thereby providing a decision basis for the flood prevention emergency system through pushing at different alarm levels.
Claims (6)
1. A method for identifying urban rail accumulated water is characterized by comprising the following steps: the method comprises the following steps:
(1) Collecting surface ponding image data under each urban rail scene based on a target use environment, labeling ponding areas by using a Labelme tool in combination with random surface ponding data on a network to form a ponding recognition deep neural network data set, and dividing a ponding recognition neural network training set and a neural network testing set based on the data set;
(2) Establishing a semantic segmentation model for ponding recognition based on a standard FCN-8s full convolution neural network;
(3) Adding a attention mechanism unit based on reflection in the step (2) to generate a novel FCN-8s full convolution neural network model, namely an FCN-8s-FL model, and further adding a loss function in the novel FCN-8s full convolution neural network model to generate an FCN-8s-FL-5RAU model;
(4) Performing neural network training based on the FCN-8s-FL model and the FCN-8s-FL-5RAU model proposed in the step (3) and the neural network training set completed in the step (1);
(5) Verifying the model trained in the step (4);
(6) Acquiring camera code stream data, and regularly framing the pictures to form a time sequence picture frame G0(x，y)、G1(x，y)……Gn(x, y) for sequence frame G by the model trained in step (4)0(x，y)、G1(x，y)……Gn(x, y) for recognition, and outputting a binary recognition result graph G0(x，y)′、G1(x，y)′……Gn(x，y)′；
(7) For the binarization result graph G0(x，y)′、G1(x，y)′……Gn(x, y)' performing pixel scanning statistics, calculating the number of accumulated water pixels, giving an accumulated water conclusion according to a threshold value, completing analysis and identification of water pits and shallow accumulated water in each frame of picture, and outputting an identification result, position information and time information of the identification point to an interface service program;
(8) For time series frame G0(x，y)、G1(x，y)……GnFiltering, ray processing, enhancing and binary identification processing are carried out on the (x, y) image to obtain an image H0(x，y)、H1(x，y)……Hn(x，y)；
(9) For image H0(x，y)、H1(x，y)......Hn(x, y) performing connected domain analysis, traversing all linear rectangular support regions R meeting the conditions that the Width is less than or equal to alpha and the Length is greater than or equal to beta in the image according to the contour pixel points of the connected domain0(x，y)、R1(x，y)……Rm(x, y) for all connected domain profiles R0(x，y)、R1(x，y)……RmTraversing the (x, y) pixel points, finding out the linear rectangular support area where each pixel point is located, preferentially extracting the linear rectangular support area to the linear rectangular support area with a larger Length value, and updating the linear rectangular support area to be R0(x，y)′、R1(x，y)′……Rm(x，y)′；
(10) For update region R0(x，y)′、R1(x，y)′......RmPerforming neighborhood detection on black point pixels in all rectangles (x, y)', removing isolated points without pixels in neighborhoods, traversing all rectangular support areas again, removing rectangular support areas with Width more than alpha or Length less than beta, and obtaining R0(x，y)″、R1(x，y)″……Rm(x, y) ", and then adding all R0(x，y)″、R1(x，y)″……RmThe line segments in the broad sides of the rectangle (x, y)' are connected and marked back to the original image G0(x，y)、G1(x，y)……Gn(x, y);
(11) All time series frames G0(x，y)、G1(x，y)……GnThe steel rail, the sleeper, the track bed and the turnout in (x, y) are locally reversely rotated according to the angle of the linear line segment, so that the horizontal setting of the key linear line segment is realized, ROI extraction can be conveniently carried out on the local linear line segment region of the steel rail, the sleeper, the track bed and the turnout through a rectangular frame after the horizontal setting, therefore, ROI extraction of different characteristics is realized, the identification and detection of depth accumulated water are completed according to a quantification method based on slope search and length accumulation of a multi-characteristic ROI region, and the identification result, the position information and the time information of the identification point are output to an interface service program;
(12) And (3) matching the sump and shallow water identification results identified and output in the steps (1) to (7) and the deep water identification results identified and output in the steps (8) to (11) by the interface service program according to a space-time relationship, outputting and storing the identification result, the position information and the time information of the matched current identification point into a database, and simultaneously sending the identification result, the position information and the time information to an intelligent alarm unit to push through different alarm levels.
2. The urban rail ponding identification method according to claim 1, characterized in that: the step (3) of identifying the related image properties reflected by the water surface based on the reflected attention mechanism unit specifically comprises searching an upper image reflected by the ponding water surface, searching reflection pixel points by matching an image area along a pixel column of the image, and traversing and searching by using multiple resolutions in vertical matching.
3. The urban rail ponding identification method according to claim 1, characterized in that: the model construction method of the attention mechanism unit based on reflection in the step (3) comprises the following steps: given a size of [ h, w, c]Is averaged in the horizontal direction and pooled to reduce to [ h, w/2, c](ii) a Then the vertical average pooling is performed to reduce it to a size of [ n, w/2]X of (2); thereafter, each row X of XiHas a size of [1, w/2 ]]Tiled or self-replicated to size [ n, w/2 ]]The feature maps from all rows are concatenated along feature axes to form a new feature map with dimensions [ n, w/2, c x n](ii) a Finally, this feature map is upsampled to [ h, w, c n [ ]]Size, and denoted X', and concatenating I n times along the characteristic axis, giving a size [ h, w, c n [ ]]Subtracting I 'from X' yields the size of [ h, w, c n]D, the subtracted feature map is connected with I' again in series, and is sent to the convolution layer to be activated by the ReLU function, so that a final output feature map with the same size as I is generated.
4. The urban rail ponding identification method according to claim 1, characterized in that: the model after the verification training in the step (5) specifically operates as follows: the FCN-8s-FL model, the FCN-8s-FL-5RAU model in step (3) are applied through the test set completed in step (1) and compared to the FCN-8s neural network model that does not use a reflex-based attention mechanism unit.
5. The urban rail ponding identification method according to claim 1, characterized in that: the quantization method based on the slope search and the length accumulation of the multi-feature ROI in the step (11) comprises the following steps: and calculating the number and length accumulation sums of all line segments with the slope values of the line segments of the steel rail, the sleeper, the track bed and the turnout being less than k in the ROI area, wherein the number and length accumulation sums of ROI linear line segments of the steel rail, the sleeper, the track bed and the turnout are respectively Counts-g, counts-d, counts-c1, counts-c2, lenth-g, lenth-d, lenth-c1 and Lenth-c2, when deep ponding occurs, the steel rail, the sleeper, the track bed and the turnout are covered by ponding, the linear line segments in the ROI area corresponding to the accumulated ponding disappear, and the Counts-g, counts-d, counts-c1, counts-c2, lenth-g, lenth-d, lenth-c1 and Lenth-c2 are suddenly reduced, and when the number n frames are continuously smaller than a threshold value, the deep ponding state is judged.
6. The utility model provides a city rail ponding identification system which characterized in that: the intelligent monitoring system comprises an image acquisition unit, a model training unit, a visual identification unit, a linear segment detection unit, an ROI (region of interest) quantization processing unit, an interface service program and an intelligent alarm unit, wherein the image acquisition unit is a camera which accords with a standard code stream output protocol; the model training unit is used for constructing an improved full convolution integral neural network, completing the training of a ponding recognition model and providing a basic model for visual recognition; the visual recognition unit carries out image recognition on a water pit and shallow ponding under the monitoring area through the model trained by the model training unit; the linear line segment detection unit is internally provided with an original linear line segment identification algorithm to realize the identification of the linear line segment characteristics of a fixed scene in an urban rail scene monitoring image; the ROI quantization processing unit is used for processing the image, extracting ROI from local linear line segment regions of a steel rail, a sleeper, a track bed and a turnout, and completing the identification and detection of depth accumulated water by a quantization method based on multi-feature ROI slope search and length accumulation; the interface service program is used for providing data flow service for each subsystem, matching according to the time-space relation of result data and outputting matched information to the intelligent alarm unit; and the intelligent alarm unit is used for sending the identification result to an urban rail emergency management center platform and pushing the identification result through different alarm levels.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210856178.0A CN115272849A (en) | 2022-07-19 | 2022-07-19 | Urban rail accumulated water identification method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210856178.0A CN115272849A (en) | 2022-07-19 | 2022-07-19 | Urban rail accumulated water identification method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115272849A true CN115272849A (en) | 2022-11-01 |
Family
ID=83766510
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210856178.0A Pending CN115272849A (en) | 2022-07-19 | 2022-07-19 | Urban rail accumulated water identification method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115272849A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117012006A (en) * | 2023-08-28 | 2023-11-07 | 浪潮智慧科技有限公司 | Flood disaster early warning method, equipment and medium for urban road |
-
2022
- 2022-07-19 CN CN202210856178.0A patent/CN115272849A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117012006A (en) * | 2023-08-28 | 2023-11-07 | 浪潮智慧科技有限公司 | Flood disaster early warning method, equipment and medium for urban road |
| CN117012006B (en) * | 2023-08-28 | 2024-03-08 | 浪潮智慧科技有限公司 | Flood disaster early warning method, equipment and medium for urban road |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111368687B (en) | Sidewalk vehicle illegal parking detection method based on target detection and semantic segmentation | |
| CN110232380B (en) | Fire night scene restoration method based on Mask R-CNN neural network | |
| CN111582029B (en) | Traffic sign identification method based on dense connection and attention mechanism | |
| CN112200172B (en) | Driving region detection method and device | |
| CN104298976A (en) | License plate detection method based on convolutional neural network | |
| CN109919883B (en) | Traffic video data acquisition method based on gray level conversion | |
| CN101022505A (en) | Mobile target in complex background automatic testing method and device | |
| CN111489339A (en) | Method for detecting defects of bolt spare nuts of high-speed railway positioner | |
| CN111080613B (en) | Image recognition method for damage fault of wagon bathtub | |
| CN112907597A (en) | Railway track line detection method based on deep convolutional neural network | |
| CN111080606A (en) | Method for identifying falling fault image of lower pull rod of railway wagon | |
| CN115272849A (en) | Urban rail accumulated water identification method and system | |
| CN112819000A (en) | Streetscape image semantic segmentation system, streetscape image semantic segmentation method, electronic equipment and computer readable medium | |
| CN111079629A (en) | Railway wagon center plate out-of-position fault image identification method | |
| CN116597270A (en) | Road damage target detection method based on attention mechanism integrated learning network | |
| CN111260631B (en) | Efficient rigid contact line structure light bar extraction method | |
| Yuliandoko et al. | Automatic vehicle counting using Raspberry pi and background subtractions method in the sidoarjo toll road | |
| CN112712552A (en) | Fault detection method for vehicle tread scratch | |
| CN111897993A (en) | Efficient target person track generation method based on pedestrian re-recognition | |
| CN112184669A (en) | Method for detecting faults of contact net suspension device on large-gradient line | |
| CN112991396B (en) | Target tracking method and device based on monitoring camera | |
| CN116402782A (en) | Road potential safety hazard detection method, system and storable medium | |
| CN114187550A (en) | Bow net core part identification method based on improved YOLO V3 network | |
| CN113763326A (en) | Pantograph detection method based on Mask scaling R-CNN network | |
| CN116778398B (en) | Bimodal track bed foreign matter detection method and system based on feature reconstruction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |