CN117218858A - Traffic safety early warning system and method for expressway - Google Patents
Traffic safety early warning system and method for expressway Download PDFInfo
- Publication number
- CN117218858A CN117218858A CN202311392183.1A CN202311392183A CN117218858A CN 117218858 A CN117218858 A CN 117218858A CN 202311392183 A CN202311392183 A CN 202311392183A CN 117218858 A CN117218858 A CN 117218858A
- Authority
- CN
- China
- Prior art keywords
- expressway
- real
- road
- sequence
- road state
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 43
- 238000007781 pre-processing Methods 0.000 claims abstract description 18
- 238000012549 training Methods 0.000 claims description 82
- 239000011159 matrix material Substances 0.000 claims description 53
- 230000007246 mechanism Effects 0.000 claims description 47
- 238000010586 diagram Methods 0.000 claims description 21
- 239000013598 vector Substances 0.000 claims description 18
- 238000000605 extraction Methods 0.000 claims description 13
- 238000003709 image segmentation Methods 0.000 claims description 11
- 230000008569 process Effects 0.000 claims description 8
- 238000005457 optimization Methods 0.000 claims description 7
- 230000004913 activation Effects 0.000 claims description 6
- 230000004931 aggregating effect Effects 0.000 claims description 5
- 238000011176 pooling Methods 0.000 claims description 4
- 238000012545 processing Methods 0.000 claims description 4
- 238000012546 transfer Methods 0.000 claims description 2
- 230000006870 function Effects 0.000 description 11
- 238000013527 convolutional neural network Methods 0.000 description 6
- 238000004458 analytical method Methods 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 230000004927 fusion Effects 0.000 description 3
- 238000002372 labelling Methods 0.000 description 3
- 238000001514 detection method Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 238000007477 logistic regression Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000000750 progressive effect Effects 0.000 description 2
- 206010039203 Road traffic accident Diseases 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000005094 computer simulation Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000012804 iterative process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
Landscapes
- Traffic Control Systems (AREA)
- Image Analysis (AREA)
Abstract
A traffic safety pre-warning system and method for expressway is disclosed. Firstly, acquiring expressway real-time road images of road sections to be analyzed, which are acquired by unmanned aerial vehicles, then, carrying out data preprocessing on the expressway real-time road images to obtain a sequence of expressway real-time road image blocks, then, extracting local real-time road state features of the sequence of expressway real-time road image blocks to obtain a sequence of expressway real-time road state feature matrices, then, extracting road section road state association features among the sequence of expressway real-time road state feature matrices to obtain an expressway road section road state association feature map, and finally, determining whether expressway traffic safety early warning is generated or not based on the expressway road section road state association feature map. Thus, whether the highway traffic safety pre-warning needs to be generated or not can be automatically judged.
Description
Technical Field
The present application relates to the field of traffic safety, and more particularly, to a traffic safety early warning system and method for expressways.
Background
The development of the expressway promotes the high-speed running of passenger flow, logistics and information flow, greatly improves the transportation efficiency and brings remarkable social and economic benefits. However, highways also present a risk of traffic safety. In order to effectively prevent and reduce traffic accidents on the expressway, real-time monitoring and analysis of the road state of the expressway are required, and traffic safety early warning information is timely provided for drivers or traffic management personnel.
However, the conventional road state monitoring method based on the fixed camera or the sensor has the problems of limited coverage, easiness in being influenced by weather and illumination and the like, and cannot meet the requirements of highway traffic safety early warning. Therefore, an optimized traffic safety precaution method for highways is desired.
Disclosure of Invention
In view of the above, the present application provides a traffic safety warning system and method for expressways, which can automatically determine whether to generate an expressway traffic safety warning, and inform drivers to adjust driving routes or inform traffic management personnel to take corresponding measures.
According to an aspect of the present application, there is provided a traffic safety precaution method for an expressway, including:
acquiring a real-time expressway road image of a road section to be analyzed, which is acquired by an unmanned aerial vehicle;
carrying out data preprocessing on the expressway real-time road image to obtain a sequence of expressway real-time road image blocks;
extracting local real-time road state characteristics of the sequence of the expressway real-time road image blocks to obtain a sequence of expressway real-time road state characteristic matrixes;
extracting road state association features among road segments among the sequences of the expressway real-time road state feature matrix to obtain an expressway road state association feature map; and
and determining whether to generate the expressway traffic safety early warning or not based on the expressway inter-road-section road state association feature map.
According to another aspect of the present application, there is provided a traffic safety precaution system for an expressway, comprising:
the image acquisition module is used for acquiring expressway real-time road images of the road section to be analyzed, which are acquired by the unmanned aerial vehicle;
the data preprocessing module is used for preprocessing the data of the expressway real-time road image to obtain a sequence of expressway real-time road image blocks;
the local real-time road state feature extraction module is used for extracting local real-time road state features of the sequence of the expressway real-time road image blocks to obtain a sequence of an expressway real-time road state feature matrix;
the inter-road state association feature extraction module is used for extracting inter-road state association features among the sequences of the expressway real-time road state feature matrix to obtain an expressway inter-road state association feature map; and
and the safety early warning module is used for determining whether to generate the expressway traffic safety early warning or not based on the expressway road state association characteristic diagram among expressway road sections.
According to the embodiment of the application, firstly, the expressway real-time road image of the road section to be analyzed, which is acquired by an unmanned plane, is acquired, then, the expressway real-time road image is subjected to data preprocessing to obtain a sequence of expressway real-time road image blocks, then, local real-time road state characteristics of the sequence of expressway real-time road image blocks are extracted to obtain a sequence of expressway real-time road state characteristic matrices, then, road section road state association characteristics among the sequence of expressway real-time road state characteristic matrices are extracted to obtain an expressway road section road state association characteristic diagram, and finally, whether expressway traffic safety early warning is generated or not is determined based on the expressway road section road state association characteristic diagram. Thus, whether the highway traffic safety pre-warning needs to be generated or not can be automatically judged.
Other features and aspects of the present application will become apparent from the following detailed description of exemplary embodiments, which proceeds with reference to the accompanying drawings.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments, features and aspects of the application and together with the description, serve to explain the principles of the application.
Fig. 1 shows a flow chart of a traffic safety precaution method for a highway according to an embodiment of the present application.
Fig. 2 shows a schematic architecture diagram of a traffic safety warning method for expressways according to an embodiment of the application.
Fig. 3 shows a block diagram of a traffic safety warning system for highways according to an embodiment of the application.
Fig. 4 shows an application scenario diagram of a traffic safety warning method for expressways according to an embodiment of the application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some, but not all embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are also within the scope of the application.
As used in the specification and in the claims, the terms "a," "an," "the," and/or "the" are not specific to a singular, but may include a plurality, unless the context clearly dictates otherwise. In general, the terms "comprises" and "comprising" merely indicate that the steps and elements are explicitly identified, and they do not constitute an exclusive list, as other steps or elements may be included in a method or apparatus.
Various exemplary embodiments, features and aspects of the application will be described in detail below with reference to the drawings. In the drawings, like reference numbers indicate identical or functionally similar elements. Although various aspects of the embodiments are illustrated in the accompanying drawings, the drawings are not necessarily drawn to scale unless specifically indicated.
In addition, numerous specific details are set forth in the following description in order to provide a better illustration of the application. It will be understood by those skilled in the art that the present application may be practiced without some of these specific details. In some instances, well known methods, procedures, components, and circuits have not been described in detail so as not to obscure the present application.
Aiming at the technical problems, the technical concept of the application is to collect the real-time road image of the expressway by utilizing the flexibility and the real-time property of the unmanned plane, and automatically judge whether the expressway traffic safety early warning needs to be generated or not by carrying out image analysis based on deep learning on the real-time road image of the expressway, and timely inform a driver to adjust a driving route or inform traffic management personnel to take corresponding measures.
Based on this, fig. 1 shows a flowchart of a traffic safety precaution method for an expressway according to an embodiment of the application. Fig. 2 shows a schematic architecture diagram of a traffic safety warning method for expressways according to an embodiment of the application. As shown in fig. 1 and 2, the traffic safety pre-warning method for expressways according to an embodiment of the present application includes the steps of: s110, acquiring expressway real-time road images of road sections to be analyzed, which are acquired by unmanned aerial vehicles; s120, carrying out data preprocessing on the expressway real-time road image to obtain a sequence of expressway real-time road image blocks; s130, extracting local real-time road state characteristics of the sequence of the expressway real-time road image blocks to obtain a sequence of an expressway real-time road state characteristic matrix; s140, extracting road state association features among road segments between the sequences of the expressway real-time road state feature matrix to obtain an expressway road state association feature map; and S150, determining whether to generate expressway traffic safety early warning or not based on the expressway inter-road-section road state association feature map.
It should be understood that the purpose of step S110 is to acquire the expressway road image through the unmanned aerial vehicle to acquire real-time road conditions, and the unmanned aerial vehicle can provide a viewing angle from top view, so as to capture the traffic conditions, vehicle density, road conditions and other information on the road. In step S120, the collected expressway road image is subjected to data preprocessing, where the preprocessing may include operations such as image denoising, image enhancement, and image segmentation, so as to extract a sequence of road image blocks, and dividing the image into blocks may make the subsequent feature extraction and analysis more efficient. In step S130, local real-time road status features, which may include information on vehicle speed, lane departure, vehicle density, etc., are extracted from the sequence of expressway real-time road image blocks, which may help to analyze traffic conditions and safety risks of the road. In step S140, road state association features between road segments, which may include information of relative positions between vehicles, speed differences between vehicles, etc., are extracted from the sequence of the expressway real-time road state feature matrix. By analyzing the road state association characteristics among road segments, the traffic flow condition and traffic safety risk among different road segments can be known. In step S150, based on the road state association feature map between expressway road segments, a decision of traffic safety early warning is made, and whether the current road state has safety risk is judged according to a preset early warning rule and model, if the safety risk exists, a corresponding traffic safety early warning is generated, and relevant departments and drivers are reminded to take corresponding measures so as to ensure traffic safety. The steps combine unmanned aerial vehicle image acquisition, data preprocessing, feature extraction and analysis and other technologies, and can help monitor and early warn traffic safety conditions on the expressway.
Specifically, in the technical scheme of the application, firstly, a real-time expressway road image of a road section to be analyzed, which is acquired by an unmanned aerial vehicle, is acquired; and image segmentation is carried out on the expressway real-time road image along the extending direction of the road section to be analyzed so as to obtain a sequence of expressway real-time road image blocks. Here, the real-time road image of the expressway is decomposed into smaller units by image segmentation, so that the road state feature extraction is conveniently performed on each unit. That is, in this way, it is possible to guide the subsequent network model to the local road state of the expressway real-time road image. In addition, the road generally has a certain continuity in the extending direction, and the geometric shape and structural information of the road can be better reserved by cutting the road image according to the extending direction. Doing so may divide the road into a plurality of smaller image blocks, making the road features in each image block more definite and prominent.
Accordingly, in step S120, the data preprocessing is performed on the expressway real-time road image to obtain a sequence of expressway real-time road image blocks, including: and carrying out image segmentation on the expressway real-time road image along the extending direction of the road section to be analyzed to obtain a sequence of expressway real-time road image blocks.
And then, the sequence of the expressway real-time road image blocks is passed through a road state feature extractor using a spatial attention mechanism to obtain a sequence of expressway real-time road state feature matrices. That is, local real-time road status features of the sequence of highway real-time road image blocks are extracted. Wherein, the network model can be induced to automatically pay attention to the most important areas in the image, such as information of vehicles, obstacles, road surface conditions and the like by using a spatial attention mechanism.
Accordingly, in step S130, extracting the local real-time road status feature of the sequence of the expressway real-time road image blocks to obtain a sequence of expressway real-time road status feature matrices, including: and the sequence of the expressway real-time road image blocks is processed by a road state feature extractor using a spatial attention mechanism to obtain the sequence of the expressway real-time road state feature matrix.
It is worth mentioning that the spatial attention mechanism (Spatial Attention Mechanism) is a common attention mechanism in deep learning, and is used to enhance the attention of the model to different positions in the input data. It can help the model more efficiently capture and utilize spatial information when processing spatially structured data (e.g., images, video, or speech, etc.). In conventional convolutional neural networks (Convolutional Neural Network, CNN), the convolutional layer extracts local features by means of sliding windows, but the feature processing for different locations is the same, without taking into account the relationship between locations. While the spatial attention mechanism enables the model to adaptively adjust the weights of features according to different locations of the input data by introducing a learnable weight. The spatial attention mechanism generally includes the following steps: 1. feature extraction: features are extracted from the input data by a series of convolution and pooling operations. 2. Attention weight calculation: from the extracted features, an attention weight for each location is calculated. This may be achieved by different methods, for example using a full connection layer, a convolution layer, or a self-attention mechanism (self-attention), etc. 3. Feature weighting: attention weights are applied to the extracted features, weighting the features at different locations. Typically, features are multiplied or weighted summed element by element with the attention weights. 4. Feature fusion: and fusing the weighted features to obtain a final representation or a feature map. By introducing a spatial attention mechanism, the model can automatically learn the attention degree of different positions in the input data, thereby better capturing the spatial structure and important characteristics. The method has important roles in tasks such as image classification, target detection, image segmentation and the like, and can improve the performance and generalization capability of the model.
Wherein more specifically, passing the sequence of highway real-time road image blocks through a road state feature extractor using a spatial attention mechanism to obtain the sequence of highway real-time road state feature matrices comprises: each layer of the road state feature extractor using the spatial attention mechanism performs the following steps on input data in the forward transfer process of the layer: respectively carrying out convolution processing on input data to generate a sequence of convolution characteristic diagrams; respectively carrying out pooling treatment on the sequences of the convolution feature graphs to generate sequences of pooled feature graphs; respectively carrying out nonlinear activation on the sequences of the pooled feature maps to generate sequences of activated feature maps; respectively calculating the average value of each position of each activation feature graph in the sequence of the activation feature graphs along the channel dimension to generate a sequence of a spatial feature matrix; respectively calculating a class Softmax function value of each position of each spatial feature matrix in the sequence of the spatial feature matrix to obtain a sequence of spatial score matrices; respectively calculating the sequence of the space feature matrix and the position-wise dot multiplication of the sequence of the space score matrix to obtain a plurality of feature matrices; wherein the plurality of feature matrices output by the last layer of the road state feature extractor using a spatial attention mechanism is a sequence of the highway real-time road state feature matrices.
And then, aggregating the sequence of the expressway real-time road state feature matrix into an expressway real-time road state input tensor along the channel dimension, and obtaining an expressway inter-road state association feature map by using an inter-road state association feature extractor of a channel attention mechanism. That is, the inter-road state association features between the sequences of the expressway real-time road state feature matrix are extracted. It should be appreciated that by capturing road state changes and associations between different image tiles, such as changes in traffic flow between image tiles, etc., the safety conditions and road traffic conditions of the entire road segment to be analyzed may be reflected. In addition, the channel attention mechanism can be adaptively enhanced or suppressed for different characteristic channels, so that the characteristic distribution of the image blocks with high road safety risks is highlighted. And the road state association characteristic diagram among the expressway sections is used for obtaining a classification result through a classifier, and the classification result is used for indicating whether expressway traffic safety early warning is generated or not.
Accordingly, in step S140, extracting the inter-road state association feature between the sequences of the real-time road state feature matrix of the expressway to obtain an inter-road state association feature map of the expressway, including: and aggregating the sequence of the expressway real-time road state feature matrix into an expressway real-time road state input tensor along a channel dimension, and obtaining the expressway inter-road state association feature map through an inter-road state association feature extractor using a channel attention mechanism.
It is noted that the channel attention mechanism (Channel Attention Mechanism) is a common attention mechanism used in deep learning to enhance the attention of the model to different channels in the input data. It can help the model adaptively adjust the weights of the channels as it processes features in the channel dimension to extract more useful and important information. In a conventional convolutional neural network (Convolutional Neural Network, CNN), the convolutional layer extracts features in the channel dimension by operating on filters for each channel. However, the importance of features may vary from channel to channel, and the features of some channels may be more critical to a particular task. The channel attention mechanism enables the model to automatically learn the importance of different channels by introducing a learnable weight, and adjusts the weight of the channel. The channel attention mechanism typically includes the following steps: 1. feature extraction: features are extracted from the input data by a series of convolution and pooling operations. 2. And (3) channel characteristic statistics: the extracted features are counted, and the statistical features of each channel, such as average value, maximum value and the like, are calculated. 3. Attention weight calculation: the attention weight of each channel is calculated from the statistical characteristics of the channel. This calculation is typically accomplished using a fully connected or convolutional layer. 4. Feature weighting: attention weights are applied to the extracted features, weighting the features of the different channels. Typically, features are multiplied element by element with attention weights. 5. Feature fusion: and fusing the weighted features to obtain a final representation or a feature map. By introducing a channel attention mechanism, the model can adaptively adjust the weights of the channels so that channels that are more useful for a particular task are given higher weights, thereby improving the performance and generalization ability of the model. The channel attention mechanism plays an important role in tasks such as image classification, target detection, image segmentation and the like, and can help the model to better utilize information in channel dimension.
Accordingly, in step S150, determining whether to generate the highway traffic safety precaution based on the highway inter-road state association feature map includes: and the road state association feature diagram among the expressway sections is used for obtaining a classification result through a classifier, and the classification result is used for indicating whether expressway traffic safety early warning is generated or not.
More specifically, the expressway inter-road state association feature map is passed through a classifier to obtain a classification result, wherein the classification result is used for indicating whether expressway traffic safety precaution is generated or not, and the method comprises the following steps: expanding the road state association feature map among expressway road sections into classification feature vectors according to row vectors or column vectors; performing full-connection coding on the classification feature vectors by using a full-connection layer of the classifier to obtain coded classification feature vectors; and inputting the coding classification feature vector into a Softmax classification function of the classifier to obtain the classification result.
That is, in the technical solution of the present application, the labeling of the classifier includes generating an expressway traffic safety precaution (first labeling) and not generating an expressway traffic safety precaution (second labeling), where the classifier determines to which classification label the expressway inter-road-section road state association feature map belongs through a soft maximum function. It should be noted that the first tag p1 and the second tag p2 do not include a manually set concept, and in fact, during the training process, the computer model does not have a concept of "whether to generate highway traffic safety precautions", which is simply that there are two kinds of classification tags and the probability that the output features are under the two classification tags, i.e., the sum of p1 and p2 is one. Therefore, the classification result of whether to generate the expressway traffic safety precaution is actually converted into the classified probability distribution conforming to the natural rule through the classification label, and the physical meaning of the natural probability distribution of the label is essentially used instead of the language text meaning of whether to generate the expressway traffic safety precaution.
It should be appreciated that the role of the classifier is to learn the classification rules and classifier using a given class, known training data, and then classify (or predict) the unknown data. Logistic regression (logistics), SVM, etc. are commonly used to solve the classification problem, and for multi-classification problems (multi-class classification), logistic regression or SVM can be used as well, but multiple bi-classifications are required to compose multiple classifications, but this is error-prone and inefficient, and the commonly used multi-classification method is the Softmax classification function.
Further, in the technical scheme of the application, the traffic safety early warning method for the expressway further comprises the training steps of: training the road state feature extractor using the spatial attention mechanism, the inter-road state association feature extractor using the channel attention mechanism and the classifier. It should be understood that the training step plays a role in the highway traffic safety early warning method in learning and optimizing parameters of the model by using the noted data, so that the model can better understand and predict the road state, thereby realizing accurate traffic safety early warning. Specifically, the training step includes training a road state feature extractor, an inter-road state associated feature extractor, and a classifier, wherein a spatial attention mechanism and a channel attention mechanism are used. 1. Training of the road state feature extractor: the road state feature extractor is a model for extracting road state features from the expressway real-time road image. By training the road condition feature extractor, the model can learn to extract useful features about the road condition from the image. These features may include information about vehicles, pedestrians, road signs, road markings, etc. In the training process, the marked road state data is used as a training sample, and the parameters of the road state feature extractor are optimized by minimizing the difference between the prediction result and the real label. 2. Training of a road state association feature extractor between road segments: the inter-road-state-associated feature extractor is a model for extracting inter-road-associated features from road-state features. It can help the model capture relationships between different road segments, such as changes in traffic flow, flow of vehicles, etc. By training the inter-road-state-association feature extractor, the model can learn to extract useful features of the inter-road association from the road-state features. In the training process, the marked road state data and the road section association information are used as training samples, and parameters of the road section road state association feature extractor are optimized by minimizing the difference between the prediction result and the real label. 3. Training of a classifier: the classifier is a model for mapping the extracted features to specific traffic safety precaution categories. By training the classifier, the model can learn the mapping relation between the road state characteristics, the inter-road-section association characteristics and the traffic safety early warning categories. In the training process, the marked road state data and the corresponding traffic safety early warning category are used as training samples, and the parameters of the classifier are optimized by minimizing the difference between the prediction result and the real label. Through the training step, the model can gradually learn to extract useful features related to road states and inter-road section association from the road images, and use the features for accurate traffic safety precaution. The training can improve the performance and generalization capability of the model, so that the model can be better adapted to different road conditions and traffic scenes.
Wherein, in one example, the training step comprises: acquiring training data, wherein the training data comprises training expressway real-time road images of road sections to be analyzed, which are acquired by unmanned aerial vehicles, and whether a true value of expressway traffic safety precaution is generated or not; image segmentation is carried out on the training expressway real-time road image along the extending direction of the road section to be analyzed so as to obtain a sequence of training expressway real-time road image blocks; passing the sequence of the training expressway real-time road image blocks through the road state feature extractor using a spatial attention mechanism to obtain a sequence of a training expressway real-time road state feature matrix; aggregating the sequence of the training expressway real-time road state feature matrix into a training expressway real-time road state input tensor along a channel dimension, and then obtaining a training expressway inter-road state association feature map through the inter-road state association feature extractor using a channel attention mechanism; the road state association feature diagram among the training expressway sections is passed through a classifier to obtain a classification loss function value; and training the road state feature extractor using the spatial attention mechanism, the road state association feature extractor using the channel attention mechanism and the classifier by using the classification loss function value, wherein in each iteration of training, iterative optimization of a weight matrix in a weight space is performed on the road state association feature vector between training expressway road segments obtained after the road state association feature map between training expressway segments is developed.
In the technical scheme of the application, after a sequence of the training expressway real-time road image blocks is processed by using a road state feature extractor of a spatial attention mechanism, each obtained training expressway real-time road state feature matrix in the sequence of the training expressway real-time road state feature matrix expresses the spatial distribution reinforced image semantic features of the corresponding training expressway real-time road image blocks, the training expressway real-time road state feature matrix is aggregated into a training expressway real-time road state input tensor along a channel dimension, then the obtained training expressway road state association feature map is further processed by using a road state association feature extractor of the channel attention mechanism on the basis of the spatial distribution reinforced in the local image semantic space of the image semantic features, so that the road state association feature map of the training expressway has a spatial distribution dimension corresponding to the local image semantic space and the global image semantic space and a channel distribution dimension fusion dimension, and each obtained by using the road state association feature extractor of the road state association feature map of the channel attention mechanism also has a dense distribution weight corresponding to the spatial distribution dimension corresponding to the local image semantic space and the global image semantic space, and the training expressway state association feature map is further trained in the global image semantic space, and the training expressway state association feature map is obtained by the training expressway state association feature matrix is reduced by the training expressway state association feature matrix.
Based on the above, when the applicant performs classification regression training on the training expressway inter-road-section road state association feature map through a classifier, the applicant performs iterative optimization of a weight matrix in a weight space based on the training expressway inter-road-section road state association feature map after developing the training expressway inter-road-section road state association feature map.
Accordingly, in a specific example, in each iteration of the training, performing iterative optimization of a weight matrix in a weight space on the training expressway inter-road state association feature vector obtained after the training expressway inter-road state association feature map is developed, where the method includes: in each iteration of training, carrying out iterative optimization of a weight matrix in a weight space based on the training expressway inter-road state association feature vector obtained after the training expressway inter-road state association feature map is unfolded according to the following optimization formula; wherein, the optimization formula is:wherein (1)>And->The weight matrix of the last iteration and the current iteration are respectively adopted, wherein, during the first iteration, different initialization strategies are adopted to set +.>And->(e.g.)>Set as a unitary matrix->Set as the diagonal matrix of the mean value of the feature vector to be classified),>is to be treated withClassified training expressway inter-road-section road-state-associated feature vector +.>And->Respectively represent feature vector +>And->Global mean of (2), and->Is a bias matrix, e.g. initially set as a unity matrix, the vectors being in the form of column vectors, +.>Representing vector multiplication, ++>Representing matrix addition, ++>Representing the multiplication by the position point,representing a transpose operation->Representing maximum function +.>Representing the optimized weight matrix.
That is, consider that the feature vector is associated with the road state between the expressway sections based on the training to be classifiedDuring the dense prediction task of (1), the high-resolution representation of the weight matrix is required to be associated with the feature vector of the road state between the training expressway sections to be classified/>The image semantic feature distribution dimension dense association context is integrated, so that progressive integration (progressive integrity) is realized based on iterative association representation resource-aware (resource-aware) by maximizing a distribution boundary of a weight space in an iterative process, thereby improving the training effect of a weight matrix and improving the training efficiency of the whole model.
In summary, according to the traffic safety early warning method for the expressway provided by the embodiment of the application, whether the expressway traffic safety early warning needs to be generated or not can be automatically judged, and a driver is timely informed of adjusting a driving route or informing traffic management staff of taking corresponding measures.
Fig. 3 shows a block diagram of a traffic safety warning system 100 for highways according to an embodiment of the application. As shown in fig. 3, a traffic safety precaution system 100 for an expressway according to an embodiment of the application includes: the image acquisition module 110 is used for acquiring expressway real-time road images of the road section to be analyzed, which are acquired by the unmanned aerial vehicle; a data preprocessing module 120, configured to perform data preprocessing on the expressway real-time road image to obtain a sequence of expressway real-time road image blocks; a local real-time road state feature extraction module 130, configured to extract local real-time road state features of the sequence of expressway real-time road image blocks to obtain a sequence of expressway real-time road state feature matrices; the inter-road state association feature extraction module 140 is configured to extract inter-road state association features between the sequences of the real-time road state feature matrix of the expressway to obtain an inter-road state association feature map of the expressway; and a safety pre-warning module 150, configured to determine whether to generate a highway traffic safety pre-warning based on the highway section road state association feature map.
In one possible implementation, the data preprocessing module 120 is configured to: and carrying out image segmentation on the expressway real-time road image along the extending direction of the road section to be analyzed to obtain a sequence of expressway real-time road image blocks.
In one possible implementation, the local real-time road state feature extraction module 130 is configured to: and the sequence of the expressway real-time road image blocks is processed by a road state feature extractor using a spatial attention mechanism to obtain the sequence of the expressway real-time road state feature matrix.
Here, it will be understood by those skilled in the art that the specific functions and operations of the respective units and modules in the above-described traffic safety early warning system for expressways 100 have been described in detail in the above description of the traffic safety early warning method for expressways with reference to fig. 1 to 2, and thus, repetitive descriptions thereof will be omitted.
As described above, the traffic safety early warning system 100 for expressways according to the embodiment of the present application may be implemented in various wireless terminals, such as a server or the like having a traffic safety early warning algorithm for expressways. In one possible implementation, the traffic safety warning system 100 for highways according to an embodiment of the application may be integrated into the wireless terminal as a software module and/or hardware module. For example, the traffic safety warning system 100 for highways may be a software module in the operating system of the wireless terminal, or may be an application developed for the wireless terminal; of course, the traffic safety warning system 100 for an expressway may also be one of a plurality of hardware modules of the wireless terminal.
Alternatively, in another example, the traffic safety pre-warning system 100 for expressways and the wireless terminal may be separate devices, and the traffic safety pre-warning system 100 for expressways may be connected to the wireless terminal through a wired and/or wireless network and transmit interactive information in an agreed data format.
Fig. 4 shows an application scenario diagram of a traffic safety warning method for expressways according to an embodiment of the application. As shown in fig. 4, in this application scenario, first, an expressway real-time road image of a road section to be analyzed (for example, D illustrated in fig. 4) acquired by an unmanned aerial vehicle is acquired, and then, the expressway real-time road image is input to a server (for example, S illustrated in fig. 4) in which a traffic safety warning algorithm for expressways is deployed, wherein the server can process the expressway real-time road image using the traffic safety warning algorithm for expressways to obtain a classification result for indicating whether or not to generate an expressway traffic safety warning.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The foregoing description of embodiments of the application has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the various embodiments described. The terminology used herein was chosen in order to best explain the principles of the embodiments, the practical application, or the improvement of technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.
Claims (10)
1. A traffic safety pre-warning method for a highway, comprising:
acquiring a real-time expressway road image of a road section to be analyzed, which is acquired by an unmanned aerial vehicle;
carrying out data preprocessing on the expressway real-time road image to obtain a sequence of expressway real-time road image blocks;
extracting local real-time road state characteristics of the sequence of the expressway real-time road image blocks to obtain a sequence of expressway real-time road state characteristic matrixes;
extracting road state association features among road segments among the sequences of the expressway real-time road state feature matrix to obtain an expressway road state association feature map; and
and determining whether to generate the expressway traffic safety early warning or not based on the expressway inter-road-section road state association feature map.
2. The traffic safety pre-warning method for an expressway according to claim 1, characterized in that the data pre-processing of the expressway real-time road image to obtain a sequence of expressway real-time road image blocks comprises:
and carrying out image segmentation on the expressway real-time road image along the extending direction of the road section to be analyzed to obtain a sequence of expressway real-time road image blocks.
3. The traffic safety pre-warning method for an expressway according to claim 2, wherein extracting local real-time road state features of the sequence of expressway real-time road image blocks to obtain a sequence of expressway real-time road state feature matrices comprises:
and the sequence of the expressway real-time road image blocks is processed by a road state feature extractor using a spatial attention mechanism to obtain the sequence of the expressway real-time road state feature matrix.
4. A traffic safety precaution method for a highway according to claim 3 and wherein passing the sequence of highway real-time road image blocks through a road state feature extractor using a spatial attention mechanism to obtain the sequence of highway real-time road state feature matrices comprises:
each layer of the road state feature extractor using the spatial attention mechanism performs the following steps on input data in the forward transfer process of the layer:
respectively carrying out convolution processing on input data to generate a sequence of convolution characteristic diagrams;
respectively carrying out pooling treatment on the sequences of the convolution feature graphs to generate sequences of pooled feature graphs;
respectively carrying out nonlinear activation on the sequences of the pooled feature maps to generate sequences of activated feature maps;
respectively calculating the average value of each position of each activation feature graph in the sequence of the activation feature graphs along the channel dimension to generate a sequence of a spatial feature matrix;
respectively calculating a class Softmax function value of each position of each spatial feature matrix in the sequence of the spatial feature matrix to obtain a sequence of spatial score matrices; and
respectively calculating the sequence of the space feature matrix and the sequence of the space score matrix, and multiplying the position points of the sequence of the space score matrix to obtain a plurality of feature matrices;
wherein the plurality of feature matrices output by the last layer of the road state feature extractor using a spatial attention mechanism is a sequence of the highway real-time road state feature matrices.
5. The traffic safety pre-warning method for expressway according to claim 4, wherein extracting inter-road state association features between sequences of the expressway real-time road state feature matrix to obtain an expressway inter-road state association feature map comprises:
and aggregating the sequence of the expressway real-time road state feature matrix into an expressway real-time road state input tensor along a channel dimension, and obtaining the expressway inter-road state association feature map through an inter-road state association feature extractor using a channel attention mechanism.
6. The traffic safety warning method for expressway according to claim 5, characterized in that determining whether to generate an expressway traffic safety warning based on the expressway inter-road-section road state association feature map includes:
and the road state association feature diagram among the expressway sections is used for obtaining a classification result through a classifier, and the classification result is used for indicating whether expressway traffic safety early warning is generated or not.
7. The traffic safety precaution method for a highway according to claim 6, further comprising the training step of: training the road state feature extractor using a spatial attention mechanism, the inter-road state association feature extractor using a channel attention mechanism and the classifier;
wherein the training step comprises:
acquiring training data, wherein the training data comprises training expressway real-time road images of road sections to be analyzed, which are acquired by unmanned aerial vehicles, and whether a true value of expressway traffic safety precaution is generated or not;
image segmentation is carried out on the training expressway real-time road image along the extending direction of the road section to be analyzed so as to obtain a sequence of training expressway real-time road image blocks;
passing the sequence of the training expressway real-time road image blocks through the road state feature extractor using a spatial attention mechanism to obtain a sequence of a training expressway real-time road state feature matrix;
aggregating the sequence of the training expressway real-time road state feature matrix into a training expressway real-time road state input tensor along a channel dimension, and then obtaining a training expressway inter-road state association feature map through the inter-road state association feature extractor using a channel attention mechanism;
the road state association feature diagram among the training expressway sections is passed through a classifier to obtain a classification loss function value; and
and training the road state feature extractor using the spatial attention mechanism, the road state association feature extractor using the channel attention mechanism and the classifier by using the classification loss function value, wherein in each iteration of training, the training expressway road state association feature vector obtained after the training expressway road state association feature map is unfolded carries out iterative optimization of a weight matrix in a weight space.
8. A traffic safety warning system for an expressway, comprising:
the image acquisition module is used for acquiring expressway real-time road images of the road section to be analyzed, which are acquired by the unmanned aerial vehicle;
the data preprocessing module is used for preprocessing the data of the expressway real-time road image to obtain a sequence of expressway real-time road image blocks;
the local real-time road state feature extraction module is used for extracting local real-time road state features of the sequence of the expressway real-time road image blocks to obtain a sequence of an expressway real-time road state feature matrix;
the inter-road state association feature extraction module is used for extracting inter-road state association features among the sequences of the expressway real-time road state feature matrix to obtain an expressway inter-road state association feature map; and
and the safety early warning module is used for determining whether to generate the expressway traffic safety early warning or not based on the expressway road state association characteristic diagram among expressway road sections.
9. The traffic safety precaution system for a highway according to claim 8, wherein said data preprocessing module is configured to:
and carrying out image segmentation on the expressway real-time road image along the extending direction of the road section to be analyzed to obtain a sequence of expressway real-time road image blocks.
10. The traffic safety warning system for highways of claim 9, wherein said local real-time road status feature extraction module is configured to:
and the sequence of the expressway real-time road image blocks is processed by a road state feature extractor using a spatial attention mechanism to obtain the sequence of the expressway real-time road state feature matrix.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311392183.1A CN117218858A (en) | 2023-10-25 | 2023-10-25 | Traffic safety early warning system and method for expressway |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311392183.1A CN117218858A (en) | 2023-10-25 | 2023-10-25 | Traffic safety early warning system and method for expressway |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117218858A true CN117218858A (en) | 2023-12-12 |
Family
ID=89039207
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311392183.1A Pending CN117218858A (en) | 2023-10-25 | 2023-10-25 | Traffic safety early warning system and method for expressway |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117218858A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190108384A1 (en) * | 2017-10-05 | 2019-04-11 | TuSimple | System and method for aerial video traffic analysis |
| CN112560609A (en) * | 2020-12-03 | 2021-03-26 | 北京百度网讯科技有限公司 | Road condition estimation method, method for establishing road condition estimation model and corresponding device |
| US11222217B1 (en) * | 2020-08-14 | 2022-01-11 | Tsinghua University | Detection method using fusion network based on attention mechanism, and terminal device |
| CN116631190A (en) * | 2023-06-02 | 2023-08-22 | 中国计量大学现代科技学院 | Intelligent traffic monitoring system and method thereof |
| CN116909317A (en) * | 2023-09-14 | 2023-10-20 | 飞客工场科技(北京)有限公司 | Unmanned aerial vehicle control system and method based on terminal Internet of vehicles |
-
2023
- 2023-10-25 CN CN202311392183.1A patent/CN117218858A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190108384A1 (en) * | 2017-10-05 | 2019-04-11 | TuSimple | System and method for aerial video traffic analysis |
| US11222217B1 (en) * | 2020-08-14 | 2022-01-11 | Tsinghua University | Detection method using fusion network based on attention mechanism, and terminal device |
| CN112560609A (en) * | 2020-12-03 | 2021-03-26 | 北京百度网讯科技有限公司 | Road condition estimation method, method for establishing road condition estimation model and corresponding device |
| CN116631190A (en) * | 2023-06-02 | 2023-08-22 | 中国计量大学现代科技学院 | Intelligent traffic monitoring system and method thereof |
| CN116909317A (en) * | 2023-09-14 | 2023-10-20 | 飞客工场科技(北京)有限公司 | Unmanned aerial vehicle control system and method based on terminal Internet of vehicles |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Alghmgham et al. | Autonomous traffic sign (ATSR) detection and recognition using deep CNN | |
| CN110147763B (en) | Video semantic segmentation method based on convolutional neural network | |
| Yang et al. | Image-based visibility estimation algorithm for intelligent transportation systems | |
| CN110263786B (en) | Road multi-target identification system and method based on feature dimension fusion | |
| CN114092917B (en) | MR-SSD-based shielded traffic sign detection method and system | |
| Vaiyapuri et al. | Automatic Vehicle License Plate Recognition Using Optimal Deep Learning Model. | |
| Al Mamun et al. | Lane marking detection using simple encode decode deep learning technique: SegNet | |
| Dias et al. | Anonymous vehicle detection for secure campuses: A framework for license plate recognition using deep learning | |
| Kuchkorov et al. | Traffic and road sign recognition using deep convolutional neural network | |
| BARODI et al. | Improved deep learning performance for real-time traffic sign detection and recognition applicable to intelligent transportation systems | |
| CN114495060A (en) | Road traffic marking identification method and device | |
| Bailke et al. | Traffic sign classification using CNN | |
| Saravanarajan et al. | Improving semantic segmentation under hazy weather for autonomous vehicles using explainable artificial intelligence and adaptive dehazing approach | |
| Zhang et al. | A front vehicle detection algorithm for intelligent vehicle based on improved gabor filter and SVM | |
| CN116872961A (en) | Control system for intelligent driving vehicle | |
| CN117218858A (en) | Traffic safety early warning system and method for expressway | |
| CN113344121B (en) | Method for training a sign classification model and sign classification | |
| Jabeen et al. | Weather classification on roads for drivers assistance using deep transferred features | |
| Choda et al. | A critical survey on real-time traffic sign recognition by using cnn machine learning algorithm | |
| CN114707567A (en) | Trajectory classification method, trajectory classification model training method and computer program product | |
| Rao et al. | A Deep Learning Approach Towards Building Intelligent Transport System | |
| Rani et al. | Driver Assistant System using YOLO V3 and VGGNET | |
| CN116503406B (en) | Hydraulic engineering information management system based on big data | |
| Ng et al. | Real-Time Detection of Objects on Roads for Autonomous Vehicles Using Deep Learning | |
| CN115375978B (en) | Behavior information determination method and apparatus, storage medium, and electronic apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |