CN111310607B - Highway safety risk identification method and system based on computer vision and artificial intelligence - Google Patents
Highway safety risk identification method and system based on computer vision and artificial intelligence Download PDFInfo
- Publication number
- CN111310607B CN111310607B CN202010073871.1A CN202010073871A CN111310607B CN 111310607 B CN111310607 B CN 111310607B CN 202010073871 A CN202010073871 A CN 202010073871A CN 111310607 B CN111310607 B CN 111310607B
- Authority
- CN
- China
- Prior art keywords
- road
- information
- image
- safety risk
- knowledge representation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/251—Fusion techniques of input or preprocessed data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a road safety risk identification method and a system based on computer vision and artificial intelligence, wherein the method comprises the following steps: extracting image characteristics of road image information, and identifying a target area containing a preset potential safety hazard target in a road image, a target type in the target area, scene information of a road section in the road image and first safety risk information in the road image; calling feature information of each road section; carrying out information fusion on the corresponding road images to obtain knowledge representation of each fused road image and the road video sequence; identifying second security risk information for the road video sequence; and identifying final safety risk information of the road video sequence based on a pre-trained second decision model according to the knowledge representation of the road video sequence, the first safety risk information and the second safety risk information. The invention can improve the efficiency, accuracy and reliability of highway safety risk identification.
Description
Technical Field
The invention relates to the field of road safety risk detection, in particular to a road safety risk identification method and system based on computer vision and artificial intelligence.
Background
With the continuous increase of automobile holding capacity and the acceleration of urbanization process, the problem of road traffic safety has become one of the main problems affecting the production safety level of the national people in China. According to statistical data, the total number of traffic accidents in China before 2002 is on the trend of increasing year by year, and in 2002-2015, the number of the safety accidents is gradually reduced from the highest 773137 times in 2002 to 187781 times in 2015, and the reduction rate is 75.71%, which shows that the road traffic safety situation in China is gradually improved. However, according to the statistical data in 2015, 4 indexes of national road traffic accident number, injured people number, dead people number and direct economic loss respectively reach 18.8 thousands, 20.0 thousands, 5.8 thousands and 10.4 million yuan, and a certain distance is still formed between the indexes and the international leading road safety level, so that the national road traffic safety situation is still severe.
By evaluating the traffic safety of the domestic highway and timely modifying and protecting the highway with irregular construction and risk, the frequency of traffic accidents can be effectively reduced, and casualties and economic losses caused by the traffic accidents are greatly reduced.
The conventional road safety risk assessment method mainly depends on the following means:
(1) the method comprises the steps of collecting video information and other related information of a road, watching the collected road video through an expert, and extracting dozens of factors (whether guardrail ends exist or not and whether the type of the middle-zone guardrail is reasonable or not) in the video. Each evaluation of the method needs to rely on manual identification and extraction of a large amount of information from the acquired images (for example, ten to dozens of unequal factors need to be manually searched from each image), which is huge for labor consumption. Meanwhile, the method has high professional requirements on workers, and the problems that different workers have different subjective judgments, the standards are not uniform, the working time is increased at any time, the working quality and the working efficiency are seriously reduced and the like exist.
(2) Some information is obtained by other means (e.g. statistics of daily accumulation such as traffic accident rate, traffic flow). And finally, calculating to obtain the road danger safety judgment through a calculation model, an analysis method and the like. In such a means, the information sources are usually through traffic accident investigation, traffic volume statistics and the like, and the information usually cannot reflect the road condition intuitively, and meanwhile, the information has certain hysteresis. Therefore, the result of the determination based on such information has a great limitation in accuracy or interpretability.
Disclosure of Invention
The invention aims to provide a road safety risk identification method and system based on computer vision and artificial intelligence, so as to solve the problems of low efficiency, accuracy and reliability of the conventional road safety risk identification.
According to a first aspect of the invention, a road safety risk identification method based on computer vision and artificial intelligence comprises the following steps: extracting image features of road image information by using a pre-trained convolutional neural network model, and identifying a target area containing a preset potential safety hazard target in a road image, a target type in the target area, scene information of a road section in the road image and first safety risk information in the road image according to the image features; calling feature information of each road section according to the longitude and latitude information of each road section in the road image information; performing information fusion on corresponding road images according to the target area of each road image, the target category in the target area, the scene information of each road section and the characteristic information obtained by identification to obtain the knowledge representation of each road image after fusion, and obtaining the knowledge representation of a road video sequence containing each road image by fusion based on the knowledge representation of each road image after fusion according to the time sequence relation among the road images; identifying second safety risk information of the road video sequence based on a pre-established expert knowledge base and a first decision model according to the knowledge representation of the road video sequence; and identifying final safety risk information of the road video sequence based on a pre-trained second decision model according to the knowledge representation of the road video sequence, the first safety risk information and the second safety risk information.
Further, the step of identifying second security risk information of the road video sequence based on a pre-established expert knowledge base and a first decision model according to the knowledge representation of the road video sequence comprises: inputting the knowledge representation of the road video sequence into a pre-constructed expert knowledge base, and identifying to obtain third safety risk information and corresponding confidence coefficient, wherein the confidence coefficient value corresponding to the third safety risk information is 0 or 1; inputting a feature vector extracted based on knowledge representation of a road video sequence into a first decision model, and identifying and obtaining fourth safety risk information and corresponding confidence coefficient of the road video sequence, wherein the value range of the confidence coefficient corresponding to the fourth safety risk information is 0-1; and according to the confidence degree of the third safety risk information and the confidence degree of the fourth safety risk information, fusing the third safety risk information and the fourth safety risk information to obtain the second safety risk information.
Further, the step of identifying a target area containing a preset target in the road image and a target category in the target area, scene information of a road section in the road image and first safety risk information in the road image according to the image features comprises: inputting the image characteristics into a target detection network, and identifying a target area containing a preset target in a road image, a target type in the target area and a confidence coefficient of the target type; and classifying the image features, and identifying scene information of road sections in the road image, the confidence coefficient of the scene information, first safety risk information in the road image and the confidence coefficient of the first safety risk information.
Further, the step of calling the feature information of each road section according to the longitude and latitude information of each road section in the road image information comprises: and inputting the longitude and latitude information of each road section in the road image information into a third-party interface, and calling the characteristic information of each road section, wherein the characteristic information comprises speed limit information, curve information or/and design drawing information of the road section.
Further, the step of performing information fusion on the corresponding road images according to the target area of each road image, the target category in the target area, the scene information of each road section and the characteristic information obtained by identification to obtain the knowledge representation of each road image after fusion comprises: correcting the knowledge representation of the road sections of which the confidence degrees of the target categories in the road sections to be corrected are lower than a first threshold value, the confidence degrees of the scene information are lower than a second threshold value or the confidence degree of the first safety risk information is lower than a third threshold value according to the feature information of the corresponding road sections to obtain the knowledge representation of the road images after fusion; the road section to be corrected is a road section which contains the characteristic information and image identification information in the road image information at the same time, and the image identification information comprises the target area, the target category and scene information; for the characteristic information of the road section of which the confidence coefficient of the target category in each road section to be corrected is higher than a fourth threshold value, the confidence coefficient of the scene information is higher than a fifth threshold value or the confidence coefficient of the first safety risk information is higher than a sixth threshold value, correcting according to the target category, the scene information or the first safety risk information of the corresponding road section to obtain the knowledge representation of the road image after fusion; and for the road sections which do not have the feature information and the image identification information in the road image information at the same time, acquiring the knowledge representation of the road image after fusion according to the feature information or the image identification information.
Further, the step of obtaining a knowledge representation of a road video sequence including each road image by fusion based on the fused knowledge representation of each road image includes: and sequencing the road images according to the sampling time, and correcting the knowledge representation of each road image by using the knowledge representation of the front road image and the rear road image to obtain the knowledge representation of the road video sequence comprising each road image.
According to a second aspect of the invention, a computer vision and artificial intelligence based road safety risk identification system comprises: the visual information module is used for extracting image characteristics of road image information by using a pre-trained convolutional neural network model, and identifying a target area containing a preset potential safety hazard target in a road image, a target type in the target area, scene information of a road section in the road image and first safety risk information in the road image according to the image characteristics; the characteristic calling module is used for calling the characteristic information of each road section according to the longitude and latitude information of each road section in the road image information; the knowledge fusion module is used for carrying out information fusion on corresponding road images according to the target area of each road image, the target category in the target area, the scene information of each road section and the characteristic information which are obtained through identification to obtain the knowledge representation of each road image after fusion, and obtaining the knowledge representation of a road video sequence containing each road image through fusion based on the knowledge representation of each road image after fusion according to the time sequence relation among the road images; the risk decision module is used for identifying second safety risk information of the road video sequence based on a pre-constructed expert knowledge base and a first decision model according to the knowledge representation of the road video sequence; and the risk identification module is used for identifying the final safety risk information of the road video sequence based on a pre-trained second decision model according to the knowledge representation of the road video sequence, the first safety risk information and the second safety risk information.
Further, the risk decision module comprises: the expert knowledge base is used for receiving a knowledge table of the road video sequence, identifying and obtaining third safety risk information and corresponding confidence coefficient, wherein the value of the confidence coefficient corresponding to the third safety risk information is 0 or 1; the first decision-making model is used for receiving a feature vector extracted based on knowledge representation of a road video sequence, identifying and obtaining fourth safety risk information and corresponding confidence coefficient of the road video sequence, wherein the value range of the confidence coefficient corresponding to the fourth safety risk information is 0-1; and a risk decision unit, configured to fuse the third security risk information and the fourth security risk information according to the confidence level of the third security risk information and the confidence level of the fourth security risk information to obtain the second security risk information.
Further, the knowledge fusion module comprises: the knowledge fusion unit is used for carrying out correction according to the characteristic information of the corresponding road section for the knowledge representation of the road section of which the confidence coefficient of the target category in each road section to be corrected in the road image information is lower than a first threshold value, the confidence coefficient of the scene information is lower than a second threshold value or the confidence coefficient of the first safety risk information is lower than a third threshold value, so as to obtain the knowledge representation of the road image after fusion; the road section to be corrected is a road section which contains the characteristic information and image identification information in the road image information at the same time, and the image identification information comprises the target area, the target category and scene information; for the characteristic information of the road section of which the confidence coefficient of the target category in each road section to be corrected is higher than a fourth threshold value, the confidence coefficient of the scene information is higher than a fifth threshold value or the confidence coefficient of the first safety risk information is higher than a sixth threshold value, correcting according to the target category, the scene information or the first safety risk information of the corresponding road section to obtain the knowledge representation of the road image after fusion; and for the road sections which do not have the feature information and the image identification information in the road image information at the same time, acquiring the knowledge representation of the road image after fusion according to the feature information or the image identification information.
Further, the knowledge fusion module further comprises: and the time sequence modeling unit is used for sequencing the road images according to the sampling time, and correcting the knowledge representation of each road image by using the knowledge representation of the front road image and the rear road image to obtain the knowledge representation of the road video sequence comprising each road image.
The invention relates to a road safety risk identification method and a system based on computer vision and artificial intelligence, which extracts high-level semantic information from an acquired road image through a convolutional neural network based on an acquired video and corresponding longitude and latitude information, calls characteristic information of each road section according to the longitude and latitude information, simultaneously combines time sequence information to jointly form road knowledge representation, performs image-level safety discrimination on the image through the convolutional neural network and a first decision model method based on the knowledge representation, performs comprehensive discrimination based on a second decision model, and gives road danger factors.
Other characteristic features and advantages of the invention will become apparent from the following description of exemplary embodiments, which is to be read in connection with the accompanying drawings.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and together with the description, serve to explain the principles of the invention. In the drawings, like reference numerals are used to indicate like elements. The drawings in the following description are directed to some, but not all embodiments of the invention. For a person skilled in the art, other figures can be derived from these figures without inventive effort.
FIG. 1 is a flow chart of an embodiment of a road safety risk identification method based on computer vision and artificial intelligence of the invention;
FIG. 2 is a flow chart of another embodiment of the road safety risk identification method based on computer vision and artificial intelligence, wherein for the convenience of understanding, the execution subjects of various modules are shown in the flow chart;
FIG. 3 is a schematic diagram of the visual information module of the method of FIG. 2 utilizing a convolutional neural network for image analysis;
fig. 4 is a block diagram of a road safety risk identification system based on computer vision and artificial intelligence according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention. It should be noted that the embodiments and features of the embodiments in the present application may be arbitrarily combined with each other without conflict.
As shown in fig. 1, the invention relates to a road safety risk identification method based on computer vision and artificial intelligence, comprising:
step 101: extracting image features of road image information by using a pre-trained convolutional neural network model, and identifying a target area containing a preset potential safety hazard target in a road image, a target type in the target area, scene information of a road section in the road image and first safety risk information in the road image according to the image features;
step 102: calling feature information of each road section according to the longitude and latitude information of each road section in the road image information; for example, obtaining external information through a third party API, a design drawing and the like;
step 103: performing information fusion on corresponding road images according to the target area of each road image, the target category in the target area, the scene information of each road section and the characteristic information obtained by identification to obtain the knowledge representation of each road image after fusion, and obtaining the knowledge representation of a road video sequence containing each road image by fusion based on the knowledge representation of each road image after fusion according to the time sequence relation among the road images;
step 104: identifying second safety risk information of the road video sequence based on a pre-established expert knowledge base and a first decision model according to the knowledge representation of the road video sequence;
step 105: and identifying final safety risk information of the road video sequence based on a pre-trained second decision model according to the knowledge representation of the road video sequence, the first safety risk information and the second safety risk information.
The invention discloses a road safety risk identification method based on computer vision and artificial intelligence, which is based on the collected video and the corresponding longitude and latitude information, after frame extraction processing, extracting high-level semantic information from the collected road image through a convolutional neural network, according to the longitude and latitude information, the characteristic information of each road section is called, simultaneously, the road knowledge representation is formed by combining with the time sequence information, the safety discrimination of the image level is respectively carried out on the image by a convolution neural network and a method of a first decision model (such as a decision tree) on the basis of the knowledge representation, the road safety evaluation method based on the second decision model is used for comprehensively judging and providing road risk factors, solves the problems that the conventional road safety evaluation method is mainly based on the manual subjective evaluation of experts, wastes time and labor, has high labor cost and the like, and improves the efficiency, accuracy and reliability of road safety risk identification.
As shown in fig. 2, the present invention provides a road safety risk identification method based on computer vision and artificial intelligence, which is a preferred implementation of the embodiment of the method shown in fig. 1, and the explanation of the embodiment shown in fig. 1 can be applied to this embodiment, and the road safety risk identification method based on computer vision and artificial intelligence of this embodiment includes:
1: as shown in fig. 2, an input image sequence or video is received, and longitude and latitude information corresponding to each frame of each image or video is received. Specifically, frames can be extracted at intervals according to requirements for received video information to obtain an image sequence, and longitude and latitude information corresponding to the extracted image sequence is obtained.
2: the step 101 is executed by the visual information module, and specifically, the visual information module analyzes the input image to obtain the following three types of output results:
a) the region of the target existing in the image and the category of the target in the region are collectively called target information;
b) scenes, namely scene information, of the safety of the relation roads existing in the images;
c) probability that the road section shown in the image belongs to a dangerous road section, i.e. safety/danger discrimination (i.e. danger factor, including confidence)
The specific working process of the visual information module is explained as follows: a multitask convolutional neural network is designed on the basis of a target detection network fast-rcnn and is used for simultaneously generating targets existing on a road shown by an image, scenes contained in the road and safety hazard conditions of the road. The convolutional neural network takes a single image as input. The network structure is shown in fig. 3 and is divided into a backbone network, a target detection branch, and a classification branch. The main network part is divided into two parts, which are mainly used for extracting features from an input image, and specifically can delete ResNet-101 of the last full connection layer (ResNet is an image classification network proposed by He-Caimen et al in 2015, and deepens the depth of the convolutional neural network by designing a jump connection structure). In the convolutional neural network, the characteristic semantic information of the lower layer is less, but the target position is accurate; the feature semantic information of the high layer is rich, but the target position is rough. The FPN network structure (proposed by Tsung-Yi Lin et al in 2016 of FPN network structure) performs prediction at different layers by fusing feature maps of the bottom layer and the upper layer to obtain more robust semantic information. The specific method is a bottom-up mode and a top-down mode.
The main body of the bottom-up mode is the forward calculation of the ResNet101 network, which calculates a feature hierarchy composed of a plurality of scale feature maps, each hierarchy composed of different sets of different convolution chunks, and the scaling size of each step is 2. There are typically many layers that produce the same size feature map, and these layers are defined as the same network stage. The final outputs of the residual blocks at each stage (i.e., the outputs of conv2, conv3, conv4, and conv 5) are expressed as { C2, C3, C4, C5}, and their sizes are reduced by 4, 8, 16, and 32 times, respectively, compared to the input image.
Top-down approach: in the top-down approach, the spatial resolution of the higher-level low-resolution feature map is increased by a factor of 2 (using bilinear interpolation). The upsampled feature map and the corresponding bottom-up feature map are then fused by means of pixel addition, and the number of channels is reduced by 1 × 1 convolution. In order to reduce aliasing effects caused by the upsampling process, for the feature maps obtained in the top-down mode, the final feature map is generated by a 3 × 3 convolution, and the final feature map set is called { P2, P3, P4, P5}, and corresponds to { C2, C3, C4, C5}, which have the same spatial size respectively.
The features obtained by the backbone network will be used for the target detection branch and the classification branch, respectively. The target detection branch is based on a target detection network, fast-rcnn (Ross b.girshick proposed in 2016), and is mainly divided into two stages: RPN network (area proposed network) and ROI-Head section:
the RPN takes a feature map with an arbitrary scale as an input and outputs a series of rectangular object proposals (object suggestions), and each object proposal has an object score (object score).
In this embodiment, a target region having a relationship with a road safety factor in a road image may be obtained through the RPN network, and a corresponding confidence score may be obtained. To generate the area proposal, a small network is slid over the feature maps of different scales output by the FPN network. This small network takes as input an n × n spatial window on the feature map. Each sliding window is mapped to a low-dimensional feature and activated by a ReLU function, and the low-dimensional feature is input to two sub-fully-connected layers: a bounding box regression layer and a classification layer. The classification layer can obtain the confidence scores of the foreground (the target area of the relation road safety factor) and the background for each preset anchor point (anchor), and the boundary frame regression layer can correct the position of each preset anchor point (anchor) to enable the anchor point (anchor) to be closer to the target area of the relation road safety factor.
At each sliding window position, multiple area proposals are predicted simultaneously, where the maximum number of possible proposals per position is denoted as k. Thus, the bounding box regression layer has 4k outputs, encoding the coordinates of the k bounding boxes, and the classification layer will output 2k scores, estimating the probability that each offer is a target region of the road safety factor. The k proposals are parameterized with respect to k reference bounding boxes called anchor points. The anchor point is located at the center of the sliding window and is associated with a predetermined scale and aspect ratio. The RPN network will get the target area of the image that is related to the road safety factor (zero to many are possible and will be passed to the ROI-Head part in the form of coordinate position relative to the whole image).
The ROI-Head part mainly has the functions of further analyzing the target area of the relation road safety factor obtained in the previous part, subdividing the target area into specific categories, and further correcting the position of each target area. The specific operation is as follows: by using ROI Align to extract the target area of the relation road safety factor obtained by RPN from the corresponding characteristic map and unifying the same size, a plurality of small characteristic maps of the target area corresponding to the road safety factor are obtained. Two full-connection layers are continuously used for the obtained small feature map corresponding to each target area, then a one-dimensional feature vector is obtained, and a full-connection layer for distinguishing the types of the target areas and a full-connection layer for correcting the positions of the target areas are respectively connected for the vector. Which regions in the road have which objects (including confidence) through the object detection branch, and the object types are shown in table 1.
TABLE 1 target detection target List
The classification branch part takes the feature map obtained by the main network part as input, pools the feature map to a fixed size through ROI Align, and takes the obtained features as the input of a plurality of different scene classification branches and danger safety judgment branches respectively, and each branch carries out classification through 3 connected full connection layers, namely whether a certain scene exists in the road section and whether the road section is safe or not. The classification branch can obtain which scenes exist in the road (see table 2 for scene classification), and visually judge whether the road is safe (including confidence).
Table 2 image scene classification list
3: the step 102 is executed by the information retrieving module, and specifically, for the input longitude and latitude information corresponding to the image, three types of output results may be obtained by the information retrieving module (for different road sections, the result obtained by the information retrieving module is not fixed, and several or none of the three types of results may be obtained): speed limit information of the road section, actual curve radius of the road section and a design drawing of the road section. It should be noted that the external information obtained by the information retrieving module is not limited to the three types described above, and is not limited herein, and any external feature information may not be obtained for each road segment.
And the information calling module acquires supplement of the knowledge representation obtained by the visual information part through a third party API interface and a construction drawing of a road. Information fusion is realized by acquiring longitude and latitude of the image to correspond to external information. The information obtained by the design drawings may comprise a plurality of types (see table 3 for details, the data shown in table 3 are only partial results and should not be understood as limiting).
TABLE 3 design paper extraction information
4: the step 103 is executed by the knowledge fusion module, and specifically, the information obtained by the visual information module and the information retrieval module is fused and corrected to form a knowledge representation about the road segment, and the knowledge representation is characterized to form a feature vector. Because the information obtained based on the image and the information obtained through the outside are overlapped, the overlapped information is mutually corrected by the way of obtaining the information and the confidence. And performing related operations of the visual information module and the information retrieval module on a plurality of images in sequence in time sequence to obtain knowledge representation and feature vectors of a plurality of continuous road sections, and correcting the obtained knowledge representation and feature vectors according to the spatial relationship of the road sections. For example, the far end of the current image can see the next image, the target on the next image can be detected, and the false detection and the missing detection of the target can be corrected by combining the knowledge information of several images.
Specifically, the knowledge fusion part functions to fuse a knowledge representation from the visual information part with a knowledge representation from the external information part. The method mainly comprises two aspects: 1) mutual complementation between knowledge 2) fusion of the knowledge of the two parts to form new knowledge.
The knowledge obtained by the visual information module indicates that all the information is not necessarily correct, and false detection and missing detection are possible to occur no matter whether the information is used for detecting the target with the safety of the relation road in the road or detecting the scene with the safety of the relation road in the road. Knowledge representation obtained through the feature calling module also has problems, for example, the obtained external information is inaccurate due to errors in the collection of longitude and latitude, and meanwhile, the external information has certain hysteresis, and particularly, a design drawing may deviate from the actual situation along with the situations of road maintenance, maintenance and the like. The knowledge fusion part not only corrects the knowledge with higher hysteresis in the external information according to the knowledge with high confidence level in the visual information part, but also corrects the knowledge with lower confidence level in the visual information according to the information in the external information, and the parts with non-overlapping knowledge of the two parts are spliced together to form the knowledge of a single image.
For example, the part obtained from the visual information has higher confidence representing that the road has a sharp curve, but the information representation obtained from the google map in the external information is a straight line section, and the curve radius given by the design drawing belongs to a sharper curve, so that the section obtained through mutual rectification is a sharp curve section, and the curve radius is the curve radius given by the design drawing. Thereby a knowledge representation of a single image is obtained by the knowledge fusion part.
In addition, the knowledge fusion part also comprises a time sequence modeling function, and specifically, the knowledge of the current road section is corrected according to the knowledge of the front road and the rear road through the precedence relationship between the roads. The acquisition of the image sequence will typically acquire images at intervals of every 200 meters, and the current road image will contain partial information of the next road image. By mutually correcting the information of the parts, the wrong parts in the knowledge are corrected, and meanwhile, the confidence coefficient of the correct parts is strengthened. For example, the object detection of the safety of the relation road existing in the visual information module part and the scene detection of the safety of the relation road existing in the road are detected, missed and mistakenly detected. For example, when the current image finds that the next image has an access port, the access port is not detected due to the fact that the next image is blocked by a passing vehicle or a blind area of a camera is detected, and the information of the previous image can be supplemented. Through time sequence modeling, knowledge of each discrete road section is spliced into knowledge of the whole road, and a plurality of information of the current road section is shared. For example, if a certain road segment finds a speed limit sign, the speed limit information of the whole road can be taken as the speed limit information of the road segment, and the road segment finds continuous side ditches (generally, continuous existence) and can be supplemented as the information of one road. And (4) obtaining a final knowledge representation through time sequence modeling. After the final knowledge representation is obtained, the obtained knowledge representation is coded into a feature vector, and the coding rule adopted by the knowledge representation is as follows:
1) knowledge of object detection for visual information module per category ni6 dimensions (i ═ 1,2 … N, N being the number of target species present), NiAn upper limit on how often each category object can appear in a section of road. The 6 dimensions respectively represent the upper left and lower right coordinates (four values in total) of the object, the categories of the object (each category corresponds to 1-N), and the confidence. n isiFixed for all images. (the number of detected i-th type objects in each image may not reach niThis case is set to 0 all over).
2) For the knowledge obtained by the classification of the visual information module, each class occupies two dimensions, and the value is the confidence coefficient of the existence and the non-existence of the class respectively.
3) For the feature extraction module, the knowledge of the numerical type (e.g., curve radius) each takes one dimension, and the value is the value that the knowledge represents. Knowledge of presence (e.g., whether it is a highway) is one-dimensional each, with values of 0,1 (0 indicating absence and 1 indicating presence). By the feature representation section, a one-dimensional feature representation vector for each image can be obtained.
5: the above step 103 is performed by the risk decision module, and the following operations are synchronously performed with respect to the knowledge representation and feature vector of each road segment:
and 5a, reasoning the risk factors (such as a sharp curve section, and the risk factors with poor linearity of the curve section can be obtained due to the lack of guardrails) in a knowledge base which is constructed by a knowledge representation sender based on expert knowledge.
And 5b, inputting a plurality of XGboost models (namely danger factor reasoning models, namely first decision models) which are trained in the model generation stage and used for distinguishing whether each type of danger factors appear for the obtained characteristic representation, and reasoning the danger factors.
That is, the risk decision module is divided into two parts, risk factor reasoning and safety risk determination. The risk factor reasoning is divided into two parts, namely risk factor reasoning based on a knowledge base and risk factor reasoning based on a decision tree model. For risk factor reasoning based on the knowledge base, the expert knowledge base is constructed by a large amount of knowledge and experience of expert level in the road safety field, the expert knowledge base internally contains a large amount of knowledge and experience of the road safety field, and the reasoning logic of the road safety risk factor can be processed by utilizing the knowledge of human experts and a problem solving method.
Specifically, the present embodiment constructs a knowledge base based on expert knowledge by collecting a large amount of knowledge, methods, and experience used by experts in the field of road security to deal with security risk assessment. In the reasoning stage of the model, the knowledge representation obtained in the first parts is used as input, and risk factors existing in the road are obtained through logical reasoning in a knowledge base. And (4) risk factor reasoning based on the decision tree model. And taking the feature vector obtained by the feature representation part as an input, and training an XGboost model aiming at each risk factor. The idea of Boosting is to integrate many weak classifiers together to form one strong classifier. XGboost is a strong classifier formed by integrating a plurality of tree models. And the method is used for judging whether the dangerous conditions exist in the road. The risk decision module obtains two types of risk factors, 1) risk factors inferred from the knowledge base and 2) risk factors (including confidence) obtained from the decision model. After the two types of results are mutually verified (the confidence coefficient of the existing category is set to be 1 and the confidence coefficient of the nonexistent category is set to be 0 in the result obtained by inference of the knowledge base, the confidence coefficient of each category 0-1 is output by the decision model, the confidence coefficient of the decision model is set to be 0 for the category smaller than the threshold, and then the confidence coefficients obtained by inference of the knowledge base and the decision model are added) to be output as the final risk factor. The predetermined types of risk factors are shown in table 4 (for illustrative purposes only and should not be construed as limiting).
TABLE 4 Risk factors categories
6: the step 104 is executed by the risk identification module, and the risk factors obtained by the visual information module, the knowledge base and the first decision model are input into the XGBoost model (i.e., the risk safety discrimination model, i.e., the second decision model) trained at the model generation stage for discriminating the road safety risk, so as to discriminate the road risk/safety.
The risk identification module is used for finally judging the safety risk, and on the basis of the feature representation obtained by the feature representation part, the dimensionality of the feature representation is expanded: adding 2 dimensions, namely obtaining confidence coefficients of road safety and road danger by a visual information module; and adding a dimension aiming at each type of risk factors, adopting 0,1 coding for the inference result obtained by the knowledge base, and taking the inference result value obtained by the decision model as the confidence coefficient. And training an XGboost model according to the newly obtained feature representation for finally judging the road danger safety.
The road safety risk identification method based on computer vision and artificial intelligence identifies a target area concerned in road safety evaluation in an image according to the acquired video and image information, carries out detailed analysis on part of the target, and converts the image information into high-level semantic knowledge. And simultaneously, a multitask convolutional neural network is designed, and the safety/danger judgment is carried out on the road from the perspective of the image while the target area of the image is given. And acquiring knowledge supplement aiming at each image through a third API (application programming interface) interface, a road design drawing and other ways according to the longitude and latitude information corresponding to the collected road image. Fusing visual information, time sequence information, spatial information, supplementary information and the like to construct a multi-cue fused knowledge representation: and mutually correcting and fusing the visual information module and the knowledge representation and knowledge supplement obtained by the information module to form a knowledge representation for describing the road corresponding to each image. And correcting the knowledge representation of the road with the sequential relation in the time sequence through the time sequence information of the video sequence to obtain a new knowledge representation. And deducing the danger factors existing on the road through a decision model and a knowledge base based on expert knowledge according to the knowledge representation. And (3) integrating the knowledge representation, the risk factors and the image-based risk judgment obtained by the neural network, and finally giving out the risk safety judgment of the corresponding road through a decision model, so that the efficiency, accuracy and reliability of the highway safety risk identification are improved.
As shown in fig. 4, the present invention further provides a road safety risk identification system based on computer vision and artificial intelligence, which is a corresponding device embodiment of the method embodiments shown in fig. 1 and fig. 2, and the explanation of the embodiments shown in fig. 1 to fig. 3 can be applied to this embodiment, and the road safety risk identification system based on computer vision and artificial intelligence comprises:
the visual information module 401 is configured to extract image features of road image information by using a pre-trained convolutional neural network model, and identify a target area containing a preset potential safety hazard target in a road image, a target category in the target area, scene information of a road segment in the road image, and first safety risk information in the road image according to the image features.
Specifically, the visual information module 401 extracts features from the acquired single frame image through a convolutional neural network to obtain a plurality of feature maps with different scales, and performs two branch predictions on the feature maps in parallel. As shown in fig. 3, the area proposed network (RPN) is used on the first branch to extract areas where targets (some targets of road safety concern such as guardrails, pillars, etc.) potentially exist. And sending the obtained characteristic areas corresponding to the areas with the potential targets to a detection network, and finally obtaining which areas in the single-frame image have the targets concerned by the road safety. For the features obtained, a plurality of classification networks dedicated to different tasks are entered: predicting whether the dangerous factors exist in the road section from the image level; and judging whether some scene factors related to road safety exist from the image level, such as curves, adjacent cliffs, adjacent water and the like.
A feature retrieving module 402, configured to retrieve feature information of each road segment according to the longitude and latitude information of each road segment in the road image information; the method comprises the steps of obtaining supplementary information (such as speed limit information of a road section, road design drawing information and the like) of the road section except visual information through longitude and latitude information of each image, and forming external knowledge of the road.
The knowledge fusion module 403 is configured to perform information fusion on corresponding road images according to the identified target area of each road image, the target category in the target area, the scene information of each road segment, and the feature information, to obtain a knowledge representation of each road image after fusion, and obtain a knowledge representation of a road video sequence including each road image based on the knowledge representation of each road image after fusion according to a time sequence relationship between the road images.
Specifically, the knowledge fusion module 403 has three functions:
1. and for a single-frame image, matching and fusing information acquired by the visual information module and the external information module, removing redundant information, and constructing knowledge representation of the single-frame image.
2. And modeling knowledge representation information of continuous multi-frame images according to the video sequence to form richer knowledge representation facing road safety evaluation.
3. And constructing a one-dimensional feature representation vector according to the obtained knowledge representation.
And a risk decision module 404, configured to identify second security risk information of the road video sequence based on a pre-constructed expert knowledge base and a first decision model according to the knowledge representation of the road video sequence. Specifically, the risk decision module 404 constructs a knowledge base for knowledge inference based on a safety discrimination requirement criterion, a road construction requirement criterion, professional knowledge, and expert experience in the field of road safety, infers risk factors existing in a road through knowledge representation obtained in the previous sections, and constructs a plurality of decision tree models to obtain probabilities of various risk factors existing through the obtained feature representation vectors.
And a risk identification module 405, configured to identify final safety risk information of the road video sequence based on a pre-trained second decision model according to the knowledge representation of the road video sequence, the first safety risk information, and the second safety risk information. The method is characterized in that a comprehensive decision tree model is constructed by combining knowledge representation, reasoning results of risk factors and safety risk judgment of images given by a visual information partial convolution neural network, and finally judgment of road risk safety is obtained.
Preferably, the risk decision module 404 comprises:
an expert knowledge base (not shown in the figure) for receiving a knowledge table of the road video sequence, and identifying to obtain third safety risk information and a corresponding confidence coefficient, wherein the confidence coefficient value corresponding to the third safety risk information is 0 or 1;
the first decision-making model (not shown in the figure) is used for receiving a feature vector extracted based on knowledge representation of a road video sequence, identifying fourth safety risk information and a corresponding confidence coefficient of the road video sequence, wherein the confidence coefficient value range corresponding to the fourth safety risk information is 0-1;
a risk decision unit (not shown in the figure), configured to fuse the third security risk information and the fourth security risk information according to the confidence level of the third security risk information and the confidence level of the fourth security risk information to obtain the second security risk information.
Preferably, the knowledge fusion module 403 includes;
a knowledge fusion unit (not shown in the figure), configured to correct, according to feature information of a corresponding road segment, knowledge representation of the road image after fusion, for a road segment whose confidence coefficient of a target category in each road segment to be corrected is lower than a first threshold, whose confidence coefficient of scene information is lower than a second threshold, or whose confidence coefficient of first safety risk information is lower than a third threshold in the road image information; the road section to be corrected is a road section which contains the characteristic information and image identification information in the road image information at the same time, and the image identification information comprises the target area, the target category and scene information; for the characteristic information of the road section of which the confidence coefficient of the target category in each road section to be corrected is higher than a fourth threshold value, the confidence coefficient of the scene information is higher than a fifth threshold value or the confidence coefficient of the first safety risk information is higher than a sixth threshold value, correcting according to the target category, the scene information or the first safety risk information of the corresponding road section to obtain the knowledge representation of the road image after fusion; and for the road sections which do not have the feature information and the image identification information in the road image information at the same time, acquiring the knowledge representation of the road image after fusion according to the feature information or the image identification information.
Preferably, the knowledge fusion module 403 further comprises;
and the time sequence modeling unit (not shown in the figure) is used for sequencing the road images according to the sampling time, and correcting the knowledge representation of each road image by using the knowledge representations of the front road image and the rear road image to obtain the knowledge representation of the road video sequence comprising each road image.
It should be noted that, in the present embodiment, there is also a model training process in the design and development stage: mainly, a prediction model, a detection model, a decision model and a classification model used in the stage of an operation process are trained through marking data. The following models are mainly constructed: training the multitask convolutional neural network of the visual information module 401; the training risk decision module 404 is used to decide a decision tree model of the risk factors; the training risk recognition module 405 is a decision tree model for determining the level of security risk; and constructing a knowledge base based on expert knowledge for risk factor reasoning. For the model generation part of the visual information module 401, firstly, 60000 collected images are labeled according to the labeling condition of artificial data, and the labeling content includes:
a) in each road image, there is a minimum bounding rectangle and corresponding class for the object shown in Table one (regularized for some images that do not easily define a minimum bounding image)
b) Which of the scene categories shown in Table 3 are present in each image
c) The safety risk identified by the expert in each image is determined (for the visual information module 401 and the risk decision module 404)
d) Which risk factors are present in each image (annotation for risk decision module 404)
And training the network of the visual information module 401 according to the label and the image, learning network parameters according to the loss back propagation gradient, finally converging the network, and ending the model generation stage.
The embodiment is divided into four main modules through the idea of applying modular design: a visual information module based on deep learning and computer vision, a characteristic retrieval module, a knowledge fusion module, a risk decision module and a risk identification module, the video and image information collected by the road safety information collecting vehicle and the longitude and latitude information corresponding to each frame of image synchronously obtained are taken as input, the whole process does not need manual intervention, can automatically and efficiently evaluate the road safety directly through video and image data and some supplementary information, simultaneously, the dangerous reasons are given, the expenses of manpower and material resources are greatly reduced, the safety evaluation of the road can be realized with high efficiency and high quality, meanwhile, the evaluation result is interpretable, the problem that the existing method is too dependent on factors and characteristics of manual design when evaluating the road safety is solved, the generalization capability and robustness of the method are not strong for different scenes and situations, and the updating and iteration capability of the method is poor.
The above-described aspects may be implemented individually or in various combinations, and such variations are within the scope of the present invention.
Finally, it should be noted that: the above examples are only for illustrating the technical solutions of the present invention, and are not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A road safety risk identification method based on computer vision and artificial intelligence is characterized by comprising the following steps:
extracting image features of road image information by using a pre-trained convolutional neural network model, and identifying a target area containing a preset potential safety hazard target in a road image, a target type in the target area, scene information of a road section in the road image and first safety risk information in the road image according to the image features;
calling feature information of each road section according to the longitude and latitude information of each road section in the road image information;
performing information fusion on corresponding road images according to the target area of each road image, the target category in the target area, the scene information of each road section and the characteristic information obtained by identification to obtain the knowledge representation of each road image after fusion, and obtaining the knowledge representation of a road video sequence containing each road image by fusion based on the knowledge representation of each road image after fusion according to the time sequence relation among the road images;
identifying second safety risk information of the road video sequence based on a pre-established expert knowledge base and a first decision model according to the knowledge representation of the road video sequence;
and identifying final safety risk information of the road video sequence based on a pre-trained second decision model according to the knowledge representation of the road video sequence, the first safety risk information and the second safety risk information.
2. The computer vision and artificial intelligence based road safety risk identification method of claim 1, wherein: the method comprises the following steps of identifying second safety risk information of the road video sequence based on a pre-constructed expert knowledge base and a first decision model according to the knowledge representation of the road video sequence, wherein the steps comprise:
inputting the knowledge representation of the road video sequence into a pre-constructed expert knowledge base, and identifying to obtain third safety risk information and corresponding confidence coefficient, wherein the confidence coefficient value corresponding to the third safety risk information is 0 or 1;
inputting a feature vector extracted based on knowledge representation of a road video sequence into a first decision model, and identifying and obtaining fourth safety risk information and corresponding confidence coefficient of the road video sequence, wherein the value range of the confidence coefficient corresponding to the fourth safety risk information is 0-1;
and according to the confidence degree of the third safety risk information and the confidence degree of the fourth safety risk information, fusing the third safety risk information and the fourth safety risk information to obtain the second safety risk information.
3. The computer vision and artificial intelligence based road safety risk identification method of claim 2, wherein: the method for identifying the target area containing the preset potential safety hazard target in the road image, the target category in the target area, the scene information of the road section in the road image and the first safety risk information in the road image according to the image characteristics comprises the following steps:
inputting the image characteristics into a target detection network, and identifying a target area containing a preset potential safety hazard target in a road image, a target category in the target area and a confidence coefficient of the target category;
and classifying the image features, and identifying scene information of road sections in the road image, the confidence coefficient of the scene information, first safety risk information in the road image and the confidence coefficient of the first safety risk information.
4. The computer vision and artificial intelligence based road safety risk identification method of claim 3, wherein: the step of calling the characteristic information of each road section according to the longitude and latitude information of each road section in the road image information comprises the following steps:
and inputting the longitude and latitude information of each road section in the road image information into a third-party interface, and calling the characteristic information of each road section, wherein the characteristic information comprises speed limit information, curve information or/and design drawing information of the road section.
5. The computer vision and artificial intelligence based road safety risk identification method of claim 4, wherein: according to the target area of each road image, the target category in the target area, the scene information of each road section and the characteristic information obtained through identification, information fusion is carried out on the corresponding road image, and the step of obtaining knowledge representation of each road image after fusion comprises the following steps:
correcting the knowledge representation of the road sections of which the confidence degrees of the target categories in the road sections to be corrected are lower than a first threshold value, the confidence degrees of the scene information are lower than a second threshold value or the confidence degree of the first safety risk information is lower than a third threshold value according to the feature information of the corresponding road sections to obtain the knowledge representation of the road images after fusion; the road section to be corrected is a road section which contains the characteristic information and image identification information in the road image information at the same time, and the image identification information comprises the target area, the target category and scene information;
for the characteristic information of the road section of which the confidence coefficient of the target category in each road section to be corrected is higher than a fourth threshold value, the confidence coefficient of the scene information is higher than a fifth threshold value or the confidence coefficient of the first safety risk information is higher than a sixth threshold value, correcting according to the target category, the scene information or the first safety risk information of the corresponding road section to obtain the knowledge representation of the road image after fusion;
and for the road sections which do not have the feature information and the image identification information in the road image information at the same time, acquiring the knowledge representation of the road image after fusion according to the feature information or the image identification information.
6. The computer vision and artificial intelligence based road safety risk identification method of claim 5, wherein: the step of obtaining the knowledge representation of the road video sequence containing each road image by fusion based on the fused knowledge representation of each road image comprises the following steps:
and sequencing the road images according to the sampling time, and correcting the knowledge representation of each road image by using the knowledge representation of the front road image and the rear road image to obtain the knowledge representation of the road video sequence comprising each road image.
7. A computer vision and artificial intelligence based road safety risk identification system, comprising:
the visual information module is used for extracting image characteristics of road image information by using a pre-trained convolutional neural network model, and identifying a target area containing a preset potential safety hazard target in a road image, a target type in the target area, scene information of a road section in the road image and first safety risk information in the road image according to the image characteristics;
the characteristic calling module is used for calling the characteristic information of each road section according to the longitude and latitude information of each road section in the road image information;
the knowledge fusion module is used for carrying out information fusion on corresponding road images according to the target area of each road image, the target category in the target area, the scene information of each road section and the characteristic information which are obtained through identification to obtain the knowledge representation of each road image after fusion, and obtaining the knowledge representation of a road video sequence containing each road image through fusion based on the knowledge representation of each road image after fusion according to the time sequence relation among the road images;
the risk decision module is used for identifying second safety risk information of the road video sequence based on a pre-constructed expert knowledge base and a first decision model according to the knowledge representation of the road video sequence;
and the risk identification module is used for identifying the final safety risk information of the road video sequence based on a pre-trained second decision model according to the knowledge representation of the road video sequence, the first safety risk information and the second safety risk information.
8. The computer vision and artificial intelligence based road safety risk identification system of claim 7, wherein: the risk decision module comprises:
the expert knowledge base is used for receiving knowledge representation of the road video sequence, identifying and obtaining third safety risk information and corresponding confidence coefficient, wherein the value of the confidence coefficient corresponding to the third safety risk information is 0 or 1;
the first decision-making model is used for receiving a feature vector extracted based on knowledge representation of a road video sequence, identifying and obtaining fourth safety risk information and corresponding confidence coefficient of the road video sequence, wherein the value range of the confidence coefficient corresponding to the fourth safety risk information is 0-1;
and a risk decision unit, configured to fuse the third security risk information and the fourth security risk information according to the confidence level of the third security risk information and the confidence level of the fourth security risk information to obtain the second security risk information.
9. The computer vision and artificial intelligence based road safety risk identification system of claim 8, wherein: the knowledge fusion module comprises:
the knowledge fusion unit is used for carrying out correction according to the characteristic information of the corresponding road section for the knowledge representation of the road section of which the confidence coefficient of the target category in each road section to be corrected in the road image information is lower than a first threshold value, the confidence coefficient of the scene information is lower than a second threshold value or the confidence coefficient of the first safety risk information is lower than a third threshold value, so as to obtain the knowledge representation of the road image after fusion; the road section to be corrected is a road section which contains the characteristic information and image identification information in the road image information at the same time, and the image identification information comprises the target area, the target category and scene information; for the characteristic information of the road section of which the confidence coefficient of the target category in each road section to be corrected is higher than a fourth threshold value, the confidence coefficient of the scene information is higher than a fifth threshold value or the confidence coefficient of the first safety risk information is higher than a sixth threshold value, correcting according to the target category, the scene information or the first safety risk information of the corresponding road section to obtain the knowledge representation of the road image after fusion; and for the road sections which do not have the feature information and the image identification information in the road image information at the same time, acquiring the knowledge representation of the road image after fusion according to the feature information or the image identification information.
10. The computer vision and artificial intelligence based road safety risk identification system of claim 9, wherein: the knowledge fusion module further comprises:
and the time sequence modeling unit is used for sequencing the road images according to the sampling time, and correcting the knowledge representation of each road image by using the knowledge representation of the front road image and the rear road image to obtain the knowledge representation of the road video sequence comprising each road image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010073871.1A CN111310607B (en) | 2020-01-22 | 2020-01-22 | Highway safety risk identification method and system based on computer vision and artificial intelligence |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010073871.1A CN111310607B (en) | 2020-01-22 | 2020-01-22 | Highway safety risk identification method and system based on computer vision and artificial intelligence |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111310607A CN111310607A (en) | 2020-06-19 |
| CN111310607B true CN111310607B (en) | 2021-01-01 |
Family
ID=71154592
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010073871.1A Active CN111310607B (en) | 2020-01-22 | 2020-01-22 | Highway safety risk identification method and system based on computer vision and artificial intelligence |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111310607B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112286200B (en) * | 2020-11-06 | 2023-09-29 | 湖北师范大学 | Automatic driving method, automatic driving device, vehicle and computer readable storage medium |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2085279B1 (en) * | 2008-01-29 | 2011-05-25 | Ford Global Technologies, LLC | A system for collision course prediction |
| AU2015347259A1 (en) * | 2014-08-26 | 2017-03-30 | Polaris Sensor Technologies, Inc. | Polarization-based mapping and perception method and system |
| CN108764042B (en) * | 2018-04-25 | 2021-05-28 | 深圳市科思创动科技有限公司 | Abnormal road condition information identification method and device and terminal equipment |
| CN109671274B (en) * | 2019-01-24 | 2020-09-25 | 交通运输部公路科学研究所 | Highway risk automatic evaluation method based on feature construction and fusion |
| CN109977812B (en) * | 2019-03-12 | 2023-02-24 | 南京邮电大学 | Vehicle-mounted video target detection method based on deep learning |
-
2020
- 2020-01-22 CN CN202010073871.1A patent/CN111310607B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN111310607A (en) | 2020-06-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109977812B (en) | Vehicle-mounted video target detection method based on deep learning | |
| US11315345B2 (en) | Method for dim and small object detection based on discriminant feature of video satellite data | |
| CN108470172B (en) | Text information identification method and device | |
| CN110210475B (en) | License plate character image segmentation method based on non-binarization and edge detection | |
| Poulain et al. | High-resolution optical and SAR image fusion for building database updating | |
| CN111813997B (en) | Intrusion analysis method, device, equipment and storage medium | |
| CN112149547B (en) | Remote sensing image water body identification method based on image pyramid guidance and pixel pair matching | |
| CN113065578B (en) | Image visual semantic segmentation method based on double-path region attention coding and decoding | |
| CN111582129A (en) | Real-time monitoring and alarming method and device for working state of shield machine driver | |
| CN113034444A (en) | Pavement crack detection method based on MobileNet-PSPNet neural network model | |
| CN112906481A (en) | Method for realizing forest fire detection based on unmanned aerial vehicle | |
| CN113313031B (en) | Deep learning-based lane line detection and vehicle transverse positioning method | |
| CN112102250B (en) | Method for establishing and detecting pathological image detection model with training data as missing label | |
| CN114332473A (en) | Object detection method, object detection device, computer equipment, storage medium and program product | |
| CN116091937A (en) | High-resolution remote sensing image ground object recognition model calculation method based on deep learning | |
| CN110659572B (en) | Video motion detection method based on bidirectional feature pyramid | |
| CN113724286A (en) | Method and device for detecting saliency target and computer-readable storage medium | |
| CN111310607B (en) | Highway safety risk identification method and system based on computer vision and artificial intelligence | |
| CN113408550B (en) | Intelligent weighing management system based on image processing | |
| CN114926791A (en) | Method and device for detecting abnormal lane change of vehicles at intersection, storage medium and electronic equipment | |
| CN116959099A (en) | Abnormal behavior identification method based on space-time diagram convolutional neural network | |
| CN111340001A (en) | Traffic jam identification method based on remote sensing image deep learning | |
| CN115511280A (en) | Urban flood toughness evaluation method based on multi-mode data fusion | |
| CN110378337A (en) | Metal cutting tool drawing identification information vision input method and system | |
| CN115100681A (en) | Clothes identification method, system, medium and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |