CN113673484A - Road condition identification and decision-making method in unmanned driving scene - Google Patents
Road condition identification and decision-making method in unmanned driving scene Download PDFInfo
- Publication number
- CN113673484A CN113673484A CN202111058219.3A CN202111058219A CN113673484A CN 113673484 A CN113673484 A CN 113673484A CN 202111058219 A CN202111058219 A CN 202111058219A CN 113673484 A CN113673484 A CN 113673484A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- map
- environment
- decision
- road condition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 22
- 238000005516 engineering process Methods 0.000 claims abstract description 6
- 238000013528 artificial neural network Methods 0.000 claims abstract description 5
- 238000010276 construction Methods 0.000 claims description 10
- 238000001914 filtration Methods 0.000 claims description 5
- 230000000007 visual effect Effects 0.000 claims description 5
- 230000004927 fusion Effects 0.000 claims description 2
- 230000011218 segmentation Effects 0.000 abstract description 9
- 238000012549 training Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000004075 alteration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/05—Geographic models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/32—Indexing scheme for image data processing or generation, in general involving image mosaicing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Geometry (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Molecular Biology (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Computer Graphics (AREA)
- Computing Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Remote Sensing (AREA)
- Evolutionary Biology (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
Abstract
The invention discloses a road condition identification and decision-making method in an unmanned driving scene, which comprises the following steps: firstly, a vehicle trains a picture expansion model by building a deep convolution neural network, and obtains a trained model; secondly, the vehicle realizes the relative positioning of the position of the vehicle in an unknown environment, and pictures are shot through a sensor; thirdly, the vehicle identifies and classifies the environment through a model which is matched and trained by pictures shot by the sensor; by adopting the technology of the invention, under the condition of adopting the same data set of the road condition picture in the unmanned scene, the road condition types in the picture can be identified, less storage space is occupied, the algorithm has better precision in a dynamic environment, decision opinions for guiding unmanned driving can be generated and returned to an unmanned control system, and the global semantic information obtained by semantic segmentation can help the robot to navigate and plan the path, thereby obviously improving the intelligent level of the mobile vehicle.
Description
Technical Field
The invention belongs to the technical field of computer image processing, and particularly relates to a road condition identification and decision-making method in an unmanned driving scene.
Background
In the field of computer image processing, a map expression mode based on a feature descriptor can complete a visual positioning task of a vehicle, but when the method is applied to road condition recognition and exploration in an unmanned scene, the method has the following defects: the traditional map based on the feature descriptor occupies more storage space and has larger estimation error, so the traditional map based on the feature descriptor cannot be directly used in the development of an unmanned control system.
Disclosure of Invention
The present invention is directed to a method for road condition identification and decision-making in an unmanned driving scene, so as to solve the problems mentioned in the background art.
In order to achieve the purpose, the invention provides the following technical scheme: a road condition identification and decision-making method in an unmanned driving scene is characterized in that: the method comprises the following steps:
firstly, a vehicle trains a picture expansion model by building a deep convolution neural network, and obtains a trained model;
secondly, the vehicle realizes the relative positioning of the position of the vehicle in an unknown environment, and pictures are shot through a sensor;
thirdly, the vehicle identifies and classifies the environment through a model which is matched and trained by pictures shot by the sensor;
step four, starting moving exploration of the vehicle through an unknown map path exploration algorithm, carrying out self-positioning through observation of the environment, and simultaneously building an incremental map;
fifthly, the vehicle realizes real-time construction and updating of the map during exploration through multi-sensor fusion data until the map construction is completed;
and step six, the vehicle control system makes a decision according to the sensed information.
Preferably, in the third step, the pose estimation of the visual SLAM algorithm is adopted.
Preferably, in the fourth step, local map splicing and global map model construction are realized through point cloud splicing and filtering technology.
Preferably, the semantic map is constructed in a dynamic environment by combining the semantic information with the SLAM algorithm adopted by the invention in the fifth step.
Compared with the prior art, the invention has the beneficial effects that: by adopting the technology of the invention, under the condition of adopting the same data set of the road condition picture in the unmanned scene, the road condition types in the picture can be identified, less storage space is occupied, the algorithm has better precision in a dynamic environment, decision opinions for guiding unmanned driving can be generated and returned to an unmanned control system, and the global semantic information obtained by semantic segmentation can help the robot to navigate and plan the path, thereby obviously improving the intelligent level of the mobile vehicle.
Drawings
FIG. 1 is a schematic diagram of the principles of the present invention;
FIG. 2 is a schematic view of the workflow of pose estimation of the visual SLAM algorithm of the present invention;
FIG. 3 is a schematic diagram of a method for local map stitching and global map model construction according to the present invention;
FIG. 4 is a schematic diagram of a single frame semantic point generation algorithm of the present invention;
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Example 1
Referring to fig. 1 to 4, the present invention provides a technical solution: a method for road condition identification and decision-making in an unmanned driving scene, comprising 1) convolutional neural network-based image semantic segmentation: building a full convolution neural network model for semantic segmentation, and training through a data set, wherein the data set comprises 11335 semantic segmentation images, the semantic segmentation images are divided into 20 types of objects, and the resolution ratio is about 500 x 300; for semantic segmentation tasks, a training set and a verification set of the data set both have corresponding labels, and the labels store corresponding semantic categories of each pixel in an image; if other categories need to be identified, the trained network can be finely adjusted by using an additional data stage, pictures are cut, and the resolution of the input image is unified to 360 x 480; adopting a multi-distribution learning rate, taking a larger learning rate at the early stage of neural network training, and attenuating the learning rate to be 0.1 time of the original learning rate when reaching a certain set training frequency along with the training; the method can enable the neural network to reach the vicinity of the optimal value quickly without fluctuating back and forth around the optimal value.
2) Pose estimation of visual SLAM algorithm: referring to the work flow of fig. 2, the position and orientation of the camera are estimated by using the SLAM algorithm of the feature point method, the projection of the same space point in different images is obtained by matching the features, the relation between the position and the posture of the images is calculated, and the motion track of the moving vehicle of the camera is obtained.
3) Local map splicing and global map model construction are realized through point cloud splicing and filtering technology: after the SLAM algorithm is used for processing a moving object in a dynamic scene and a pose graph with the pose as a variable is optimized, the optimized globally consistent mobile robot key frame pose can be obtained; accordingly, the point clouds of the key frames are spliced to obtain a local map of the environment, and the method is as shown in fig. 3; converting the point cloud map corresponding to each key frame into a world coordinate system by using the pose of the key frame, and connecting to obtain a local global point cloud map; filtering outliers in the point cloud map by adopting a filtering technology to realize map optimization and complete map construction; the mathematical expression is as follows:
in the formula:
Ck-key(s)A single frame point cloud formed by the frame images,
Tkthe pose of the camera at the moment of the frame,
and (4) local point cloud under the m-camera coordinate system.
4) Combining the semantic information with the SLAM algorithm adopted by the invention to complete the construction of the semantic map under the dynamic environment: adding semantic information to the acquired point cloud information, identifying the class information of an object in the image by semantic segmentation of the image, fusing the segmentation result of the two-dimensional image into the point cloud, and giving semantic information to each point in the point cloud so as to obtain a semantic point cloud map with rich information; adopting different colors to correspond to objects of a specific category, namely semantic information of scenes; the single frame semantic point generation algorithm is shown in fig. 4.
5) And generating the unmanned decision-making opinions according to the obtained road condition types, returning the unmanned decision-making opinions to the unmanned control system, and generating the unmanned decision-making opinions according to the road condition types.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (4)
1. A road condition identification and decision-making method in an unmanned driving scene is characterized in that: the method comprises the following steps:
firstly, a vehicle trains a picture expansion model by building a deep convolution neural network, and obtains a trained model;
secondly, the vehicle realizes the relative positioning of the position of the vehicle in an unknown environment, and pictures are shot through a sensor;
thirdly, the vehicle identifies and classifies the environment through a model which is matched and trained by pictures shot by the sensor;
step four, starting moving exploration of the vehicle through an unknown map path exploration algorithm, carrying out self-positioning through observation of the environment, and simultaneously building an incremental map;
fifthly, the vehicle realizes real-time construction and updating of the map during exploration through multi-sensor fusion data until the map construction is completed;
and step six, the vehicle control system makes a decision according to the sensed information.
2. The method of claim 1, wherein the method comprises the steps of: and in the third step, the pose estimation of the visual SLAM algorithm is adopted.
3. The method of claim 1, wherein the method comprises the steps of: and in the fourth step, local map splicing and global map model construction are realized through point cloud splicing and filtering technology.
4. The method of claim 1, wherein the method comprises the steps of: and fifthly, building the semantic map in the dynamic environment by combining the semantic information with the SLAM algorithm adopted by the invention.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111058219.3A CN113673484A (en) | 2021-09-09 | 2021-09-09 | Road condition identification and decision-making method in unmanned driving scene |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111058219.3A CN113673484A (en) | 2021-09-09 | 2021-09-09 | Road condition identification and decision-making method in unmanned driving scene |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113673484A true CN113673484A (en) | 2021-11-19 |
Family
ID=78549224
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111058219.3A Pending CN113673484A (en) | 2021-09-09 | 2021-09-09 | Road condition identification and decision-making method in unmanned driving scene |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113673484A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114445593A (en) * | 2022-01-30 | 2022-05-06 | 重庆长安汽车股份有限公司 | Aerial view semantic segmentation label generation method based on multi-frame semantic point cloud splicing |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109117718A (en) * | 2018-07-02 | 2019-01-01 | 东南大学 | A kind of semantic map structuring of three-dimensional towards road scene and storage method |
| CN110363816A (en) * | 2019-06-25 | 2019-10-22 | 广东工业大学 | A kind of mobile robot environment semanteme based on deep learning builds drawing method |
| CN111368759A (en) * | 2020-03-09 | 2020-07-03 | 河海大学常州校区 | Monocular vision-based semantic map construction system for mobile robot |
| US20200364554A1 (en) * | 2018-02-09 | 2020-11-19 | Baidu Usa Llc | Systems and methods for deep localization and segmentation with a 3d semantic map |
-
2021
- 2021-09-09 CN CN202111058219.3A patent/CN113673484A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200364554A1 (en) * | 2018-02-09 | 2020-11-19 | Baidu Usa Llc | Systems and methods for deep localization and segmentation with a 3d semantic map |
| CN109117718A (en) * | 2018-07-02 | 2019-01-01 | 东南大学 | A kind of semantic map structuring of three-dimensional towards road scene and storage method |
| CN110363816A (en) * | 2019-06-25 | 2019-10-22 | 广东工业大学 | A kind of mobile robot environment semanteme based on deep learning builds drawing method |
| CN111368759A (en) * | 2020-03-09 | 2020-07-03 | 河海大学常州校区 | Monocular vision-based semantic map construction system for mobile robot |
Non-Patent Citations (1)
| Title |
|---|
| 何松 等: "基于激光SLAM和深度学习的语义地图构建", 《计算机技术与发展》, vol. 30, no. 9, pages 88 - 94 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114445593A (en) * | 2022-01-30 | 2022-05-06 | 重庆长安汽车股份有限公司 | Aerial view semantic segmentation label generation method based on multi-frame semantic point cloud splicing |
| CN114445593B (en) * | 2022-01-30 | 2024-05-10 | 重庆长安汽车股份有限公司 | Bird's eye view semantic segmentation label generation method based on multi-frame semantic point cloud splicing |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Toft et al. | Long-term visual localization revisited | |
| Sahu et al. | Artificial intelligence (AI) in augmented reality (AR)-assisted manufacturing applications: a review | |
| US10854006B2 (en) | AR-enabled labeling using aligned CAD models | |
| CN110135249B (en) | Human behavior identification method based on time attention mechanism and LSTM (least Square TM) | |
| CN110119148B (en) | Six-degree-of-freedom attitude estimation method and device and computer readable storage medium | |
| CN109815847B (en) | Visual SLAM method based on semantic constraint | |
| Liu et al. | RGB-D inertial odometry for a resource-restricted robot in dynamic environments | |
| CN107967457A (en) | A kind of place identification for adapting to visual signature change and relative positioning method and system | |
| Wen et al. | Hybrid semi-dense 3D semantic-topological mapping from stereo visual-inertial odometry SLAM with loop closure detection | |
| Li et al. | Overview of deep learning application on visual SLAM | |
| WO2019175532A1 (en) | Urban environment labelling | |
| Cheng et al. | Improving dense mapping for mobile robots in dynamic environments based on semantic information | |
| Liu et al. | D-lc-nets: Robust denoising and loop closing networks for lidar slam in complicated circumstances with noisy point clouds | |
| Civera et al. | Rgb-d odometry and slam | |
| Li et al. | Multi-modal neural feature fusion for automatic driving through perception-aware path planning | |
| Dai et al. | A review of common techniques for visual simultaneous localization and mapping | |
| CN113673484A (en) | Road condition identification and decision-making method in unmanned driving scene | |
| CN114399515A (en) | Language description-based class-level target object 6D pose acquisition method and storage medium | |
| CN117788544A (en) | Image depth estimation method based on lightweight attention mechanism | |
| Zhang et al. | Diff-tracker: Text-to-image diffusion models are unsupervised trackers | |
| CN112348854A (en) | Visual inertial mileage detection method based on deep learning | |
| Liedtke et al. | Use of explicit knowledge for the reconstruction of 3-D object geometry | |
| CN116721139A (en) | Generating depth images of image data | |
| CN114943785A (en) | Map construction method, map construction device, map construction equipment and storage medium | |
| Liu et al. | An efficient edge-feature constraint visual SLAM |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |