CN105488459A - Vehicle-mounted 3D road real-time reconstruction method and apparatus - Google Patents
Vehicle-mounted 3D road real-time reconstruction method and apparatus Download PDFInfo
- Publication number
- CN105488459A CN105488459A CN201510819904.1A CN201510819904A CN105488459A CN 105488459 A CN105488459 A CN 105488459A CN 201510819904 A CN201510819904 A CN 201510819904A CN 105488459 A CN105488459 A CN 105488459A
- Authority
- CN
- China
- Prior art keywords
- cloud data
- point
- data set
- data acquisition
- depth image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 38
- 239000011159 matrix material Substances 0.000 claims abstract description 63
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 32
- 230000009466 transformation Effects 0.000 claims abstract description 22
- 238000004364 calculation method Methods 0.000 claims abstract description 12
- 238000012545 processing Methods 0.000 claims abstract description 10
- 238000005070 sampling Methods 0.000 claims abstract description 5
- 238000013519 translation Methods 0.000 claims description 27
- 238000003909 pattern recognition Methods 0.000 claims description 24
- 238000010168 coupling process Methods 0.000 claims description 13
- 238000005859 coupling reaction Methods 0.000 claims description 13
- 230000008878 coupling Effects 0.000 claims description 12
- 230000001143 conditioned effect Effects 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 10
- 230000009467 reduction Effects 0.000 claims description 8
- 238000005286 illumination Methods 0.000 abstract description 7
- 230000004927 fusion Effects 0.000 abstract description 5
- 238000003384 imaging method Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000003860 storage Methods 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/05—Geographic models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Remote Sensing (AREA)
- Computer Graphics (AREA)
- Image Analysis (AREA)
Abstract
Embodiments of the invention provide a vehicle-mounted 3D road real-time reconstruction method and apparatus. The method comprises: obtaining a first depth image and a second depth image of a target road at two adjacent sampling moments; performing perspective projection transformation on the first depth image and the second depth image to obtain a first point cloud data set and a second point cloud data set; processing the first set and the second set according to an ICP algorithm to obtain a rotary matrix R and a translational matrix T; and normalizing the first point cloud data set and the second point cloud data set to a same coordinate system according to the rotary matrix R and the translational matrix T to obtain a reconstructed model. According to the embodiments of the invention, the fusion of the point cloud data sets is realized, the reconstructed model is obtained, the influences of factors of color, texture, illumination intensity and the like on road information are avoided, the calculation precision of the road information is improved, and accurate, real-time and reliable front road surface information is provided for drivers.
Description
Technology neighborhood
The embodiment of the present invention relates to technical field of image processing, particularly relates to a kind of vehicle-mounted 3D road Real-time Reconstruction method and device.
Background technology
Along with the development of auto manufacturing, automobile has become the vehicles indispensable in people's daily life.
In prior art, intelligence DAS (Driver Assistant System) and automatic Pilot accelerated development, go to obtain that road ahead information just becoming is more and more general by machine vision technique, but the technology such as traditional road Identification based on two dimensional image, lane detection, and the reason of sensor imaging mechanism, make road information be easy to be subject to the impact of the factors such as color, texture, intensity of illumination, reduce the computational accuracy of road information, can not provide accurately for driver, in real time, road surface ahead information reliably.
Summary of the invention
The embodiment of the present invention provides a kind of vehicle-mounted 3D road Real-time Reconstruction method and device, so as driver provide accurately, in real time, road surface ahead information reliably.
An aspect of the embodiment of the present invention is to provide a kind of vehicle-mounted 3D road Real-time Reconstruction method, comprising:
The first depth image M of target road is obtained respectively two neighbouring sample moment
k-1with the second depth image M
k;
By described first depth image M
k-1with described second depth image M
kthe first cloud data set N is obtained respectively through perspective projection transformation
k-1with second point cloud data acquisition N
k;
According to described first cloud data set N
k-1obtain first object cloud data set N'
k-1, and according to described second point cloud data acquisition N
kobtain the second impact point cloud data acquisition N'
k;
Calculate described first object cloud data set N'
k-1in each corresponding PFH feature histogram, and described second impact point cloud data acquisition N'
kin each the 2nd corresponding PFH feature histogram;
If a described PFH feature histogram and described 2nd PFH feature histogram coupling, then corresponding with described PFH feature histogram point is the first matching characteristic point, the point corresponding with described 2nd PFH feature histogram is the second matching characteristic point, and all described first matching characteristic points are formed the first set Q'
k-1, all described second matching characteristic points are formed the second set Q'
k;
According to the first set Q' described in ICP algorithm process
k-1with described second set Q'
kobtain rotation matrix R and translation matrix T, and Q'
k=R*Q'
k-1+ T;
According to described rotation matrix R and described translation matrix T by the first cloud data set N
k-1with second point cloud data acquisition N
kbe normalized to same coordinate system and obtain reconstruction model;
Using the input data of described reconstruction model as algorithm for pattern recognition, according to described algorithm for pattern recognition, pattern-recognition is carried out to described reconstruction model, export information of road surface corresponding to described target road to make described algorithm for pattern recognition.
Another aspect of the embodiment of the present invention is to provide a kind of vehicle-mounted 3D road Real-time Reconstruction device, comprising:
Sampling module, for obtaining the first depth image M of target road respectively two neighbouring sample moment
k-1with the second depth image M
k;
Projection module, for by described first depth image M
k-1with described second depth image M
kthe first cloud data set N is obtained respectively through perspective projection transformation
k-1with second point cloud data acquisition N
k;
Denoising module, for according to described first cloud data set N
k-1obtain first object cloud data set N'
k-1, and according to described second point cloud data acquisition N
kobtain the second impact point cloud data acquisition N'
k;
Histogram calculation module, for calculating described first object cloud data set N'
k-1in each corresponding PFH feature histogram, and described second impact point cloud data acquisition N'
kin each the 2nd corresponding PFH feature histogram;
Matching module, if for a described PFH feature histogram and described 2nd PFH feature histogram coupling, then corresponding with described PFH feature histogram point is the first matching characteristic point, the point corresponding with described 2nd PFH feature histogram is the second matching characteristic point, and all described first matching characteristic points are formed the first set Q'
k-1, all described second matching characteristic points are formed the second set Q'
k;
ICP processing module, for the first set Q' described in foundation ICP algorithm process
k-1with described second set Q'
kobtain rotation matrix R and translation matrix T, and Q'
k=R*Q'
k-1+ T;
Normalizing module, for according to described rotation matrix R and described translation matrix T by the first cloud data set N
k-1with second point cloud data acquisition N
kbe normalized to same coordinate system and obtain reconstruction model; Using the input data of described reconstruction model as algorithm for pattern recognition, according to described algorithm for pattern recognition, pattern-recognition is carried out to described reconstruction model, export information of road surface corresponding to described target road to make described algorithm for pattern recognition.
The vehicle-mounted 3D road Real-time Reconstruction method that the embodiment of the present invention provides and device, by two-dimensional depth image being converted to three-dimensional cloud data set, each corresponding PFH feature histogram in acquisition point cloud data acquisition, the matching characteristic point pair of adjacent two moment point cloud data acquisitions is found by PFH feature histogram coupling, to matching characteristic point to enforcement ICP algorithm, ask for the rotation translation matrix of adjacent two moment point cloud data acquisitions, according to rotating translation matrix, two cloud data set are normalized to the same coordinate system, achieve the fusion of cloud data set, obtain reconstruction model, road information is avoided to be subject to color, texture, the impact of the factors such as intensity of illumination, improve the computational accuracy of road information, for driver provides accurate, in real time, reliable road surface ahead information.
Accompanying drawing explanation
The vehicle-mounted 3D road Real-time Reconstruction method flow diagram that Fig. 1 provides for the embodiment of the present invention;
The schematic diagram of the neighborhood point of any point in the impact point cloud data acquisition that Fig. 2 provides for the embodiment of the present invention;
The coordinate schematic diagram of any point in the impact point cloud data acquisition that Fig. 3 provides for the embodiment of the present invention;
The schematic diagram of the PFH feature histogram that Fig. 4 provides for the embodiment of the present invention;
The structural drawing that principal and subordinate's computing method that Fig. 5 provides for the embodiment of the present invention are suitable for;
The structural drawing of the vehicle-mounted 3D road Real-time Reconstruction device that Fig. 6 provides for the embodiment of the present invention.
Embodiment
The vehicle-mounted 3D road Real-time Reconstruction method flow diagram that Fig. 1 provides for the embodiment of the present invention; The schematic diagram of the neighborhood point of any point in the impact point cloud data acquisition that Fig. 2 provides for the embodiment of the present invention.The embodiment of the present invention is for technology such as traditional road Identification based on two dimensional image, lane detection, and the reason of sensor imaging mechanism, road information is made to be easy to be subject to the impact of the factors such as color, texture, intensity of illumination, reduce the computational accuracy of road information, can not provide accurately for driver, in real time, road surface ahead information reliably, provide vehicle-mounted 3D road Real-time Reconstruction method, the concrete steps of the method are as follows:
Step S101, obtain the first depth image M of target road respectively two neighbouring sample moment
k-1with the second depth image M
k;
The embodiment of the present invention obtains the depth image of road ahead scene by the TOF imaging device being arranged on front part of vehicle, and TOF imaging device is specifically as follows 3D imaging sensor, 3D imaging sensor is with the depth image of higher employing frequency acquisition road ahead scene, such as, in the K-1 moment, 3D imaging sensor carries out single pass to road ahead scene and obtains the first depth image M
k-1, in the K moment, 3D imaging sensor carries out single pass to road ahead scene and obtains the second depth image M
k.
Step S102, by described first depth image M
k-1with described second depth image M
kthe first cloud data set N is obtained respectively through perspective projection transformation
k-1with second point cloud data acquisition N
k;
Described by described first depth image M
k-1with described second depth image M
kthe first cloud data set N is obtained respectively through perspective projection transformation
k-1with second point cloud data acquisition N
k, comprising: obtain the transformation matrix F of depth image to cloud data set; According to described transformation matrix F and described first depth image M
k-1calculate described first cloud data set N
k-1=F*M
k-1, according to described transformation matrix F and described second depth image M
kcalculate described second point cloud data acquisition N
k=F*M
k.
The parameter of demarcating according to 3D imaging sensor inside and outside location parameter obtain the transformation matrix F of depth image to cloud data set, calculate described first cloud data set N
k-1=F*M
k-1with second point cloud data acquisition N
k=F*M
k.
Step S103, according to described first cloud data set N
k-1obtain first object cloud data set N'
k-1, and according to described second point cloud data acquisition N
kobtain the second impact point cloud data acquisition N'
k;
To described first cloud data set N
k-1carry out noise reduction or simplify pre-service obtaining described first object cloud data set N'
k-1; To described second point cloud data acquisition N
kcarry out noise reduction or simplify pre-service obtaining described second impact point cloud data acquisition N'
k.
Step S104, calculate described first object cloud data set N'
k-1in each corresponding PFH feature histogram, and described second impact point cloud data acquisition N'
kin each the 2nd corresponding PFH feature histogram;
As shown in Figure 2, putting P is first object cloud data set N'
k-1in any point, apart from having H neighborhood point in this P preset range, P
tbe any one point in H the neighborhood point adjacent with a P, what be connected with a P is other neighborhood point, according to the normal n that some P is corresponding
sdeviation between normal corresponding with each the neighborhood point in H neighborhood point respectively obtains a PFH feature histogram corresponding to some P, in like manner, and first object cloud data set N'
k-1in the corresponding PFH feature histogram of each point.Described second impact point cloud data acquisition N' can be calculated according to same method
kin each the 2nd corresponding PFH feature histogram.
If the described PFH feature histogram of step S105 and described 2nd PFH feature histogram coupling, then corresponding with described PFH feature histogram point is the first matching characteristic point, the point corresponding with described 2nd PFH feature histogram is the second matching characteristic point, and all described first matching characteristic points are formed the first set Q'
k-1, all described second matching characteristic points are formed the second set Q'
k;
First object cloud data set N' can be obtained by step S104
k-1in a PFH feature histogram corresponding to each point, and the second impact point cloud data acquisition N'
kin each the 2nd corresponding PFH feature histogram, if first object cloud data set N'
k-1in a PFH feature histogram of a some correspondence and the second impact point cloud data acquisition N'
kin the 2nd PFH feature histogram of a some correspondence identical, then these two points are respectively matching characteristic point, first object cloud data set N'
k-1in matching characteristic point form first set Q'
k-1, the second impact point cloud data acquisition N'
kin matching characteristic point form second set Q'
k.
First set Q' described in step S106, foundation ICP algorithm process
k-1with described second set Q'
kobtain rotation matrix R and translation matrix T, and Q'
k=R*Q'
k-1+ T;
To described first set Q'
k-1with described second set Q'
kcarry out ICP algorithm process, obtain rotation matrix R and translation matrix T, and Q'
k=R*Q'
k-1+ T.
Step S107, according to described rotation matrix R and described translation matrix T by the first cloud data set N
k-1with second point cloud data acquisition N
kbe normalized to same coordinate system and obtain reconstruction model.
Described according to described rotation matrix R and described translation matrix T by the first cloud data set N
k-1with second point cloud data acquisition N
kbe normalized to same coordinate system and obtain reconstruction model, comprising:
According to described rotation matrix R, described translation matrix T and described first cloud data set N
k-1calculate thirdly cloud data acquisition N "
k=R*N
k-1+ T;
According to described thirdly cloud data acquisition N "
kwith described second point cloud data acquisition N
kobtain reconstruction model N "
k+ N
k.
Step S108, using the input data of described reconstruction model as algorithm for pattern recognition, according to described algorithm for pattern recognition, pattern-recognition is carried out to described reconstruction model, export information of road surface corresponding to described target road to make described algorithm for pattern recognition.
The embodiment of the present invention is by being converted to three-dimensional cloud data set by two-dimensional depth image, each corresponding PFH feature histogram in acquisition point cloud data acquisition, the matching characteristic point pair of adjacent two moment point cloud data acquisitions is found by PFH feature histogram coupling, to matching characteristic point to enforcement ICP algorithm, ask for the rotation translation matrix of adjacent two moment point cloud data acquisitions, according to rotating translation matrix, two cloud data set are normalized to the same coordinate system, achieve the fusion of cloud data set, obtain reconstruction model, road information is avoided to be subject to color, texture, the impact of the factors such as intensity of illumination, improve the computational accuracy of road information, for driver provides accurate, in real time, reliable road surface ahead information.
The coordinate schematic diagram of any point in the impact point cloud data acquisition that Fig. 3 provides for the embodiment of the present invention.The schematic diagram of the PFH feature histogram that Fig. 4 provides for the embodiment of the present invention.The structural drawing that principal and subordinate's computing method that Fig. 5 provides for the embodiment of the present invention are suitable for.On the basis of above-described embodiment, described calculating described first object cloud data set N'
k-1in each corresponding PFH feature histogram, comprising:
Define described first object cloud data set N'
k-1the coordinate of middle any point P is (u, v, w), wherein, and u=n
s,
w=u × v, n
sthe normal that a P is corresponding, P
tany one point in H the neighborhood point adjacent with a P, and the normal n that some P is corresponding
swith a P
tcorresponding normal n
tbetween deviation be expressed as
wherein, α=v*n
t,
θ=arctan (w*n
t, u*n
t); The horizontal ordinate dividing two-dimensional coordinate at equal intervals obtains multiple calibration points, and described multiple calibration points is corresponding a different set of respectively
value; Add up for each calibration points in described two-dimensional coordinate the number meeting pre-conditioned impact point in described H neighborhood point and obtain a described PFH feature histogram, it is corresponding that the deviation between the normal that the described pre-conditioned normal corresponding for described impact point is corresponding with some P equals described calibration points
value.
For first object cloud data set N'
k-1in some P, define its coordinate for (u, v, w), wherein, u=n
s,
w=u × v, n
sthe normal that a P is corresponding, P
tany one point in H the neighborhood point adjacent with a P, and the normal n that some P is corresponding
swith a P
tcorresponding normal n
tbetween deviation be expressed as
wherein, α=v*n
t,
θ=arctan (w*n
t, u*n
t).So, the normal n that P is corresponding
sa deviation is had between the normal that any one point in H the neighborhood point be adjacent is corresponding
in like manner, first object cloud data set N'
k-1in H neighborhood point being adjacent of normal corresponding to any one point in any one some correspondence normal between have a deviation
As shown in Figure 4, the horizontal ordinate dividing two-dimensional coordinate at equal intervals obtains multiple calibration points, and preferably, the embodiment of the present invention is divided into 125 calibration points, is specially 0-124, and each calibration points correspondence is a different set of
value; For each calibration points of 125 calibration points, the number of pre-conditioned impact point is met in H neighborhood point in the preset range of statistics point P, this number is as ordinate corresponding to this calibration points, and it is corresponding that the deviation between the concrete pre-conditioned normal corresponding with some P for the normal of impact point equals described calibration points
value, so obtains the PFH feature histogram that some P is corresponding, i.e. described first object cloud data set N'
k-1in each point can portray with a PFH feature histogram, in like manner, described second impact point cloud data acquisition N' can be obtained
kin each the 2nd corresponding PFH feature histogram.
On the basis of the embodiment of the present invention, it is the ability that whole system provides high-speed parallel computing power to calculate with reply complex logic.Owing to needing cloud data amount to be processed huge, as shown in Figure 5, the account form that native system adopts principal and subordinate to combine implements related algorithm, and main computation unit is taken on by universal cpu, for realizing complicated algorithm logic; Taken on by GPU or special parallel computation unit from computing unit, be mainly used in realizing a large amount of double counting; Main computation unit need converge conjunction and is divided into some subsets by point to be processed, is distributed to the sub-computing unit that each is parallel, after being disposed, by main computation unit aggregation process result.
The embodiment of the present invention specifically provides the method how calculating each corresponding PFH feature histogram in impact point cloud data acquisition, in addition, is improve the speed of data processing by principal and subordinate's computing method.
The structural drawing of the vehicle-mounted 3D road Real-time Reconstruction device that Fig. 6 provides for the embodiment of the present invention.The vehicle-mounted 3D road Real-time Reconstruction device that the embodiment of the present invention provides can perform the treatment scheme that vehicle-mounted 3D road Real-time Reconstruction embodiment of the method provides, as shown in Figure 6, vehicle-mounted 3D road Real-time Reconstruction device 50 comprises sampling module 51, projection module 52, denoising module 53, histogram calculation module 54, matching module 55, ICP processing module 56 and normalizing module 57, wherein, sampling module 51 is for obtaining the first depth image M of target road respectively two neighbouring sample moment
k-1with the second depth image M
k; Projection module 52 is for by described first depth image M
k-1with described second depth image M
kthe first cloud data set N is obtained respectively through perspective projection transformation
k-1with second point cloud data acquisition N
k; Denoising module 53 is for according to described first cloud data set N
k-1obtain first object cloud data set N'
k-1, and according to described second point cloud data acquisition N
kobtain the second impact point cloud data acquisition N'
k; Histogram calculation module 54 is for calculating described first object cloud data set N'
k-1in each corresponding PFH feature histogram, and described second impact point cloud data acquisition N'
kin each the 2nd corresponding PFH feature histogram; If matching module 55 is for a described PFH feature histogram and described 2nd PFH feature histogram coupling, then corresponding with described PFH feature histogram point is the first matching characteristic point, the point corresponding with described 2nd PFH feature histogram is the second matching characteristic point, and all described first matching characteristic points are formed the first set Q'
k-1, all described second matching characteristic points are formed the second set Q'
k; ICP processing module 56 is for the first set Q' described in foundation ICP algorithm process
k-1with described second set Q'
kobtain rotation matrix R and translation matrix T, and Q'
k=R*Q'
k-1+ T; Normalizing module 57 for according to described rotation matrix R and described translation matrix T by the first cloud data set N
k-1with second point cloud data acquisition N
kbe normalized to same coordinate system and obtain reconstruction model; Using the input data of described reconstruction model as algorithm for pattern recognition, according to described algorithm for pattern recognition, pattern-recognition is carried out to described reconstruction model, export information of road surface corresponding to described target road to make described algorithm for pattern recognition.
The embodiment of the present invention is by being converted to three-dimensional cloud data set by two-dimensional depth image, each corresponding PFH feature histogram in acquisition point cloud data acquisition, the matching characteristic point pair of adjacent two moment point cloud data acquisitions is found by PFH feature histogram coupling, to matching characteristic point to enforcement ICP algorithm, ask for the rotation translation matrix of adjacent two moment point cloud data acquisitions, according to rotating translation matrix, two cloud data set are normalized to the same coordinate system, achieve the fusion of cloud data set, obtain reconstruction model, road information is avoided to be subject to color, texture, the impact of the factors such as intensity of illumination, improve the computational accuracy of road information, for driver provides accurate, in real time, reliable road surface ahead information.
On the basis of above-described embodiment, projection module 52 is specifically for obtaining the transformation matrix F of depth image to cloud data set; According to described transformation matrix F and described first depth image M
k-1calculate described first cloud data set N
k-1=F*M
k-1, according to described transformation matrix F and described second depth image M
kcalculate described second point cloud data acquisition N
k=F*M
k.
Denoising module 53 is specifically for described first cloud data set N
k-1carry out noise reduction or simplify pre-service obtaining described first object cloud data set N'
k-1; To described second point cloud data acquisition N
kcarry out noise reduction or simplify pre-service obtaining described second impact point cloud data acquisition N'
k.
Histogram calculation module 54 is specifically for defining described first object cloud data set N'
k-1the coordinate of middle any point P is (u, v, w), wherein, and u=n
s,
w=u × v, n
sthe normal that a P is corresponding, P
tany one point in H the neighborhood point adjacent with a P, and the normal n that some P is corresponding
swith a P
tcorresponding normal n
tbetween deviation be expressed as
wherein, α=v*n
t,
θ=arctan (w*n
t, u*n
t); The horizontal ordinate dividing two-dimensional coordinate at equal intervals obtains multiple calibration points, and described multiple calibration points is corresponding a different set of respectively
value; Add up for each calibration points in described two-dimensional coordinate the number meeting pre-conditioned impact point in described H neighborhood point and obtain a described PFH feature histogram, it is corresponding that the deviation between the normal that the described pre-conditioned normal corresponding for described impact point is corresponding with some P equals described calibration points
value.
Normalizing module 57 is specifically for according to described rotation matrix R, described translation matrix T and described first cloud data set N
k-1calculate thirdly cloud data acquisition N "
k=R*N
k-1+ T; According to described thirdly cloud data acquisition N "
kwith described second point cloud data acquisition N
kobtain reconstruction model N "
k+ N
k.
The vehicle-mounted 3D road Real-time Reconstruction device that the embodiment of the present invention provides can specifically for performing the embodiment of the method that above-mentioned Fig. 1 provides, and concrete function repeats no more herein.
The embodiment of the present invention specifically provides the method how calculating each corresponding PFH feature histogram in impact point cloud data acquisition, in addition, is improve the speed of data processing by principal and subordinate's computing method.
In sum, the embodiment of the present invention is by being converted to three-dimensional cloud data set by two-dimensional depth image, each corresponding PFH feature histogram in acquisition point cloud data acquisition, the matching characteristic point pair of adjacent two moment point cloud data acquisitions is found by PFH feature histogram coupling, to matching characteristic point to enforcement ICP algorithm, ask for the rotation translation matrix of adjacent two moment point cloud data acquisitions, according to rotating translation matrix, two cloud data set are normalized to the same coordinate system, achieve the fusion of cloud data set, obtain reconstruction model, road information is avoided to be subject to color, texture, the impact of the factors such as intensity of illumination, improve the computational accuracy of road information, for driver provides accurate, in real time, reliable road surface ahead information, specifically provide the method how calculating each corresponding PFH feature histogram in impact point cloud data acquisition, in addition, improve the speed of data processing by principal and subordinate's computing method.
In several embodiment provided by the present invention, should be understood that, disclosed apparatus and method, can realize by another way.Such as, device embodiment described above is only schematic, such as, the division of described unit, be only a kind of logic function to divide, actual can have other dividing mode when realizing, such as multiple unit or assembly can in conjunction with or another system can be integrated into, or some features can be ignored, or do not perform.Another point, shown or discussed coupling each other or direct-coupling or communication connection can be by some interfaces, and the indirect coupling of device or unit or communication connection can be electrical, machinery or other form.
The described unit illustrated as separating component or can may not be and physically separates, and the parts as unit display can be or may not be physical location, namely can be positioned at a place, or also can be distributed in multiple network element.Some or all of unit wherein can be selected according to the actual needs to realize the object of the present embodiment scheme.
In addition, each functional unit in each embodiment of the present invention can be integrated in a processing unit, also can be that the independent physics of unit exists, also can two or more unit in a unit integrated.Above-mentioned integrated unit both can adopt the form of hardware to realize, and the form that hardware also can be adopted to add SFU software functional unit realizes.
The above-mentioned integrated unit realized with the form of SFU software functional unit, can be stored in a computer read/write memory medium.Above-mentioned SFU software functional unit is stored in a storage medium, comprising some instructions in order to make a computer equipment (can be personal computer, server, or the network equipment etc.) or processor (processor) perform the part steps of method described in each embodiment of the present invention.And aforesaid storage medium comprises: USB flash disk, portable hard drive, ROM (read-only memory) (Read-OnlyMemory, ROM), random access memory (RandomAccessMemory, RAM), magnetic disc or CD etc. various can be program code stored medium.
This neighborhood technique personnel can be well understood to, for convenience and simplicity of description, only be illustrated with the division of above-mentioned each functional module, in practical application, can distribute as required and by above-mentioned functions and be completed by different functional modules, inner structure by device is divided into different functional modules, to complete all or part of function described above.The specific works process of the device of foregoing description, with reference to the corresponding process in preceding method embodiment, can not repeat them here.
Last it is noted that above each embodiment is only in order to illustrate technical scheme of the present invention, be not intended to limit; Although with reference to foregoing embodiments to invention has been detailed description, the those of ordinary skill of this neighborhood is to be understood that: it still can be modified to the technical scheme described in foregoing embodiments, or carries out equivalent replacement to wherein some or all of technical characteristic; And these amendments or replacement, do not make the essence of appropriate technical solution depart from the scope of various embodiments of the present invention technical scheme.
Claims (10)
1. a vehicle-mounted 3D road Real-time Reconstruction method, is characterized in that, comprising:
The first depth image M of target road is obtained respectively two neighbouring sample moment
k-1with the second depth image M
k;
By described first depth image M
k-1with described second depth image M
kthe first cloud data set N is obtained respectively through perspective projection transformation
k-1with second point cloud data acquisition N
k;
According to described first cloud data set N
k-1obtain first object cloud data set N'
k-1, and according to described second point cloud data acquisition N
kobtain the second impact point cloud data acquisition N'
k;
Calculate described first object cloud data set N'
k-1in each corresponding PFH feature histogram, and described second impact point cloud data acquisition N'
kin each the 2nd corresponding PFH feature histogram;
If a described PFH feature histogram and described 2nd PFH feature histogram coupling, then corresponding with described PFH feature histogram point is the first matching characteristic point, the point corresponding with described 2nd PFH feature histogram is the second matching characteristic point, and all described first matching characteristic points are formed the first set Q'
k-1, all described second matching characteristic points are formed the second set Q'
k;
According to the first set Q' described in ICP algorithm process
k-1with described second set Q'
kobtain rotation matrix R and translation matrix T, and Q'
k=R*Q'
k-1+ T;
According to described rotation matrix R and described translation matrix T by the first cloud data set N
k-1with second point cloud data acquisition N
kbe normalized to same coordinate system and obtain reconstruction model;
Using the input data of described reconstruction model as algorithm for pattern recognition, according to described algorithm for pattern recognition, pattern-recognition is carried out to described reconstruction model, export information of road surface corresponding to described target road to make described algorithm for pattern recognition.
2. method according to claim 1, is characterized in that, described by described first depth image M
k-1with described second depth image M
kthe first cloud data set N is obtained respectively through perspective projection transformation
k-1with second point cloud data acquisition N
k, comprising:
Obtain the transformation matrix F of depth image to cloud data set;
According to described transformation matrix F and described first depth image M
k-1calculate described first cloud data set N
k-1=F*M
k-1, according to described transformation matrix F and described second depth image M
kcalculate described second point cloud data acquisition N
k=F*M
k.
3. method according to claim 2, is characterized in that, described according to described first cloud data set N
k-1obtain first object cloud data set N'
k-1, comprising:
To described first cloud data set N
k-1carry out noise reduction or simplify pre-service obtaining described first object cloud data set N'
k-1;
Described according to described second point cloud data acquisition N
kobtain the second impact point cloud data acquisition N'
k, comprising:
To described second point cloud data acquisition N
kcarry out noise reduction or simplify pre-service obtaining described second impact point cloud data acquisition N'
k.
4. the method according to any one of claim 1-3, is characterized in that, described calculating described first object cloud data set N'
k-1in each corresponding PFH feature histogram, comprising:
Define described first object cloud data set N'
k-1the coordinate of middle any point P is (u, v, w), wherein, and u=n
s,
w=u × v, n
sthe normal that a P is corresponding, P
tany one point in H the neighborhood point adjacent with a P, and the normal n that some P is corresponding
swith a P
tcorresponding normal n
tbetween deviation be expressed as
wherein, α=v*n
t,
θ=arctan (w*n
t, u*n
t);
The horizontal ordinate dividing two-dimensional coordinate at equal intervals obtains multiple calibration points, and described multiple calibration points is corresponding a different set of respectively
value;
Add up for each calibration points in described two-dimensional coordinate the number meeting pre-conditioned impact point in described H neighborhood point and obtain a described PFH feature histogram, it is corresponding that the deviation between the normal that the described pre-conditioned normal corresponding for described impact point is corresponding with some P equals described calibration points
value.
5. method according to claim 4, is characterized in that, described according to described rotation matrix R and described translation matrix T by the first cloud data set N
k-1with second point cloud data acquisition N
kbe normalized to same coordinate system and obtain reconstruction model, comprising:
According to described rotation matrix R, described translation matrix T and described first cloud data set N
k-1calculate thirdly cloud data acquisition N "
k=R*N
k-1+ T;
According to described thirdly cloud data acquisition N "
kwith described second point cloud data acquisition N
kobtain reconstruction model N "
k+ N
k.
6. a vehicle-mounted 3D road Real-time Reconstruction device, is characterized in that, comprising:
Sampling module, for obtaining the first depth image M of target road respectively two neighbouring sample moment
k-1with the second depth image M
k;
Projection module, for by described first depth image M
k-1with described second depth image M
kthe first cloud data set N is obtained respectively through perspective projection transformation
k-1with second point cloud data acquisition N
k;
Denoising module, for according to described first cloud data set N
k-1obtain first object cloud data set N'
k-1, and according to described second point cloud data acquisition N
kobtain the second impact point cloud data acquisition N'
k;
Histogram calculation module, for calculating described first object cloud data set N'
k-1in each corresponding PFH feature histogram, and described second impact point cloud data acquisition N'
kin each the 2nd corresponding PFH feature histogram;
Matching module, if for a described PFH feature histogram and described 2nd PFH feature histogram coupling, then corresponding with described PFH feature histogram point is the first matching characteristic point, the point corresponding with described 2nd PFH feature histogram is the second matching characteristic point, and all described first matching characteristic points are formed the first set Q'
k-1, all described second matching characteristic points are formed the second set Q'
k;
ICP processing module, for the first set Q' described in foundation ICP algorithm process
k-1with described second set Q'
kobtain rotation matrix R and translation matrix T, and Q'
k=R*Q'
k-1+ T;
Normalizing module, for according to described rotation matrix R and described translation matrix T by the first cloud data set N
k-1with second point cloud data acquisition N
kbe normalized to same coordinate system and obtain reconstruction model; Using the input data of described reconstruction model as algorithm for pattern recognition, according to described algorithm for pattern recognition, pattern-recognition is carried out to described reconstruction model, export information of road surface corresponding to described target road to make described algorithm for pattern recognition.
7. vehicle-mounted 3D road Real-time Reconstruction device according to claim 6, is characterized in that, described projection module is specifically for obtaining the transformation matrix F of depth image to cloud data set; According to described transformation matrix F and described first depth image M
k-1calculate described first cloud data set N
k-1=F*M
k-1, according to described transformation matrix F and described second depth image M
kcalculate described second point cloud data acquisition N
k=F*M
k.
8. vehicle-mounted 3D road Real-time Reconstruction device according to claim 7, is characterized in that, described denoising module is specifically for described first cloud data set N
k-1carry out noise reduction or simplify pre-service obtaining described first object cloud data set N'
k-1; To described second point cloud data acquisition N
kcarry out noise reduction or simplify pre-service obtaining described second impact point cloud data acquisition N'
k.
9. the vehicle-mounted 3D road Real-time Reconstruction device according to any one of claim 6-8, is characterized in that, described histogram calculation module is specifically for defining described first object cloud data set N'
k-1the coordinate of middle any point P is (u, v, w), wherein, and u=n
s,
w=u × v, n
sthe normal that a P is corresponding, P
tany one point in H the neighborhood point adjacent with a P, and the normal n that some P is corresponding
swith a P
tcorresponding normal n
tbetween deviation be expressed as
wherein, α=v*n
t,
θ=arctan (w*n
t, u*n
t);
The horizontal ordinate dividing two-dimensional coordinate at equal intervals obtains multiple calibration points, and described multiple calibration points is corresponding a different set of respectively
value;
Add up for each calibration points in described two-dimensional coordinate the number meeting pre-conditioned impact point in described H neighborhood point and obtain a described PFH feature histogram, it is corresponding that the deviation between the normal that the described pre-conditioned normal corresponding for described impact point is corresponding with some P equals described calibration points
value.
10. vehicle-mounted 3D road Real-time Reconstruction device according to claim 9, is characterized in that, described normalizing module is specifically for according to described rotation matrix R, described translation matrix T and described first cloud data set N
k-1calculate thirdly cloud data acquisition N "
k=R*N
k-1+ T; According to described thirdly cloud data acquisition N "
kwith described second point cloud data acquisition N
kobtain reconstruction model N "
k+ N
k.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510819904.1A CN105488459A (en) | 2015-11-23 | 2015-11-23 | Vehicle-mounted 3D road real-time reconstruction method and apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510819904.1A CN105488459A (en) | 2015-11-23 | 2015-11-23 | Vehicle-mounted 3D road real-time reconstruction method and apparatus |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105488459A true CN105488459A (en) | 2016-04-13 |
Family
ID=55675431
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510819904.1A Pending CN105488459A (en) | 2015-11-23 | 2015-11-23 | Vehicle-mounted 3D road real-time reconstruction method and apparatus |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105488459A (en) |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107798724A (en) * | 2016-09-02 | 2018-03-13 | 德尔福技术有限公司 | Automated vehicle 3D road models and lane markings define system |
| CN108254758A (en) * | 2017-12-25 | 2018-07-06 | 清华大学苏州汽车研究院（吴江） | Three-dimensional road construction method based on multi-line laser radar and GPS |
| CN108286976A (en) * | 2017-01-09 | 2018-07-17 | 北京四维图新科技股份有限公司 | The fusion method and device and hybrid navigation system of a kind of point cloud data |
| CN108627112A (en) * | 2018-05-09 | 2018-10-09 | 广州市杜格科技有限公司 | Vehicle axis pin is away from dynamic measurement method |
| CN108759665A (en) * | 2018-05-25 | 2018-11-06 | 哈尔滨工业大学 | A kind of extraterrestrial target reconstruction accuracy analysis method based on coordinate conversion |
| CN108960060A (en) * | 2018-06-01 | 2018-12-07 | 东南大学 | A kind of automatic driving vehicle pavement texture identifying system and method |
| WO2018232631A1 (en) * | 2017-06-21 | 2018-12-27 | 深圳配天智能技术研究院有限公司 | Image processing method, device and system, and computer storage medium |
| CN109523581A (en) * | 2017-09-19 | 2019-03-26 | 华为技术有限公司 | A kind of method and apparatus of three-dimensional point cloud alignment |
| CN109816784A (en) * | 2019-02-25 | 2019-05-28 | 盾钰（上海）互联网科技有限公司 | The method and system and medium of three-dimensionalreconstruction human body |
| CN110082779A (en) * | 2019-03-19 | 2019-08-02 | 同济大学 | A kind of vehicle pose localization method and system based on 3D laser radar |
| CN110415330A (en) * | 2019-04-29 | 2019-11-05 | 当家移动绿色互联网技术集团有限公司 | Road generation method, device, storage medium and electronic equipment |
| CN110738730A (en) * | 2019-10-15 | 2020-01-31 | 业成科技（成都）有限公司 | Point cloud matching method and device, computer equipment and storage medium |
| CN110796705A (en) * | 2019-10-23 | 2020-02-14 | 北京百度网讯科技有限公司 | Error elimination method, device, equipment and computer readable storage medium |
| CN110795819A (en) * | 2019-09-16 | 2020-02-14 | 腾讯科技（深圳）有限公司 | Method and device for generating automatic driving simulation scene and storage medium |
| CN111368604A (en) * | 2018-12-26 | 2020-07-03 | 北京图森智途科技有限公司 | Parking control method, equipment and system |
| CN111696144A (en) * | 2019-03-11 | 2020-09-22 | 北京地平线机器人技术研发有限公司 | Depth information determination method, depth information determination device and electronic equipment |
| WO2021103945A1 (en) * | 2019-11-27 | 2021-06-03 | Oppo广东移动通信有限公司 | Map fusion method, apparatus, device, and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101976460A (en) * | 2010-10-18 | 2011-02-16 | 胡振程 | Generating method of virtual view image of surveying system of vehicular multi-lens camera |
| CN102999892A (en) * | 2012-12-03 | 2013-03-27 | 东华大学 | Intelligent fusion method for depth images based on area shades and red green blue (RGB) images |
| CN103116757A (en) * | 2013-01-30 | 2013-05-22 | 北京科技大学 | Three-dimension information restoration and extraction method for identifying spilled articles on roads |
| CN104732581A (en) * | 2014-12-26 | 2015-06-24 | 东华大学 | Mobile context point cloud simplification algorithm based on point feature histogram |
| US20150254857A1 (en) * | 2014-03-10 | 2015-09-10 | Sony Corporation | Image processing system with registration mechanism and method of operation thereof |
-
2015
- 2015-11-23 CN CN201510819904.1A patent/CN105488459A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101976460A (en) * | 2010-10-18 | 2011-02-16 | 胡振程 | Generating method of virtual view image of surveying system of vehicular multi-lens camera |
| CN102999892A (en) * | 2012-12-03 | 2013-03-27 | 东华大学 | Intelligent fusion method for depth images based on area shades and red green blue (RGB) images |
| CN103116757A (en) * | 2013-01-30 | 2013-05-22 | 北京科技大学 | Three-dimension information restoration and extraction method for identifying spilled articles on roads |
| US20150254857A1 (en) * | 2014-03-10 | 2015-09-10 | Sony Corporation | Image processing system with registration mechanism and method of operation thereof |
| CN104732581A (en) * | 2014-12-26 | 2015-06-24 | 东华大学 | Mobile context point cloud simplification algorithm based on point feature histogram |
Non-Patent Citations (2)
| Title |
|---|
| 郭连朋 等: "基于kinect传感器的物体三维重建", 《四川兵工学报》 * |
| 黄军君: "基于PFH与信息融合的移动场景实时三维重构研究", 《中国学位论文全文数据库（万方数据）》 * |
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107798724A (en) * | 2016-09-02 | 2018-03-13 | 德尔福技术有限公司 | Automated vehicle 3D road models and lane markings define system |
| CN108286976A (en) * | 2017-01-09 | 2018-07-17 | 北京四维图新科技股份有限公司 | The fusion method and device and hybrid navigation system of a kind of point cloud data |
| WO2018232631A1 (en) * | 2017-06-21 | 2018-12-27 | 深圳配天智能技术研究院有限公司 | Image processing method, device and system, and computer storage medium |
| CN109523581A (en) * | 2017-09-19 | 2019-03-26 | 华为技术有限公司 | A kind of method and apparatus of three-dimensional point cloud alignment |
| CN108254758A (en) * | 2017-12-25 | 2018-07-06 | 清华大学苏州汽车研究院（吴江） | Three-dimensional road construction method based on multi-line laser radar and GPS |
| CN108627112A (en) * | 2018-05-09 | 2018-10-09 | 广州市杜格科技有限公司 | Vehicle axis pin is away from dynamic measurement method |
| CN108759665A (en) * | 2018-05-25 | 2018-11-06 | 哈尔滨工业大学 | A kind of extraterrestrial target reconstruction accuracy analysis method based on coordinate conversion |
| CN108759665B (en) * | 2018-05-25 | 2021-04-27 | 哈尔滨工业大学 | Spatial target three-dimensional reconstruction precision analysis method based on coordinate transformation |
| CN108960060A (en) * | 2018-06-01 | 2018-12-07 | 东南大学 | A kind of automatic driving vehicle pavement texture identifying system and method |
| CN111368604A (en) * | 2018-12-26 | 2020-07-03 | 北京图森智途科技有限公司 | Parking control method, equipment and system |
| CN111368604B (en) * | 2018-12-26 | 2023-07-18 | 北京图森智途科技有限公司 | A parking control method, device and system |
| CN109816784A (en) * | 2019-02-25 | 2019-05-28 | 盾钰（上海）互联网科技有限公司 | The method and system and medium of three-dimensionalreconstruction human body |
| CN111696144A (en) * | 2019-03-11 | 2020-09-22 | 北京地平线机器人技术研发有限公司 | Depth information determination method, depth information determination device and electronic equipment |
| CN110082779A (en) * | 2019-03-19 | 2019-08-02 | 同济大学 | A kind of vehicle pose localization method and system based on 3D laser radar |
| CN110415330A (en) * | 2019-04-29 | 2019-11-05 | 当家移动绿色互联网技术集团有限公司 | Road generation method, device, storage medium and electronic equipment |
| CN110795819A (en) * | 2019-09-16 | 2020-02-14 | 腾讯科技（深圳）有限公司 | Method and device for generating automatic driving simulation scene and storage medium |
| CN110795819B (en) * | 2019-09-16 | 2022-05-20 | 腾讯科技（深圳）有限公司 | Method and device for generating automatic driving simulation scene and storage medium |
| CN110738730A (en) * | 2019-10-15 | 2020-01-31 | 业成科技（成都）有限公司 | Point cloud matching method and device, computer equipment and storage medium |
| CN110738730B (en) * | 2019-10-15 | 2023-07-18 | 业成科技（成都）有限公司 | Point cloud matching method, device, computer equipment and storage medium |
| CN110796705A (en) * | 2019-10-23 | 2020-02-14 | 北京百度网讯科技有限公司 | Error elimination method, device, equipment and computer readable storage medium |
| WO2021103945A1 (en) * | 2019-11-27 | 2021-06-03 | Oppo广东移动通信有限公司 | Map fusion method, apparatus, device, and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105488459A (en) | Vehicle-mounted 3D road real-time reconstruction method and apparatus | |
| CN113819890B (en) | Distance measuring method, distance measuring device, electronic equipment and storage medium | |
| CN106908052B (en) | Path planning method and device for intelligent robot | |
| CN113628263A (en) | Point cloud registration method based on local curvature and neighbor characteristics thereof | |
| JP2022522385A (en) | Road sign recognition methods, map generation methods, and related products | |
| Chen et al. | SAANet: Spatial adaptive alignment network for object detection in automatic driving | |
| CN113267761B (en) | Laser radar target detection and identification method, system and computer readable storage medium | |
| CN113592015B (en) | Method and device for positioning and training feature matching network | |
| CN115082881A (en) | Target detection method, storage medium, electronic device, and vehicle | |
| CN113341396A (en) | Robot and charging pile identification method and device thereof | |
| Li et al. | Judgment and optimization of video image recognition in obstacle detection in intelligent vehicle | |
| Ajaykumar et al. | Automated lane detection by K-means clustering: A machine learning approach | |
| CN112102382A (en) | Electromechanical equipment visual information stereo matching algorithm based on multi-scale transformation and ADcensus-JWGF | |
| Yang et al. | Real-Time field road freespace extraction for agricultural machinery autonomous driving based on LiDAR | |
| CN116030445A (en) | A real-time 3D object detection method for autonomous driving combined with point cloud shape features | |
| CN119478256B (en) | A method and system for constructing a point cloud model of a green new energy lightweight vehicle | |
| DE112022005081T5 (en) | VALIDATION OF AN SFM MAP USING LIDAR POINT CLOUDS | |
| Wang et al. | Lane detection algorithm based on temporal–spatial information matching and fusion | |
| Hu et al. | R-CNN based 3D object detection for autonomous driving | |
| CN115439600A (en) | Live pig three-dimensional point cloud modeling method and device based on closest point iteration and mark point matching | |
| Jia et al. | Multi-scale cost volumes cascade network for stereo matching | |
| Lin et al. | Enhancing deep-learning object detection performance based on fusion of infrared and visible images in advanced driver assistance systems | |
| CN118898824A (en) | Radar-vision fusion target perception method, system, device, processor and computer-readable storage medium based on hybrid strategy | |
| CN113834463A (en) | Intelligent vehicle side pedestrian/vehicle monocular depth distance measuring method based on absolute size | |
| CN111860084B (en) | Image feature matching and positioning method and device and positioning system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information |
Inventor after: Huang Junjun Inventor after: Li Wuhui Inventor after: Zhou Qian Inventor after: Li Xianfei Inventor after: Ma Yage Inventor after: Yuan Liangxin Inventor after: Wang Xueying Inventor after: Liu Yueqiao Inventor before: Huang Junjun Inventor before: Li Wuhui Inventor before: Zhou Qian Inventor before: Li Xianfei Inventor before: Ma Yage Inventor before: Yuan Liangxin Inventor before: Wang Xueying Inventor before: Liu Yue |
|
| COR | Change of bibliographic data | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20160413 |