CN112395985A - Ground unmanned vehicle vision road detection method based on unmanned aerial vehicle image - Google Patents

Ground unmanned vehicle vision road detection method based on unmanned aerial vehicle image Download PDF

Info

Publication number
CN112395985A
CN112395985A CN202011290845.0A CN202011290845A CN112395985A CN 112395985 A CN112395985 A CN 112395985A CN 202011290845 A CN202011290845 A CN 202011290845A CN 112395985 A CN112395985 A CN 112395985A
Authority
CN
China
Prior art keywords
image
vehicle
road
unmanned aerial
particle
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011290845.0A
Other languages
Chinese (zh)
Other versions
CN112395985B (en
Inventor
王欢
田乐
彭晓蕊
何智静
卢逸飞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing University of Science and Technology
Original Assignee
Nanjing University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing University of Science and Technology filed Critical Nanjing University of Science and Technology
Priority to CN202011290845.0A priority Critical patent/CN112395985B/en
Publication of CN112395985A publication Critical patent/CN112395985A/en
Application granted granted Critical
Publication of CN112395985B publication Critical patent/CN112395985B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/182—Network patterns, e.g. roads or rivers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/56—Extraction of image or video features relating to colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image

Abstract

The invention discloses a ground unmanned vehicle vision road detection method assisted by an unmanned aerial vehicle, which comprises the following steps of firstly training two road detection models respectively used for identifying road areas in aerial images and vehicle-mounted images; then, respectively obtaining common interest area images of the aerial image and the vehicle-mounted image by using a color particle tracking algorithm, a direction matching method based on the straight line segment and inverse projection transformation; and respectively carrying out road detection on the two common interest area images, and carrying out weighted average on the identification results to obtain a road detection result with fused overlooking visual angles. The unmanned aerial vehicle is used for expanding the view field, the perception range of the unmanned vehicle is expanded, the road detection is carried out by combining the overlooking visual angle information, the accuracy of the detection of the distant road part is effectively improved, and the robustness of the road detection is improved.

Description

Ground unmanned vehicle vision road detection method based on unmanned aerial vehicle image
Technical Field
The invention belongs to a road detection technology based on vision, and particularly relates to a ground unmanned vehicle vision road detection method based on an unmanned aerial vehicle image.
Background
One of the bases on which a ground unmanned vehicle can autonomously travel is to accurately sense the surrounding environment, with road region information being the most basic. The traditional image analysis method and the learning-based method can effectively distinguish road areas, but the vehicle-mounted camera is limited in erection height, so that the perception range of an unmanned vehicle is limited to a great extent.
Disclosure of Invention
The invention aims to provide a ground unmanned vehicle vision road detection method based on an unmanned aerial vehicle image.
The technical scheme for realizing the purpose of the invention is as follows: the utility model provides an utilize camera that unmanned aerial vehicle carried on to increase road image information of visual angle of looking down on a large scale for unmanned aerial vehicle, includes the following step:
step 1, respectively extracting superpixel characteristics of a vehicle-mounted road image and an aerial image, and manufacturing a vehicle-mounted road image detection model training set and an aerial image road detection model training set;
step 2, training an SVM model by utilizing a vehicle-mounted road image detection model training set and an aerial image road detection model training set to respectively obtain a vehicle-mounted image road detection model and an aerial image road detection model;
step 3, setting common interest areas of the unmanned vehicle and the unmanned aerial vehicle, and respectively generating corresponding common interest area images in the original vehicle-mounted road image and the aerial photographing road image;
and 4, carrying out road detection and image fusion on the common interest area image to obtain a fused road detection result image.
Preferably, the specific method for making the training set of the vehicle-mounted road image detection model and the training set of the aerial image road detection model comprises the following steps:
segmenting the vehicle-mounted road image and the aerial photographing road image into superpixel units by using linear iterative clustering;
constructing a feature vector by taking a super pixel as a unit;
marking the road super pixel as 1 and the non-road super pixel as 0;
and forming training data of a training set by the feature vectors of the super pixels and the corresponding label information.
Preferably, color and texture feature vectors are constructed by taking the super-pixels as units, and the color feature vectors are obtained from a gray level histogram of the super-pixels; the texture feature vector is obtained by a gray level histogram of the super pixel unit after filtering through a Laplacian operator and a Prewitt operator.
Preferably, the SVM model uses a Gaussian kernel as a kernel function.
Preferably, the specific method for generating the corresponding common interest area images in the original vehicle-mounted road image and the aerial road image respectively is as follows:
determining the range of the road area concerned commonly by the aerial image of the road by adopting a color particle filter algorithm and a direction matching method based on a straight line segment to obtain an image of the common interest area;
and (4) carrying out inverse projection transformation on the vehicle-mounted road image to obtain a common interest area image.
Compared with the prior art, the invention has the following remarkable advantages: the invention utilizes the characteristic of large observation range of the unmanned aerial vehicle, adopts the unmanned aerial vehicle to expand the visual field, and combines the overlooking visual angle information to detect the road, thereby effectively improving the accuracy of the detection of the distant road part.
Drawings
Fig. 1 is a schematic view of an unmanned aerial vehicle and an unmanned vehicle field of view.
FIG. 2 is a flow chart of the present invention.
FIG. 3 is a raw image taken, and FIGS. 3(a) (b) are aerial images; FIG. 3(c) and (d) are vehicle-mounted views.
Fig. 4 is a diagram showing a result of super-pixel division, and fig. 4 (a) to (d) are corresponding diagrams showing a result of super-pixel division in fig. 3(a) to (d).
Fig. 5 is a pose information extraction diagram of a vehicle target in an aerial image, fig. 5(a) is a vehicle target image detected by particle filtering, fig. 5(b) is an edge detection diagram of a target vehicle, and fig. 5(c) is a cumulative probability hough transform result diagram.
Fig. 6 is a common region of interest image of the aerial image, and fig. 6(a) and 6(b) are the common region of interest images of fig. 2(a) and 2(b), respectively.
Fig. 7 is a common interest area map of the on-vehicle image, and fig. 7(a) and 7(b) are the common interest area images of fig. 2(c) and 2(d), respectively.
FIG. 8 is a road detection result image of a common region of interest of the vehicle-mounted image and the aerial image. Fig. 8(a) and 8(b) show road detection results of common interest areas in the aerial images, and fig. 8(c) and 8(d) show road detection results of common interest areas in the on-vehicle images.
Fig. 9 is a fused road detection result image.
Detailed Description
As shown in fig. 1 and 2, a ground unmanned vehicle vision road detection method based on unmanned aerial vehicle images comprises the following specific steps:
step 1, respectively extracting superpixel characteristics from the vehicle-mounted road image and the aerial image, and making a training set, namely making a vehicle-mounted road image detection model training set and an aerial image road detection model training set, wherein the specific method comprises the following steps:
dividing the vehicle-mounted road image and the aerial photographing road image into super pixel units by using linear iterative clustering (SLIC), and establishing a color and texture histogram for each super pixel to serve as a super pixel feature; and marking the road superpixel as 1 and the non-road superpixel as 0 in a manual marking mode. And combining the super-pixel characteristics and corresponding labeling information to form a vehicle-mounted road image detection model training set and an aerial image road detection model training set.
In one embodiment, for each frame of original image, a slic function in python is called to divide the image into 70 superpixel units, and a feature vector is constructed in the unit of superpixel. The feature vector used by the invention comprises a color part and a texture part, and the color feature is obtained by a gray level histogram of the super pixel; the texture features are obtained by the gray level histogram of the super pixel unit after filtering through the Laplacian operator and the Prewitt operator. The class label of each superpixel is determined by the fraction of road pixels therein. The road pixel proportion is greater than 50% of the superpixels, and is marked as 1, otherwise, the road pixel proportion is marked as 0. And forming training data of a training set by the feature vectors of the super pixels and the corresponding label information.
And 2, respectively training Support Vector Machine (SVM) models by using the training sets obtained in the step 1 to respectively obtain a vehicle-mounted image road detection model and an aerial image road detection model, wherein the SVM models use Gaussian kernels as kernel functions.
Step 3, as shown in fig. 3-7, setting common interest areas of the unmanned vehicle and the unmanned aerial vehicle, and respectively generating corresponding common interest area images in the original vehicle-mounted road image and the aerial photographing road image;
specifically, the area with the width of 25m and the length of 25m in front of the head of the unmanned vehicle under the overlooking visual angle is selected as the common interest area. The unmanned aerial vehicle aerial road image estimates vehicle position and attitude information through a particle filter algorithm and a straight line segment matching algorithm to generate a corresponding common interest area image; the vehicle-mounted road image is converted into an overlooking visual angle image through inverse projection transformation, and then a common interest area image is generated.
In a further embodiment, a color particle filter algorithm and a direction matching method based on straight line segments are adopted for the aerial road image to determine the common concerned road area range, so as to obtain a common interest area image, and the specific method is as follows:
the method comprises the following steps of determining the position information of a vehicle in an aerial image through a particle filter algorithm based on a color histogram, wherein the specific steps are as follows:
a) manually marking a vehicle target area of an initial frame, and calculating the histogram color distribution of the area;
b) setting the particle model to S ═ { x, y, vx，vy，Hx，HyA, where x, y are the particle center positions, vx，vyIs the speed of movement of the particles in the x, y directions, Hx，HyRepresenting the width and height of the region in which the particle is located, and a is the corresponding scale factor. For a given pointSet of particle samples S at time t-1t-1The initial weight of the particles is 1/N, and N is the sample set size. The vehicle localization algorithm for processing each frame of image is as described in steps c) -g):
c) for particle set St-1Resampling is performed. At St-1According to the weight
Figure BDA0002783753280000041
Selecting N samples, specifically: calculating a normalized cumulative probability for a weight array
Figure BDA0002783753280000042
Represents a set S of particlest-1The weight value of the ith particle in (c),
Figure BDA0002783753280000043
representing the cumulative probability of the first k particles of the current image. Generating N [0,1]]Random variables which are uniformly distributed in the interval are satisfied to form an array r; for each element r [ i ] in the array r]Searching normalized cumulative probability array ct-1When it is satisfied
Figure BDA0002783753280000044
Then, record j [ i]K. Finally obtaining an array j containing N particle element indexes, wherein all elements in the j meet the requirement
Figure BDA0002783753280000045
Updating a set of particle samples
Figure BDA0002783753280000046
d) According to the system state equation St＝ASt-1+wt-1Calculating a new set S of particlestAnd estimating the position of the new particle. Where a is the likelihood match values for the N particles with respect to the initial state color histogram and w is gaussian noise.
e) For set StIs calculated for each particle and compared to the initial vehicle target area color histogram. Updating estimation to obtain a new weight probability, specifically: for state setsAnd then StCalculates a color histogram distribution as:
Figure BDA0002783753280000047
wherein the normalization factor
Figure BDA0002783753280000048
k is a kernel density function, scale factor
Figure BDA0002783753280000049
Figure BDA00027837532800000410
Delta is the Kronecker trigonometric function, h (X)i) Is XiThe pixel value of (d); bin values of the color histogram. Calculating Bhattacharyya coefficient of each particle color distribution and target model color distribution
Figure BDA00027837532800000411
Rho is a similarity measurement function, and p and q are particles respectively
Figure BDA00027837532800000412
Color distribution of the model and the initial target model, and calculating respective weight values according to the coefficients
Figure BDA00027837532800000413
Where σ is the variance of the gaussian function.
f) According to the weight
Figure BDA0002783753280000051
Estimate set StExtracting the position coordinates as a vehicle target tracking output:
Figure BDA0002783753280000052
wherein the vector E [ S ]t]Containing the position information x, y of the vehicle target in the image at the t moment, and the width and height information H of the current vehicle target detection windowx，Hy。
g) Calculating the color distribution of a target histogram in the detection window obtained in the previous step to be used as the color distribution of the vehicle target of the next frame, and calculating the Bhattacharyya coefficient of the current target color distribution and the previous color distribution to be used as the weight of the next frame
Figure BDA0002783753280000053
The position coordinates of the unmanned vehicle in each frame of image of the image sequence and the size of a detection window on the image, namely the position information of the vehicle are obtained by the algorithm steps a) -g).
Secondly, acquiring the attitude information of the vehicle by utilizing a direction matching method based on a straight line segment for the vehicle target image acquired by the method, wherein the specific method comprises the following steps:
and (5) using a canny operator for the vehicle target image to obtain an edge information image of the vehicle. And calling a HoughLinesP function in OpenCV for the edge image to calculate cumulative probability Hough transform, and extracting all line segments meeting the conditions. And detecting a straight line segment with the minimum straight line length of one half of the image width and the maximum straight line gap of the image width. And calculating the rotation angle of the straight line segment relative to the horizontal or vertical direction to obtain the deflection angle of the target vehicle, namely the attitude information of the vehicle.
And finally, selecting a corresponding area in front of the aerial image vehicle for rotary stretching according to the vehicle position information and the vehicle posture information to obtain a common interest area image in the aerial image.
In a further embodiment, the specific method for obtaining the common interest area image by using the inverse projection transformation to the vehicle-mounted road image comprises the following steps:
and converting the original vehicle-mounted image into a depression image by adopting inverse projection transformation. The transformation from the image coordinate system [ U, V ] to the real world coordinate system [ X, Y, Z ] is as follows:
Figure BDA0002783753280000054
k is a 3 × 3 camera reference matrix, R, T is a camera reference, where R represents a 3 × 3 rotation matrix, T is a 3 × 1 translation vector, and S is a coefficient constant. Assuming that the ground is a Z ═ 0 road, K · [ R | T ] can be reduced from 3 × 4 columns to a matrix of 3 × 3 columns, and let H ═ K · [ R | T ], then:
Figure BDA0002783753280000061
the camera is calibrated to obtain the camera internal and external parameters K, R, T, and the conversion matrix H can be obtained by substituting the camera internal and external parameters into the formula (1), so that the one-to-one correspondence relationship from the point in the image plane to the point in the road surface is obtained. The original image is converted into an inverse projective transformation image by matrix transformation. And selecting a corresponding area in front of the vehicle in the vehicle-mounted road image to carry out rotary stretching to obtain a common interest area image of the vehicle-mounted road image.
Step 4, as shown in fig. 8 and 9, performing road detection on the common interest area image under the double-view angle obtained in step 3, and fusing two road detection result images to obtain a final road detection result image, wherein the specific steps are as follows:
and (3) respectively inputting the common interest area images under the double view angles obtained in the step (3) into the vehicle-mounted image road detection model and the aerial image road detection model in the step (2) to obtain two road prediction binary images. Carrying out weighted average on the two binary images, wherein the fusion strategy is as follows:
R＝α×G+(1-α)×A
g, A are output result images of the vehicle-mounted image road detection model and the aerial image road detection model respectively, wherein alpha is an image fusion weight and is 0.5. And R is the image of the road detection result finally obtained.

Claims (10)

1. The utility model provides an utilize camera that unmanned aerial vehicle carried on to increase road image information of visual angle of looking down on a large scale for unmanned aerial vehicle, its characterized in that includes following step:
step 1, respectively extracting superpixel characteristics of a vehicle-mounted road image and an aerial image, and manufacturing a vehicle-mounted road image detection model training set and an aerial image road detection model training set;
step 2, training an SVM model by utilizing a vehicle-mounted road image detection model training set and an aerial image road detection model training set to respectively obtain a vehicle-mounted image road detection model and an aerial image road detection model;
step 3, setting common interest areas of the unmanned vehicle and the unmanned aerial vehicle, and respectively generating corresponding common interest area images in the original vehicle-mounted road image and the aerial photographing road image;
and 4, carrying out road detection and image fusion on the common interest area image to obtain a fused road detection result image.
2. The method for increasing road image information of a wide-range overlooking visual angle for the unmanned aerial vehicle by using the camera carried by the unmanned aerial vehicle as claimed in claim 1, wherein the specific method for manufacturing the vehicle-mounted road image detection model training set and the aerial image road detection model training set comprises the following steps:
segmenting the vehicle-mounted road image and the aerial photographing road image into superpixel units by using linear iterative clustering;
constructing a feature vector by taking a super pixel as a unit;
marking the road super pixel as 1 and the non-road super pixel as 0;
and forming training data of a training set by the feature vectors of the super pixels and the corresponding label information.
3. The method for increasing road image information of a wide-range overlooking visual angle for the unmanned aerial vehicle by utilizing the camera carried by the unmanned aerial vehicle as claimed in claim 2, wherein color and texture feature vectors are constructed by taking super pixels as units, and the color feature vectors are obtained by a gray level histogram of the super pixels; the texture feature vector is obtained by a gray level histogram of the super pixel unit after filtering through a Laplacian operator and a Prewitt operator.
4. The method of claim 1, wherein the SVM model uses a Gaussian kernel as a kernel function to add road image information for a wide range of overhead views to the drone.
5. The method for increasing road image information of a wide-range overlooking visual angle for the unmanned aerial vehicle by using the camera carried by the unmanned aerial vehicle as claimed in claim 1, wherein the specific method for respectively generating the corresponding common interest area images in the original vehicle-mounted road image and the aerial road image is as follows:
determining the range of the road area concerned commonly by the aerial image of the road by adopting a color particle filter algorithm and a direction matching method based on a straight line segment to obtain an image of the common interest area;
and (4) carrying out inverse projection transformation on the vehicle-mounted road image to obtain a common interest area image.
6. The method for increasing road image information of a wide-range overlooking visual angle for the unmanned aerial vehicle by using the camera carried by the unmanned aerial vehicle according to any one of claims 1 to 5 is characterized in that a specific method for generating a corresponding common interest area image in an aerial photography road image is as follows:
marking a vehicle target area of an initial frame, and calculating the histogram color distribution of the area;
setting the particle model to S ═ x, y, vx,vy,Hx,HyA, where x, y are the particle center positions, vx,vy,Is the speed of movement of the particles in the x, y directions, Hx,HyRepresenting the width and height of the region in which the particle is located, a being the corresponding scale factor; set of particle samples S for a given time t-1t-1The initial weight of the particles is 1/N, and N is the size of the sample set;
for particle set St-1Resampling is carried out;
according to the system state equation St＝ASt-1+wt-1Calculating a new set S of particlestEstimating the position of a new particle, wherein A is the likelihood matching value of N particles with respect to the initial state color histogram, and w is Gaussian noise;
for set StCalculating a color histogram for each particle, andcomparing the initial color histograms of the target regions of the vehicle, updating the estimates, and obtaining new weight probabilities
Figure FDA0002783753270000021
Estimating set S according to new weight probabilitytExtracting the position coordinates as a vehicle target tracking output:
Figure FDA0002783753270000022
wherein the vector E [ S ]t]Containing the position information x, y of the vehicle target in the image at the t moment, and the width and height information H of the current vehicle target detection windowx,Hy；
Calculating the color distribution of a target histogram in the detection window as the color distribution of the vehicle target of the next frame, and calculating the Bhattacharyya coefficient of the current target color distribution and the previous color distribution as the weight of the next frame
Figure FDA0002783753270000023
7. The method of claim 6, wherein the set of particles is St-1The specific method for resampling comprises the following steps:
calculating a normalized cumulative probability for a weight array
Figure FDA0002783753270000024
Figure FDA0002783753270000025
Represents a set S of particlest-1The weight value of the ith particle in (c),
Figure FDA0002783753270000026
representing the cumulative probability of the first k particles of the current image;
generating N random variables which meet the requirement of uniform distribution in the [0,1] interval to form an array r;
for each element r [ i ] in the array r]Searching normalized cumulative probability array ct-1When it is satisfied
Figure FDA0002783753270000027
Figure FDA0002783753270000028
Then, record j [ i]K, get an array j containing N particle element indices, all elements in j satisfying
Figure FDA0002783753270000031
Updating a set of particle samples
Figure FDA0002783753270000032
8. The method for adding road image information with a wide range of overhead viewing angles to the unmanned aerial vehicle by using the camera carried by the unmanned aerial vehicle according to claim 6 is characterized in that the specific method for obtaining the new weight probability comprises the following steps: for state set StCalculating a color histogram distribution for each particle in (1)
Figure FDA0002783753270000033
Figure FDA0002783753270000034
Wherein the normalization factor
Figure FDA0002783753270000035
y is the particle center value, k is the kernel density function, scale factor
Figure FDA0002783753270000036
Delta is the Kronecker trigonometric function, h (X)i) Is XiM is the bin value of the color histogram;
calculate eachBhattacharyya coefficients of particle color distribution and target model color distribution
Figure FDA0002783753270000037
Rho is a similarity measurement function, and p and q are particles respectively
Figure FDA0002783753270000038
Color distribution of the model and the initial target model, and calculating respective weight values according to the coefficients
Figure FDA0002783753270000039
Where σ is the variance of the gaussian function.
9. The method for increasing the road image information of the wide-range overlooking visual angle for the unmanned aerial vehicle by using the camera carried by the unmanned aerial vehicle according to the claim 1 is characterized in that the specific method for acquiring the common interest area image by adopting the inverse projection transformation to the vehicle-mounted road image comprises the following steps:
the original vehicle-mounted image is converted into a depression image by adopting inverse projection transformation, and the transformation relation from an image coordinate system [ U, V ] to an actual world coordinate system [ X, Y, Z ] is as follows:
Figure FDA00027837532700000310
k is a camera internal reference matrix, and R, T is a camera external reference;
converting the original image into an inverse projection conversion image through matrix conversion; and selecting a corresponding area in front of the vehicle in the vehicle-mounted road image to carry out rotary stretching to obtain a common interest area image of the vehicle-mounted road image.
10. The method for increasing road image information of a wide-range overlooking visual angle for the unmanned aerial vehicle by using the camera carried by the unmanned aerial vehicle according to claim 1 is characterized in that road detection and image fusion are carried out on the common interest area image, and a specific method for obtaining a fused road detection result image comprises the following steps:
respectively inputting the common interest area images under the double view angles into the vehicle-mounted image road detection model and the aerial image road detection model in the step 2 to obtain two road prediction binary images;
carrying out weighted average on the two binary images, wherein the fusion strategy is as follows:
R＝α×G+(1-α)×A
g, A are output result images of the vehicle-mounted image road detection model and the aerial image road detection model respectively, alpha is an image fusion weight, alpha is 0.5, and R is a road detection result image finally obtained.
CN202011290845.0A 2020-11-17 2020-11-17 Ground unmanned vehicle vision road detection method based on unmanned aerial vehicle image Active CN112395985B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011290845.0A CN112395985B (en) 2020-11-17 2020-11-17 Ground unmanned vehicle vision road detection method based on unmanned aerial vehicle image

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011290845.0A CN112395985B (en) 2020-11-17 2020-11-17 Ground unmanned vehicle vision road detection method based on unmanned aerial vehicle image

Publications (2)

Publication Number Publication Date
CN112395985A true CN112395985A (en) 2021-02-23
CN112395985B CN112395985B (en) 2022-10-21

Family

ID=74606449

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011290845.0A Active CN112395985B (en) 2020-11-17 2020-11-17 Ground unmanned vehicle vision road detection method based on unmanned aerial vehicle image

Country Status (1)

Country Link
CN (1) CN112395985B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113361508A (en) * 2021-08-11 2021-09-07 四川省人工智能研究院(宜宾) Cross-view-angle geographic positioning method based on unmanned aerial vehicle-satellite

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140126822A1 (en) * 2011-03-11 2014-05-08 The University Of Sydney Image Processing
CN108961790A (en) * 2018-07-24 2018-12-07 河北德冠隆电子科技有限公司 Bad weather pre-warning management system and method based on four-dimensional outdoor scene traffic simulation
CN109215053A (en) * 2018-10-16 2019-01-15 西安建筑科技大学 Moving vehicle detection method containing halted state in a kind of unmanned plane video

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140126822A1 (en) * 2011-03-11 2014-05-08 The University Of Sydney Image Processing
CN108961790A (en) * 2018-07-24 2018-12-07 河北德冠隆电子科技有限公司 Bad weather pre-warning management system and method based on four-dimensional outdoor scene traffic simulation
CN109215053A (en) * 2018-10-16 2019-01-15 西安建筑科技大学 Moving vehicle detection method containing halted state in a kind of unmanned plane video

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113361508A (en) * 2021-08-11 2021-09-07 四川省人工智能研究院(宜宾) Cross-view-angle geographic positioning method based on unmanned aerial vehicle-satellite

Also Published As

Publication number Publication date
CN112395985B (en) 2022-10-21

Similar Documents

Publication Publication Date Title
CN110569704B (en) Multi-strategy self-adaptive lane line detection method based on stereoscopic vision
CN111563415B (en) Binocular vision-based three-dimensional target detection system and method
CN107767400B (en) Remote sensing image sequence moving target detection method based on hierarchical significance analysis
US8212812B2 (en) Active shape model for vehicle modeling and re-identification
CN112801074B (en) Depth map estimation method based on traffic camera
CN108171715B (en) Image segmentation method and device
CN108597009B (en) Method for detecting three-dimensional target based on direction angle information
CN108446634B (en) Aircraft continuous tracking method based on combination of video analysis and positioning information
CN109961506A (en) A kind of fusion improves the local scene three-dimensional reconstruction method of Census figure
CN106709472A (en) Video target detecting and tracking method based on optical flow features
CN108229475B (en) Vehicle tracking method, system, computer device and readable storage medium
CN111862201A (en) Deep learning-based spatial non-cooperative target relative pose estimation method
CN110021029B (en) Real-time dynamic registration method and storage medium suitable for RGBD-SLAM
CN111046843A (en) Monocular distance measurement method under intelligent driving environment
CN113781562A (en) Lane line virtual and real registration and self-vehicle positioning method based on road model
CN114120283A (en) Method for distinguishing unknown obstacles in road scene three-dimensional semantic segmentation
CN113689331A (en) Panoramic image splicing method under complex background
CN116978009A (en) Dynamic object filtering method based on 4D millimeter wave radar
CN106709432B (en) Human head detection counting method based on binocular stereo vision
CN112395985B (en) Ground unmanned vehicle vision road detection method based on unmanned aerial vehicle image
CN107944350B (en) Monocular vision road identification method based on appearance and geometric information fusion
CN113345084B (en) Three-dimensional modeling system and three-dimensional modeling method
CN113221739B (en) Monocular vision-based vehicle distance measuring method
CN111127613B (en) Image sequence three-dimensional reconstruction method and system based on scanning electron microscope
Chun-Zhao et al. Drivable road boundary detection for intelligent vehicles based on stereovision with plane-induced homography

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant