CN118131208A - Automobile safety lane changing method based on millimeter wave radar and vision fusion - Google Patents
Automobile safety lane changing method based on millimeter wave radar and vision fusion Download PDFInfo
- Publication number
- CN118131208A CN118131208A CN202410458224.0A CN202410458224A CN118131208A CN 118131208 A CN118131208 A CN 118131208A CN 202410458224 A CN202410458224 A CN 202410458224A CN 118131208 A CN118131208 A CN 118131208A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- rear vehicle
- visual
- millimeter wave
- imaging
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000004927 fusion Effects 0.000 title claims abstract description 52
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000003384 imaging method Methods 0.000 claims abstract description 77
- 230000000007 visual effect Effects 0.000 claims abstract description 67
- 238000012545 processing Methods 0.000 claims abstract description 11
- 238000005259 measurement Methods 0.000 claims description 13
- 238000001514 detection method Methods 0.000 claims description 10
- 239000011159 matrix material Substances 0.000 claims description 10
- 230000009466 transformation Effects 0.000 claims description 7
- RZVAJINKPMORJF-UHFFFAOYSA-N Acetaminophen Chemical compound CC(=O)NC1=CC=C(O)C=C1 RZVAJINKPMORJF-UHFFFAOYSA-N 0.000 claims description 6
- 208000029618 autoimmune pulmonary alveolar proteinosis Diseases 0.000 claims description 6
- 238000000354 decomposition reaction Methods 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000012216 screening Methods 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 abstract description 3
- 230000000694 effects Effects 0.000 description 2
- 206010039203 Road traffic accident Diseases 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000000205 computational method Methods 0.000 description 1
- 230000006698 induction Effects 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/02—Systems using reflection of radio waves, e.g. primary radar systems; Analogous systems
- G01S13/06—Systems determining position data of a target
- G01S13/08—Systems for measuring distance only
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/02—Systems using reflection of radio waves, e.g. primary radar systems; Analogous systems
- G01S13/50—Systems of measurement based on relative movement of target
- G01S13/58—Velocity or trajectory determination systems; Sense-of-movement determination systems
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/86—Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/88—Radar or analogous systems specially adapted for specific applications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S15/00—Systems using the reflection or reradiation of acoustic waves, e.g. sonar systems
- G01S15/88—Sonar systems specially adapted for specific applications
- G01S15/89—Sonar systems specially adapted for specific applications for mapping or imaging
Landscapes
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- General Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Electromagnetism (AREA)
- Radar Systems Or Details Thereof (AREA)
Abstract
The invention discloses an automobile safety lane changing method based on millimeter wave radar and vision fusion, which comprises the following steps: step 1: measuring the target distance D and the speed v of an incoming vehicle behind the vehicle; step 2: respectively performing ultrasonic imaging and visual imaging through an ultrasonic sensor and a camera sensor; step 3: performing visual fusion on the ultrasonic imaging and the visual imaging obtained in the step 2 to obtain a visual fusion view of a rear vehicle and fused space coordinate data of the rear vehicle; step 4: and (3) analyzing and processing the vehicle speed v and the distance data D obtained by the millimeter wave radar in the step (1) and the spatial coordinate data obtained by the step (3) after the rear vehicle is fused, calculating the time of the rear vehicle reaching one side of the vehicle, and reminding the vehicle to safely change lanes. The method ensures that the lane changing process of the vehicle in the driving process is safer and more stable by the cooperation of the millimeter wave radar and the vision fusion imaging technology.
Description
Technical Field
The invention belongs to the technical field of automobile safety, and particularly relates to an automobile safety lane changing method based on millimeter wave radar and vision fusion.
Background
Advanced Driving Assistance Systems (ADAS) have developed a system that supports driver safe and comfortable driving, providing a variety of traffic convenience systems, such as Front Collision Warning (FCW), lane Keeping Assistance (LKA), and intelligent cruise control (SCC). These driving assistance systems operate on the basis of various vehicle sensors that identify and monitor the surrounding environment, collect data required for analysis, and in order to improve detection accuracy, various assistance systems are often combined, which can be regarded as an intermediate step towards fully autonomous driving. The following problems exist in the prior art:
Because the existing automobile millimeter wave radar has a limited induction range, the existence of other vehicles can be induced only when the distance is relatively short, and thus the problem that the automobile with the millimeter wave radar cannot give an alarm in time once the speed of the rear vehicle is relatively high or the speed of the automobile is relatively high during lane change early warning, and traffic accidents can be caused after lane change.
Disclosure of Invention
The invention aims to provide an automobile safety lane changing method based on millimeter wave radar and vision fusion, which is used for judging the speed distance of a rear coming automobile more accurately through the cooperation of the millimeter wave radar and the vision fusion imaging technology, so that reasonable lane changing time is given to an own automobile, and the lane changing process of the driving automobile is safer and more stable.
The technical scheme adopted by the invention is that the automobile safety lane changing method based on millimeter wave radar and vision fusion comprises the following steps:
step 1: the method comprises the steps of transmitting electromagnetic waves through a millimeter wave radar, receiving echo signals reflected from a target, and measuring the target distance D and the speed v of an incoming vehicle behind the vehicle;
Step 2: respectively performing ultrasonic imaging and visual imaging through an ultrasonic sensor and a camera sensor;
step 3: performing visual fusion on the ultrasonic imaging and the visual imaging obtained in the step 2 to obtain a visual fusion view of a rear vehicle and fused space coordinate data of the rear vehicle;
Step 4: and (3) analyzing and processing the vehicle speed v and the distance data D obtained by the millimeter wave radar in the step (1) and the spatial coordinate data obtained by the step (3) after the rear vehicle is fused, calculating the time of the rear vehicle reaching one side of the vehicle, and reminding the vehicle to safely change lanes.
The invention is also characterized in that:
In the step (1) of the process,
The target distance D is obtained through a millimeter wave radar ranging formula, and the specific formula is as follows:
Wherein f t is the scanning signal frequency, f r is the echo signal frequency, f DEV is the scanning frequency bandwidth of the frequency-modulated wave, t s is half of the period of the frequency-modulated wave, the distance between the detected target and the radar is D, and the movement speed of light is c;
The speed v is obtained through millimeter wave radar speed measurement, and the specific formula is as follows:
Wherein f b is the frequency difference between the scanning signal and the echo signal when the detected target is stationary, v is the moving speed of the target relative to the radar, f is the center frequency of the radar scanning signal, and the moving speed of light is c.
The step 2 is specifically as follows:
Step 2.1: the ultrasonic sensor is arranged below the tail part of the automobile, ultrasonic imaging is adopted to acquire an image of a target object by utilizing the propagation characteristic of ultrasonic waves, and in the detection of a vehicle behind the automobile, the ultrasonic sensor transmits ultrasonic waves backwards, when the ultrasonic waves meet the vehicle behind, part of the ultrasonic waves are reflected back by the surface of the vehicle, a receiver in the ultrasonic sensor receives the reflected signals, the intensity of the reflected ultrasonic waves depends on the size and shape of the vehicle behind, and the ultrasonic imaging of the vehicle behind is reconstructed according to the intensity, time delay and direction of the signals;
Step 2.2: a three-eye camera sensor is selected and installed at the central position of the tail of the automobile to acquire the visual imaging of the rear vehicle.
The step 3 is specifically as follows:
step 3.1: by extracting the sift characteristic points of the ultrasonic imaging and visual imaging pictures, the RANSAC algorithm is used for screening characteristic point pairs, error points are eliminated, and the screened characteristic points basically correspond to each other one by one.
Step 3.2: using DLT algorithm to estimate perspective transformation matrix of the screened characteristic points, and in order to improve registration accuracy, APAP algorithm cuts the ultrasonic imaging and visual imaging picture into innumerable small squares, homography matrix transformation is carried out on each small square, homography matrix change is carried out on each small square, so as to obtain denoised ultrasonic imaging and visual imaging picture;
Step 3.3: directly carrying out Laplacian pyramid decomposition on the denoised ultrasonic imaging and visual imaging pictures obtained in the step 3.2 by adopting a multi-band bleing algorithm, decomposing an original image into sub-images with different scales by differentiating Gaussian pyramids of two adjacent layers, carrying out weighted average on each sub-image to obtain a fusion result of each layer, and finally carrying out reverse reconstruction of pyramids to obtain a visual fusion view of a rear vehicle; and taking the visual fusion view of the rear vehicle as the input of the SSD rapid target detection algorithm to obtain the fused space coordinate data of the rear vehicle.
The step 4 is specifically as follows:
step 4.1: the time t for the rear vehicle to reach the side of the own vehicle is calculated from the vehicle speed v and the distance data D obtained in step 1, and the calculation formula is as follows:
Wherein t is the time when the rear vehicle reaches one side of the own vehicle, the distance between the detected target and the radar, namely the distance between the rear vehicle and the own vehicle at the initial moment is D, v b is the speed of the rear vehicle, and v a is the speed of the own vehicle;
Step 4.2: when the vehicle needs to change the lane, referring to the visual fusion view of the rear vehicle and the spatial coordinate data of the rear vehicle fused obtained in the step 3, and the time of the rear vehicle reaching one side of the vehicle calculated in the step 4.1, lane changing can be performed within a reasonable time range of 10 to 20 seconds; the visual fusion view of the rear vehicle obtained in the step 3 can bring a specific number and position reference of the rear vehicle on a visual level to a vehicle owner, and specifically comprises the following steps: according to the space coordinate data, when the distance between the rear vehicle and the own vehicle is 30 meters or more, the lane change can be performed safely; when the distance between the rear vehicle and the own vehicle is 10-20 meters, the speed is required to be increased and the lane is changed; when the following vehicle is less than 10 meters away from the own vehicle, stopping lane change is recommended.
The beneficial effects of the invention are as follows:
According to the automobile safety lane changing method based on millimeter wave radar and vision fusion, the speed distance of a rear coming automobile is accurately judged through the cooperation of the millimeter wave radar and the vision fusion imaging technology, so that reasonable lane changing time is given to an own automobile, and the lane changing process of a driving automobile is safer and more stable.
Drawings
FIG. 1 is a schematic flow chart of the method of the present invention.
Detailed Description
The invention will be described in detail below with reference to the drawings and the detailed description.
The invention provides an automobile safety lane changing method based on millimeter wave radar and vision fusion, as shown in fig. 1, comprising the following steps:
step 1: the method comprises the steps of transmitting electromagnetic waves through a millimeter wave radar, receiving echo signals reflected from a target, and measuring the target distance D and the speed v of an incoming vehicle behind the vehicle;
In the step 1, millimeter wave radars of 77GHz are selected and installed at the left side and the right side of the tail of the automobile. Electromagnetic waves with specific modulation frequencies are generated by utilizing the high-frequency circuit, the electromagnetic waves are transmitted and received by the antenna, and the distance measurement, the speed measurement and the azimuth measurement can be simultaneously carried out on a plurality of targets of a plurality of lanes behind the vehicle. The millimeter wave radar can be matched with the oscillator to generate a signal with gradually increased frequency, the signal can rebound after encountering a rear vehicle obstacle, the returned waveform and the emitted waveform have a frequency difference, the frequency difference and the time delay are in a linear relation, the longer the object is, the later the time for returning the waveform is received, the larger the frequency difference with the incident wave is, the difference frequency of the two frequencies can be obtained by subtracting the two frequencies, and the distance of the obstacle can be judged by judging the difference frequency; the speed measurement is to calculate the frequency change of the radar wave returned to the receiving antenna according to the Doppler effect, so as to obtain the moving speed and time of the target relative to the radar, and obtain the target distance.
The target distance D is obtained through a millimeter wave radar ranging formula, and the specific formula is as follows:
Wherein f t is the scanning signal frequency, f r is the echo signal frequency, f DEV is the scanning frequency bandwidth of the frequency-modulated wave, t s is half of the period of the frequency-modulated wave, the distance between the detected target and the radar is D, and the movement speed of light is c;
The speed v is obtained through millimeter wave radar speed measurement, and the specific formula is as follows:
Wherein f b is the frequency difference between the scanning signal and the echo signal when the detected target is stationary, v is the moving speed of the target relative to the radar, f is the center frequency of the radar scanning signal, and the moving speed of light is c.
Step 2: respectively performing ultrasonic imaging and visual imaging through an ultrasonic sensor and a camera sensor;
Step 2.1: the ultrasonic sensor is installed below the tail part of the automobile, ultrasonic imaging is adopted to acquire an image of a target object by utilizing the propagation characteristic of ultrasonic waves, and in the detection of a vehicle behind the automobile, the ultrasonic sensor emits ultrasonic waves backwards, and when the ultrasonic waves meet the vehicle behind, part of the ultrasonic waves are reflected by the surface of the vehicle. The receiver inside the ultrasonic sensor receives the reflected signals, the intensity of the reflected sound wave depends on the size and shape of the rear vehicle, and the ultrasonic imaging of the rear vehicle is reconstructed according to the intensity, time delay and direction of the signals;
Step 2.2: a three-eye camera sensor is selected and installed at the central position of the tail part of the automobile. Visual imaging is when light is focused on a camera sensor, each pixel receives light of a certain intensity and generates a corresponding charge. These charges are converted into digital signals according to the operating principle of the image sensor. The digital signals are amplified, filtered and digitized by a signal processing circuit in the camera sensor so as to obtain clearer visual imaging of the rear vehicle.
Step 3: performing visual fusion on the ultrasonic imaging and the visual imaging obtained in the step 2 to obtain a visual fusion view of a rear vehicle and fused space coordinate data of the rear vehicle;
step 3.1: by extracting the sift characteristic points of the ultrasonic imaging and visual imaging pictures, the RANSAC algorithm is used for screening characteristic point pairs, error points are eliminated, and the screened characteristic points basically correspond to each other one by one.
Step 3.2: using DLT algorithm to estimate perspective transformation matrix of the screened characteristic points, and in order to improve registration accuracy, APAP algorithm cuts the ultrasonic imaging and visual imaging picture into innumerable small squares, homography matrix transformation is carried out on each small square, homography matrix change is carried out on each small square, so as to obtain denoised ultrasonic imaging and visual imaging picture;
Step 3.3: directly carrying out Laplacian pyramid decomposition on the denoised ultrasonic imaging and visual imaging pictures obtained in the step 3.2 by adopting a multi-band bleing algorithm, decomposing an original image into sub-images with different scales by differentiating Gaussian pyramids of two adjacent layers, carrying out weighted average on each sub-image to obtain a fusion result of each layer, and finally carrying out reverse reconstruction of pyramids to obtain a visual fusion view of a rear vehicle; and taking the visual fusion view of the rear vehicle as the input of the SSD rapid target detection algorithm to obtain the fused space coordinate data of the rear vehicle.
Step 4: and (3) analyzing and processing the vehicle speed v and the distance data D obtained by the millimeter wave radar in the step (1) and the spatial coordinate data obtained by the step (3) after the rear vehicle is fused, calculating the time of the rear vehicle reaching one side of the vehicle, and reminding the vehicle to safely change lanes.
Step 4.1: the time t for the rear vehicle to reach the side of the own vehicle is calculated from the vehicle speed v and the distance data D obtained in step 1, and the calculation formula is as follows:
Wherein t is the time when the rear vehicle reaches one side of the own vehicle, the distance between the detected target and the radar, namely the distance between the rear vehicle and the own vehicle at the initial moment is D, v b is the speed of the rear vehicle, and v a is the speed of the own vehicle;
Step 4.2: when the vehicle needs to change the lane, the lane can be changed within a reasonable time range between 10 and 20 seconds by referring to the visual fusion view of the rear vehicle and the spatial coordinate data of the rear vehicle after fusion, which are obtained in the step 3, and the time of the rear vehicle reaching one side of the vehicle, which are calculated in the step 4.1. The visual fusion view of the rear vehicle obtained in the step 3 can bring a specific number and position reference of the rear vehicle on a visual level to the vehicle owner, and the fused space coordinate data of the rear vehicle can provide a distance reference of the rear vehicle on the data level to the vehicle owner. Generally, according to the space coordinate data, when the following vehicle is 30 meters or more from the own vehicle, lane change can be performed safely; when the distance between the rear vehicle and the own vehicle is 10-20 meters, the speed is required to be increased and the lane is changed; when the following vehicle is less than 10 meters away from the own vehicle, stopping lane change is recommended.
The lane change of the automobile is a link which is easy to cause accidents in the driving process, and particularly in complex road conditions, the speed and the distance of the coming automobile at the rear are required to be considered so as to avoid collision or traffic jam. In the conventional driving mode, the driver needs to determine when to change the lane by means of own observation and judgment, but the human driver has the problems of blind areas of vision and judgment errors, which are easy to cause accidents. Millimeter wave radar is mainly used for measuring the distance and speed of a target vehicle, but the single use of a millimeter wave radar sensor can lead to insufficient acquisition of information such as the shape and color of the target vehicle, and accurate judgment is difficult to make. The camera sensor can provide rich target vehicle information, but is easily affected in bad weather or at night when the light is insufficient, and the recognition accuracy is reduced. Ultrasonic sensors have a certain accuracy in a short distance, but in the case of moving an object in a long distance and at a high speed, accuracy and reliability are limited.
The method comprehensively utilizes the advantages of the millimeter wave radar, the ultrasonic sensor and the camera sensor, and can acquire the detailed information such as the distance, the speed and the shape of the rear vehicle. The millimeter wave radar of 77GHz is selected, so that the target distance and speed of the rear vehicle can be obtained under various complex weather and road conditions, and accurate data support is provided for finally calculating the time of the rear vehicle reaching one side of the self vehicle. The ultrasonic sensor with the waterproof module is used for ultrasonic imaging, an ultrasonic imaging picture of a rear vehicle with certain precision can be provided in rainy days, and the picture can accurately show information such as the shape and the position of the rear vehicle. The three-eye camera sensor is selected to obtain a clearer visual imaging picture. The ultrasonic imaging and the visual imaging are subjected to visual fusion, and a visual fusion view of the rear vehicle is obtained through RANSAC, DLT, APAP, multi-band bleing and other algorithm processing, so that a visual blind area does not exist in the view, and more accurate information such as the shape and the position of the rear vehicle is provided. The method also adopts an SSD rapid target detection algorithm to acquire the space coordinate data of the rear vehicle, and the algorithm simultaneously predicts the category and the bounding box of the target in a single convolution network, realizes real-time target detection and outputs the space coordinate data of the rear vehicle. The algorithm used in the method has good accuracy and stability, and provides accurate data information for automobile lane changing. Based on the accurate data, more accurate lane change advice or control can be provided, and risks possibly brought by blind areas of the visual field of a driver and judgment errors are avoided, so that safer and more stable lane change of the automobile is realized.
In summary, the method of the invention obtains the information of the rear vehicle through the millimeter wave radar, the ultrasonic wave and the camera sensor, and adopts the efficient fusion algorithm, thereby realizing the omnibearing and accurate identification of the rear coming vehicle, and providing safer and more reliable technical support for the lane change of the automobile.
Example 1
As shown in fig. 1, the invention provides an automobile safety lane changing method based on millimeter wave radar and vision fusion, which comprises the following steps:
step 1: the method comprises the steps of transmitting electromagnetic waves through a millimeter wave radar, receiving echo signals reflected from a target, and measuring the target distance D and the speed v of an incoming vehicle behind the vehicle;
Step 2: respectively performing ultrasonic imaging and visual imaging through an ultrasonic sensor and a camera sensor;
step 3: performing visual fusion on the ultrasonic imaging and the visual imaging obtained in the step 2 to obtain a visual fusion view of a rear vehicle and fused space coordinate data of the rear vehicle;
Step 4: and (3) analyzing and processing the vehicle speed v and the distance data D obtained by the millimeter wave radar in the step (1) and the spatial coordinate data obtained by the step (3) after the rear vehicle is fused, calculating the time of the rear vehicle reaching one side of the vehicle, and reminding the vehicle to safely change lanes.
In the step 1, electromagnetic waves with specific modulation frequency are generated by utilizing a high-frequency circuit, and the electromagnetic waves are sent and received by an antenna, so that the distance measurement, the speed measurement and the azimuth measurement can be carried out on a plurality of targets on a plurality of lanes behind a vehicle at the same time; the speed measurement is to calculate the frequency change of radar waves returned to a receiving antenna according to the Doppler effect, so that the movement speed and the flight time of a target relative to the radar can be obtained, and the distance of the target is obtained, wherein in the step 2, the ultrasonic imaging is based on the reflection principle of the sound waves, when the sound waves strike a rear vehicle, the sound waves are reflected back, the intensity of the reflected sound waves depends on the size and the shape of the rear vehicle, in the step 3, the ultrasonic imaging and the visual imaging are visually fused through RANCAS, DLT, APAP and a multi-band bleing algorithm, so that the accuracy and the visualization capability of the rear vehicle imaging are enhanced, in the step 4, the vehicle speed v and the distance data D obtained through the millimeter wave radar in the step 1 and the fused space coordinate data of the rear vehicle obtained in the step 3 are analyzed and processed, and the time of the rear vehicle reaching one side of the vehicle is calculated, so that the vehicle is reminded of safety lane change.
Example 2
The automobile safety lane change method based on millimeter wave radar and vision fusion, as shown in fig. 1, comprises the following steps:
step 1: the method comprises the steps of transmitting electromagnetic waves through a millimeter wave radar, receiving echo signals reflected from a target, and measuring the target distance D and the speed v of an incoming vehicle behind the vehicle;
In the step (1) of the process,
The target distance D is obtained through a millimeter wave radar ranging formula, and the specific formula is as follows:
Wherein f t is the scanning signal frequency, f r is the echo signal frequency, f DEV is the scanning frequency bandwidth of the frequency-modulated wave, t s is half of the period of the frequency-modulated wave, the distance between the detected target and the radar is D, and the movement speed of light is c;
The speed v is obtained through millimeter wave radar speed measurement, and the specific formula is as follows:
Wherein f b is the frequency difference between the scanning signal and the echo signal when the detected target is stationary, v is the moving speed of the target relative to the radar, f is the center frequency of the radar scanning signal, and the moving speed of light is c.
Step 2: respectively performing ultrasonic imaging and visual imaging through an ultrasonic sensor and a camera sensor;
step 3: performing visual fusion on the ultrasonic imaging and the visual imaging obtained in the step 2 to obtain a visual fusion view of a rear vehicle and fused space coordinate data of the rear vehicle;
Step 4: and (3) analyzing and processing the vehicle speed v and the distance data D obtained by the millimeter wave radar in the step (1) and the spatial coordinate data obtained by the step (3) after the rear vehicle is fused, calculating the time of the rear vehicle reaching one side of the vehicle, and reminding the vehicle to safely change lanes.
In the first step, the millimeter wave radar is matched with the oscillator to generate a signal with gradually increased frequency, the signal can rebound after encountering a rear vehicle obstacle, the returned waveform and the emitted waveform have a frequency difference, the frequency difference and the time delay are in a linear relation, the longer the object is, the later the time the returned waveform is received, the larger the frequency difference with the incident wave is, the difference frequency of the two frequencies can be obtained by subtracting the two frequencies, and the distance of the obstacle can be judged by judging the difference frequency.
In this embodiment, millimeter wave radar compares the millimeter wave radar and has small, easy integrated and the high characteristics of spatial resolution, and the size of the required system component of processing millimeter wave signal is also less, possesses the high accuracy simultaneously, and millimeter wave system around 77GHz will be able to detect down to the removal of several tenths of a millimeter to accurate judgement rear car distance.
Example 3
The automobile safety lane change method based on millimeter wave radar and vision fusion, as shown in fig. 1, comprises the following steps:
step 1: the method comprises the steps of transmitting electromagnetic waves through a millimeter wave radar, receiving echo signals reflected from a target, and measuring the target distance D and the speed v of an incoming vehicle behind the vehicle;
In the step (1) of the process,
The target distance D is obtained through a millimeter wave radar ranging formula, and the specific formula is as follows:
Wherein f t is the scanning signal frequency, f r is the echo signal frequency, f DEV is the scanning frequency bandwidth of the frequency-modulated wave, t s is half of the period of the frequency-modulated wave, the distance between the detected target and the radar is D, and the movement speed of light is c;
The speed v is obtained through millimeter wave radar speed measurement, and the specific formula is as follows:
Wherein f b is the frequency difference between the scanning signal and the echo signal when the detected target is stationary, v is the moving speed of the target relative to the radar, f is the center frequency of the radar scanning signal, and the moving speed of light is c.
Step 2: respectively performing ultrasonic imaging and visual imaging through an ultrasonic sensor and a camera sensor;
the step 2 is specifically as follows:
Step 2.1: the ultrasonic sensor is arranged below the tail part of the automobile, ultrasonic imaging is adopted to acquire an image of a target object by utilizing the propagation characteristic of ultrasonic waves, and in the detection of a vehicle behind the automobile, the ultrasonic sensor transmits ultrasonic waves backwards, when the ultrasonic waves meet the vehicle behind, part of the ultrasonic waves are reflected back by the surface of the vehicle, a receiver in the ultrasonic sensor receives the reflected signals, the intensity of the reflected ultrasonic waves depends on the size and shape of the vehicle behind, and the ultrasonic imaging of the vehicle behind is reconstructed according to the intensity, time delay and direction of the signals;
Step 2.2: a three-eye camera sensor is selected and installed at the central position of the tail of the automobile to acquire the visual imaging of the rear vehicle.
Step 3: performing visual fusion on the ultrasonic imaging and the visual imaging obtained in the step 2 to obtain a visual fusion view of a rear vehicle and fused space coordinate data of the rear vehicle;
Step 4: and (3) analyzing and processing the vehicle speed v and the distance data D obtained by the millimeter wave radar in the step (1) and the spatial coordinate data obtained by the step (3) after the rear vehicle is fused, calculating the time of the rear vehicle reaching one side of the vehicle, and reminding the vehicle to safely change lanes.
In this embodiment, the RANSAC algorithm is a computational method for processing a sample set containing outlier data, aimed at finding valid mathematical model parameters from these data, by randomly sampling the acquired view sample set and using these samples to estimate the parameters of the view model. The DLT algorithm adopts direct linear transformation, and solves a camera projection matrix according to three-dimensional world coordinate points and corresponding two-dimensional image coordinate points. The main function of the APAP algorithm is the self-adaptive parameter adjustment of the projection algorithm, and the quality and convergence rate of image reconstruction are improved. Then the laplacian pyramid decomposition is directly carried out on the two pictures with the splice, and the second half is fused with the first half. The multi-band bleing algorithm is to directly apply laplacian pyramid decomposition to two pictures with stitching, the latter half being fused to the former half.
Claims (5)
1. The automobile safety lane changing method based on millimeter wave radar and vision fusion is characterized by comprising the following steps of:
step 1: the method comprises the steps of transmitting electromagnetic waves through a millimeter wave radar, receiving echo signals reflected from a target, and measuring the target distance D and the speed v of an incoming vehicle behind the vehicle;
Step 2: respectively performing ultrasonic imaging and visual imaging through an ultrasonic sensor and a camera sensor;
step 3: performing visual fusion on the ultrasonic imaging and the visual imaging obtained in the step 2 to obtain a visual fusion view of a rear vehicle and fused space coordinate data of the rear vehicle;
Step 4: and (3) analyzing and processing the vehicle speed v and the distance data D obtained by the millimeter wave radar in the step (1) and the spatial coordinate data obtained by the step (3) after the rear vehicle is fused, calculating the time of the rear vehicle reaching one side of the vehicle, and reminding the vehicle to safely change lanes.
2. The method for safely switching lanes of an automobile based on millimeter wave radar and vision fusion according to claim 1, wherein in step 1,
The target distance D is obtained through a millimeter wave radar ranging formula, and the specific formula is as follows:
Wherein f t is the scanning signal frequency, f r is the echo signal frequency, f DEV is the scanning frequency bandwidth of the frequency-modulated wave, t s is half of the period of the frequency-modulated wave, the distance between the detected target and the radar is D, and the movement speed of light is c;
The speed v is obtained through millimeter wave radar speed measurement, and the specific formula is as follows:
Wherein f b is the frequency difference between the scanning signal and the echo signal when the detected target is stationary, v is the moving speed of the target relative to the radar, f is the center frequency of the radar scanning signal, and the moving speed of light is c.
3. The car security lane changing method based on millimeter wave radar and vision fusion according to claim 2, wherein step 2 is specifically as follows:
Step 2.1: the ultrasonic sensor is arranged below the tail part of the automobile, ultrasonic imaging is adopted to acquire an image of a target object by utilizing the propagation characteristic of ultrasonic waves, and in the detection of a vehicle behind the automobile, the ultrasonic sensor transmits ultrasonic waves backwards, when the ultrasonic waves meet the vehicle behind, part of the ultrasonic waves are reflected back by the surface of the vehicle, a receiver in the ultrasonic sensor receives the reflected signals, the intensity of the reflected ultrasonic waves depends on the size and shape of the vehicle behind, and the ultrasonic imaging of the vehicle behind is reconstructed according to the intensity, time delay and direction of the signals;
Step 2.2: a three-eye camera sensor is selected and installed at the central position of the tail of the automobile to acquire the visual imaging of the rear vehicle.
4. The method for safely switching the lane of the automobile based on the millimeter wave radar and vision fusion of claim 3, wherein the step 3 is specifically as follows:
Step 3.1: the sift characteristic points of the ultrasonic imaging and visual imaging pictures are extracted, a RANSAC algorithm is used for screening characteristic point pairs, error points are eliminated, and the screened characteristic points basically correspond to each other one by one;
Step 3.2: using DLT algorithm to estimate perspective transformation matrix of the screened characteristic points, and in order to improve registration accuracy, APAP algorithm cuts the ultrasonic imaging and visual imaging picture into innumerable small squares, homography matrix transformation is carried out on each small square, homography matrix change is carried out on each small square, so as to obtain denoised ultrasonic imaging and visual imaging picture;
Step 3.3: directly carrying out Laplacian pyramid decomposition on the denoised ultrasonic imaging and visual imaging pictures obtained in the step 3.2 by adopting a multi-band bleing algorithm, decomposing an original image into sub-images with different scales by differentiating Gaussian pyramids of two adjacent layers, carrying out weighted average on each sub-image to obtain a fusion result of each layer, and finally carrying out reverse reconstruction of pyramids to obtain a visual fusion view of a rear vehicle; and taking the visual fusion view of the rear vehicle as the input of the SSD rapid target detection algorithm to obtain the fused space coordinate data of the rear vehicle.
5. The method for safely switching lanes of an automobile based on millimeter wave radar and vision fusion according to claim 4, wherein the step 4 is specifically as follows:
step 4.1: the time t for the rear vehicle to reach the side of the own vehicle is calculated from the vehicle speed v and the distance data D obtained in step 1, and the calculation formula is as follows:
Wherein t is the time when the rear vehicle reaches one side of the own vehicle, the distance between the detected target and the radar, namely the distance between the rear vehicle and the own vehicle at the initial moment is D, v b is the speed of the rear vehicle, and v a is the speed of the own vehicle;
Step 4.2: when the vehicle needs to change the lane, referring to the visual fusion view of the rear vehicle and the spatial coordinate data of the rear vehicle fused obtained in the step 3, and the time of the rear vehicle reaching one side of the vehicle calculated in the step 4.1, lane changing can be performed within a reasonable time range of 10 to 20 seconds; the visual fusion view of the rear vehicle obtained in the step 3 can bring a specific number and position reference of the rear vehicle on a visual level to a vehicle owner, and specifically comprises the following steps: according to the space coordinate data, when the distance between the rear vehicle and the own vehicle is 30 meters or more, the lane change can be performed safely; when the distance between the rear vehicle and the own vehicle is 10-20 meters, the speed is required to be increased and the lane is changed; when the following vehicle is less than 10 meters away from the own vehicle, stopping lane change is recommended.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410458224.0A CN118131208A (en) | 2024-04-17 | 2024-04-17 | Automobile safety lane changing method based on millimeter wave radar and vision fusion |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410458224.0A CN118131208A (en) | 2024-04-17 | 2024-04-17 | Automobile safety lane changing method based on millimeter wave radar and vision fusion |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118131208A true CN118131208A (en) | 2024-06-04 |
Family
ID=91244128
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410458224.0A Pending CN118131208A (en) | 2024-04-17 | 2024-04-17 | Automobile safety lane changing method based on millimeter wave radar and vision fusion |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118131208A (en) |
-
2024
- 2024-04-17 CN CN202410458224.0A patent/CN118131208A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11276189B2 (en) | Radar-aided single image three-dimensional depth reconstruction | |
| EP3885794A1 (en) | Track and road obstacle detecting method | |
| US10739438B2 (en) | Super-resolution radar for autonomous vehicles | |
| US11475678B2 (en) | Lane marker detection and lane instance recognition | |
| CN112946628A (en) | Road running state detection method and system based on radar and video fusion | |
| CN104237881B (en) | FMCW anti-collision radar multi-target detecting and tracking system and method | |
| CN109085570A (en) | Automobile detecting following algorithm based on data fusion | |
| CN103176185B (en) | Method and system for detecting road barrier | |
| US20200041612A1 (en) | Recurrent super-resolution radar for autonomous vehicles | |
| JP5047290B2 (en) | Device and method for estimating the size of a parking space, and automobile equipped with the device | |
| CN113850102B (en) | Vehicle-mounted vision detection method and system based on millimeter wave radar assistance | |
| Cui et al. | 3D detection and tracking for on-road vehicles with a monovision camera and dual low-cost 4D mmWave radars | |
| CN110568433A (en) | High-altitude parabolic detection method based on millimeter wave radar | |
| US11867829B2 (en) | Continuous visualization of beam steering vehicle radar scans | |
| US11587204B2 (en) | Super-resolution radar for autonomous vehicles | |
| CN112784679A (en) | Vehicle obstacle avoidance method and device | |
| CN112572430A (en) | Collision risk determination method and device | |
| EP4166984A1 (en) | Distributed microwave radar imaging method and apparatus | |
| CN112633101A (en) | Obstacle speed detection method and device | |
| CN115343713A (en) | Vehicle-mounted millimeter wave radar obstacle detection method and system, radar and automobile | |
| CN117724089B (en) | Ground and underground integrated intelligent mobile detection system | |
| WO2024109347A1 (en) | Method and apparatus for detecting travelable area, and vehicle and storage medium | |
| CN118131208A (en) | Automobile safety lane changing method based on millimeter wave radar and vision fusion | |
| CN115965847A (en) | Three-dimensional target detection method and system based on multi-modal feature fusion under cross view angle | |
| CN112651405B (en) | Target detection method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |