CN105480227A - Information fusion method based on infrared radar and video image in active driving technique - Google Patents
Information fusion method based on infrared radar and video image in active driving technique Download PDFInfo
- Publication number
- CN105480227A CN105480227A CN201511008952.9A CN201511008952A CN105480227A CN 105480227 A CN105480227 A CN 105480227A CN 201511008952 A CN201511008952 A CN 201511008952A CN 105480227 A CN105480227 A CN 105480227A
- Authority
- CN
- China
- Prior art keywords
- infra
- vehicle
- video image
- target
- region
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 67
- 238000007500 overflow downdraw method Methods 0.000 title abstract 2
- 238000001514 detection method Methods 0.000 claims abstract description 24
- 238000012360 testing method Methods 0.000 claims description 14
- 238000006073 displacement reaction Methods 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 6
- 238000012545 processing Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 238000012790 confirmation Methods 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
- B60W30/08—Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences
- B60W30/095—Predicting travel path or likelihood of collision
- B60W30/0956—Predicting travel path or likelihood of collision the prediction being responsive to traffic or environmental parameters
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2420/00—Indexing codes relating to the type of sensors based on the principle of their operation
- B60W2420/40—Photo, light or radio wave sensitive means, e.g. infrared sensors
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2420/00—Indexing codes relating to the type of sensors based on the principle of their operation
- B60W2420/40—Photo, light or radio wave sensitive means, e.g. infrared sensors
- B60W2420/403—Image sensing, e.g. optical camera
Landscapes
- Engineering & Computer Science (AREA)
- Automation & Control Theory (AREA)
- Transportation (AREA)
- Mechanical Engineering (AREA)
- Traffic Control Systems (AREA)
- Radar Systems Or Details Thereof (AREA)
Abstract
The invention relates to an information fusion method based on an infrared radar and a video image in an active driving technique. The method comprises a vehicle detecting process and a vehicle tracking process, wherein the vehicle detecting process comprises the following step of determining a region-of-interest ROI according to the video image, the step of detecting the target vehicle in the region-of-interest ROI, and the step of detecting the target vehicle in the region-of-interest ROI through an infrared radar module; the vehicle tracking process comprises the step of sending the information of the target region to the infrared radar module through the video image, and the step of detecting the target vehicle in the target region through the infrared radar module. According to the method disclosed by the invention, detection on the target vehicle is reliable, and assessment on the distance and the relative velocity is also precise.
Description
Technical field
The invention belongs to initiatively driving technique field, is based on the method that infra-red radar and video image information merge specifically in a kind of active driving technique.
Background technology
Automobile active safety plays a part very crucial in the active driving technique of automobile.At present, by detecting front vehicles, carry out early warning very useful to help driver or supplementary controlled system decision-making whether to change the speed of a motor vehicle etc.Traditional detection method, is combined with sensors such as laser radars the vehicle detecting front by vision sensor, by mutual for both Detection Information reference, can improve accuracy of detection each other, thus actv. detects front vehicles information more.But the scope that scope and laser radar sensor that vision sensor detects detect has very big-difference, causing the dual distance in life period and spatially when mutually confirming each other, affecting final detection efficiency and positioning precision.
Summary of the invention
For solving the problems referred to above that prior art exists, the invention provides in a kind of active driving technique based on the method that infra-red radar and video image information merge, vision sensor testing result is projected in vehicle axis system, laser radar sensor testing result is also projected in vehicle axis system simultaneously, the advantage that both utilizations are respective, information fusion is carried out, to obtain accurate information more comprehensively in detecting test of vehicle, vehicle odometry, vehicle tracking and side vehicle lane change etc. in overlapping region.
For achieving the above object, technical scheme of the present invention is, the method initiatively merged based on infra-red radar and video image information in driving technique, comprising: detecting test of vehicle process and vehicle tracking process; Described detecting test of vehicle process, comprising: the step detecting target vehicle according to the step of video image determination region of interest ROI, the step detecting target vehicle in region of interest ROI and infra-red radar module in region of interest ROI; Described vehicle tracking process, comprising: target area information sends to the step of infra-red radar module and infra-red radar module in target area, carry out the step of target vehicle detection by video image.
Further, the step according to video image determination region of interest ROI is specially: in the process of running car, utilizes the partial 3 d information reconstruction in video image, obtains the three-dimensional information of front space in vehicle traveling process; According to track, lane mark determination vehicle body place wherein, and the region in the lane mark choosing this front, track below sky ground wire is as region of interest ROI.
Further, the step detecting target vehicle in region of interest ROI is specially: detect in region of interest ROI, as target vehicle detected, carries out three-dimensional information mark, and testing result is issued infra-red radar module and confirm.
Further, the step that infra-red radar module detects target vehicle in region of interest ROI is specially: infra-red radar module is in region of interest ROI, if this target vehicle also detected, then measuring distance is carried out to this vehicle, and judge whether to need to slow down in conjunction with current vehicle speed.
Further, target area information sends to the step of infra-red radar module to be specially by video image: after video images detection to target vehicle, go out target area according to objects ahead vehicle extraction, and the target area three-dimensional information of extraction is passed to infra-red radar module.
Further, the step that infra-red radar module carries out target vehicle detection in target area is specially: infra-red radar module carries out the detection of target in target area, as target vehicle detected, the marginal information of target vehicle is extracted, in conjunction with the target area three-dimensional information received, simultaneously according to relative displacement, and then extract real target area, reject shadow, measuring distance is carried out to this vehicle, and judge whether to need to slow down in conjunction with current vehicle speed; If target vehicle do not detected, then illustrate in this target area there is no target vehicle, before video images detection to be shadow.
As further, region of interest ROI is carried out interpolation expansion, and the detection of target vehicle is carried out to the area image after expansion.
As further, infra-red radar module carries out 1/N expansion doubly to the target area three-dimensional information received, and carries out the detection of target vehicle in the target area after expansion.
As further, the span of described N is 3-4.
As further, described relative displacement refer to video image acquisition to target vehicle and target vehicle time difference of arriving of infra-red radar module acquires, determine in conjunction with map and speedometer.
The present invention is owing to adopting above technical scheme, following technique effect can be obtained: the present invention utilizes method of video image processing, inspection vehicle diatom, and confirm track belonging to self, intercept a part very little in original image as region of interest ROI according to image sky ground wire information simultaneously.So greatly reduce the scope that succeeding target detects, and the region of search specifying to infra-red radar processing module, decrease its live load.Meanwhile, the target decision after infra-red radar process is more reliable, and distance is also more accurate with the assessment of relative velocity.
On the other hand, in tracing process, video image is followed the tracks of and after the infrared combination of radar, can be removed the interference of front automobile shadow and other shadow regions further, thus improve the accuracy of tracking.
Accompanying drawing explanation
The present invention has accompanying drawing 2 width:
Fig. 1 is detecting test of vehicle process flow block diagram of the present invention;
Fig. 2 is vehicle tracking process flow block diagram of the present invention.
Detailed description of the invention
Below by embodiment, and by reference to the accompanying drawings, technical scheme of the present invention is described in further detail.
Embodiment 1
The method initiatively merged based on infra-red radar and video image information in driving technique, comprising: detecting test of vehicle process and vehicle tracking process;
Described detecting test of vehicle process is: in the process of running car, utilizes the partial 3 d information reconstruction in video image, obtains the three-dimensional information of front space in vehicle traveling process, according to track, lane mark determination vehicle body place wherein, and the region in the lane mark choosing this front, track below sky ground wire is as region of interest ROI, region of interest ROI is carried out interpolation expansion, and the area image after expansion is carried out to the detection of target vehicle, as target vehicle detected, carry out three-dimensional information mark, and testing result is issued infra-red radar module and confirm, in the region of infra-red radar module after expansion, if this target vehicle also detected, then measuring distance is carried out to this vehicle, and judge whether to need to slow down in conjunction with current vehicle speed, to avoid rear-end collision,
In vehicle tracking process, be very easy to occur that the target of following the tracks of may be the shadow (or part is shadow, is partly automobile entity) of automobile, this scene, utilizes the means of graphic processing just cumbersome comparatively speaking, in the present embodiment, vehicle tracking process is: described vehicle tracking process is for after video images detection to target vehicle, target area is gone out according to objects ahead vehicle extraction, and the target area three-dimensional information of extraction is passed to infra-red radar module, infra-red radar module carries out the expansion of 1/3 times to the target area three-dimensional information received, the detection of target vehicle is carried out in target area after expansion, as target vehicle detected, the marginal information of target vehicle is extracted, in conjunction with the target area three-dimensional information received, simultaneously according to relative displacement, and then extract real target area, reject shadow, measuring distance is carried out to this vehicle, and judge whether to need to slow down in conjunction with current vehicle speed, if target vehicle do not detected, then illustrate in this target area there is no target vehicle, before video images detection to be shadow.Described relative displacement refer to video image acquisition to target vehicle and target vehicle time difference of arriving of infra-red radar module acquires, determine in conjunction with map and speedometer.
The present invention utilizes the video image collected, and confirms track belonging to self, and utilizes local ROI region to amplify, and reaches the object detecting super distant view vehicle.Meanwhile, carry out the confirmation of physical object in conjunction with infra-red radar sensor, in order to avoid cause error detection, and carry out the Relative ranging between vehicle body and target vehicle.Afterwards, then according to key feature points on vehicle body and target vehicle, carry out the evaluation and test of relative velocity between vehicle, carry out alarm whether judge in conjunction with two spacings.In addition, the present invention, in the process of target tracking, utilizes image tracking algorithm in conjunction with infra-red radar to reject intended shadow subregion, thus carries out the tracking of target and the calculating of relative distance more accurately.
The above; be only the present invention's preferably detailed description of the invention; but protection scope of the present invention is not limited thereto; anyly be familiar with those skilled in the art in the technical scope that the present invention discloses; be equal to according to technical scheme of the present invention and inventive concept thereof and replace or change, all should be encompassed within protection scope of the present invention.
Claims (10)
1. the method merged based on infra-red radar and video image information in active driving technique, is characterized in that, comprising: detecting test of vehicle process and vehicle tracking process; Described detecting test of vehicle process, comprising: the step detecting target vehicle according to the step of video image determination region of interest ROI, the step detecting target vehicle in region of interest ROI and infra-red radar module in region of interest ROI; Described vehicle tracking process, comprising: target area information sends to the step of infra-red radar module and infra-red radar module in target area, carry out the step of target vehicle detection by video image.
2. the method merged based on infra-red radar and video image information in active driving technique according to claim 1, it is characterized in that, step according to video image determination region of interest ROI is specially: in the process of running car, utilize the partial 3 d information reconstruction in video image, obtain the three-dimensional information of front space in vehicle traveling process; According to track, lane mark determination vehicle body place wherein, and the region in the lane mark choosing this front, track below sky ground wire is as region of interest ROI.
3. the method merged based on infra-red radar and video image information in active driving technique according to claim 2, it is characterized in that, the step detecting target vehicle in region of interest ROI is specially: detect in region of interest ROI, as target vehicle detected, carry out three-dimensional information mark, and testing result is issued infra-red radar module and confirm.
4. the method merged based on infra-red radar and video image information in active driving technique according to claim 3, it is characterized in that, the step that infra-red radar module detects target vehicle in region of interest ROI is specially: infra-red radar module is in region of interest ROI, if this target vehicle also detected, then measuring distance is carried out to this vehicle, and judge whether to need to slow down in conjunction with current vehicle speed.
5. the method merged based on infra-red radar and video image information in active driving technique according to claim 1, it is characterized in that, target area information sends to the step of infra-red radar module to be specially by video image: after video images detection to target vehicle, go out target area according to objects ahead vehicle extraction, and the target area three-dimensional information of extraction is passed to infra-red radar module.
6. the method merged based on infra-red radar and video image information in active driving technique according to claim 5, it is characterized in that, the step that infra-red radar module carries out target vehicle detection in target area is specially: infra-red radar module carries out the detection of target in target area, as target vehicle detected, the marginal information of target vehicle is extracted, in conjunction with the target area three-dimensional information received, simultaneously according to relative displacement, and then extract real target area, reject shadow, measuring distance is carried out to this vehicle, and judge whether to need to slow down in conjunction with current vehicle speed, if target vehicle do not detected, then illustrate in this target area there is no target vehicle, before video images detection to be shadow.
7. based on the method that infra-red radar and video image information merge in the active driving technique according to any one of claim 2-4, it is characterized in that, region of interest ROI is carried out interpolation expansion, and the detection of target vehicle is carried out to the area image after expansion.
8. based on the method that infra-red radar and video image information merge in the active driving technique according to claim 5 or 6, it is characterized in that, infra-red radar module carries out 1/N expansion doubly to the target area three-dimensional information received, and carries out the detection of target vehicle in the target area after expansion.
9. the method merged based on infra-red radar and video image information in active driving technique according to claim 8, is characterized in that, the span of described N is 3-4.
10. the method merged based on infra-red radar and video image information in active driving technique according to claim 6, it is characterized in that, described relative displacement refer to video image acquisition to target vehicle and target vehicle time difference of arriving of infra-red radar module acquires, determine in conjunction with map and speedometer.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201511008952.9A CN105480227B (en) | 2015-12-29 | 2015-12-29 | The method merged in active driving technology based on infrared radar with video image information |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201511008952.9A CN105480227B (en) | 2015-12-29 | 2015-12-29 | The method merged in active driving technology based on infrared radar with video image information |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105480227A true CN105480227A (en) | 2016-04-13 |
| CN105480227B CN105480227B (en) | 2017-12-15 |
Family
ID=55667605
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201511008952.9A Active CN105480227B (en) | 2015-12-29 | 2015-12-29 | The method merged in active driving technology based on infrared radar with video image information |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105480227B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106373394A (en) * | 2016-09-12 | 2017-02-01 | 深圳尚桥交通技术有限公司 | Vehicle detection method and system based on video and radar |
| CN106527425A (en) * | 2016-09-30 | 2017-03-22 | 张家港长安大学汽车工程研究院 | Omnibearing intelligent following trolley and controlling method thereof |
| WO2018035711A1 (en) * | 2016-08-23 | 2018-03-01 | 深圳市速腾聚创科技有限公司 | Target detection method and system |
| CN110443819A (en) * | 2018-05-03 | 2019-11-12 | 比亚迪股份有限公司 | A kind of track detection method and device of monorail train |
| CN110781816A (en) * | 2019-10-25 | 2020-02-11 | 北京行易道科技有限公司 | Method, device, equipment and storage medium for transverse positioning of vehicle in lane |
| CN111409584A (en) * | 2020-02-27 | 2020-07-14 | 广汽蔚来新能源汽车科技有限公司 | Pedestrian protection method, device, computer equipment and storage medium |
| CN111856611A (en) * | 2019-04-17 | 2020-10-30 | 阿里巴巴集团控股有限公司 | Trigger detection method, device and system |
| WO2021233154A1 (en) * | 2020-05-20 | 2021-11-25 | 华为技术有限公司 | Drivable region detection method and apparatus, and device, and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060178789A1 (en) * | 2005-02-07 | 2006-08-10 | Nobuyuki Kuge | Driving intention estimation system, vehicle operation assistance system, and vehicle equipped therewith |
| CN102303605A (en) * | 2011-06-30 | 2012-01-04 | 中国汽车技术研究中心 | Multi-sensor information fusion-based collision and departure pre-warning device and method |
| CN102508246A (en) * | 2011-10-13 | 2012-06-20 | 吉林大学 | Method for detecting and tracking obstacles in front of vehicle |
| CN102542843A (en) * | 2010-12-07 | 2012-07-04 | 比亚迪股份有限公司 | Early warning method for preventing vehicle collision and device |
| CN104899855A (en) * | 2014-03-06 | 2015-09-09 | 株式会社日立制作所 | Three-dimensional obstacle detection method and apparatus |
-
2015
- 2015-12-29 CN CN201511008952.9A patent/CN105480227B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060178789A1 (en) * | 2005-02-07 | 2006-08-10 | Nobuyuki Kuge | Driving intention estimation system, vehicle operation assistance system, and vehicle equipped therewith |
| CN102542843A (en) * | 2010-12-07 | 2012-07-04 | 比亚迪股份有限公司 | Early warning method for preventing vehicle collision and device |
| CN102303605A (en) * | 2011-06-30 | 2012-01-04 | 中国汽车技术研究中心 | Multi-sensor information fusion-based collision and departure pre-warning device and method |
| CN102508246A (en) * | 2011-10-13 | 2012-06-20 | 吉林大学 | Method for detecting and tracking obstacles in front of vehicle |
| CN104899855A (en) * | 2014-03-06 | 2015-09-09 | 株式会社日立制作所 | Three-dimensional obstacle detection method and apparatus |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018035711A1 (en) * | 2016-08-23 | 2018-03-01 | 深圳市速腾聚创科技有限公司 | Target detection method and system |
| US10891499B2 (en) | 2016-08-23 | 2021-01-12 | Suteng Innovation Technology Co., Ltd. | Target detection method and system |
| CN106373394A (en) * | 2016-09-12 | 2017-02-01 | 深圳尚桥交通技术有限公司 | Vehicle detection method and system based on video and radar |
| CN106527425A (en) * | 2016-09-30 | 2017-03-22 | 张家港长安大学汽车工程研究院 | Omnibearing intelligent following trolley and controlling method thereof |
| CN110443819A (en) * | 2018-05-03 | 2019-11-12 | 比亚迪股份有限公司 | A kind of track detection method and device of monorail train |
| CN110443819B (en) * | 2018-05-03 | 2022-04-15 | 比亚迪股份有限公司 | Method and device for detecting track of monorail train |
| CN111856611A (en) * | 2019-04-17 | 2020-10-30 | 阿里巴巴集团控股有限公司 | Trigger detection method, device and system |
| CN110781816A (en) * | 2019-10-25 | 2020-02-11 | 北京行易道科技有限公司 | Method, device, equipment and storage medium for transverse positioning of vehicle in lane |
| CN111409584A (en) * | 2020-02-27 | 2020-07-14 | 广汽蔚来新能源汽车科技有限公司 | Pedestrian protection method, device, computer equipment and storage medium |
| WO2021233154A1 (en) * | 2020-05-20 | 2021-11-25 | 华为技术有限公司 | Drivable region detection method and apparatus, and device, and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105480227B (en) | 2017-12-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105480227A (en) | Information fusion method based on infrared radar and video image in active driving technique | |
| US11953599B2 (en) | Vehicle navigation based on aligned image and LIDAR information | |
| US10260889B2 (en) | Position estimation device and position estimation method | |
| US10605896B2 (en) | Radar-installation-angle calculating device, radar apparatus, and radar-installation-angle calculating method | |
| CN104793202B (en) | The object emerging system of more radar imagery sensors | |
| CN105489035A (en) | Detection method of traffic lights applied to active drive technology | |
| JP7052543B2 (en) | Vehicle position estimation device | |
| CN108473131A (en) | Parking assistance method and device | |
| US20050278112A1 (en) | Process for predicting the course of a lane of a vehicle | |
| JP2018092483A (en) | Object recognition device | |
| CN104943694A (en) | System and method for determining of and compensating for misalignment of sensor | |
| JP2004531424A (en) | Sensing device for cars | |
| JP2004534947A (en) | Object location system for road vehicles | |
| EP3364336B1 (en) | A method and apparatus for estimating a range of a moving object | |
| CN106164930A (en) | Method for lane detection | |
| CN104620297A (en) | Speed calculating device and speed calculating method, and collision determination device | |
| JP5821276B2 (en) | Detection device for position and traveling direction of moving body | |
| JP6263453B2 (en) | Momentum estimation device and program | |
| KR101980509B1 (en) | Inter-vehicle distance estimation method and inter-vehicle distance estimation device | |
| CN109427212A (en) | Vehicle running detection method and vehicle running detection system | |
| US11205081B2 (en) | Arithmetic apparatus | |
| JP6604052B2 (en) | Runway boundary estimation device and runway boundary estimation method | |
| JP5162962B2 (en) | Object detection device | |
| Cario et al. | Predictive time-to-lane-crossing estimation for lane departure warning systems | |
| Bauer et al. | Comparison of mono camera-based static obstacle position estimation methods for automotive application |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |