CN109901156A - A kind of subject fusion method and apparatus of vehicle millimetre-wave radar and camera - Google Patents

A kind of subject fusion method and apparatus of vehicle millimetre-wave radar and camera Download PDF

Info

Publication number
CN109901156A
CN109901156A CN201910075572.9A CN201910075572A CN109901156A CN 109901156 A CN109901156 A CN 109901156A CN 201910075572 A CN201910075572 A CN 201910075572A CN 109901156 A CN109901156 A CN 109901156A
Authority
CN
China
Prior art keywords
point
millimetre
camera
wave radar
target point
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910075572.9A
Other languages
Chinese (zh)
Inventor
刘兴亮
付会通
刘世东
方锐
高博麟
宋攀
崔东
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Automotive Technology and Research Center Co Ltd
CATARC Tianjin Automotive Engineering Research Institute Co Ltd
Original Assignee
China Automotive Technology and Research Center Co Ltd
CATARC Tianjin Automotive Engineering Research Institute Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Automotive Technology and Research Center Co Ltd, CATARC Tianjin Automotive Engineering Research Institute Co Ltd filed Critical China Automotive Technology and Research Center Co Ltd
Priority to CN201910075572.9A priority Critical patent/CN109901156A/en
Publication of CN109901156A publication Critical patent/CN109901156A/en
Pending legal-status Critical Current

Links

Landscapes

  • Radar Systems Or Details Thereof (AREA)
  • Traffic Control Systems (AREA)

Abstract

The present invention provides the subject fusion method and apparatus of a kind of vehicle millimetre-wave radar and camera, for solving the problems, such as the goal congruence confirmation of vehicle millimetre-wave radar and camera sensing device in terms of target level data fusion, to improve vehicle environmental detecting sensor to the precision of target acquisition.Method includes: the data information for obtaining first object point and the second target point, and first object point represents the object that millimetre-wave radar sensor detects, and the second target point represents the object that camera detects;The distance between first object point and second target point are determined less than the first preset threshold, first object point is thought at this time and the second target point is same object;The target level data information as the object is combined in the classification for the object that the distance for the object that millimetre-wave radar detects and speed, camera are detected and lateral position.

Description

A kind of subject fusion method and apparatus of vehicle millimetre-wave radar and camera
Technical field
The invention belongs to vehicle target Fusion Features field, more particularly, to a kind of vehicle millimetre-wave radar and camera Subject fusion method and apparatus.
Background technique
The fast development of advanced driving assistance system or automated driving system proposes vehicle perceptual performance more harsh Requirement, it may be said that the promotion of perceptual performance is revolutionary for the promotion of system performance.But it is each to be limited to technology, cost etc. Aspect reason, single-sensor is also difficult to meet complicated vehicle-surroundings environmental information perception at present, therefore uses a variety of sensings Device, which carries out data fusion, becomes the direction that each research institution makes great efforts.And the environment sensing sensor used on vehicle at present is main There are laser radar (LiDAR), millimetre-wave radar (Radar) and camera, laser radar has accurate, resolution ratio height of detection etc. excellent Point, but it is limited to the cost of laser radar, the main sensors that current and following a period of time uses on volume production vehicle are still Millimetre-wave radar and camera.Millimetre-wave radar have the characteristics that ranging, test the speed precisely, influenced by weather small, but its target Resolution capability and target angle resolution capability are weaker, and camera is angle-resolved for target type and object with higher Identification capability, if it is possible to millimetre-wave radar and camera are subjected to subject fusion, then the raising for system senses performance Help with highly significant.
Carrying out sensor fusion perception at present has three big target directions, is image element level data fusion, feature level data respectively Fusion and target level data fusion.Image element level data fusion just carries out Data Matching fusion at sensor raw data end, needs The data of magnanimity are handled, very high requirement is proposed for vehicle computing resource;Feature level data fusion is each single Sensor carries out initial data to carry out object matching after simply filtering again, and the data volume of this amalgamation mode processing is compared to original Data volume is not much less, in addition it is also possible to filtering some key messages;And target level data fusion is each single biography Sensor is filtered initial data, carries out object matching fusion again after feature extraction, and data volume is smaller at this time, target information Compare and define, data fusion can be carried out by sensor of the integrated side such as main engine plants to millimetre-wave radar and camera Liang Ge producer It interprets, improves the perceptual performance of sensor.
In the prior art, it would be highly desirable to which the technical issues of solving is: how to realize vehicle millimetre-wave radar and camera sensing device It is respective to play the two sensor to same object with well for goal congruence confirmation in terms of target level data fusion Advantage namely the resolution of the target of camera and angular resolution, the ranging of millimetre-wave radar and the ability that tests the speed, improve vehicle ring Precision of the border detecting sensor to target acquisition.
Summary of the invention
In view of this, the present invention is directed to propose the subject fusion method and dress of a kind of vehicle millimetre-wave radar and camera It sets, the advantage of the two sensor respectively can be played to same object well, improve vehicle environmental detecting sensor to mesh Mark the precision of detection.
In order to achieve the above objectives, the technical scheme of the present invention is realized as follows:
A kind of subject fusion method of vehicle millimetre-wave radar and camera:
The data information of first object point and the second target point is obtained, data information includes location information, first object point The object that millimetre-wave radar sensor detects is represented, the second target point represents the object that camera detects;
Determine that the distance between first object point and second target point less than the first preset threshold, think first object at this time Point and the second target point are same objects;
The classification for the object that the distance for the object that millimetre-wave radar detects and speed, camera are detected The target level data information as the object is combined with lateral position.
Further, determine that the distance between first object point and second target point less than the first preset threshold, are recognized at this time It include: the left side up and down in first object point and the second target point before being same object for first object point and the second target point Right each extended range of four direction;There are intersections for first object point and the second target point after determining extension, think first at this time Target point and the second target point are same objects.
Further, when the type of the object detected when camera is vehicle, in first object point and the second target The distance of the width direction extension of point is the half of vehicle width, and the distance of length direction extension is the half of Vehicle length;When When the type for the object that camera detects is single pedestrian, at four up and down of first object point and the second target point The distance of Directional Extension is 0.3-0.5m;When the type for the object that camera detects is two wheeler, in first object The distance that the four direction up and down of point and the second target point extends is 0.4-0.6m.
Further, determine extension after first object point and the second target point there are intersections, it is believed that first object point and If the second target point is that same object includes: that the speed for the object that millimetre-wave radar detects and camera detect Difference between the speed of object thinks first object point at this time and the second target point is same mesh less than the second preset threshold Mark object;If the directional velocity for the object that the directional velocity and camera of the object that millimetre-wave radar detects detect exists It is not identical in limiting time, then directly determine first object point and the second target point is not same object.
Further, the calculation formula of the second preset threshold △ v is as follows:
va≤ 30km/h, △ v=0.1va
30km/h<va≤ 80km/h, △ v=0.09va
80km/h<va≤ 100km/h, △ v=0.085va
100km/h<va≤ 120km/h, △ v=0.075va
va> 120km/h, △ v=0.07va
Wherein, vaIt is the speed for the object that millimetre-wave radar detects.
A kind of subject fusion device of vehicle millimetre-wave radar and camera, comprising:
Module is obtained, for obtaining the data information of first object point and the second target point, wherein data information includes position Confidence breath, first object point represent the object that millimetre-wave radar sensor detects, and the second target point represents camera detection The object arrived;
Determining module, for determine the distance between first object point and second target point less than the first preset threshold, this When think first object point and the second target point is same object;
Composite module, the distance and speed of the object for detecting millimetre-wave radar, camera detect The target level data information as the object is combined in the classification of the object and lateral position;
It further, further include expansion module, in four sides up and down of first object point and the second target point To each extension a certain distance;
Determining module, there are intersections for the first object point and the second target point for being further used for determining after extending, and recognize at this time It is same object for first object point and the second target point.
Further, expansion module is when the type for the object that camera detects is vehicle, in first object point and The distance of the width direction extension of second target point is the half of vehicle width, and the distance of length direction extension is Vehicle length Half;When the type for the object that camera detects is single pedestrian, first object point and the second target point up and down The distance of left and right four direction extension is 0.3-0.5m;When the type for the object that camera detects is two wheeler, The distance of the four direction up and down of first object point and the second target point extension is 0.4-0.6m;
Determining module, there are intersections for the first object point and the second target point for being further used for determining after extending, and in the least Difference between the speed for the object that the speed and camera for the object that metre wave radar detects detect is pre- less than second If threshold value, first object point is thought at this time and the second target point is same object;
It determines that determining module includes judging submodule, judge the directional velocity for the object that millimetre-wave radar detects and takes the photograph Whether the directional velocity of the object detected as head is identical whithin a period of time, if different, directly determines the first mesh Punctuate and the second target point are not same objects;
The calculation formula of second preset threshold △ v is as follows:
va≤ 30km/h, △ v=0.1va
30km/h<va≤ 80km/h, △ v=0.09va
80km/h<va≤ 100km/h, △ v=0.085va
100km/h<va≤ 120km/h, △ v=0.075va
va> 120km/h, △ v=0.07va
Wherein, vaIt is the speed for the object that millimetre-wave radar detects.
A kind of electronic equipment, comprising: shell, processor, memory, circuit board and power circuit, wherein circuit board placement In the space interior that shell surrounds, processor and memory setting are on circuit boards;Power circuit, for being above-mentioned electronic equipment Each circuit or device power supply;Memory is for storing executable program code;Processor is stored by reading in memory Executable program code run program corresponding with executable program code, for execute above-mentioned vehicle millimetre-wave radar and The subject fusion method of camera.
A kind of computer readable storage medium, computer-readable recording medium storage have one or more program, and one Or multiple programs can be executed by one or more processor, for executing the mesh of above-mentioned vehicle millimetre-wave radar and camera Mark fusion method.
Compared with the existing technology, the subject fusion method and apparatus tool of a kind of vehicle millimetre-wave radar of the present invention and camera There is following advantage:
A kind of subject fusion method and apparatus of vehicle millimetre-wave radar and camera of the present invention, by judging first object Between point (object that millimetre-wave radar sensor detects) and the second target point (object that camera detects) away from From whether less than the first preset threshold, think that first object point and the second target point meet object consistency when being less than, i.e., It is same object, the distance and speed of the object for later detecting millimetre-wave radar, camera detection in position The target level data information as the object is combined in the classification of the object arrived and lateral position, and the present invention can be well The target that same object plays the respective advantage namely camera of both millimetre-wave radar sensor and camera is differentiated and Angular resolution, the ranging of millimetre-wave radar and the ability that tests the speed improve vehicle environmental detecting sensor to the essence of target acquisition Accuracy.
Detailed description of the invention
The attached drawing for constituting a part of the invention is used to provide further understanding of the present invention, schematic reality of the invention It applies example and its explanation is used to explain the present invention, do not constitute improper limitations of the present invention.
In the accompanying drawings:
Fig. 1 is the flow chart of the subject fusion embodiment of the method one of vehicle millimetre-wave radar of the present invention and camera;
Fig. 2 is the schematic diagram of the subject fusion embodiment of the method one of vehicle millimetre-wave radar of the present invention and camera;
Fig. 3 is the flow chart of the subject fusion embodiment of the method two of vehicle millimetre-wave radar of the present invention and camera;
Fig. 4 be in the subject fusion embodiment of the method two of vehicle millimetre-wave radar of the present invention and camera millimetre-wave radar with Detection schematic diagram of the camera sensing device to same object;
Fig. 5 is sensor detection in the subject fusion embodiment of the method two of vehicle millimetre-wave radar of the present invention and camera The schematic diagram of target point extension;
Fig. 6 is object consistency in the subject fusion embodiment of the method two of vehicle millimetre-wave radar of the present invention and camera The flow diagram of confirmation;
Fig. 7 is the flow chart of the subject fusion embodiment of the method three of vehicle millimetre-wave radar of the present invention and camera;
Fig. 8 is object fusion letter in the subject fusion embodiment of the method three of vehicle millimetre-wave radar of the present invention and camera The schematic diagram of breath;
Fig. 9 is the structural schematic diagram of the subject fusion Installation practice one of vehicle millimetre-wave radar of the present invention and camera;
Figure 10 is the structural schematic diagram of the subject fusion Installation practice two of vehicle millimetre-wave radar of the present invention and camera;
Figure 11 is the structural schematic diagram of electronic equipment one embodiment of the present invention.
It should be noted that in the absence of conflict, the feature in embodiment and embodiment in the present invention can phase Mutually combination.
In the description of the present invention, it is to be understood that, term " center ", " longitudinal direction ", " transverse direction ", "upper", "lower", The orientation or positional relationship of the instructions such as "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outside" is It is based on the orientation or positional relationship shown in the drawings, is merely for convenience of description of the present invention and simplification of the description, rather than instruction or dark Show that signified device or element must have a particular orientation, be constructed and operated in a specific orientation, therefore should not be understood as pair Limitation of the invention.In addition, term " first ", " second " etc. are used for description purposes only, it is not understood to indicate or imply phase To importance or implicitly indicate the quantity of indicated technical characteristic.The feature for defining " first ", " second " etc. as a result, can To explicitly or implicitly include one or more of the features.In the description of the present invention, unless otherwise indicated, " multiple " It is meant that two or more.
In the description of the present invention, it should be noted that unless otherwise clearly defined and limited, term " installation ", " phase Even ", " connection " shall be understood in a broad sense, for example, it may be being fixedly connected, may be a detachable connection, or be integrally connected;It can To be mechanical connection, it is also possible to be electrically connected;It can be directly connected, can also can be indirectly connected through an intermediary Connection inside two elements.For the ordinary skill in the art, above-mentioned term can be understood by concrete condition Concrete meaning in the present invention.
The present invention will be described in detail below with reference to the accompanying drawings and embodiments.
As shown in Figs. 1-2, the present embodiment provides a kind of subject fusion method of vehicle millimetre-wave radar and camera, packets It includes:
Step 101: obtaining the data information of first object point and the second target point, wherein data information includes position letter Breath, first object point represent the object that millimetre-wave radar sensor detects, and the second target point represents what camera detected Object;
Step 102: determining that the distance between first object point and second target point less than the first preset threshold, are thought at this time First object point and the second target point are same objects;
In this step, the size of the first preset threshold can flexibly be set according to the actual situation, specifically, can be according to mesh The size of mark object is set, such as is set as the 1/5-1/2 of object size.If in this step first object point and The distance between second target point is more than or equal to the first preset threshold, then it is assumed that first object point and the second target point are not same Object.
Step 103: the target that the distance for the object that millimetre-wave radar detects and speed, camera are detected The target level data information as the object is combined in the classification of object and lateral position.
The present embodiment, by judging first object point (object that millimetre-wave radar sensor detects) and the second target Whether the distance between point (object that camera detects) when be less than thinks first object point less than the first preset threshold Meet object consistency with the second target point, i.e., is same object in position, later detects millimetre-wave radar The classification for the object that the distance and speed of the object, camera detect and lateral position combination are as the object Target level data information, the present embodiment can play both millimetre-wave radar sensor and camera to same object well The target of respective advantage namely camera is differentiated and angular resolution, the ranging of millimetre-wave radar and the ability that tests the speed, and improves Precision of the vehicle environmental detecting sensor to target acquisition.
The present embodiment carries out consistency confirmation to detection target first, and object passes through the consistency detection of two sensors Afterwards, the object distance and speed for using millimetre-wave radar to detect use camera as the final distance and speed of object The classification and lateral distance of the object of detection are as the final classification of object and lateral position, distance, speed in summary To obtain the information of object, the algorithm that may be used as decision-making level executes foundation for degree, classification and lateral distance.
The present embodiment is suitable for carrying out the scheme that fusion perceives detection with first two sensor is imaged using millimetre-wave radar, This method has algorithm simple, efficiently, subject fusion feature with high accuracy.
The present embodiment has the advantages that
1) vehicle millimetre-wave radar and camera are improved to the synthesis sensing capability of vehicle-surroundings environment;
2) improve vehicle to the sensing capability of environmental goals object, object classification, speed, in terms of improve essence Exactness;
3) method is simple and effective, it is possible to reduce camera and Radar Data Fusion calculation amount improve computation rate.
Fig. 3 is the flow chart of the subject fusion embodiment of the method two of vehicle millimetre-wave radar of the present invention and camera, such as Fig. 3 Shown, the method for the present embodiment may include:
Step 201: obtaining the data information of first object point and the second target point, wherein data information includes position letter Breath, first object point represent the object that millimetre-wave radar sensor detects, and the second target point represents what camera detected Object;
Step 202: respectively extending a certain distance in the four direction up and down of first object point and the second target point;
Step 203: there are intersections for first object point and the second target point after determining extension, think first object point at this time It is same object with the second target point;
In this step 202-203, the confirmation of the consistency of object is mainly realized by method that target point extends, such as Shown in Fig. 4-6, wherein representing the object that millimetre-wave radar sensor detects with point A, represent what camera detected with point B Same object.Due to the physics and performance difference of two sensors, point A and point B are not overlapped in most cases, this When point A is extended in vehicle detection coordinate system, be specifically exactly that four direction is each up and down in point A in the present embodiment A certain distance is extended, is denoted as w, this time point A is extended to a square;Point B is also carried out simultaneously similarly to extend behaviour with point A Make, point B is also extended to a square, as shown in Figure 5.Just there will be two kinds of situations after the completion of two point extensions, a kind of situation It is rectangular A and rectangular B there are intersection, another situation is that there is no intersections by rectangular A and rectangular B, such as Fig. 5 (S1) and (S2) Situation.It selects suitable threshold value w to be extended point A and point B, if rectangular A and rectangular B have intersection at this moment, is considered as Point A and point B is matched in orientation meets object consistency, i.e., is same object in position.
It should be noted that can be four direction up and down when being extended to point A, B and extend identical distance w, It is also possible to up and down direction and extends identical distance w1, left and right directions extends identical distance w2, wherein w1 ≠ w2;Meanwhile w, The value of w1, w2 can flexible choice according to the actual situation, the type of the object preferably detected according to camera it is different and It is different, to improve the accuracy of object consistency judgement.For example, first distinguishing object is vehicle, pedestrian or two Vehicle is taken turns, is judged according to the output of the target classification of camera, i.e. the marker characteristic of target, if target label feature is vehicle , at this moment taking w1 is the half of vehicle width, i.e. w1=0.9m;Taking w2 is the half of Vehicle length, i.e. w2=2.5m;If mesh Mark marker characteristic is single pedestrian, then extending identical distance w up and down, w value is 0.3-0.5m, preferably 0.4m;Such as Fruit target is two wheeler, will equally extend identical distance w up and down, and w value is 0.4-0.6m, preferably 0.5m;If mesh Mark marker characteristic be crowd (number >=2, and be closer), at this moment do not do extended arithmetic, with camera export location parameter Location parameter as target.
Specifically, such as first coordinate (the x of point A is calculateda,ya), i.e. xa=l × sin θ, ya=l × cos θ, wherein L and θ is respectively the target level data information that millimetre-wave radar detects, i.e. distance and angle of the object relative to this vehicle. And coordinate (the x of point Bb,yb) directly provided by camera sensing device, as shown in Figure 5.Then square expansion is carried out to point A and point B Exhibition, as shown in figure 5, square shaped A and rectangular B carries out intersection inspection after the completion of extension, i.e., if xa≤xb, and xa+w≥xb-w;Or If person xa>xb, and xa-w≤xb+ w, it is believed that point A and point B meet consistency on lateral position, next to point A and point B into Row vertical consistency inspection, method is similar with laterally uniform property inspection method, as shown in fig. 6, wherein extended range w can basis The method of test is obtained.If horizontal and vertical consistency check passes through, then it is assumed that the orientation of point A and point B match at Function, i.e., in position it is considered that point A and point B are same objects.
Step 204: the target that the distance for the object that millimetre-wave radar detects and speed, camera are detected The target level data information as the object is combined in the classification of object and lateral position.
In the present embodiment, realize that the consistency of object confirms that recognition speed is fast by the method for target point extension, it is quasi- True rate is high.
Fig. 7 is the flow chart of the subject fusion embodiment of the method three of vehicle millimetre-wave radar of the present invention and camera, such as Fig. 7 Shown, the method for the present embodiment includes:
Step 301: obtaining the data information of first object point and the second target point, wherein data information includes position letter Breath, first object point represent the object that millimetre-wave radar sensor detects, and the second target point represents what camera detected Object;
Step 302: respectively extending a certain distance in the four direction up and down of first object point and the second target point;
Step 303: there are intersections for first object point and the second target point after determining extension, and millimetre-wave radar detects To object speed and the speed of object that detects of camera between difference less than the second preset threshold, recognize at this time It is same object for first object point and the second target point;
In this step, in order to more accurately confirm that point A and point B is same object, need the two in motion state It is matched, is specifically exactly the speed difference of comparison point A He point B.If the speed of this time point A and the speed of point B are very close, I.e. the speed difference of the two is less than the second preset threshold △ v set, that is, thinks that point A and point B pass through object in speeds match Consistency check, namely same object is considered in speed.
Specifically, such as by millimetre-wave radar the object relative velocity detected, i.e. the speed v of point AaIt is detected with camera Object relative velocity, i.e. the speed v of point BbTake difference, then take the difference absolute value and the second preset threshold △ v into Row compares, if the difference is less than threshold value △ v, i.e., | va-vb|≤△ v, then it is assumed that point A and point B successful match in speed, i.e., It is considered that point A and point B is same object, as shown in Figure 6 in motion state.
In order to improve accuracy of judgement degree and efficiency, the directional velocity of object can be judged first, if vaWith vb Directional velocity it is different, and still direction is different after continuous a period of time such as 5 processor step-lengths, then directly judgement two A object is not same target;If directional velocity is identical, then calculates vaAnd vbDifference, if the difference be less than threshold value △ v, i.e., | va-vb|≤△ v, then it is assumed that the two is same object.
Specifically, target relative velocity v of the value of the second preset threshold △ v preferably with millimetre-wave radar detectionaCorrelation, That is:
va≤ 30km/h, △ v=0.1va
30km/h<va≤ 80km/h, △ v=0.09va
80km/h<va≤ 100km/h, △ v=0.085va
100km/h<va≤ 120km/h, △ v=0.075va
va> 120km/h, △ v=0.07va
In this way, passing through the target relative velocity v for detecting the second preset threshold △ v according to millimetre-wave radaraNot same order Ladder type changes value, and after measured, the case where relative to the second preset threshold is set as fixed value, the present embodiment is to object one The False Rate of cause property has significant decrease, can drop within 3% from 10%.
Step 304: the target that the distance for the object that millimetre-wave radar detects and speed, camera are detected The target level data information as the object is combined in the classification of object and lateral position.
It will be apparent to a skilled person that the target that two sensors (millimetre-wave radar and camera) are detected Object may have N (N >=2) a, and 2 are at this moment selected in N number of object, carry out position and speed consistency check, then recycle ratio To inspection until completing, all consistency targets carry out target data fusion after being detected again, as shown in Figure 6.
When the target object point A of millimetre-wave radar detection and the target object point B of camera detection pass through location consistency and speed It spends after consistency check i.e. by as same object T.At this moment the target information of object T can be detected by millimetre-wave radar Distance and speed, the object classification of camera detection, lateral position are constituted, i.e., object information is T (l, va,xa, c), wherein L and vaThe respectively object distance and relative velocity of millimetre-wave radar detection, xaIt is respectively the object of camera detection with c Lateral position and object classification, as shown in figure 8, these information, which are used decision-making level, does planning and decision.
The present embodiment three on the basis of example 2, increases the matching inspection of motion state, to more ensure The accuracy of goal congruence confirmation.
Fig. 9 is the structural schematic diagram of the subject fusion Installation practice one of vehicle millimetre-wave radar of the present invention and camera, As shown in figure 9, the device of the present embodiment may include:
Module 11 is obtained, for obtaining the data information of first object point and the second target point, wherein data information includes Location information, first object point represent the object that millimetre-wave radar sensor detects, and the second target point represents camera spy The object measured;
Determining module 12, for determine the distance between first object point and second target point less than the first preset threshold, First object point is thought at this time and the second target point is same object;
Composite module 13, the distance and speed of the object for detecting millimetre-wave radar, camera detect The object classification and lateral position combine target level data information as the object.
The device of the present embodiment can be used for executing the technical solution of embodiment of the method shown in Fig. 1, realization principle and skill Art effect is similar, and details are not described herein again.
Figure 10 is the structural schematic diagram of the subject fusion Installation practice two of vehicle millimetre-wave radar of the present invention and camera, As shown in Figure 10, the device of the present embodiment may include:
Module 21 is obtained, for obtaining the data information of first object point and the second target point, wherein data information includes Location information, first object point represent the object that millimetre-wave radar sensor detects, and the second target point represents camera spy The object measured;
Expansion module 22 respectively extends centainly for the four direction up and down in first object point and the second target point Distance;
Determining module 23, for determining the first object point after extension and the second target point, there are intersections, think the at this time One target point and the second target point are same objects;
Composite module 24, the distance and speed of the object for detecting millimetre-wave radar, camera detect The object classification and lateral position combine target level data information as the object.
The device of the present embodiment can be used for executing the technical solution of embodiment of the method shown in Fig. 3, realization principle and skill Art effect is similar, and details are not described herein again.
Preferably, in expansion module 22, in the extension of the four direction up and down of first object point and the second target point The type of distance, the object detected according to camera is different and different, to improve the standard of object consistency judgement True property.For example, when the type for the object that camera detects is vehicle, in the width of first object point and the second target point The distance of Directional Extension is the half of vehicle width, and the distance of length direction extension is the half of Vehicle length;When camera is visited When the type of the object measured is single pedestrian, extended in the four direction up and down of first object point and the second target point Distance be 0.3-0.5m;When the type for the object that camera detects is two wheeler, in first object point and second The distance of the extension of four direction up and down of target point is 0.4-0.6m.
Further, on the basis of Figure 10 shown device structure, determining module 23 can also be further used for determining expansion First object point and the second target point after exhibition are there are intersection, and the speed and camera shooting of object that millimetre-wave radar detects Difference between the speed for the object that head detects thinks first object point and the second target less than the second preset threshold at this time Point is same object.
The device of the present embodiment can be used for executing the technical solution of embodiment of the method shown in Fig. 7, realization principle and skill Art effect is similar, and details are not described herein again.
In order to improve accuracy of judgement degree and efficiency, determining module 23 is preferably included:
Judging submodule judges the target that the directional velocity for the object that millimetre-wave radar detects and camera detect Whether the directional velocity of object is identical whithin a period of time, if different, directly determines first object point and the second target point It is not same object.
Further, the calculation formula of the second preset threshold △ v is as follows:
va≤ 30km/h, △ v=0.1va
30km/h<va≤ 80km/h, △ v=0.09va
80km/h<va≤ 100km/h, △ v=0.085va
100km/h<va≤ 120km/h, △ v=0.075va
va> 120km/h, △ v=0.07va
Wherein, vaIt is the speed for the object that millimetre-wave radar detects.
In this way, passing through the target relative velocity v for detecting the second preset threshold △ v according to millimetre-wave radaraNot same order Ladder type changes value, and after measured, the case where relative to the second preset threshold is set as fixed value, the present embodiment is to object one The False Rate of cause property has significant decrease.
The embodiment of the present invention also provides a kind of electronic equipment, as shown in figure 11, may be implemented shown in Fig. 1,3 and 7 of the present invention The process of embodiment of the method, the electronic equipment may include: shell 41, processor 42, memory 43, circuit board 44 and power supply electricity Road 45, wherein circuit board 44 is placed in the space interior that shell 41 surrounds, and processor 42 and memory 43 are arranged in circuit board 44 On;Power circuit 45, for each circuit or the device power supply for above-mentioned electronic equipment;Memory 43 is for storing executable journey Sequence code;Processor 42 is run and executable program code pair by reading the executable program code stored in memory 43 The program answered, for executing aforementioned either method embodiment method.
Processor 42 to the specific implementation procedures of above-mentioned steps and processor 42 by operation executable program code come The step of further executing may refer to the description of embodiment of the method shown in Fig. 1,3 and 7 of the present invention, and details are not described herein.
The electronic equipment exists in a variety of forms, including but not limited to:
(1) mobile communication equipment: the characteristics of this kind of equipment is that have mobile communication function, and to provide speech, data Communication is main target.This Terminal Type includes: smart phone (such as iPhone), multimedia handset, functional mobile phone and low Hold mobile phone etc..
(2) super mobile personal computer equipment: this kind of equipment belongs to the scope of personal computer, there is calculating and processing function Can, generally also have mobile Internet access characteristic.This Terminal Type includes: PDA, MID and UMPC equipment etc., such as iPad.
(3) portable entertainment device: this kind of equipment can show and play multimedia content.Such equipment include: audio, Video player (such as iPod), handheld device, e-book and intelligent toy and portable car-mounted navigation equipment.
(4) server: providing the equipment of the service of calculating, and the composition of server includes that processor, hard disk, memory, system are total Line etc., server is similar with general computer architecture, but due to needing to provide highly reliable service, in processing energy Power, stability, reliability, safety, scalability, manageability etc. are more demanding.
(5) other electronic equipments with data interaction function.
The embodiment of the present invention also provides a kind of computer readable storage medium, and meter is stored in computer readable storage medium Calculation machine program realizes any of the above-described embodiment of the method method and step when computer program is executed by processor.
The above is merely preferred embodiments of the present invention, be not intended to limit the invention, it is all in spirit of the invention and Within principle, any modification, equivalent replacement, improvement and so on be should all be included in the protection scope of the present invention.

Claims (10)

1. a kind of subject fusion method of vehicle millimetre-wave radar and camera, it is characterised in that:
The data information of first object point and the second target point is obtained, the data information includes location information, first mesh Punctuate represents the object that millimetre-wave radar sensor detects, and second target point represents the target that camera detects Object;
The distance between the first object point and described second target point are determined less than the first preset threshold, thinks described at this time First object point and second target point are same objects;
The classification and cross for the object that the distance for the object that millimetre-wave radar detects and speed, camera are detected Target level data information to position grouping as the object.
2. a kind of subject fusion method of vehicle millimetre-wave radar and camera according to claim 1, it is characterised in that: Determine that the distance between the first object point and described second target point less than the first preset threshold, think described first at this time Target point and second target point include: in the first object point and second target point before being same object Each extended range of four direction up and down;The first object point and second target point after determining extension, which exist, to be handed over Collection, thinks the first object point at this time and second target point is same object.
3. a kind of subject fusion method of vehicle millimetre-wave radar and camera according to claim 2, it is characterised in that: When the type for the object that camera detects is vehicle, in the width side of the first object point and second target point It is the half of vehicle width to the distance of extension, the distance of length direction extension is the half of Vehicle length;When camera detects When the type of the object arrived is single pedestrian, in four sides up and down of the first object point and second target point It is 0.3-0.5m to the distance of extension;When the type for the object that camera detects is two wheeler, in first mesh The distance of the four direction up and down of punctuate and second target point extension is 0.4-0.6m.
4. a kind of subject fusion method of vehicle millimetre-wave radar and camera according to claim 2, it is characterised in that: There are intersections for the first object point and second target point after determining extension, it is believed that the first object point and described the If two target points are that same object includes: the speed for the object that millimetre-wave radar detects and the mesh that camera detects The difference between the speed of object is marked less than the second preset threshold, the first object point is thought at this time and second target point is Same object;If the speed for the object that the directional velocity and camera of the object that millimetre-wave radar detects detect Direction is not identical within a limited period of time, then directly determines the first object point and second target point is not same target Object.
5. a kind of subject fusion method of vehicle millimetre-wave radar and camera according to claim 4, it is characterised in that: The calculation formula of the second preset threshold △ v is as follows:
va≤ 30km/h, △ v=0.1va
30km/h<va≤ 80km/h, △ v=0.09va
80km/h<va≤ 100km/h, △ v=0.085va
100km/h<va≤ 120km/h, △ v=0.075va
va> 120km/h, △ v=0.07va
Wherein, vaIt is the speed for the object that millimetre-wave radar detects.
6. the subject fusion device of a kind of vehicle millimetre-wave radar and camera, it is characterised in that: include:
Module is obtained, for obtaining the data information of first object point and the second target point, wherein the data information includes position Confidence breath, the first object point represent the object that millimetre-wave radar sensor detects, and the second target point representative is taken the photograph The object detected as head;
Determining module, for determine the distance between the first object point and second target point less than the first preset threshold, this When think first object point and the second target point is same object;
Composite module, the mesh that the distance and speed of the object for detecting millimetre-wave radar, camera detect The classification and lateral position of marking object combine the target level data information as the object.
7. the subject fusion device of a kind of vehicle millimetre-wave radar and camera according to claim 6, it is characterised in that:
Expansion module, for the four direction up and down of the first object point and the second target point respectively extend it is certain away from From;
The determining module, there are intersections for the first object point and the second target point for being further used for determining after extending, and recognize at this time It is same object for first object point and the second target point.
8. the subject fusion device of a kind of vehicle millimetre-wave radar and camera according to claim 7, it is characterised in that: Expansion module is when the type for the object that camera detects is vehicle, in the width of the first object point and the second target point The distance for spending Directional Extension is the half of vehicle width, and the distance of length direction extension is the half of Vehicle length;Work as camera When the type of the object detected is single pedestrian, in four sides up and down of the first object point and the second target point It is 0.3-0.5m to the distance of extension;When the type for the object that camera detects is two wheeler, in first mesh The distance of the four direction up and down of punctuate and the second target point extension is 0.4-0.6m;
The determining module, there are intersections for the first object point and the second target point for being further used for determining after extending, and in the least Difference between the speed for the object that the speed and camera for the object that metre wave radar detects detect is pre- less than second If threshold value, first object point is thought at this time and the second target point is same object;
The determining determining module includes judging submodule, judges the directional velocity for the object that millimetre-wave radar detects and takes the photograph Whether the directional velocity of the object detected as head is identical whithin a period of time, if different, directly determines the first mesh Punctuate and the second target point are not same objects;
The calculation formula of the second preset threshold △ v is as follows:
va≤ 30km/h, △ v=0.1va
30km/h<va≤ 80km/h, △ v=0.09va
80km/h<va≤ 100km/h, △ v=0.085va
100km/h<va≤ 120km/h, △ v=0.075va
va> 120km/h, △ v=0.07va
Wherein, vaIt is the speed for the object that millimetre-wave radar detects.
9. a kind of electronic equipment characterized by comprising shell, processor, memory, circuit board and power circuit, wherein Circuit board is placed in the space interior that shell surrounds, and processor and memory setting are on circuit boards;Power circuit, for being State each circuit or the device power supply of electronic equipment;Memory is for storing executable program code;Processor is deposited by reading The executable program code stored in reservoir runs program corresponding with executable program code, wants for executing aforementioned right Seek method described in any claim in 1-5.
10. a kind of computer readable storage medium, which is characterized in that the computer-readable recording medium storage have one or Multiple programs, one or more of programs can be executed by one or more processor, to realize preceding claims 1-5 Method described in middle any claim.
CN201910075572.9A 2019-01-25 2019-01-25 A kind of subject fusion method and apparatus of vehicle millimetre-wave radar and camera Pending CN109901156A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910075572.9A CN109901156A (en) 2019-01-25 2019-01-25 A kind of subject fusion method and apparatus of vehicle millimetre-wave radar and camera

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910075572.9A CN109901156A (en) 2019-01-25 2019-01-25 A kind of subject fusion method and apparatus of vehicle millimetre-wave radar and camera

Publications (1)

Publication Number Publication Date
CN109901156A true CN109901156A (en) 2019-06-18

Family

ID=66944144

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910075572.9A Pending CN109901156A (en) 2019-01-25 2019-01-25 A kind of subject fusion method and apparatus of vehicle millimetre-wave radar and camera

Country Status (1)

Country Link
CN (1) CN109901156A (en)

Cited By (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110456321A (en) * 2019-08-21 2019-11-15 森思泰克河北科技有限公司 Filter out method, terminal device and the storage medium of radar wrong report
CN110865367A (en) * 2019-11-30 2020-03-06 山西禾源科技股份有限公司 Intelligent fusion method for radar video data
CN111191697A (en) * 2019-12-21 2020-05-22 武汉光庭信息技术股份有限公司 ADAS road test verification optimization method and device based on sensor fusion
CN111289969A (en) * 2020-03-27 2020-06-16 北京润科通用技术有限公司 Vehicle-mounted radar moving target fusion method and device
CN111566505A (en) * 2019-06-28 2020-08-21 深圳市大疆创新科技有限公司 Calibration method and system for installation angle of millimeter wave radar component and movable platform
CN111695619A (en) * 2020-06-05 2020-09-22 中国第一汽车股份有限公司 Multi-sensor target fusion method and device, vehicle and storage medium
CN111796286A (en) * 2020-06-24 2020-10-20 中国第一汽车股份有限公司 Brake grade evaluation method and device, vehicle and storage medium
CN112162275A (en) * 2020-09-29 2021-01-01 安徽江淮汽车集团股份有限公司 Target object identification method, device, equipment and storage medium
CN112837527A (en) * 2019-11-22 2021-05-25 罗伯特·博世有限公司 Object recognition system and method thereof
CN112882048A (en) * 2021-01-11 2021-06-01 广州合智瑞达科技有限公司 Radar device for high-precision distance measurement of moving object and measurement method thereof
CN113325415A (en) * 2021-04-20 2021-08-31 武汉光庭信息技术股份有限公司 Fusion method and system for vehicle radar data and camera data
CN113386742A (en) * 2021-05-28 2021-09-14 晋城鸿智纳米光机电研究院有限公司 Brake auxiliary system, method and automobile
CN113567973A (en) * 2021-06-02 2021-10-29 北京木牛领航科技有限公司 Target monitoring system, method, warning system and storage medium
CN113611112A (en) * 2021-07-29 2021-11-05 中国第一汽车股份有限公司 Target association method, device, equipment and storage medium
CN114624711A (en) * 2020-12-11 2022-06-14 中寰卫星导航通信有限公司 Fusion method and device for different radar information, storage medium and equipment
CN115436899A (en) * 2022-08-31 2022-12-06 中国第一汽车股份有限公司 Method, device, equipment and storage medium for processing millimeter wave radar detection data
CN115453504A (en) * 2022-08-02 2022-12-09 珠海云洲智能科技股份有限公司 Target detection method, target detection device and electronic equipment

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102303605A (en) * 2011-06-30 2012-01-04 中国汽车技术研究中心 Multi-sensor information fusion-based collision and departure pre-warning device and method
CN102542843A (en) * 2010-12-07 2012-07-04 比亚迪股份有限公司 Early warning method for preventing vehicle collision and device
CN105522998A (en) * 2014-10-15 2016-04-27 本田技研工业株式会社 Object recognition apparatus
CN106842188A (en) * 2016-12-27 2017-06-13 上海思致汽车工程技术有限公司 A kind of object detection fusing device and method based on multisensor
CN109031302A (en) * 2017-04-12 2018-12-18 福特全球技术公司 Analyze the method and apparatus of vehicle environmental and the vehicle equipped with this device
CN109073778A (en) * 2016-04-22 2018-12-21 株式会社电装 Article detection device, object detecting method

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102542843A (en) * 2010-12-07 2012-07-04 比亚迪股份有限公司 Early warning method for preventing vehicle collision and device
CN102303605A (en) * 2011-06-30 2012-01-04 中国汽车技术研究中心 Multi-sensor information fusion-based collision and departure pre-warning device and method
CN105522998A (en) * 2014-10-15 2016-04-27 本田技研工业株式会社 Object recognition apparatus
CN109073778A (en) * 2016-04-22 2018-12-21 株式会社电装 Article detection device, object detecting method
CN106842188A (en) * 2016-12-27 2017-06-13 上海思致汽车工程技术有限公司 A kind of object detection fusing device and method based on multisensor
CN109031302A (en) * 2017-04-12 2018-12-18 福特全球技术公司 Analyze the method and apparatus of vehicle environmental and the vehicle equipped with this device

Cited By (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111566505A (en) * 2019-06-28 2020-08-21 深圳市大疆创新科技有限公司 Calibration method and system for installation angle of millimeter wave radar component and movable platform
CN110456321B (en) * 2019-08-21 2021-07-30 森思泰克河北科技有限公司 Method for filtering false alarm of radar, terminal equipment and storage medium
CN110456321A (en) * 2019-08-21 2019-11-15 森思泰克河北科技有限公司 Filter out method, terminal device and the storage medium of radar wrong report
CN112837527A (en) * 2019-11-22 2021-05-25 罗伯特·博世有限公司 Object recognition system and method thereof
CN110865367A (en) * 2019-11-30 2020-03-06 山西禾源科技股份有限公司 Intelligent fusion method for radar video data
CN110865367B (en) * 2019-11-30 2023-05-05 山西禾源科技股份有限公司 Intelligent radar video data fusion method
CN111191697A (en) * 2019-12-21 2020-05-22 武汉光庭信息技术股份有限公司 ADAS road test verification optimization method and device based on sensor fusion
CN111191697B (en) * 2019-12-21 2023-04-28 武汉光庭信息技术股份有限公司 ADAS road test verification optimization method and device based on sensor fusion
CN111289969A (en) * 2020-03-27 2020-06-16 北京润科通用技术有限公司 Vehicle-mounted radar moving target fusion method and device
CN111289969B (en) * 2020-03-27 2022-03-04 北京润科通用技术有限公司 Vehicle-mounted radar moving target fusion method and device
CN111695619A (en) * 2020-06-05 2020-09-22 中国第一汽车股份有限公司 Multi-sensor target fusion method and device, vehicle and storage medium
WO2021259260A1 (en) * 2020-06-24 2021-12-30 中国第一汽车股份有限公司 Braking level assessment method and apparatus, vehicle, and storage medium
CN111796286A (en) * 2020-06-24 2020-10-20 中国第一汽车股份有限公司 Brake grade evaluation method and device, vehicle and storage medium
CN111796286B (en) * 2020-06-24 2023-01-17 中国第一汽车股份有限公司 Brake grade evaluation method and device, vehicle and storage medium
CN112162275B (en) * 2020-09-29 2024-05-14 安徽江淮汽车集团股份有限公司 Target object identification method, device, equipment and storage medium
CN112162275A (en) * 2020-09-29 2021-01-01 安徽江淮汽车集团股份有限公司 Target object identification method, device, equipment and storage medium
CN114624711B (en) * 2020-12-11 2024-05-17 中寰卫星导航通信有限公司 Fusion method, device, storage medium and equipment of different radar information
CN114624711A (en) * 2020-12-11 2022-06-14 中寰卫星导航通信有限公司 Fusion method and device for different radar information, storage medium and equipment
CN112882048A (en) * 2021-01-11 2021-06-01 广州合智瑞达科技有限公司 Radar device for high-precision distance measurement of moving object and measurement method thereof
CN113325415A (en) * 2021-04-20 2021-08-31 武汉光庭信息技术股份有限公司 Fusion method and system for vehicle radar data and camera data
CN113325415B (en) * 2021-04-20 2023-10-13 武汉光庭信息技术股份有限公司 Fusion method and system of vehicle radar data and camera data
CN113386742A (en) * 2021-05-28 2021-09-14 晋城鸿智纳米光机电研究院有限公司 Brake auxiliary system, method and automobile
CN113567973A (en) * 2021-06-02 2021-10-29 北京木牛领航科技有限公司 Target monitoring system, method, warning system and storage medium
CN113611112A (en) * 2021-07-29 2021-11-05 中国第一汽车股份有限公司 Target association method, device, equipment and storage medium
CN115453504A (en) * 2022-08-02 2022-12-09 珠海云洲智能科技股份有限公司 Target detection method, target detection device and electronic equipment
CN115436899A (en) * 2022-08-31 2022-12-06 中国第一汽车股份有限公司 Method, device, equipment and storage medium for processing millimeter wave radar detection data

Similar Documents

Publication Publication Date Title
CN109901156A (en) A kind of subject fusion method and apparatus of vehicle millimetre-wave radar and camera
CN109886308A (en) One kind being based on the other dual sensor data fusion method of target level and device
US10867189B2 (en) Systems and methods for lane-marker detection
CN102208012B (en) Landscape coupling reference data generation system and position measuring system
CN101447019B (en) Image recognition apparatuses and methods
CN104296756B (en) Run the method and motor vehicle of motor vehicle
CN110136058B (en) Drawing construction method based on overlook spliced drawing and vehicle-mounted terminal
CN103927508A (en) Target vehicle tracking method and device
CN106407315A (en) Vehicle self-positioning method based on street view image database
CN111179162B (en) Positioning initialization method under special environment and vehicle-mounted terminal
CN107545770A (en) A kind of air navigation aid, apparatus and system
US11288887B2 (en) Object tracking method and apparatus
CN103208008A (en) Fast adaptation method for traffic video monitoring target detection based on machine vision
CN103150786A (en) Non-contact type unmanned vehicle driving state measuring system and measuring method
CN102831766A (en) Multi-source traffic data fusion method based on multiple sensors
CN106569245A (en) Vehicle positioning method and device
US11238735B2 (en) Parking lot information management system, parking lot guidance system, parking lot information management program, and parking lot guidance program
CN116734828A (en) Determination of road topology information, electronic map data processing method and electronic equipment
CN110659548A (en) Vehicle and target detection method and device thereof
CN107193820A (en) Location information acquisition method, device and equipment
CN112036427A (en) Simulation of realistic sensor fusion detection estimation with objects
CN103196440B (en) M sequence discrete-type artificial signpost arrangement method and related mobile robot positioning method
CN111160132A (en) Method and device for determining lane where obstacle is located, electronic equipment and storage medium
CN113985465A (en) Sensor fusion positioning method and system, readable storage medium and computer equipment
CN103903253A (en) Mobile terminal positioning method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20190618

RJ01 Rejection of invention patent application after publication