CN103559791A - Vehicle detection method fusing radar and CCD camera signals - Google Patents

Abstract

The invention discloses a vehicle detection method fusing radar and CCD camera signals. The method includes the steps of inputting the radar and CCD camera signals, correcting a camera to obtain a projection matrix of a road plane coordinate and a projection matrix of an image coordinate, converting a road plane world coordinate into an image plane coordinate, building positive and negative sample sets suitable for a vehicle HOG describer, carrying out batch feature extraction on the vehicle sample sets to build an HOG sample set, building an SVM classification model of a linear support vector machine to train an SVM, extracting regions of interest of barriers detected by radar in a video image, inputting the regions of interest into an SVM classifier to judge a target category, outputting an identifying result, and measuring the distance of a target which is judged to be a vehicle by means of the radar. Combination detection is carried out by means of the radar and CCD camera signals, depth information of the vehicle is obtained, meanwhile, profile information of the vehicle can be well detected, and reliability and accuracy of vehicle detection and positioning are improved.

Description

A kind of vehicle checking method that merges radar and ccd video camera signal

Technical field

The invention belongs to computer vision field, relate to a kind of intelligent vehicle front vehicles detection method based on multi-sensor information fusion technology, particularly a kind of vehicle checking method that merges radar and ccd video camera signal.

Background technology

In intelligent driving technology, the environmental detection sensor of main flow is inertial navigation, laser radar, millimetre-wave radar, infrared and colourful CCD video camera etc.

Adopt radar can obtain quickly and accurately the vehicle distances information on the two-dimensional level face of intelligent vehicle the place ahead, the frequency of operation of radar is high simultaneously, and the range data precision of measurement is high, and price is relatively cheap, can meet the real-time of vehicle detection.But it is less that radar obtains the quantity of information of vehicle, use separately radar can only detect the depth information on the vehicle plane of scanning motion.

Video sensor can provide two-dimensional visible light image, by the known obvious characteristic of certain objects (vehicle) (as characteristic informations such as edge, angle point, texture, position and shapes) is detected to judgement, under some specific conditions, can effectively detect vehicle.Use video sensor to detect vehicle and conventionally comprise three processes: first from image, determining target object, then to object classification identification, is finally to follow the tracks of vehicle.The shortcoming of video sensor is the range information that cannot obtain vehicle.

At actual intelligent vehicle, detect in application, use separately a certain detecting sensor to be all difficult to complete vehicle detection and location comprehensively and accurately, so be necessary to utilize the data of multiple sensors to merge, realize and have complementary advantages to improve reliability, the accuracy of vehicle detection and location.

Summary of the invention

For a kind of detecting sensor of independent use existing in prior art, be difficult to complete vehicle detection and orientation problem comprehensively and accurately, the present invention proposes a kind of vehicle checking method that merges radar and ccd video camera signal.

Multisensor of the present invention comprises single line laser radar and video sensor.Single line laser radar can obtain the range information of vehicle on the two-dimensional level face of intelligent vehicle the place ahead quickly and accurately; Video sensor can provide the two-dimensional visible light image of vehicle, can detect the type information of target vehicle according to image.

A vehicle checking method that merges radar and ccd video camera signal, comprises the following steps:

Step 1, input is from the road obstacle information signal of radar with from the road horizontal image signal of ccd video camera.

Step 2, correcting video sensor camera, the projection matrix ，Jiang road plane world coordinates that obtains road planimetric coordinates and image coordinate converts plane of delineation coordinate to.Single line laser radar road planimetric coordinates is converted to the reference frame coordinate at selected scaling board place, obtain the image coordinate corresponding to vehicle coordinate of radar monitoring Dao road plane.

Step 3, set up to be applicable to vehicle HOG(Histogram of Oriented Gradient, gradient orientation histogram) the positive and negative sample set of feature describer.

HOG feature describer is to be applied to computer vision and image processing field, for the feature describer of target detection.Utilize HOG to calculate the statistical value of the directional information of topography's gradient.

Step 4, adopts HOG algorithm to carry out feature extraction in batches to vehicle sample set, thereby sets up HOG feature samples collection.

Step 5, sets up linear SVM svm classifier model, and use characteristic sample set is trained SVM.

Step 6, extract detections of radar to the region of barrier in video image, and carry out HOG feature extraction, in the svm classifier device that input training obtains, carry out target type judgement.

Step 7, the target object recognition result of output svm classifier device.

Step 8, intelligent vehicle the place ahead that output single line laser radar is measured is judged as the distance of the target of vehicle.

Beneficial effect of the present invention is mainly manifested in: the range information of the image information of the vehicle that utilization is obtained by video sensor and the vehicle being recorded by single line laser radar carries out joint-detection, not only obtained the depth information of vehicle, also can detect preferably the profile information of vehicle, realize and have complementary advantages to improve reliability and the accuracy of vehicle detection, location simultaneously.

Accompanying drawing explanation

Fig. 1 is for merging the vehicle checking method process flow diagram of radar and ccd video camera signal;

Fig. 2 is the conversion schematic diagram that world coordinates is tied to camera coordinate system;

Fig. 3 is the conversion schematic diagram that camera coordinate is tied to image coordinate system;

Fig. 4 is the horizontal vertical gradient schematic diagram of vehicle image.

Embodiment

Below in conjunction with drawings and Examples, the present invention will be further described.

Vehicle checking method of the present invention is realized by the software being stored in computer, and computer is arranged in vehicle trunk.Radar horizon is fixed on automobile leading portion car plate position, and ccd video camera is arranged on room mirror position.

Fig. 1, for merging the vehicle checking method process flow diagram of radar and ccd video camera signal, comprises the following steps:

Step 1, input radar and ccd video camera signal.

Step 2, carries out camera correction (internal reference and the outer ginseng that comprise camera), and the projection matrix ，Jiang road plane world coordinates that obtains road planimetric coordinates and image coordinate converts plane of delineation coordinate to.

Point in world coordinate system is realized in two steps to the projection process of the point in image coordinate:

(1) by the coordinate (X of world coordinate system (or reference frame) mid point _w, Y _w, Z _w) transform to camera coordinate system (X _c, Y _c, Z _c).As shown in Figure 2, transformation for mula is conversion process:

$\left[\begin{array}{c}{X}_C\\ Y_C\\ Z_C\\ 1\end{array}\right]=\left[\begin{array}{cccc}{r}_{11}& r_{12}& r_{13}& t_1\\ r_{21}& r_{22}& r_{23}& t_2\\ r_{31}& r_{32}& r_{32}& t_3\\ 0& 0& 0& 1\end{array}\right]\left[\begin{array}{c}{X}_W\\ Y_W\\ Z_W\\ 1\end{array}\right]$

In formula, $\left[\begin{array}{ccc}{r}_{11}& r_{12}& r_{13}\\ r_{21}& r_{22}& r_{23}\\ r_{31}& r_{32}& r_{33}\end{array}\right]$ For be tied to the rotation matrix of camera coordinate system by world coordinates, $\left[\begin{array}{c}{t}_1\\ t_2\\ t_3\end{array}\right]$ For be tied to the translation matrix of camera coordinate system by world coordinates.

(2) by camera coordinate system, transform to image coordinate system, conversion process is as Fig. 3, and transformation for mula is:

$s\left[\begin{array}{c}x\\ y\\ 1\end{array}\right]=\left[\begin{array}{ccc}{f}_x& 0& u_0\\ 0& f_y& v_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{c}{X}_C\\ Y_C\\ Z_C\end{array}\right]$

In formula, f _x, f _yhorizontal direction and the vertical direction focal length that pixel is unit take in representative, u ₀, v ₀the horizontal stroke, the ordinate that represent respectively principal point (intersection point of video camera main shaft and the plane of delineation), s is projective parameter (is a procedure parameter, is cancelled in computation process).

Point in world coordinate system to the projection formula of the point in image coordinate is:

$s\left[\begin{array}{c}x\\ y\\ 1\end{array}\right]=\left[\begin{array}{ccc}{f}_x& 0& u_0\\ 0& f_y& v_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}{r}_{11}& r_{12}& r_{13}& t_1\\ r_{21}& r_{22}& r_{23}& t_2\\ r_{31}& r_{32}& r_{33}& t_3\end{array}\right]\left[\left[\begin{array}{c}{X}_W\\ Y_W\\ Z_W\\ 1\end{array}\right]\right]$

Select the coordinate system that defines in gridiron pattern as with reference to coordinate system, corresponding rigid body translation is set up at each visual angle, by given camera intrinsic parameter, obtains the initial value of solution procedure, makes the internal reference of the camera of trying to achieve make re-projection error minimum as far as possible.When calibrating the intrinsic parameter of camera, the gridiron pattern coordinate system of the picture of a scaling board in last selectively plane is as with reference to coordinate system.

By radar is converted between road plane coordinate system and the coordinate system of selected scaling board, obtain the transition matrix of the vehicle coordinate of radar monitoring Dao road plane and transition matrix, height of car and the image coordinate of image coordinate, thereby can determine the height of the vehicle of the position of vehicle on the plane of delineation and position.

Step 3, sets up the positive and negative sample set that is applicable to vehicle HOG feature describer.

The vehicle Sample Storehouse that the present embodiment adopts CaltechGraz to provide, the size of vehicle is all the HOG feature of 64 * 64 batch extracting training samples, the characteristic of HOG is from CCD.

Step 4, adopts HOG algorithm to carry out batch feature extraction to vehicle sample set, thereby sets up HOG feature samples collection.

HOG feature is for the intensity statistics on the gradient direction in rectangular area.The horizontal vertical gradient schematic diagram of vehicle as shown in Figure 4.

Adopting car modal size is 64*64, the block piece that template samples is divided into 16*16 size, if the height of block is H, wide is W, the present invention adopts H:W=1:1 block feature extracting method: each block piece is divided into 4 identical cell unit, the size of each cell unit is 8*8, and the feature of each unit is the proper vector sum of its inner 64 pixels.

With I (x, y) presentation video, I locates the gray-scale value of pixel at (x, y), is calculated as follows the intensity statistics feature of the gradient direction in rectangular area:

G _x(x,y)＝I(x+1,y)-I(x-1,y)

G _y(x,y)＝I(x,y+1)-I(x,y-1)

$G(x,y)=\sqrt{{G^2}_x(x,y)+{G^2}_y(x,y)}$

α(x,y)＝arctan(G _y(x,y),G _x(x,y))

Wherein, G _x(x, y), G _y(x, y) represents that respectively (x, y) locates the gradient magnitude of horizontal direction and the vertical direction of pixel, and G (x, y) is the gradient intensity that (x, y) locates pixel, and α (x, y) represents that (x, y) locates the gradient direction of pixel.

HOG feature will

gradient direction be evenly divided into 9 bin (interval), the gradient magnitude of k direction size A _k(x, y) is:

$A_k(x,y)=\left\{\begin{array}{cc}G(x,y),& \mathrm{\α}(x,y)\∈{\mathrm{bin}}_k\\ 0,& \mathrm{others}\end{array}\right.,1\≤k\≤9$

Wherein, bin _k(x, y) represents k Direction interval of gradient direction.Like this, (x, y) locate Gradient Features in each direction of pixel can be with the vectorial A of one 9 dimension _k(x, y) represents.

In order to eliminate the factor impacts such as illumination, each unit in piece is normalized:

$f(c_m,k)=\frac{{\mathrm{\Σ}}_{(x,y)\∈c_m}{A}_k(x,y)+\mathrm{\ϵ}}{{\mathrm{\Σ}}_{(x,y)\∈B}{A}_k(x,y)+\mathrm{\ϵ}},m=\mathrm{1,2,3,4}$

Wherein, f (c _m, k) represent m unit c _min k interval normalized intensity, ε is to be the zero less number arranging for fear of denominator.

By f (c _m, expression formula k) is known, the proper vector that each unit extracts is 9 dimensions, each piece be characterized as 36 dimensional vectors that the feature cascade in 4 cell unit is obtained.

Step 5, sets up Linear SVM sorter, uses the feature samples training in step 4 to practice svm classifier device.

Step 6, target type judgement.The positional information of the vehicle that radar is obtained in the plane of road converts image coordinate information to by matrixing, and target area image is carried out to HOG feature extraction, and the svm classifier device that adopts step 5 training to obtain is predicted, judges whether target is vehicle.

Step 7, the target object recognition result of output svm classifier device.

Step 8, intelligent vehicle the place ahead that output single line laser radar is measured is judged as the distance of the target of vehicle.

Claims (1)

1. a vehicle checking method that merges radar and ccd video camera signal, is characterized in that comprising the following steps:

Step 1, input is from the road obstacle information signal of radar with from the road horizontal image signal of ccd video camera;

Step 2, comprises the internal reference of camera and the correction of the camera of outer ginseng, and the projection matrix ，Jiang road plane world coordinates that obtains road planimetric coordinates and image coordinate converts plane of delineation coordinate to;

Point in world coordinate system is realized in two steps to the projection process of the point in image coordinate:

(1) by the coordinate (X of world coordinate system (or reference frame) mid point _w, Y _w, Z _w) transform to camera coordinate system (X _c, Y _c, Z _c), transformation for mula is:

$\left[\begin{array}{c}{X}_C\\ Y_C\\ Z_C\\ 1\end{array}\right]=\left[\begin{array}{cccc}{r}_{11}& r_{12}& r_{13}& t_1\\ r_{21}& r_{22}& r_{23}& t_2\\ r_{31}& r_{32}& r_{32}& t_3\\ 0& 0& 0& 1\end{array}\right]\left[\begin{array}{c}{X}_W\\ Y_W\\ Z_W\\ 1\end{array}\right]$

In formula, $\left[\begin{array}{ccc}{r}_{11}& r_{12}& r_{13}\\ r_{21}& r_{22}& r_{23}\\ r_{31}& r_{32}& r_{33}\end{array}\right]$ For be tied to the rotation matrix of camera coordinate system by world coordinates, $\left[\begin{array}{c}{t}_1\\ t_2\\ t_3\end{array}\right]$ For be tied to the translation matrix of camera coordinate system by world coordinates;

(2) by camera coordinate system, transform to image coordinate system, transformation for mula is:

$s\left[\begin{array}{c}x\\ y\\ 1\end{array}\right]=\left[\begin{array}{ccc}{f}_x& 0& u_0\\ 0& f_y& v_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{c}{X}_C\\ Y_C\\ Z_C\end{array}\right]$

In formula, f _x, f _yhorizontal direction and the vertical direction focal length that pixel is unit take in representative, u ₀, v ₀the horizontal stroke, the ordinate that represent respectively the intersection point of video camera main shaft and the plane of delineation, s is projective parameter;

Point in world coordinate system to the projection formula of the point in image coordinate is:

$s\left[\begin{array}{c}x\\ y\\ 1\end{array}\right]=\left[\begin{array}{ccc}{f}_x& 0& u_0\\ 0& f_y& v_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}{r}_{11}& r_{12}& r_{13}& t_1\\ r_{21}& r_{22}& r_{23}& t_2\\ r_{31}& r_{32}& r_{33}& t_3\end{array}\right]\left[\left[\begin{array}{c}{X}_W\\ Y_W\\ Z_W\\ 1\end{array}\right]\right]$

Select the coordinate system that defines in gridiron pattern as with reference to coordinate system, corresponding rigid body translation is set up at each visual angle, by given camera intrinsic parameter, obtains the initial value of solution procedure, makes the internal reference of the camera of trying to achieve make re-projection error minimum as far as possible; When calibrating the intrinsic parameter of camera, the gridiron pattern coordinate system of the picture of a scaling board in last selectively plane is as with reference to coordinate system;

By radar is converted between road plane coordinate system and the coordinate system of selected scaling board, obtain the transition matrix of the vehicle coordinate of radar monitoring Dao road plane and transition matrix, height of car and the image coordinate of image coordinate, thereby can determine the height of the vehicle of the position of vehicle on the plane of delineation and position;

Step 3, sets up the positive and negative sample set that is applicable to vehicle HOG feature describer;

Step 4, adopts HOG algorithm to carry out batch feature extraction to vehicle sample set, thereby sets up HOG feature samples collection;

HOG feature is for the intensity statistics on the gradient direction in rectangular area;

Adopting pedestrian's template size is 64*128, the block piece that pedestrian's template samples is divided into 16*16 size, if the height of block is H, wide is W, the present invention adopts H:W=1:1 block feature extracting method: each block piece is divided into 4 identical cell unit, the size of each cell unit is 8*8, and the feature of each unit is the proper vector sum of its inner 64 pixels;

With I (x, y) presentation video, I locates the gray-scale value of pixel at (x, y), is calculated as follows the intensity statistics feature of the gradient direction in rectangular area:

G _x(x,y)＝I(x+1,y)-I(x-1,y)

G _y(x,y)＝I(x,y+1)-I(x,y-1)

$G(x,y)=\sqrt{{G^2}_x(x,y)+{G^2}_y(x,y)}$

α(x,y)＝arctan(G _y(x,y),G _x(x,y))

Wherein, G _x(x, y), G _y(x, y) represents that respectively (x, y) locates the gradient magnitude of horizontal direction and the vertical direction of pixel, and G (x, y) is the gradient intensity that (x, y) locates pixel, and α (x, y) represents that (x, y) locates the gradient direction of pixel;

HOG feature will

gradient direction be evenly divided into 9 bin (interval), the gradient magnitude of k direction size A _k(x, y) is:

$A_k(x,y)=\left\{\begin{array}{cc}G(x,y),& \mathrm{\α}(x,y)\∈{\mathrm{bin}}_k\\ 0,& \mathrm{others}\end{array}\right.,1\≤k\≤9$

Wherein, bin _k(x, y) represents k Direction interval of gradient direction; Like this, (x, y) locate Gradient Features in each direction of pixel can be with the vectorial A of one 9 dimension _k(x, y) represents;

In order to eliminate the factor impacts such as illumination, each unit in piece is normalized:

$f(c_m,k)=\frac{{\mathrm{\Σ}}_{(x,y)\∈c_m}{A}_k(x,y)+\mathrm{\ϵ}}{{\mathrm{\Σ}}_{(x,y)\∈B}{A}_k(x,y)+\mathrm{\ϵ}},m=\mathrm{1,2,3,4}$

Wherein, f (c _m, k) represent m unit c _min k interval normalized intensity, ε is to be the zero less number arranging for fear of denominator;

By f (c _m, expression formula k) is known, the proper vector that each unit extracts is 9 dimensions, each piece be characterized as 36 dimensional vectors that the feature cascade in 4 cell unit is obtained;

Step 5, sets up Linear SVM sorter, uses the feature samples training in step 4 to practice svm classifier device;

Step 6, target type judgement; The positional information of the vehicle that radar is obtained in the plane of road converts image coordinate information to by matrixing, and region of interest area image is carried out to feature extraction, and the svm classifier device that adopts step 5 training to obtain is predicted, judges whether target is vehicle;

Step 7, the target object recognition result of output svm classifier device;

Step 8, utilizes single line laser radar to measure the distance that intelligent vehicle the place ahead is judged as the target of vehicle.

Images