CN115164827A - Monocular distance measurement method based on self-adaptive target detection network and license plate detection - Google Patents

Abstract

The invention relates to a monocular distance measurement method and a monocular distance measurement system based on a self-adaptive target detection network and license plate detection, wherein the method comprises the following steps: s1: acquiring a forward-looking traffic image of a vehicle by using a camera, identifying the shape of a license plate in the forward-looking traffic image by using a license plate detection algorithm, and obtaining a depression gamma of the camera by using a perspective transformation principle; s2: inputting the forward-looking traffic image into a target detection network to obtain a lower edge central point P of a predicted target detection frame, and predicting to obtain a target distance by utilizing gamma and P; s3: and constructing a loss function by using the target distance and the real distance for training the target detection network. The method provided by the invention can directly detect the acquired image and measure the distance of the target, thereby avoiding the steps of image fusion and the like and reducing the manufacturing cost.

Description

A Monocular Ranging Method Based on Adaptive Target Detection Network and License Plate Detection

technical field

The invention relates to the field of automatic driving, in particular to a monocular ranging method and system based on an adaptive target detection network and license plate detection.

Background technique

With the continuous advancement of autonomous driving technology, more and more technologies have been put into use, and various types of autonomous vehicles have been put into the market. Distance measurement is a very important part of autonomous driving technology. Accurately measuring the distance of obstacles is of great significance for important functions such as path planning of autonomous driving and early warning systems, and can even be said to be the cornerstone.

The ranging method is mainly divided into active measurement and passive measurement. Active measurement is the main research direction of many researchers at present. Active measurement is ranging through ultrasonic sensors, lidars, cameras and other in-vehicle equipment. The cost of ultrasonic sensors is low, but the accuracy is not good, especially when driving at high speed, the error is large, and the applicable scenarios are relatively limited. Lidar has the highest accuracy, but it is expensive. And these methods are not easy to fuse the target with the image collected by the camera. The monocular camera can directly detect the collected images, and then measure the distance of the target, avoiding steps such as image fusion, and also reducing the cost, which is conducive to its popularization and application in smart cars.

Monocular vision ranging generally uses the corresponding point calibration method to obtain the depth information of the image. The corresponding point calibration method refers to solving the transformation relationship of the coordinate system through the corresponding coordinates of the corresponding points in different coordinate systems, but the corresponding point calibration method, in the calibration process Due to the limitation of equipment, it is still impossible to record the corresponding coordinates of a point in the world coordinate system and the image coordinate system very accurately. If its coordinates are not accurate enough, the accuracy of the obtained transformation matrix will also be restricted, and the coordinate transformation The accuracy of the results will also fluctuate because of the corresponding point calibration method. The calibration of the camera is carried out under the condition that the various angles and heights of the camera have been determined. When any parameter of the camera changes, it must be re-calibrated. In order to obtain the transformation matrix in this specific case, this method is only applicable to the case of the camera with a fixed position, and for the camera applied on the moving carrier, because the camera carrier will cause the camera parameters to occur during the movement process. changes, so applicability is limited. Therefore, how to perform ranging in the moving process has become an urgent problem to be solved.

SUMMARY OF THE INVENTION

In order to solve the above technical problems, the present invention provides a monocular ranging method and system based on an adaptive target detection network and license plate detection.

The technical solution of the present invention is: a monocular ranging method based on adaptive target detection network and license plate detection, comprising:

Step S1: use a camera to obtain a forward-looking traffic image of the vehicle, use a license plate detection algorithm to identify the shape of the license plate in the forward-looking traffic image, and obtain a top-down angle γ of the camera through a perspective transformation principle;

Step S2: Input the forward-looking traffic image into the target detection network to obtain the lower center point P of the predicted target detection frame, and use γ and P to predict the target distance;

Step S3: constructing a loss function using the target distance and the real distance for training the target detection network.

Compared with the prior art, the present invention has the following advantages:

1. The present invention discloses a monocular ranging method based on an adaptive target detection network and license plate detection. Using a monocular camera, the collected images can be directly detected, and then the distance of the target is measured to avoid image fusion, etc. It also reduces the cost, which is conducive to its popularization and application in smart cars.

2. The present invention uses the license plate detection network to detect the shape of the license plate, and then estimates the angle of the camera through the principle of perspective transformation, so as to obtain a more accurate top-view angle of the camera.

3. When training the target detection network, the present invention further satisfies the requirements of the imaging point of the geometric ranging principle by using the real-time measurement ranging result as a constraint, and increases the accuracy of the ranging result.

Description of drawings

1 is a flowchart of a monocular ranging method based on an adaptive target detection network and license plate detection in an embodiment of the present invention;

FIG. 2 is a schematic diagram of a camera imaging geometric relationship in an embodiment of the present invention;

3 is a schematic diagram of visualization after calculating the target distance in an embodiment of the present invention;

FIG. 4 is a structural block diagram of a monocular ranging system based on an adaptive target detection network and license plate detection in an embodiment of the present invention.

Detailed ways

The invention provides a monocular ranging method based on an adaptive target detection network and license plate detection, which can directly detect the collected image and measure the distance of the target, avoid steps such as image fusion, and reduce the cost.

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below through specific implementation and in conjunction with the accompanying drawings.

Example 1

As shown in FIG. 1 , a monocular ranging method based on an adaptive target detection network and license plate detection provided by an embodiment of the present invention includes the following steps:

Step S1: use the camera to obtain the front-view traffic image of the vehicle, use the license plate detection algorithm to identify the shape of the license plate in the front-view traffic image, and obtain the top-down angle γ of the camera through the principle of perspective transformation;

Step S2: Input the forward-looking traffic image into the target detection network to obtain the lower center point P of the predicted target detection frame, and use γ and P to predict the target distance;

Step S3: Use the target distance and the real distance to construct a loss function for training the target detection network.

In one embodiment, the above step S1: using a camera to obtain a forward-looking traffic image of the vehicle, and using a license plate detection algorithm to identify the shape of the license plate in the forward-looking traffic image; obtaining the top-down angle γ of the camera through the principle of perspective transformation, which specifically includes:

Step S11: use the license plate detection network to identify the license plate in the forward-looking traffic image and obtain the license plate shape;

Using the trained license plate detection network, input the forward-looking traffic image obtained by the vehicle camera, and identify the license plate shape of the vehicle in the forward-looking traffic image;

Step S12: Calculate the perspective transformation matrix H according to the four sets of points at the four corners of the license plate shape, and obtain the top-view angle γ of the camera according to H.

According to the shape of the license plate, four sets of points are taken to calculate the perspective transformation matrix H. As shown in Figure 2, A is the plane where the license plate is located, and B is the plane parallel to the camera phase plane. The function of the license plate detection network is to detect that the license plate is in the Pixel coordinates in the camera, usually a trapezoid.

It can be seen from the figure that the length of the bottom edge of the trapezoid is equal to the length of the long side of the rectangular license plate. Since the aspect ratio of the license plate is known, the rectangular pixel coordinates of the rectangular license plate on the B plane can be obtained through this ratio. Four pairs of corresponding points are collected by this method, and the perspective transformation matrix H of the B plane under the pixel coordinates of the A plane is calculated by the four-point correspondence method. After obtaining the perspective transformation matrix H between the A and B planes, let the image points of the same point on A and B in the real space be x _A and x _B respectively, as shown in formula (1):

Among them, K _B and K _A are the internal parameter matrices corresponding to the two plane cameras respectively;

Since the two internal parameter matrices K _B and K _A and the perspective transformation matrix H are known parameters, the orthogonal matrix R can be calculated. The third row of R is the camera corresponding to the B plane in the world coordinate system (that is, the A plane). The unit vector of the main axis pointing straight ahead in the corresponding camera's coordinate system). From this, the angle between the A and B planes can be obtained. According to the geometric relationship, the angle between A and B is the top-down angle γ of the camera and the horizontal plane.

In one embodiment, the above step S2: input the forward-looking traffic image into the target detection network to obtain the lower center point P of the predicted target detection frame, and use γ and P to predict the target distance, which specifically includes:

Input the forward-looking traffic image into the pre-trained weighted target detection network YOLOv4, use it to detect the forward-looking traffic map, and output the predicted target detection frame. Let the center point of the lower edge of the predicted target detection frame be P, according to formula (2), Predict the target distance O ₃ P between the camera and P:

Among them, γ is the top-down angle between the camera and the ground, and Height is the height of the camera from the ground.

In one embodiment, the above step S3: using the target distance and the real distance to construct a loss function for training the target detection network, specifically including:

Construct the loss function value Giou, as shown in formulas (3) to (4):

Among them, IOU is the ratio of the overlapping area of the predicted target detection frame and the real detection frame to the sum of the two areas; p is the distance between the predicted target detection frame and the center of the real detection frame; x ^t and x ^p are the real detection frame and the real detection frame, respectively. Predict the center point coordinates of the target frame; GT _x is the GroundTruth of the target detection network, and c is the preset parameter;

x _i is the predicted distance O ₃ P, μ is the calibrated real distance; N is the total number of samples.

In this step, the target detection network needs to be retrained. During the training process, the difference between the ranging result _xi and the calibrated real distance μ is input into the target detection network as loss, so that the P point obtained by the target detection network is Further corrections are made to bring it closer to the true value.

After calculating the target distance through the trained target detection network, you can use the OpenCV tool to display the result on the picture, so as to realize the visualization of the result, as shown in Figure 3.

In order to verify the effectiveness of the method of the present invention, the results before and after the improvement using the method of the present invention were compared respectively:

Table 1: Different results presentation using original YOLOv4 and BAOD of the present invention

GroundTruth(m) BEVDE+YOLOv4 BEVDE+BAOD 4.10 4.42 4.18 6.02 6.54 6.45 7.13 7.64 7.63 8.55 9.28 8.83

It can be seen that the method BAOD of the present invention is closer to GroundTruth than the original YOLOv4.

Table 2: Comparison of before and after camera lens pose correction using LPD

GroundTruth(m) MDE+YOLOv4 MDE+YOLOv4+BAOD 3.00 2.83 3.00 4.20 4.01 4.11 4.80 4.53 4.70 6.00 8.92 5.82 7.20 12.36 6.81 8.40 \ 7.69 9.30 \ 8.62 20.00 \ 19.82 30.00 \ 29.05 40.00 \ 34.58 50.00 \ 37.96

It can also be seen that the method BAOD of the present invention is closer to GroundTruth than the original YOLOv4.

The invention discloses a monocular ranging method based on an adaptive target detection network and license plate detection. A monocular camera can be used to directly detect the collected image, and then the distance of the target is measured, avoiding steps such as image fusion, etc. It also reduces the cost, which is conducive to its popularization and application in smart cars. The invention uses the license plate detection network to detect the shape of the license plate, and then estimates the angle of the camera through the principle of perspective transformation, so as to obtain a more accurate top-view angle of the camera. When training the target detection network, the present invention further satisfies the requirements of the geometric ranging principle imaging point by taking the real-time measurement ranging result as a constraint, and increases the accuracy of the ranging result.

Embodiment 2

As shown in FIG. 4 , an embodiment of the present invention provides a monocular ranging system based on an adaptive target detection network and license plate detection, including the following modules:

The module 41 for obtaining the top-view angle of the camera is used to obtain the front-view traffic image of the vehicle by using the camera, use the license plate detection algorithm to identify the shape of the license plate in the front-view traffic image, and obtain the top-view angle γ of the camera through the principle of perspective transformation;

The predicted target distance module 42 is used to input the forward-looking traffic image into the target detection network to obtain the lower center point P of the predicted target detection frame, and use γ and P to predict the target distance;

The training target detection network module 43 is used for constructing a loss function by using the target distance and the real distance for training the target detection network.

The above embodiments are provided for the purpose of describing the present invention only, and are not intended to limit the scope of the present invention. The scope of the invention is defined by the appended claims. Various equivalent replacements and modifications made without departing from the spirit and principle of the present invention should be included within the scope of the present invention.

Claims (5)

1. A monocular distance measurement method based on a self-adaptive target detection network and license plate detection is characterized by comprising the following steps:

step S1: acquiring a forward-looking traffic image of a vehicle by using a camera, identifying the shape of a license plate in the forward-looking traffic image by using a license plate detection algorithm, and obtaining a depression gamma of the camera by using a perspective transformation principle;

step S2: inputting the forward-looking traffic image into a target detection network to obtain a lower edge central point P of a predicted target detection frame, and predicting to obtain a target distance by utilizing gamma and P;

and step S3: and constructing a loss function by using the target distance and the real distance for training the target detection network.

2. The monocular distance measuring method based on the adaptive target detection network and the license plate detection according to claim 1, wherein the step S1: acquiring a forward-looking traffic image of a vehicle by using a camera, and identifying the shape of a license plate in the forward-looking traffic image by using a license plate detection algorithm; obtaining a depression gamma of the camera through a perspective transformation principle, specifically comprising:

step S11: identifying the license plate in the forward-looking traffic image by using a license plate detection network and obtaining the shape of the license plate;

step S12: and calculating a perspective transformation matrix H according to four groups of points at four corners of the license plate shape, and acquiring a depression angle gamma of the camera according to H.

3. The monocular distance measuring method based on adaptive target detection network and license plate detection according to claim 1, wherein the step S2: inputting the forward-looking traffic image into a target detection network to obtain a lower edge central point P of a predicted target detection frame, and predicting to obtain a target distance by utilizing gamma and P, wherein the method specifically comprises the following steps:

inputting the forward-looking traffic image into a target detection network, outputting a predicted target detection frame, setting the central point of the lower edge of the predicted target detection frame as P, and predicting to obtain the target distance O between the camera and the P according to a formula (2) ₃ P：

Wherein gamma is the depression angle of the camera and the ground, and Height is the Height of the camera from the ground.

4. The monocular distance measuring method based on the adaptive target detection network and the license plate detection according to claim 1, wherein the step S3: constructing a loss function by using the target distance and the real distance for training the target detection network, specifically comprising:

constructing a loss function value Giou as shown in formulas (3) to (4):

the IOU is the proportion of the overlapping area of the prediction target detection frame and the real detection frame to the sum of the areas of the prediction target detection frame and the real detection frame; p is the distance between the center of the predicted target detection frame and the center of the real detection frame; x is the number of ^t 、x ^p Respectively a central point coordinate of a real detection frame and a central point coordinate of the prediction target frame; GT & lt/EN & GT _x Detecting the GroudTruth of the network for the target, wherein c is a preset parameter;

x _i is a predicted distance O ₃ P, mu is a calibrated real distance; and N is the total number of samples.

5. A monocular distance measuring system based on self-adaptive target detection network and license plate detection is characterized by comprising the following modules:

the system comprises a camera depression angle acquisition module, a license plate detection module and a depression angle acquisition module, wherein the camera depression angle acquisition module is used for acquiring a forward-looking traffic image of a vehicle, recognizing the shape of a license plate in the forward-looking traffic image by using a license plate detection algorithm and obtaining a depression angle gamma of the camera through a perspective transformation principle;

the predicted target distance module is used for inputting the forward-looking traffic image into a target detection network to obtain a lower edge central point P of a predicted target detection frame, and the target distance is obtained through prediction by utilizing gamma and P;

and the training target detection network module is used for constructing a loss function by using the target distance and the real distance and training the target detection network.

Images