CN115164827A - Monocular distance measurement method based on self-adaptive target detection network and license plate detection - Google Patents
Monocular distance measurement method based on self-adaptive target detection network and license plate detection Download PDFInfo
- Publication number
- CN115164827A CN115164827A CN202210897606.4A CN202210897606A CN115164827A CN 115164827 A CN115164827 A CN 115164827A CN 202210897606 A CN202210897606 A CN 202210897606A CN 115164827 A CN115164827 A CN 115164827A
- Authority
- CN
- China
- Prior art keywords
- license plate
- target
- target detection
- distance
- camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 104
- 238000000691 measurement method Methods 0.000 title claims abstract description 13
- 238000000034 method Methods 0.000 claims abstract description 19
- 230000009466 transformation Effects 0.000 claims abstract description 16
- 238000012549 training Methods 0.000 claims abstract description 10
- 239000011159 matrix material Substances 0.000 claims description 8
- 230000003044 adaptive effect Effects 0.000 claims description 7
- 238000005259 measurement Methods 0.000 abstract description 15
- 230000004927 fusion Effects 0.000 abstract description 4
- 238000004519 manufacturing process Methods 0.000 abstract description 3
- 238000006243 chemical reaction Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000003384 imaging method Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 230000004888 barrier function Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000004575 stone Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C3/00—Measuring distances in line of sight; Optical rangefinders
Landscapes
- Physics & Mathematics (AREA)
- Electromagnetism (AREA)
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention relates to a monocular distance measurement method and a monocular distance measurement system based on a self-adaptive target detection network and license plate detection, wherein the method comprises the following steps: s1: acquiring a forward-looking traffic image of a vehicle by using a camera, identifying the shape of a license plate in the forward-looking traffic image by using a license plate detection algorithm, and obtaining a depression gamma of the camera by using a perspective transformation principle; s2: inputting the forward-looking traffic image into a target detection network to obtain a lower edge central point P of a predicted target detection frame, and predicting to obtain a target distance by utilizing gamma and P; s3: and constructing a loss function by using the target distance and the real distance for training the target detection network. The method provided by the invention can directly detect the acquired image and measure the distance of the target, thereby avoiding the steps of image fusion and the like and reducing the manufacturing cost.
Description
Technical Field
The invention relates to the field of automatic driving, in particular to a monocular distance measurement method and a monocular distance measurement system based on a self-adaptive target detection network and license plate detection.
Background
With the continuous progress of the automatic driving technology, more and more technologies are put into use, and various automatic driving automobiles are put into the market. Distance measurement is a very important part of automatic driving technology. The distance of accurate measurement barrier all has great significance to important functions such as route planning and early warning system of autopilot, can say even be the foundation stone.
The distance measurement method is mainly divided into active measurement and passive measurement, and active measurement is the main research direction of many researchers at present. The active measurement is to measure the distance by vehicle-mounted equipment such as an ultrasonic sensor, a laser radar and a camera. The ultrasonic sensor has low cost, but poor precision, large error especially when driving at high speed, and limited applicable scenes. Lidar has the highest accuracy but is expensive. And these methods are not easy to perform target fusion with the images acquired by the camera. The monocular camera can directly detect the acquired image, then measure the distance of the target, avoid the steps of image fusion and the like, reduce the manufacturing cost and be beneficial to popularization and application on the intelligent vehicle.
Monocular vision ranging generally adopts a corresponding point calibration method to obtain depth information of an image, the corresponding point calibration method is to solve a conversion relation of a coordinate system through corresponding coordinates of corresponding points in different coordinate systems, however, in the calibration process, due to the limitation of receiving materials, if the coordinates of corresponding coordinates of a point in a world coordinate system and an image coordinate system are not accurate enough, the accuracy of an obtained conversion matrix is also limited, and the accuracy of a coordinate conversion result fluctuates accordingly. Therefore, how to perform ranging in the moving process becomes an urgent problem to be solved.
Disclosure of Invention
In order to solve the technical problems, the invention provides a monocular distance measurement method and a monocular distance measurement system based on a self-adaptive target detection network and license plate detection.
The technical solution of the invention is as follows: a monocular distance measurement method based on a self-adaptive target detection network and license plate detection comprises the following steps:
step S1: acquiring a forward-looking traffic image of a vehicle by using a camera, identifying the shape of a license plate in the forward-looking traffic image by using a license plate detection algorithm, and obtaining a depression gamma of the camera by using a perspective transformation principle;
step S2: inputting the forward-looking traffic image into a target detection network to obtain a lower edge central point P of a predicted target detection frame, and predicting to obtain a target distance by utilizing gamma and P;
and step S3: and constructing a loss function by using the target distance and the real distance for training the target detection network.
Compared with the prior art, the invention has the following advantages:
1. the invention discloses a monocular distance measurement method based on a self-adaptive target detection network and license plate detection.
2. The invention detects the shape of the license plate through the license plate detection network and estimates the angle of the camera through the principle of perspective transformation, thereby obtaining more accurate overlooking angle of the camera.
3. When the target detection network is trained, the invention further meets the requirement of imaging points of a geometric distance measurement principle by taking the distance measurement result measured in real time as constraint, thereby increasing the accuracy of the distance measurement result.
Drawings
Fig. 1 is a flowchart of a monocular distance measurement method based on an adaptive target detection network and license plate detection in an embodiment of the present invention;
FIG. 2 is a diagram illustrating camera imaging geometry according to an embodiment of the present invention;
FIG. 3 is a schematic view of visualization after calculating a target distance according to an embodiment of the present invention;
fig. 4 is a block diagram of a monocular distance measuring system based on an adaptive target detection network and license plate detection in an embodiment of the present invention.
Detailed Description
The invention provides a monocular distance measurement method based on a self-adaptive target detection network and license plate detection, which can directly detect the acquired image and measure the distance of the target, avoids the steps of image fusion and the like, and reduces the manufacturing cost.
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings.
Example one
As shown in fig. 1, a monocular distance measurement method based on an adaptive target detection network and license plate detection provided by an embodiment of the present invention includes the following steps:
step S1: acquiring a forward-looking traffic image of a vehicle by using a camera, identifying the shape of a license plate in the forward-looking traffic image by using a license plate detection algorithm, and obtaining a depression gamma of the camera by using a perspective transformation principle;
step S2: inputting the forward-looking traffic image into a target detection network to obtain a lower edge central point P of a predicted target detection frame, and predicting to obtain a target distance by utilizing gamma and P;
and step S3: and constructing a loss function by using the target distance and the real distance for training a target detection network.
In one embodiment, the step S1: acquiring a forward-looking traffic image of a vehicle by using a camera, and identifying the shape of a license plate in the forward-looking traffic image by using a license plate detection algorithm; obtaining a depression gamma of the camera through a perspective transformation principle, specifically comprising:
step S11: identifying the license plate in the forward-looking traffic image by using a license plate detection network and obtaining the shape of the license plate;
inputting a forward-looking traffic image acquired by a vehicle-mounted camera by using a trained license plate detection network, and identifying and obtaining the shape of a license plate of a vehicle in the forward-looking traffic image;
step S12: and calculating a perspective transformation matrix H according to four groups of points at four corners of the shape of the license plate, and acquiring a depression angle gamma of the camera according to H.
And (3) calculating a perspective transformation matrix H by taking four groups of points according to the shape of the license plate, wherein A is a plane where the license plate is located and B is a plane parallel to a camera phase plane as shown in figure 2, and the license plate detection network is used for detecting pixel coordinates of the license plate in the camera, and the pixel coordinates are usually trapezoidal.
As can be seen from the figure, the length of the bottom side of the trapezoid is equal to the length of the long side of the rectangular license plate, and the aspect ratio of the license plate is known, so that the rectangular pixel coordinates of the rectangular license plate on the B plane can be obtained by the ratio. Four pairs of corresponding points are collected by the method, and a perspective transformation matrix H of the B plane under the pixel coordinates of the A plane is calculated by a four-point corresponding method. After obtaining the perspective transformation matrix H between the A and B planes, setting the image points of the same point in the real space on the A and B as x respectively A And x B As shown in equation (1):
wherein, K B And K A Respectively corresponding internal reference matrixes of the two plane cameras;
due to two internal reference matrixes K B And K A And the perspective transformation matrix H are known parameters, the orthogonal matrix R can be calculated, and the third row of R is the unit vector of the principal axis of the camera corresponding to the B plane pointing to the right front in the world coordinate system (i.e., the coordinate system of the camera corresponding to the a plane). Thus, the included angle between the A and B planes can be obtained. According to the geometrical relationship, the included angle between A and B is the depression angle gamma between the camera and the horizontal plane.
In one embodiment, the step S2: inputting the forward-looking traffic image into a target detection network to obtain a lower edge central point P of a predicted target detection frame, and predicting to obtain a target distance by utilizing gamma and P, wherein the method specifically comprises the following steps:
inputting the forward-looking traffic image into a pre-trained weight target detection network YOLOv4, and inputting the forward-looking traffic image into a forward-looking traffic imageLine detection, outputting a predicted target detection frame, making the central point of the lower edge of the predicted target detection frame be P, and predicting to obtain the target distance O between the camera and P according to a formula (2) 3 P：
Wherein gamma is the depression angle of the camera and the ground, and Height is the Height of the camera from the ground.
In one embodiment, the step S3: constructing a loss function by using the target distance and the real distance for training a target detection network, which specifically comprises the following steps:
constructing a loss function value Giou as shown in formulas (3) to (4):
the IOU is the ratio of the overlapping area of the predicted target detection frame and the real detection frame to the sum of the areas of the predicted target detection frame and the real detection frame; p is the distance between the center of the predicted target detection frame and the center of the real detection frame; x is the number of t 、x p Respectively representing the coordinates of the central points of the real detection frame and the prediction target frame; GT & lt/EN & GT x Detecting the GroudTruth of the network for the target, wherein c is a preset parameter;
x i is a predicted distance O 3 P, mu is a calibrated real distance; and N is the total number of samples.
In this step, the target detection network is retrained, and the ranging result x is used in the training process i And inputting the difference value of the calibrated real distance mu into the target detection network as loss, and further correcting the P point acquired by the target detection network to enable the P point to be close to a real value.
After the target distance is calculated by the trained target detection network, the result can be displayed on a picture by using an OpenCV tool, so that the result is visualized, and an example is shown in fig. 3.
To verify the effectiveness of the method of the invention, the results before and after improvement using the method of the invention were compared, respectively:
table 1: different results using original YOLOv4 and the BAOD of the invention show
| GroundTruth(m) | BEVDE+YOLOv4 | BEVDE+BAOD |
| 4.10 | 4.42 | 4.18 |
| 6.02 | 6.54 | 6.45 |
| 7.13 | 7.64 | 7.63 |
| 8.55 | 9.28 | 8.83 |
It can be seen that the method BAOD of the present invention is closer to GroundTruth than the original YOLOv 4.
Table 2: forward and backward correction comparison of camera lens pose using LPD
| GroundTruth(m) | MDE+YOLOv4 | MDE+YOLOv4+BAOD |
| 3.00 | 2.83 | 3.00 |
| 4.20 | 4.01 | 4.11 |
| 4.80 | 4.53 | 4.70 |
| 6.00 | 8.92 | 5.82 |
| 7.20 | 12.36 | 6.81 |
| 8.40 | \ | 7.69 |
| 9.30 | \ | 8.62 |
| 20.00 | \ | 19.82 |
| 30.00 | \ | 29.05 |
| 40.00 | \ | 34.58 |
| 50.00 | \ | 37.96 |
It can likewise be seen that the method BAOD of the invention is closer to GroudTruth than the original YOLOv 4.
The invention discloses a monocular distance measurement method based on a self-adaptive target detection network and license plate detection. The invention detects the shape of the license plate through a license plate detection network and estimates the angle of the camera through the principle of perspective transformation, thereby obtaining more accurate overlooking angle of the camera. When the target detection network is trained, the invention further meets the requirement of imaging points of a geometric distance measurement principle by taking the distance measurement result measured in real time as constraint, thereby increasing the accuracy of the distance measurement result.
Example two
As shown in fig. 4, an embodiment of the present invention provides a monocular distance measuring system based on an adaptive target detection network and license plate detection, including the following modules:
the depression angle acquisition module 41 of the camera is used for acquiring a forward-looking traffic image of the vehicle by using the camera, identifying the shape of a license plate in the forward-looking traffic image by using a license plate detection algorithm, and obtaining a depression angle gamma of the camera by a perspective transformation principle;
a predicted target distance module 42, configured to input the forward-looking traffic image into the target detection network to obtain a lower edge central point P of the predicted target detection frame, and predict to obtain a target distance by using γ and P;
and a training target detection network module 43, configured to construct a loss function using the target distance and the real distance, and train a target detection network.
The above examples are provided for the purpose of describing the present invention only and are not intended to limit the scope of the present invention. The scope of the invention is defined by the appended claims. Various equivalent substitutions and modifications can be made without departing from the spirit and principles of the invention, and are intended to be within the scope of the invention.
Claims (5)
1. A monocular distance measurement method based on a self-adaptive target detection network and license plate detection is characterized by comprising the following steps:
step S1: acquiring a forward-looking traffic image of a vehicle by using a camera, identifying the shape of a license plate in the forward-looking traffic image by using a license plate detection algorithm, and obtaining a depression gamma of the camera by using a perspective transformation principle;
step S2: inputting the forward-looking traffic image into a target detection network to obtain a lower edge central point P of a predicted target detection frame, and predicting to obtain a target distance by utilizing gamma and P;
and step S3: and constructing a loss function by using the target distance and the real distance for training the target detection network.
2. The monocular distance measuring method based on the adaptive target detection network and the license plate detection according to claim 1, wherein the step S1: acquiring a forward-looking traffic image of a vehicle by using a camera, and identifying the shape of a license plate in the forward-looking traffic image by using a license plate detection algorithm; obtaining a depression gamma of the camera through a perspective transformation principle, specifically comprising:
step S11: identifying the license plate in the forward-looking traffic image by using a license plate detection network and obtaining the shape of the license plate;
step S12: and calculating a perspective transformation matrix H according to four groups of points at four corners of the license plate shape, and acquiring a depression angle gamma of the camera according to H.
3. The monocular distance measuring method based on adaptive target detection network and license plate detection according to claim 1, wherein the step S2: inputting the forward-looking traffic image into a target detection network to obtain a lower edge central point P of a predicted target detection frame, and predicting to obtain a target distance by utilizing gamma and P, wherein the method specifically comprises the following steps:
inputting the forward-looking traffic image into a target detection network, outputting a predicted target detection frame, setting the central point of the lower edge of the predicted target detection frame as P, and predicting to obtain the target distance O between the camera and the P according to a formula (2) 3 P：
Wherein gamma is the depression angle of the camera and the ground, and Height is the Height of the camera from the ground.
4. The monocular distance measuring method based on the adaptive target detection network and the license plate detection according to claim 1, wherein the step S3: constructing a loss function by using the target distance and the real distance for training the target detection network, specifically comprising:
constructing a loss function value Giou as shown in formulas (3) to (4):
the IOU is the proportion of the overlapping area of the prediction target detection frame and the real detection frame to the sum of the areas of the prediction target detection frame and the real detection frame; p is the distance between the center of the predicted target detection frame and the center of the real detection frame; x is the number of t 、x p Respectively a central point coordinate of a real detection frame and a central point coordinate of the prediction target frame; GT & lt/EN & GT x Detecting the GroudTruth of the network for the target, wherein c is a preset parameter;
x i is a predicted distance O 3 P, mu is a calibrated real distance; and N is the total number of samples.
5. A monocular distance measuring system based on self-adaptive target detection network and license plate detection is characterized by comprising the following modules:
the system comprises a camera depression angle acquisition module, a license plate detection module and a depression angle acquisition module, wherein the camera depression angle acquisition module is used for acquiring a forward-looking traffic image of a vehicle, recognizing the shape of a license plate in the forward-looking traffic image by using a license plate detection algorithm and obtaining a depression angle gamma of the camera through a perspective transformation principle;
the predicted target distance module is used for inputting the forward-looking traffic image into a target detection network to obtain a lower edge central point P of a predicted target detection frame, and the target distance is obtained through prediction by utilizing gamma and P;
and the training target detection network module is used for constructing a loss function by using the target distance and the real distance and training the target detection network.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210897606.4A CN115164827A (en) | 2022-07-28 | 2022-07-28 | Monocular distance measurement method based on self-adaptive target detection network and license plate detection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210897606.4A CN115164827A (en) | 2022-07-28 | 2022-07-28 | Monocular distance measurement method based on self-adaptive target detection network and license plate detection |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115164827A true CN115164827A (en) | 2022-10-11 |
Family
ID=83476586
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210897606.4A Pending CN115164827A (en) | 2022-07-28 | 2022-07-28 | Monocular distance measurement method based on self-adaptive target detection network and license plate detection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115164827A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118297819A (en) * | 2024-06-06 | 2024-07-05 | 比亚迪股份有限公司 | Image processing method, storage medium, controller and vehicle |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110517349A (en) * | 2019-07-26 | 2019-11-29 | 电子科技大学 | A kind of 3D vehicle target detection method based on monocular vision and geometrical constraint |
| CN113465572A (en) * | 2021-06-25 | 2021-10-01 | 中国科学技术大学 | Monocular distance measurement method and system based on camera imaging geometric relation |
-
2022
- 2022-07-28 CN CN202210897606.4A patent/CN115164827A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110517349A (en) * | 2019-07-26 | 2019-11-29 | 电子科技大学 | A kind of 3D vehicle target detection method based on monocular vision and geometrical constraint |
| CN113465572A (en) * | 2021-06-25 | 2021-10-01 | 中国科学技术大学 | Monocular distance measurement method and system based on camera imaging geometric relation |
Non-Patent Citations (1)
| Title |
|---|
| 贾海涛: "基于PyTorch的浓度学习与计算机视觉", 30 June 2022, 电子科技大学出版社, pages: 162 - 164 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118297819A (en) * | 2024-06-06 | 2024-07-05 | 比亚迪股份有限公司 | Image processing method, storage medium, controller and vehicle |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112669393B (en) | Laser radar and camera combined calibration method | |
| EP3792660B1 (en) | Method, apparatus and system for measuring distance | |
| CN110031829B (en) | Target accurate distance measurement method based on monocular vision | |
| CN103487034B (en) | Method for measuring distance and height by vehicle-mounted monocular camera based on vertical type target | |
| CN112880642B (en) | Ranging system and ranging method | |
| CN109752701A (en) | A kind of road edge detection method based on laser point cloud | |
| CN103499337B (en) | Vehicle-mounted monocular camera distance and height measuring device based on vertical target | |
| CN109343041B (en) | Monocular distance measuring method for advanced intelligent auxiliary driving | |
| CN111508027A (en) | Method and device for calibrating external parameters of camera | |
| CN112363167A (en) | Extended target tracking method based on fusion of millimeter wave radar and monocular camera | |
| JP2006053890A (en) | Obstacle detection apparatus and method therefor | |
| CN112017248B (en) | 2D laser radar camera multi-frame single-step calibration method based on dotted line characteristics | |
| CN111678518B (en) | Visual positioning method for correcting automatic parking path | |
| CN114200442B (en) | Road target detection and association method based on millimeter wave radar and vision | |
| CN114037762B (en) | Real-time high-precision positioning method based on registration of image and high-precision map | |
| CN111382591B (en) | Binocular camera ranging correction method and vehicle-mounted equipment | |
| KR20100066952A (en) | Apparatus for tracking obstacle using stereo vision and method thereof | |
| Adamshuk et al. | On the applicability of inverse perspective mapping for the forward distance estimation based on the HSV colormap | |
| CN111476798B (en) | Vehicle space morphology recognition method and system based on contour constraint | |
| CN115079143B (en) | Multi-radar external parameter quick calibration method and device for double-bridge steering mine card | |
| CN114413958A (en) | Monocular vision distance and speed measurement method of unmanned logistics vehicle | |
| CN115588040A (en) | System and method for counting and positioning coordinates based on full-view imaging points | |
| CN114494466B (en) | External parameter calibration method, device and equipment and storage medium | |
| CN112068567A (en) | Positioning method and positioning system based on ultra-wideband and visual image | |
| CN115267756A (en) | Monocular real-time distance measurement method based on deep learning target detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |