CN115511975A - Distance measurement method of monocular camera and computer program product - Google Patents
Distance measurement method of monocular camera and computer program product Download PDFInfo
- Publication number
- CN115511975A CN115511975A CN202211215921.0A CN202211215921A CN115511975A CN 115511975 A CN115511975 A CN 115511975A CN 202211215921 A CN202211215921 A CN 202211215921A CN 115511975 A CN115511975 A CN 115511975A
- Authority
- CN
- China
- Prior art keywords
- target object
- determining
- value
- initial
- projection area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/521—Depth or shape recovery from laser ranging, e.g. using interferometry; from the projection of structured light
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30244—Camera pose
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Biomedical Technology (AREA)
- Optics & Photonics (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Multimedia (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The application provides a distance measuring method of a monocular camera and a computer program product, wherein the distance measuring method of the monocular camera comprises the following steps: acquiring a detection image and internal parameters of a camera; determining an initial distance measurement value, a detection frame of a target object and the real size of the target object according to the detection image; determining a direction angle of a target object according to the initial ranging value and the detection image; determining the three-dimensional coordinates of the target object contour according to the real size, the initial ranging value and the direction angle; generating a projection area of the target object according to the three-dimensional coordinates of the contour of the target object and the internal parameters; and determining a final ranging value according to the projection area and the detection frame. The distance measurement method of the embodiment can adjust the monocular camera in real time through the determined final distance measurement value, is convenient and rapid, has higher reliability, and improves the driving safety of vehicles.
Description
Technical Field
The present application relates to the field of vehicle ranging technologies, and in particular, to a method and a computer program product for ranging a monocular camera.
Background
In advanced driving assistance systems, it is required that the vehicle be able to perceive the surrounding environment using information obtained by sensors, such as: and acquiring the distance between the surrounding obstacles and the self-vehicle. For ranging, commonly used methods mainly employ monocular cameras, millimeter-wave radars, laser radars, and binocular cameras. For the distance measurement with the monocular camera, the accuracy of the measured distance measurement value is generally considered to be poor. If the distance measurement value of the monocular camera is evaluated by adopting binocular or other means, the collected scenes are different, more errors may exist, and the accurate distance measurement value of the monocular camera cannot be obtained.
Disclosure of Invention
The embodiment of the application provides a distance measurement method of a monocular camera and a computer program product, which are used for solving the problems in the related technology, and the technical scheme is as follows:
in a first aspect, an embodiment of the present application provides a monocular camera ranging method, including:
acquiring a detection image and internal parameters of a camera;
determining an initial ranging value, a detection frame of a target object and the real size of the target object according to the detection image;
determining a direction angle of the target object according to the initial ranging value and the detection image;
determining a three-dimensional coordinate of the contour of the target object according to the real size, the initial distance measurement value and the direction angle;
generating a projection area of the target object according to the three-dimensional coordinates and the internal parameters of the contour of the target object;
and determining a final ranging value according to the projection area and the detection frame.
In a second aspect, the present application provides a computer program product, and the computer program/instructions when executed by a processor alone or in cooperation with a plurality of processors implement any one of the methods provided by the embodiments of the present disclosure.
The advantages or beneficial effects in the above technical solution at least include:
in the present embodiment, the initial ranging value is determined by detecting the image, and the detection frame of the target object and the real size of the target object are determined according to the detected image. Since the direction angle of the target object can affect the three-dimensional coordinates of the contour of the target object, the direction angle of the target object needs to be determined first, and after the direction angle is determined, the three-dimensional coordinates of the contour of the target object are determined by combining the real size of the target object and the initial ranging value. After the three-dimensional coordinates of the contour of the target object are determined, the three-dimensional coordinates of the contour of the target object are converted into two-dimensional coordinates of an image, namely the projection area of the target object, through the internal parameters of the camera. The detection frame of the target object is determined by detecting the image, the projection area of the target object determined by the initial ranging value can be evaluated and adjusted by taking the detection frame of the target object as a reference, and therefore the accurate ranging value can be effectively determined as the final ranging value. That is, the distance measurement method of the embodiment does not need to add additional equipment, and effectively obtains the accurate final distance measurement value of the monocular camera under the same scene. The process of confirming the final range finding value of monocular camera can be gone on in real time at the in-process that the vehicle went, can adjust the monocular camera in real time according to final range finding value moreover, and convenient and fast, the reliability is higher moreover, has promoted the security that the vehicle went.
The foregoing summary is provided for the purpose of description only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of the present application will be readily apparent by reference to the drawings and following detailed description.
Drawings
In the drawings, like reference numerals refer to the same or similar parts or elements throughout the several views unless otherwise specified. The figures are not necessarily to scale. It is appreciated that these drawings depict only some embodiments in accordance with the disclosure and are therefore not to be considered limiting of its scope.
FIG. 1 is a diagram illustrating a back projection distance measurement method according to an embodiment of the related art;
FIG. 2 is a diagram illustrating a back projection distance measurement method according to another embodiment of the related art;
FIG. 3 is a diagram illustrating a target size ranging method according to one embodiment of the related art;
FIG. 4 is a flowchart illustrating a distance measuring method of a monocular camera according to an embodiment of the present application;
fig. 5 is a schematic flowchart of a distance measuring method of a monocular camera according to another embodiment of the present application;
FIG. 6 is a flowchart illustrating a distance measuring method of a monocular camera according to another embodiment of the present application;
Detailed Description
In the following, only certain exemplary embodiments are briefly described. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present application. Accordingly, the drawings and description are to be regarded as illustrative in nature, and not as restrictive.
In the related art, a monocular camera generally performs ranging using an inverse projection algorithm or an object size ranging algorithm.
The ideal coordinate system of each vehicle is that an X axis and a Y axis of the vehicle are parallel to the road surface, and a Z axis is perpendicular to the road surface, wherein the X axis is directed to the head and the tail of the vehicle, and the Y axis is directed to the doors of the two sides of the vehicle. There is also an ideal coordinate system for the monocular camera, the ideal coordinate system for the monocular camera, X c Axis and Z c The axis being parallel to the vehicle chassis, Y c The axis being perpendicular to the vehicle chassis, wherein X c The axis is in the direction of the head and tail of the vehicle, Z c The axis is the direction of the vehicle pointing to the doors on both sides. There is also a corresponding image pixel coordinate system (u, v) for the image taken by the monocular camera. The ideal coordinate system of the vehicle, the ideal coordinate system of the monocular camera and the image pixel coordinate system can be converted according to the internal parameters of the camera and the real-time calibration parameters of the camera.
In the back projection algorithm, the back projection algorithm is generally an image captured by a monocular camera, and the image includes a target object to be measured, and the target object may be a vehicle ahead, a person ahead, an article ahead, or the like. The detection frame of the target object can be obtained by recognition of an image recognition model obtained through training, can be a frame for defining the outline of the image, and can also be a maximum circumscribed rectangle for defining the outline of the image or a maximum circumscribed circle for defining the outline of the image. Taking the example that the target object is a front vehicle, a key point of the target object in an image shot by the monocular camera is extracted from the image shot by the monocular camera, the key point can also be called a grounding point, the current vehicle and the front target object are subjected to distance measurement by using the key point, and the distance measurement can be realized through formulas (1) - (3).
As shown in fig. 1, in formula (1) and formula (2), the road on which the current vehicle and the target object in front are located has no slope, that is, the current vehicle and the target object in front are on the same plane, at this time, the pitch angle θ of the monocular camera is 0, and the attitude of the current camera is located under an ideal camera coordinate system (the optical axis is parallel to the ground):
wherein d is the range finding value of the monocular camera, F c Focal length of monocular camera, H c Height of monocular camera to ground, v b As coordinates of the ground point of the vehicle, v v V coordinate, z, of vanishing line c As Z-axis coordinate, y, of the coordinate of the grounding point in the coordinate system of the monocular camera c Is the Y-axis coordinate of the ground point in the coordinate system of the monocular camera, where z c ＝1。
As shown in fig. 2, for equation (3), the road on which the current vehicle and the target object in front are located has a slope, that is, the current vehicle and the target object in front are not located on the same plane. At the moment, the pitch angle theta of the monocular camera is not 0, the real-time calibration parameters of the monocular camera can be directly obtained from the monocular camera, and the pitch angle theta of the monocular camera is determined from the real-time calibration parameters of the monocular camera.
Wherein d is the range finding value of the monocular camera, F c Focal length of monocular camera, H c Height of monocular camera to ground, v b As coordinates of the vehicle's ground point, v v And theta is a v coordinate of the vanishing line and is a pitch angle.
The range value of the monocular camera range can be calculated through the formula (1), the formula (2) and the formula (3), wherein the advantage of adopting the back projection algorithm is that a more accurate estimation value can be obtained for a near object; however, the method has the problems that the assumption based on the same plane is needed, the pitch angle of the camera has a large influence on the distance measurement, and the measured distance measurement value is easily inaccurate.
For the target size ranging algorithm, as shown in fig. 3, the target size ranging algorithm mainly estimates the distance between the current vehicle and the target object, i.e. the ranging value, by using the similar triangle according to the known physical size of the object (the height or width of the object) and the camera focal length of the monocular camera.
If the true width (or height) of the target object is known, the true width (or height) of the target object can be determined by a trained deep learning model. According to the imaging pixel of the width (or height) of the target object in the image, the longitudinal distance from the target object to the self vehicle can be calculated according to the similar triangle, and the calculation formula is shown as a formula (4) or a formula (5).
Wherein d is the range finding value of the monocular camera, F c Is the focal length of the monocular camera, W is the true width of the target object, H is the true height of the target object, Δ u is the width of the detection box of the target object on the image taken by the monocular camera, and Δ v is the height of the detection box of the target object on the image taken by the monocular camera. The width of the detection frame of the target object and the height of the detection frame of the target object can be determined through a trained deep learning model or can be directly determined through pixels.
The distance measurement of the monocular camera can be realized through the target size distance measurement algorithm, the corresponding distance measurement value is obtained, and the distance measurement value of the monocular camera obtained through the back projection algorithm can be adjusted according to the distance measurement value of the monocular camera obtained through the target size distance measurement algorithm. The advantages of using the target size ranging algorithm are: the ranging result is insensitive to the pitch angle. However, the distance measurement result depends on the used variables, such as estimation of the physical size of the target and the pixel size of the target in the image, and if the used variables have errors, the measured distance measurement value of the monocular camera is prone to be inaccurate.
The two methods have respective advantages and disadvantages, but in the actual use process, the problem that the measured distance measurement value is inaccurate may exist, and if the millimeter wave radar, the laser radar and the binocular camera are used for distance measurement, although the accuracy of the obtained distance measurement value is improved, the cost is greatly improved, the obtained environmental information is not rich enough, and the calculation complexity is also improved.
Fig. 4 shows a flowchart of a distance measuring method of a monocular camera according to an embodiment of the present application. As shown in fig. 4, the ranging method of the monocular camera may include:
step S410: the inspection image and the internal parameters of the camera are acquired.
Step S420: and determining the initial ranging value, a detection frame of the target object and the real size of the target object according to the detection image.
Step S430: and determining the direction angle of the target object according to the initial ranging value and the detection image.
Step S440: and determining the three-dimensional coordinates of the target object contour according to the real size, the initial ranging value and the direction angle.
Step S450: and generating a projection area of the target object according to the three-dimensional coordinates and the internal parameters of the contour of the target object.
Step S460: and determining a final ranging value according to the projection area and the detection frame.
In the present embodiment, the initial ranging value is determined by detecting the image, and at the same time, the detection frame of the target object and the real size of the target object are determined according to the detected image. Since the direction angle of the target object affects the three-dimensional coordinates of the contour of the target object, the direction angle of the target object needs to be determined first, and after the direction angle is determined, the three-dimensional coordinates of the contour of the target object are determined by combining the real size of the target object and the initial ranging value. After the three-dimensional coordinates of the contour of the target object are determined, the three-dimensional coordinates of the contour of the target object are converted into two-dimensional coordinates of an image through internal parameters of a camera, namely a projection area of the target object, a detection frame of the target object is determined through detecting the image, the detection frame of the target object is used as a reference, the projection area of the target object determined by the initial ranging value can be evaluated and adjusted, and therefore the accurate ranging value can be effectively determined to be used as a final ranging value. That is, the distance measurement method of the embodiment does not need to add additional equipment, and effectively obtains the accurate final distance measurement value of the monocular camera under the same scene. The process of confirming the final range finding value of monocular camera can go on in real time at the in-process that the vehicle went, can adjust in real time according to final range finding value to the monocular camera moreover, and convenient and fast, the reliability is higher moreover, has promoted the security that the vehicle went.
The distance measurement method of the monocular camera in the embodiment can be executed on a domain controller or an Electronic Control Unit (ECU) of the current vehicle or on a distance measurement processor of the vehicle, the execution is executed at the vehicle end, the distance measurement accuracy of the monocular camera can be evaluated more conveniently and rapidly from the vehicle end, the monocular camera can be adjusted more timely and effectively according to the accuracy, the distance measurement process of the monocular camera is adjusted, and the accuracy of the monocular camera is improved. In some embodiments, the ranging method of the monocular camera may be executed at the server by acquiring data of the detected image and the internal parameters of the camera from the vehicle end, the server may be a cloud terminal, a road side device or other terminal devices, and the server has a stronger processing capability, so that the ranging evaluation of the monocular camera does not occupy a memory of the vehicle end, and is more efficient and more accurate. In the following embodiments, the car end is taken as an example for explanation, and the server end execution can be obtained in the same manner.
In step S410, the vehicle end acquires the inspection image and the internal parameters of the camera.
The detection image may be captured by a monocular camera provided on the current vehicle. Internal parameters of the camera, such as an internal reference matrix of the monocular camera, an ideal coordinate system of the monocular camera and the like, are all solidified on the monocular camera when the monocular camera leaves a factory, and as attributes of the monocular camera, the internal parameters of the camera can be obtained directly through the monocular camera. For the monocular camera, under the condition that the current vehicle and the target object are in the same plane, the pitch angle of the monocular camera is 0; under the condition that the current vehicle and the target object are not in the same plane, the support of the monocular camera adjusts the pitch angle of the monocular camera according to the actual situation, and then the pitch angle of the monocular camera is used as a parameter which needs to be calibrated in real time. Under the condition that the current vehicle and the target object are in the same plane, the pitch angle of the monocular camera at the moment is 0, so that the above back projection algorithm can know that the pitch angle of the monocular camera can not be considered at the moment, and the distance between the current vehicle and the target object, namely the distance measurement value can be calculated by using the back projection algorithm.
In step S420, the vehicle end determines an initial distance measurement value, a detection frame of the target object, and a real size of the target object according to the detection image.
The vehicle end can perform calculation through an inverse projection algorithm or a target size ranging algorithm according to a detection image acquired from the monocular camera, and an initial ranging value between the current vehicle and the target object is determined, wherein the initial ranging value is a ranging value with accuracy to be evaluated. The initial ranging value may or may not be accurate, and needs to be verified by evaluation. In the back projection algorithm, when the pitch angle is 0, the formula (1) and the formula (2) are adopted for calculation, and when the pitch angle is not 0, the formula (3) is adopted for calculation.
According to the detection image acquired from the monocular camera, the vehicle end can determine the detection frame of the target object through the trained deep learning model, for example, the image detection model, and can directly use the outline of the target object as the detection frame of the target object or use the maximum circumscribed rectangle or the maximum circumscribed circle of the outline of the target object as the detection frame of the target object by identifying the target object and determining the outline of the target object and framing the target object based on the outline of the target object.
The real size of the target object can be determined through the trained deep learning model according to the detection image acquired from the monocular camera by the vehicle end, and the real size of the target object can be directly identified through the detection image through the trained deep learning model. The real size of the target object and the detection image can be trained in advance by collecting a training set and a testing set to obtain a trained deep learning model, and the real size of the target object can be directly obtained according to the detection image. Wherein the real size of the target object comprises the parameters of length, width, height and the like of the target object.
In step S430, the vehicle end determines the direction angle of the target object according to the initial distance measurement value and the detection image.
According to the initial ranging value of the detected image, at least two key points (grounding points) of the target object are determined, the key points are all on the same straight line, taking the two key points as an example, the connecting line of the two key points to the front and back directions of the target object, namely the connecting line of the two key points and the X axis of the ideal coordinate system of the target object, one of the two key points can be located at the head of the vehicle, and the other key point can be located at the tail of the vehicle, so that the selection and the calculation are convenient.
And calculating and determining plane coordinates of the two key points according to an inverse projection algorithm and the initial ranging value. In the process of the back projection algorithm, in the back projection algorithm, when the pitch angle is 0, the formula (1) and the formula (2) are adopted for calculation, and under the condition that the pitch angle is not 0, the formula (3) is adopted for calculation.
The R point is a key point (grounding point) at the tail part of the target object, and R is y Is the coordinate value of the Y axis of R under the ideal coordinate system which is back projected to the current vehicle, R x And the coordinate value of the X axis under the ideal coordinate system which is back projected to the current vehicle is R.
Point B is a key point (grounding point) of the header of the target object, point B y Is the coordinate value of the Y axis of B under the inverse projection to the coordinate system of the vehicle body, B x And B is the coordinate value of the X axis under the ideal coordinate system which is back projected to the current vehicle.
Wherein R is y 、R x 、B y And B x Can be determined by the back projection algorithm formula (1), formula (2) or formula (3) in the case of determining the initial ranging value and detecting the image.
Where α is the azimuth of the target object.
If the target object is a vehicle, the direction angle of the target object is the heading angle of the vehicle.
In step S440, the vehicle end determines the three-dimensional coordinates of the contour of the target object according to the real size, the initial distance measurement value, and the direction angle;
and the vehicle end can determine the three-dimensional coordinates of the reference contour of the target object according to the acquired real size of the target object, wherein the three-dimensional coordinates of the reference contour of the target object are based on the three-dimensional coordinates of the current vehicle in an ideal coordinate system. For example, with the coordinate of the center of the target object as the origin in the ideal coordinate system of the current vehicle, the three-dimensional coordinates of eight vertices of the target object can be determined by the real size of the target object, and the eight vertices can form the outline of the target object, wherein the number of the vertices can be selected according to practical situations, and is at least four or less, and is lower than four outlines which cannot form the most basic cube.
After the three-dimensional coordinates of the reference contour of the target object are determined, the three-dimensional coordinates of the reference contour of the target object are adjusted through the initial ranging value and the direction angle of the target object to obtain the adjusted three-dimensional coordinates, namely the three-dimensional coordinates of the contour of the target object.
In step 450, the vehicle end generates a projection area of the target object according to the three-dimensional coordinates and the internal parameters of the target object contour.
The vehicle end converts the three-dimensional coordinates of the target object contour according to the internal parameters of the monocular camera, the internal parameters are the internal parameter matrix of the camera, the three-dimensional coordinates of the target object contour are converted into the two-dimensional coordinates of the image shot by the monocular camera, and the two-dimensional coordinate connecting lines of the image shot by the monocular camera form the projection area of the target object.
The outline of the projection area, that is, the maximum bounding box of the projection area, can be determined by directly projecting on the detection image or by projecting on a blank detection image.
In step S460, the vehicle end determines a final ranging value according to the projection area and the detection frame.
The projection area of the target object and the detection frame of the target object are determined in the above manner, and the detection frame of the target object is directly determined by detecting the image and is not calculated by the initial ranging value and is used as a reference for comparison. And the projection area of the target object is determined by the initial ranging value, if the initial ranging value is inaccurate, the projection area of the target object and the detection frame of the target object have size difference, for example, the projection area of the target object is larger than the detection frame of the target object or the projection area of the target object is smaller than the detection of the target object. That is, whether the measured initial ranging value is accurate or not can be effectively determined by the method, and if the initial ranging value is determined to be accurate under the condition that the maximum external frame of the projection area is consistent with the detection frame, the initial ranging value is determined to be the final ranging value; and under the condition that the maximum external frame of the projection area is inconsistent with the detection frame, determining that the initial ranging value is inaccurate, and re-evaluating by adjusting the initial ranging value until the adjusted initial ranging value is determined to be the final ranging value under the condition that the maximum external frame of the projection area is consistent with the detection frame.
The evaluation criterion may be set for comparison between the maximum circumscribed frame of the projection area of the target object and the detection frame of the target object, and since there may be an error in the calculated projection area of the target object, it may be determined that the initial ranging value is accurate when a difference between the maximum circumscribed frame of the projection area of the target object and the detection frame of the target object is within a certain range. Under the condition that the difference between the maximum external frame of the projection area of the target object and the detection frame of the target object exceeds a certain range, the initial ranging value can be determined to be inaccurate, so that the initial ranging value can be evaluated, and the final ranging value can be determined. The certain range can be set according to actual conditions.
Through the range finding method of the monocular camera of this embodiment, need not additionally to increase equipment such as binocular camera or laser radar, can confirm accurate final range finding value effectively to can adjust or not adjust the vehicle based on this final range finding value, thereby can promote the security of vehicle. Compared with the related art, the cost of the equipment can be effectively reduced through the monocular camera, and a wider range and abundant images can be obtained, so that the influence of a certain single variable on a ranging value in target size ranging is reduced, the ranging error of monocular ranging is reduced, and the ranging robustness is improved.
In one embodiment, evaluating the accuracy of the initial ranging value based on the projection area and the detection box comprises:
and under the condition that the maximum external frame of the projection area is consistent with the detection frame, determining that the initial ranging value is accurate.
Whether the initial ranging value is accurate is determined by judging whether the maximum circumscribing frame of the projection area of the target object is consistent with the detection frame of the target object, namely whether the maximum circumscribing frame of the projection area of the target object and the detection frame of the target object can be coincided is judged. If the initial ranging values can be coincided, the initial ranging values are considered to be accurate. If the target object is not overlapped, the initial ranging value is inaccurate, at this time, the three-dimensional coordinate of the contour of the target object needs to be determined again by adjusting and adjusting the initial ranging value, so that the maximum external frame of the projection area of the target object is determined again, the evaluation comparison between the maximum external frame of the projection area of the target object and the detection frame of the target object is performed again, and after multiple iterations, under the condition that the maximum external frame of the projection area of the target object is determined again to be consistent with the detection frame of the target object, the adjusted ranging value is the final ranging value. The initial distance measurement value can be effectively adjusted by judging whether the maximum external frame of the projection area of the target object is consistent with the detection frame of the target object, so that the final distance measurement value of the monocular camera with better precision is obtained, the accurate distance measurement value between the monocular camera and the target object can be ensured in the driving process of the vehicle, and the driving safety of the vehicle is ensured.
As shown in fig. 5, in one embodiment, determining a final ranging value according to the projection area and the detection frame includes:
step S510: and under the condition that the maximum external frame of the projection area is inconsistent with the detection frame, adjusting the initial ranging value.
Step S520: and re-determining the three-dimensional coordinates of the contour of the target object according to the real size of the target object, the direction angle of the target object and the adjusted initial distance measurement value.
Step S530: and generating the projection area of the adjusted target object according to the internal parameters of the camera and the three-dimensional coordinates of the contour of the target object which is determined again.
Step S540: and under the condition that the projection area of the adjusted target object is consistent with the detection frame, determining the adjusted initial ranging value as a final ranging value.
Through the comparison between the maximum external frame and the detection frame in the projection area, the initial ranging value is directly determined as the final ranging value under the condition that the maximum external frame and the detection frame in the projection area are consistent, and the ranging value is not required to be adjusted.
Under the condition that the maximum external frame of the projection area is not consistent with the detection frame, the initial ranging value needs to be adjusted again at the moment, so that the initial ranging value measured by the monocular camera can be adjusted to obtain an accurate final ranging value.
The initial ranging value is adjusted by increasing or decreasing the initial ranging value, and after the initial ranging value is adjusted, whether the adjusted initial ranging value is accurate needs to be determined again. The adjusted initial ranging value has a large influence on the three-dimensional coordinates of the contour of the target object, and the real size of the target object and the direction angle of the target object are changed little or even not after the initial ranging value is changed. Therefore, in the three-dimensional coordinates of the target object contour, the initial ranging value is determined as a unique variable according to the real size, the initial ranging value and the direction angle, and the three-dimensional coordinates of different target object contours are determined again through adjustment of the initial ranging value. And converting the three-dimensional coordinates of the target object contour into two-dimensional coordinates of an image coordinate system according to the internal parameters of the camera through the re-determined three-dimensional coordinates of the target object contour, and projecting the two-dimensional coordinates on an image, wherein the two-dimensional coordinates can be directly projected on a detection image. And judging whether the maximum external frame and the detection frame of the projection area are consistent or not again, and if so, determining that the adjusted initial ranging value is an accurate initial ranging value. If the initial ranging value is inconsistent with the initial ranging value, the adjusted initial ranging value is determined to be inaccurate, the initial ranging value is adjusted again, and the accurate initial ranging value can be determined after multiple iterations.
The ranging method of the monocular camera of this affair example can promote the accuracy of monocular camera range finding effectively, compares in millimeter wave radar, laser radar and binocular camera moreover, and its equipment is simple more cheap, can obtain more extensive scene information moreover, can have lower operand simultaneously, reduces range error, obtains the range finding value of accurate monocular camera, improves the range finding robustness.
In one embodiment, adjusting the initial ranging value comprises:
and adjusting the initial ranging value according to the comparison result between the maximum external frame and the detection frame of the projection area.
The adjustment of the initial ranging value may be adjusted regularly, that is, according to a comparison result between the maximum bounding box of the projection area and the detection box, for example, if the maximum bounding box of the projection area is greater than the detection box, it is determined that the initial ranging value is greater than the final ranging value, and at this time, the initial ranging value is adjusted by reducing the initial ranging value. Similarly, if the maximum circumscribed frame of the projection area is smaller than the detection frame, it is determined that the initial ranging value is smaller than the final ranging value, and the initial ranging value is adjusted by increasing the initial ranging value.
The comparison result (such as area difference) between the maximum external frame and the detection frame of the projection area and the value of the initial distance measurement value increase and decrease can be collected to be used as a training set, the deep learning model is trained through the training set to obtain a trained depth model, and the value of the initial distance measurement value increase or decrease can be directly determined by determining the comparison result between the maximum external frame and the detection frame of the projection area. The final ranging value can be quickly and effectively found, and the calculation amount is reduced.
In one embodiment, determining the direction angle of the target object based on the initial ranging value and the detection image comprises:
acquiring real-time calibration parameters of a camera;
and determining the direction angle of the target object according to the initial distance measurement value, the detection image and the real-time calibration parameter.
Under the condition that the current vehicle and the target object are not on the same plane, the pitch angle of the monocular camera needs to be acquired at the moment, and the real-time calibration parameters of the camera comprise the real-time pitch angle of the monocular camera, wherein the real-time calibration parameters (namely the pitch angle) of the camera can be acquired through real-time dynamic calibration of the camera, specifically, the pose change of the monocular camera is calculated by the monocular camera or the vehicle end according to the pixel change of the front frame and the rear frame of the image, and the real-time calibration parameters of the camera, namely the pitch angle of the monocular camera, are output.
As for the real-time calibration parameters of the camera, as can be known from the above formula (3), the pitch angle of the monocular camera may affect the back-projection algorithm, that is, the initial distance measurement value determined by using the back-projection algorithm or the direction angle of the target object may be affected by the pitch angle of the monocular camera, and therefore, the accurate direction angle of the target object may be obtained only by adding the pitch angle of the monocular camera to the calculation of the direction angle of the target object.
In the embodiment, the real-time calibration parameters of the camera are added into the back projection algorithm, so that the application range of the evaluation method of the monocular camera can be effectively widened, the initial distance measurement value can be effectively evaluated under the condition that the current vehicle and the target object are not on the same plane, and the method is convenient and rapid and has a wide application range.
As shown in fig. 6, in one embodiment, determining the direction angle of the target object from the initial ranging value and the detection image includes:
step S610: determining a first key point and a second key point of the target object according to the detection image, wherein a connecting line of the first key point and the second key point points to the front and back directions of the target object;
step S620: determining the coordinates of the first key point and the coordinates of the second key point according to the back projection algorithm and the initial distance measurement value;
step S630: and determining the direction angle of the target object according to the coordinates of the first key point and the coordinates of the second key point.
For the direction angle of the target object, it is actually the angle formed by the X-axis of the ideal coordinate system of the target object and the X-axis of the ideal coordinate system of the current vehicle. The target object heading angle is determined in the ideal coordinate system of the current vehicle. The coordinates of the first key point and the second key point of the target object are determined, and the connecting line of the first key point and the second key point and the X axis of the ideal coordinate system of the target object, namely the connecting line of the first key point and the second key point points to the front and back directions of the target object. In order to use the back projection algorithm conveniently, the first key point and the second key point which are determined on the detection image are grounding points of the target object.
The coordinates of the first key point and the coordinates of the second key point can be calculated through an inverse projection algorithm and an initial ranging value, the coordinates of the two key points can be three-dimensional coordinates or planar two-dimensional coordinates, and the direction angle of the target object can be determined directly by determining the planar two-dimensional coordinates because the direction angle of the target object has no substantive relation with the Z axis. The coordinate of the first key point and the coordinate of the second key point are put into an ideal coordinate system of the current vehicle, and an included angle between a connecting line of the coordinate of the first key point and the coordinate of the second key point and the ideal coordinate system of the current vehicle is a direction angle of the target object. The direction angle of the target object has a large influence on restoring the contour of the target object, and by determining the direction angle of the target object, the three-dimensional coordinates of the contour of the target object can be effectively determined.
In one embodiment, determining the detection frame of the target object and the real size of the target object according to the detection image comprises:
and performing target object recognition on the detection image according to the trained deep learning model, and acquiring a detection frame of the target object and the real size of the target object.
The deep learning model is trained from the cloud or the vehicle end, the training set can be trained through data acquired by the vehicle end, the detection frame of the target object and the real size of the target object can be effectively determined from the detection image, the accuracy is high, and convenience and rapidness are realized.
In one embodiment, determining the initial ranging value from the detected image comprises:
and performing ranging calculation on the detection image by using an inverse projection algorithm or a target size ranging algorithm to determine an initial ranging value.
The vehicle end can directly use the back projection algorithm or the target size ranging algorithm to carry out ranging on a detection image obtained by shooting of the monocular camera. In the case of using the back projection algorithm, the pitch angle calibrated in real time by the monocular camera is calculated when the pitch angle is 0 or not 0.
The initial ranging value can be effectively determined through an inverse projection algorithm or a target size ranging algorithm, and the determined initial ranging value is used as the ranging value to be evaluated.
In one embodiment, determining the three-dimensional coordinates of the target object contour based on the true size, the initial range value, and the orientation angle comprises:
determining the three-dimensional coordinates of the reference contour of the target object according to the real size;
and adjusting the three-dimensional coordinate of the reference contour according to the initial ranging value and the direction angle, and determining the three-dimensional coordinate of the target object contour.
The vehicle end can construct the contour of the target object according to the real size of the target object by using the ideal coordinate system of the current vehicle, and determine the three-dimensional coordinates of the reference contour of the target object. For example, with the coordinate of the center of the target object as the origin in the ideal coordinate system of the current vehicle, the three-dimensional coordinates of eight vertices of the target object can be determined by the real size of the target object, and the eight vertices can form the outline of the target object, wherein the number of the vertices can be selected according to practical situations, and is at least four or less, and is lower than four outlines which cannot form the most basic cube.
After the three-dimensional coordinates of the reference contour of the target object are determined, the adjusted three-dimensional coordinates are obtained for the three-dimensional coordinates of the reference contour of the target object through the initial ranging value and the direction angle of the target object, and the adjusted three-dimensional coordinates are the three-dimensional coordinates of the contour of the target object. For example, after the three-dimensional coordinates of the reference contour of the target object are determined, the three-dimensional coordinates of the reference contour of the target object are translated through the initial ranging value, the three-dimensional coordinates of the translated reference contour of the target object are adjusted according to the direction angle of the target object, and finally the adjusted three-dimensional coordinates are obtained, namely the three-dimensional coordinates of the contour of the target object.
In this embodiment, the three-dimensional coordinates of the contour of the target object can be determined again by the real size, the initial distance measurement value and the direction angle, and the contour of the target object can be determined in an ideal coordinate system of the current vehicle, so that a projection area can be generated to evaluate the initial distance measurement value.
In one embodiment, generating the projection region of the target object based on the three-dimensional coordinates and the internal parameters of the contour of the target object comprises:
and projecting the three-dimensional coordinates of the outline of the target object on the detection image through the internal parameters to generate a projection area of the target object.
The vehicle end acquires internal parameters of the monocular camera, wherein the internal parameters can include an internal parameter matrix K of the camera, the internal parameter matrix K is an inherent attribute of a lens of the camera, and the internal parameter matrix K of the camera can be acquired directly through the monocular camera as follows.
Wherein f is x Is the focal length of the camera in the direction of the u-axis of the image coordinate system of the camera, f y Is the focal length of the camera in the direction of the v-axis of the image coordinate system of the camera, c x Is the u-axis coordinate of the origin (optical center) of the image coordinate system of the camera in the pixel coordinate system, c y Is the v-axis coordinate of the origin (optical center) of the image coordinate system of the camera in the pixel coordinate system, i.e. c x Coordinates representing the optical center in the u-axis direction of the camera's image coordinate system, c y Representing the coordinates of the light point in the direction of the v-axis of the camera's image coordinate system.
And converting the three-dimensional coordinates of the contour of the target object into two-dimensional coordinates of the image through an internal reference matrix K of the camera, and determining the contour surrounded by all the two-dimensional coordinates as a projection area of the target object. In order to compare the initial ranging value with the detection frame of the target object on the detection image, the outline surrounded by all the two-dimensional coordinates can be directly projected onto the detection image, so that the projection area of the target object and the detection frame of the target object can be evaluated and compared, and the accuracy of the initial ranging value can be judged.
Embodiments of the present application also provide a computer program product, and the computer program/instructions can be executed by one processor or by a plurality of processors to realize any one of the methods provided by the embodiments of the present application.
It should be understood that the processor may be a Central Processing Unit (CPU), other general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or any conventional processor or the like. It is noted that the processor may be a processor supporting Advanced reduced instruction set machine (ARM) architecture.
The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, training device, or data center to another website site, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.).
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other physical types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Disks (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage media, or any other non-transmission medium, that can be used to store information that can be accessed by a computing device. As defined herein, computer readable Media does not include non-Transitory computer readable Media (transient Media), such as modulated data signals and carrier waves.
Program code for implementing the methods of the present application may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
It should be understood that various forms of the flows shown above, reordering, adding or deleting steps, may be used. For example, the steps described in the present application may be executed in parallel, sequentially, or in different orders, and are not limited herein as long as the desired results of the technical solutions of the present application can be achieved. The above embodiments are only specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive various changes or substitutions within the technical scope of the present application, and these should be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (11)
1. A distance measurement method of a monocular camera is characterized by comprising the following steps:
acquiring a detection image and internal parameters of a camera;
determining an initial ranging value, a detection frame of a target object and the real size of the target object according to the detection image;
determining a direction angle of a target object according to the initial ranging value and the detection image;
determining a three-dimensional coordinate of the contour of the target object according to the real size, the initial distance measurement value and the direction angle;
generating a projection area of the target object according to the three-dimensional coordinates of the target object contour and the internal parameters;
and determining a final ranging value according to the projection area and the detection frame.
2. The method of claim 1, wherein determining a final ranging value according to the projection area and the detection block comprises:
and under the condition that the maximum external frame of the projection area is consistent with the detection frame, determining the initial ranging value as a final ranging value.
3. The method of claim 1, wherein determining a final ranging value according to the projection area and the detection block comprises:
adjusting an initial distance measurement value under the condition that the maximum external frame of the projection area is inconsistent with the detection frame;
re-determining the three-dimensional coordinates of the contour of the target object according to the real size of the target object, the direction angle of the target object and the adjusted initial distance measurement value;
generating a projection area of the adjusted target object according to the internal parameters of the camera and the three-dimensional coordinates of the contour of the target object which is determined again;
and under the condition that the projection area of the adjusted target object is consistent with the detection frame, determining the adjusted initial ranging value as a final ranging value.
4. The method of claim 3, wherein the adjusting the initial ranging value comprises:
and adjusting the initial ranging value according to a comparison result between the maximum external frame of the projection area and the detection frame.
5. The method of claim 1, wherein determining the directional angle of the target object based on the initial range value and the detected image comprises:
acquiring real-time calibration parameters of a camera;
and determining the direction angle of the target object according to the initial distance measurement value, the detection image and the real-time calibration parameter.
6. The method of claim 1, wherein determining the directional angle of the target object based on the initial range value and the detected image comprises:
determining a first key point and a second key point of a target object according to the detection image, wherein a connecting line of the first key point and the second key point points to the front and back directions of the target object;
determining the coordinates of the first key point and the coordinates of the second key point according to an inverse projection algorithm and the initial ranging value;
and determining the direction angle of the target object according to the coordinates of the first key point and the coordinates of the second key point.
7. The method of claim 1, wherein determining a detection frame of a target object and a real size of the target object from the detection image comprises:
and carrying out target object identification on the detection image according to the trained deep learning model to obtain a detection frame of the target object and the real size of the target object.
8. The method of claim 1, wherein determining an initial range value from the detected image comprises:
and performing ranging calculation on the detection image by using an inverse projection algorithm or a target size ranging algorithm to determine an initial ranging value.
9. The method of claim 1, wherein determining three-dimensional coordinates of a target object contour based on the real size, the initial range value, and the orientation angle comprises:
determining the three-dimensional coordinates of the reference contour of the target object according to the real size;
and adjusting the three-dimensional coordinate of the reference contour according to the initial distance measurement value and the direction angle, and determining the three-dimensional coordinate of the contour of the target object.
10. The method of claim 1, wherein generating a projection region of a target object based on the three-dimensional coordinates of the target object contour and the internal parameters comprises:
and projecting the three-dimensional coordinates of the target object outline on the detection image through the internal parameters to generate a projection area of the target object.
11. A computer program product comprising computer program/instructions, characterized in that the computer program/instructions, when executed by a processor alone or in combination with a processor, implement the method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211215921.0A CN115511975A (en) | 2022-09-30 | 2022-09-30 | Distance measurement method of monocular camera and computer program product |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211215921.0A CN115511975A (en) | 2022-09-30 | 2022-09-30 | Distance measurement method of monocular camera and computer program product |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115511975A true CN115511975A (en) | 2022-12-23 |
Family
ID=84507351
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211215921.0A Pending CN115511975A (en) | 2022-09-30 | 2022-09-30 | Distance measurement method of monocular camera and computer program product |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115511975A (en) |
-
2022
- 2022-09-30 CN CN202211215921.0A patent/CN115511975A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11719788B2 (en) | Signal processing apparatus, signal processing method, and program | |
| WO2022022694A1 (en) | Method and system for sensing automated driving environment | |
| CN110146869B (en) | Method and device for determining coordinate system conversion parameters, electronic equipment and storage medium | |
| CN110148185B (en) | Method and device for determining coordinate system conversion parameters of imaging equipment and electronic equipment | |
| CN110119698B (en) | Method, apparatus, device and storage medium for determining object state | |
| CN110794406B (en) | Multi-source sensor data fusion system and method | |
| CN110969055B (en) | Method, apparatus, device and computer readable storage medium for vehicle positioning | |
| CN114705121B (en) | Vehicle pose measurement method and device, electronic equipment and storage medium | |
| CN112106111A (en) | Calibration method, calibration equipment, movable platform and storage medium | |
| CN115797454B (en) | Multi-camera fusion sensing method and device under bird's eye view angle | |
| CN111080784A (en) | Ground three-dimensional reconstruction method and device based on ground image texture | |
| US20230206500A1 (en) | Method and apparatus for calibrating extrinsic parameter of a camera | |
| US11244473B2 (en) | Positioning method, positioning apparatus of mobile device and electronic device | |
| CN112927309A (en) | Vehicle-mounted camera calibration method and device, vehicle-mounted camera and storage medium | |
| CN115097419A (en) | External parameter calibration method and device for laser radar IMU | |
| CN113256709A (en) | Target detection method, target detection device, computer equipment and storage medium | |
| CN112529011A (en) | Target detection method and related device | |
| KR102003387B1 (en) | Method for detecting and locating traffic participants using bird's-eye view image, computer-readerble recording medium storing traffic participants detecting and locating program | |
| CN114648639B (en) | Target vehicle detection method, system and device | |
| CN115511975A (en) | Distance measurement method of monocular camera and computer program product | |
| US11210535B1 (en) | Sensor fusion | |
| CN113409376A (en) | Method for filtering laser radar point cloud based on depth estimation of camera | |
| CN115147612B (en) | Processing method for estimating vehicle size in real time based on accumulated point cloud | |
| CN112733697B (en) | Method and device for determining yaw angle of vehicle | |
| CN113537161B (en) | Obstacle identification method, system and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |