CN117058210A - Distance calculation method and device based on vehicle-mounted sensor, storage medium and vehicle - Google Patents
Distance calculation method and device based on vehicle-mounted sensor, storage medium and vehicle Download PDFInfo
- Publication number
- CN117058210A CN117058210A CN202311309292.2A CN202311309292A CN117058210A CN 117058210 A CN117058210 A CN 117058210A CN 202311309292 A CN202311309292 A CN 202311309292A CN 117058210 A CN117058210 A CN 117058210A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- distance
- image
- target object
- reference point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004364 calculation method Methods 0.000 title claims abstract description 45
- 238000003860 storage Methods 0.000 title claims abstract description 8
- 238000000034 method Methods 0.000 claims abstract description 55
- 238000004590 computer program Methods 0.000 claims description 9
- 238000010586 diagram Methods 0.000 description 11
- 230000008569 process Effects 0.000 description 10
- 241000283070 Equus zebra Species 0.000 description 8
- 238000013178 mathematical model Methods 0.000 description 8
- 238000005259 measurement Methods 0.000 description 8
- 239000011159 matrix material Substances 0.000 description 5
- 230000008859 change Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 239000003550 marker Substances 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 238000012625 in-situ measurement Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012067 mathematical method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 238000011002 quantification Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/02—Measuring arrangements characterised by the use of optical techniques for measuring length, width or thickness
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
Abstract
The invention discloses a distance calculating method and device based on a vehicle-mounted sensor, a storage medium and a vehicle, wherein the method comprises the following steps: acquiring a vehicle environment image and determining the position of a target object in the vehicle environment image; acquiring corresponding calibration information based on the position; and determining the distance information between the target object and the vehicle based on the calibration information. According to the method and the device and the vehicle, the distance calculation is simpler and faster, and the driving safety of the vehicle is improved.
Description
Technical Field
The invention relates to the technical field of vehicles, in particular to a distance calculating method based on an on-vehicle sensor, a distance calculating device based on the on-vehicle sensor, a computer readable storage medium and a vehicle.
Background
Depth estimation is a fundamental problem in the field of computer vision, which can be applied in the fields of robot navigation, augmented reality, three-dimensional reconstruction, autopilot, etc. At present, monocular distance calculation is mostly based on a camera calibration mode, an internal reference matrix and an external reference matrix of a camera are calculated, image coordinates and world coordinates are converted with each other, then distance estimation is carried out, and the calculation method is complex and low in efficiency.
Disclosure of Invention
The present invention aims to solve at least one of the technical problems existing in the prior art. Therefore, a first object of the present invention is to provide a distance calculating method based on a vehicle-mounted sensor, which does not depend on camera parameter calibration, improves the efficiency of calculating the distance, and is simple and quick.
A second object of the present invention is to provide a distance calculating device based on an in-vehicle sensor.
A third object of the present invention is to propose a computer readable storage medium.
A fourth object of the present invention is to propose a vehicle.
In order to achieve the above object, a distance calculating method based on an on-vehicle sensor according to an embodiment of the first aspect of the present invention includes: acquiring a vehicle environment image and determining the position of a target object in the vehicle environment image; acquiring corresponding calibration information based on the position; and determining the distance information between the target object and the vehicle based on the calibration information.
According to the distance calculating method based on the vehicle-mounted sensor, the distance information of the target object and the vehicle is determined based on the calibration information, the calculating method is simple and quick, and the distance calculating efficiency is improved.
In some embodiments, the calibration information at least includes a correspondence between pixels of each position of the image acquired by the vehicle-mounted sensor and a target associated with pixels of each position of the image, and distance information between the target associated with pixels of each position of the image and a vehicle.
In some embodiments, the distance information between the target object associated with the pixel points at each position of the image and the vehicle is obtained based on actual distance calibration between the reference points which are arranged in parallel and at equal intervals on the road surface and the vehicle.
In some embodiments, the distance information of the object associated with the pixels of each position of the image from the vehicle includes a longitudinal distance of the object associated with the pixels of each position of the image from the vehicle; the longitudinal distance between the target object associated with the pixel points at each position of the image and the vehicle is calibrated by the following steps: acquiring an image comprising reference points which are arranged in parallel and at equal intervals on the road surface, wherein the arrangement direction of the reference points is the longitudinal direction of the vehicle; determining the position of any first pixel point along the arrangement direction of the reference points on the pavement in the image of the reference points; determining a proportional relation satisfied by a reference point in an image of the arbitrary first pixel point and the reference point based on the position of the arbitrary first pixel point; and obtaining the longitudinal distance between the target object associated with any one first pixel point and the vehicle according to the proportional relation satisfied by the reference point in the image of the reference point and the actual distance between the reference point and the vehicle.
In some embodiments, the distance information of the object associated with the pixel point at each position of the image and the vehicle further includes a lateral distance of the object associated with the pixel point at each position of the image and the vehicle; the lateral distance between the target object associated with the pixel points at each position of the image and the vehicle is calibrated by the following steps: the image of the reference point is perpendicular to the reference point and is provided with a perpendicular bisector, the perpendicular bisector is correspondingly determined according to the distance of the perpendicular bisector of the connecting line of the two front wheels of the vehicle, and the vehicle-mounted sensor is arranged on the perpendicular bisector; determining the position of any second pixel point on two sides of the perpendicular bisector in the image of the reference point; determining a proportional relation between any second pixel point and the reference point in the image of the reference point based on the position of the any second pixel point; and obtaining the transverse distance from the target object associated with any second pixel point to the vehicle according to the proportional relation between the any second pixel point and the reference point in the image of the reference point and the actual distance from the reference point to the perpendicular bisector.
In some embodiments, determining distance information of the target object from the vehicle based on the calibration information includes: obtaining a first longitudinal distance from a first object in the vehicle environment image to a vehicle, and obtaining a second longitudinal distance from a second object in the vehicle environment image to the vehicle; obtaining a first lateral distance of the first object to the vehicle and a second lateral distance of the second object to the vehicle; and obtaining the relative distance between the first target object and the second target object according to the first longitudinal distance, the second longitudinal distance, the first transverse distance and the second transverse distance.
In some embodiments, determining distance information of the target object from the vehicle based on the calibration information includes: identifying that the target object in the vehicle environment image is a target object in the height direction; and determining the relative height of the target object and the vehicle based on the calibration information.
In some embodiments, the distance information of the target object associated with the pixel points of the respective positions of the image and the vehicle includes the relative height of the target object associated with the pixel points of the respective positions of the image and the vehicle; the relative heights of the target objects associated with the pixel points at each position of the image and the vehicle are calibrated through the following steps: the reference points which are arranged in parallel and at intervals on the road surface are provided with heights which are perpendicular to the road surface, and the actual relative heights of the reference points and the vehicle-mounted sensor are measured; determining the position of any third pixel point in the image of the reference point; determining the relative height of any third pixel point and the road surface based on the position of the any third pixel point; determining the ratio relation between the relative height of any third pixel point and the road surface and the relative height of the reference point and the road surface; and obtaining the relative height from the target object associated with any one third pixel point to the vehicle according to the proportional relation satisfied by the relative height and the actual relative height between the reference point and the vehicle-mounted sensor.
In some embodiments, the distance calculation method further comprises: acquiring vehicle type information of a vehicle; obtaining a calibration configuration file according to the vehicle type information; and acquiring the calibration information in the calibration configuration file.
In order to achieve the above object, an in-vehicle sensor-based distance calculation device according to a second aspect of the present invention includes: a processor; a memory communicatively coupled to the processor; the memory stores a computer program executed by the processor, and the distance calculating method based on the vehicle-mounted sensor is realized when the computer program is executed by the processor.
According to the distance calculating device based on the vehicle-mounted sensor, the distance calculating method based on the vehicle-mounted sensor is simple and quick, and the efficiency of distance calculation is improved.
The computer readable storage medium of the embodiment of the third aspect of the present invention stores thereon a computer program that when executed implements the distance calculation method based on the vehicle-mounted sensor.
An embodiment of a fourth aspect of the present invention includes: the vehicle-mounted sensor is used for collecting vehicle environment images; the distance calculating device based on the vehicle-mounted sensor of the above embodiment is connected to the vehicle-mounted sensor.
According to the vehicle provided by the embodiment of the invention, the distance calculating device based on the vehicle-mounted sensor is adopted, so that the distance calculating method is simple and quick, and the driving safety is improved.
Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
The foregoing and/or additional aspects and advantages of the invention will become apparent and may be better understood from the following description of embodiments taken in conjunction with the accompanying drawings in which:
FIG. 1 is a flow chart of a calibration process for an actual calibration object according to one embodiment of the invention;
FIG. 2 is a flow chart of a method of vehicle sensor-based distance calculation according to one embodiment of the invention;
FIG. 3 is a schematic diagram of a mathematical model of calculating longitudinal distances according to one embodiment of the invention;
FIG. 4 is a schematic diagram of a mathematical model for calculating lateral distance according to one embodiment of the invention;
FIG. 5 is a flow chart of a method of vehicle sensor-based distance calculation according to one embodiment of the invention;
FIG. 6 is a schematic diagram of a mathematical model for a target height less than a camera height according to one embodiment of the invention;
FIG. 7 is a schematic diagram of a mathematical model for a target height above the camera height according to one embodiment of the invention;
FIG. 8 is a schematic diagram of an in-vehicle sensor-based distance calculation apparatus according to one embodiment of the invention;
fig. 9 is a block diagram of a vehicle according to one embodiment of the invention.
Detailed Description
Embodiments of the present invention will be described in detail below, by way of example with reference to the accompanying drawings.
In order to solve the problems of low distance calculating efficiency and low accuracy of the existing monocular distance measuring method, the method is based on the fact that the actual object is used as a reference point to perform calibration, in an embodiment, at least two objects, drawn lines or marked points or other objects which are arranged at intervals on a road surface in practice are used as the reference point, preferably, the actual calibration object used as the reference point can be a specific calibration object which is favorable for the calibration of the actual distance, for example, the actual calibration object with the condition of spacing can be presented in one direction by adopting the actual calibration objects such as zebra stripes, regional speed reducing belts and roadblocks, the actual distance calibration is convenient, the corresponding image is obtained to perform the calibration of an image coordinate system, namely, the scene of the actual calibration object forms a calibration scale, and the accuracy of the distance calculation is improved.
The calibration based on the actual reference point according to the embodiment of the present invention will be described first.
For example, taking the actual calibration object as the zebra stripes as the reference points, the zebra stripes have the advantages of being equidistant, parallel and easy to distinguish, and in an embodiment, the actual calibration object is not necessarily limited to the zebra stripes. Any reference points arranged at intervals on the road surface can be used, for example, the distance can be measured, and the distance objects which can form auxiliary lines parallel to the upper edge and the lower edge of the picture can meet the condition.
Due to the perspective principle, the farther the road sign spacing is, the closer the corresponding pixel values on the screen projection are, and in the worst case, the adjacent two road signs may coincide. I.e. the farther away the calculated value error must be. From an examination of the above quantitative procedure, it can be found that: after the road sign is placed, the real world scale is defined and is invariable; the camera height changes necessarily result in changes in the coordinates of the measurement target and calibration object in the actual picture. Adjusting the camera resolution necessarily results in a change in the coordinates of the measurement target and the calibration object in the actual picture. After the camera height and resolution are determined, the coordinates of the actual calibration object and the actual physical distance can be saved.
Further, after the on-site quantification process is finished, a set of fixed parameters such as road sign parameters, camera height and the like can be obtained, and the parameters can be encoded into a configuration file, namely calibration information of an actual calibration object can be configured in the configuration file.
FIG. 1 is a flow chart of a calibration process for an actual calibration object according to one embodiment of the invention, as shown in FIG. 1, comprising:
s1, making a horizontal vertical line perpendicular to the connecting line of two front wheels of the vehicle.
S2, placing the target camera on the horizontal vertical line, and measuring the height of the camera.
S3, ensuring that the camera view is facing to the front.
S4, measuring the distance between the calibration object in the camera field of view and the horizontal vertical line.
S5, measuring the distance between the calibration object in the visual field of the camera and the camera in the longitudinal axis direction.
And S6, saving the obtained data as a configuration file.
The calibration information of the actual reference points is calibrated by a preset vehicle model, so that the camera position (height, angle and the like) can be ensured to be fixed, errors caused by position change do not need to be considered in the corresponding relation in the generated configuration file, or position parameters only need to be adjusted by a small amount according to the vehicle model change, and meanwhile, the certainty of zebra crossing data brings convenience to scheme implementation.
Therefore, in some embodiments, calibration information of the actual reference point may be calibrated corresponding to different vehicle types, and configuration files corresponding to various vehicle types may be pre-stored. Therefore, in practical application, the vehicle type information of the vehicle can be obtained, the calibration configuration file is obtained according to the vehicle type information, and the calibration information of the actual reference point corresponding to the vehicle type in the calibration configuration file is obtained more accurately.
Further, based on calibration information of the actual reference point, when the distance calculation is performed, the physical distance corresponding to the given image coordinate can be obtained based on a specific mathematical method. The essence of the operation step is to find the mapping of any point coordinate on the screen in the real world (larger errors are caused if the mapping is simulated by adopting a linear interpolation method). After the predetermined distance data is obtained, the depth distance/horizontal distance corresponding to the screen pixel point can be obtained after the finite mathematical operation step; and these mathematical operations are easily implemented in mainstream programming languages. And if it is desired to eliminate the calculation step, it is possible to consider laying as many reference points as possible on the road surface and then taking samples. When the program is implemented, only a binary search is used for finding the closest reference point through the vertical axis value of the target object, and the physical distance of the reference point is returned. In principle, the more reference points the smaller the error. For example, if the project requires less than 5 cm error, then a reference point is set every 5 cm, and then the error in reading from the screen to obtain the actual distance must be less than 5 cm.
A distance calculating method based on an in-vehicle sensor according to an embodiment of the present invention is described below with reference to the accompanying drawings. The distance calculation method according to the embodiment of the invention can be applied to the related distance calculation in the driving process, for example, the distance between vehicles and obstacles, the distance between two objects in front of the vehicles, and the like are calculated, and in the embodiment, the vehicles, the obstacles, and the like can be referred to as targets.
Fig. 2 is a flowchart of a distance calculating method based on an in-vehicle sensor according to an embodiment of the present invention, as shown in fig. 2, including:
s11, acquiring a vehicle environment image and determining the position of a target object in the vehicle environment image.
The vehicle environment image may be acquired by an onboard sensor, for example, an onboard camera placed as in the calibration process described above, to more effectively utilize the calibration information.
The target may be other vehicles, obstacles, pedestrians, etc. The coordinate information of the target object in the image can be directly determined by identifying and analyzing the image.
S12, acquiring corresponding calibration information based on the position.
S13, determining the distance information between the target object and the vehicle based on the calibration information.
Specifically, as described above, calibration information is stored in advance, and can be directly called when the distance calculation is performed on the vehicle, where the calibration information may include information of relevant distance mapped by the object associated with each pixel point in the acquisition view of the vehicle-mounted sensor, such as height, lateral distance, longitudinal distance, and the like.
Therefore, the corresponding calibration information can be obtained by determining the position of the target object in the image, and the distance information corresponding to the target object and corresponding to the vehicle is determined.
According to the distance calculating method based on the vehicle-mounted sensor, the distance information of the target object and the vehicle is determined based on the calibration information, the calculating method is simple and quick, and the efficiency of distance calculation is improved.
In an embodiment, the calibration information at least includes a correspondence between pixels at each position of the image acquired by the vehicle-mounted sensor and a target associated with pixels at each position of the image, and distance information between the target associated with pixels at each position of the image and the vehicle. Therefore, when the driving distance calculation is carried out, the position of the target object is determined, and the distance information between the target object and the vehicle can be obtained based on the calibration information, so that the method is simple and quick.
In the embodiment of the invention, the distance information between the target object associated with the pixel points at each position of the image and the vehicle is obtained based on the actual distance calibration between at least two reference points which are arranged at intervals on the road surface and the vehicle. The reference points, i.e. the actual calibration objects, may include, but are not limited to, zebra stripes, roadblocks arranged at intervals, regional speed-reducing zones, marks or drawn lines or objects on a road surface at intervals, and in an embodiment, the reference points, i.e. the actual calibration objects, preferably have characteristics of equal distance, parallel, and the like, and specifically, calibration information of the actual calibration objects may be obtained through configuration files.
According to the distance calculation method based on the vehicle-mounted sensor, the actual distance of the target object is obtained based on the collection of the image pixel points and the measurement of the real world distance, the two tasks can obtain a high-precision reliable result through a mature and effective method, and the actual distance is conveniently measured and the numerical value is accurate by using the calibration object actually existing in the physical world as a scale, so that the calculated target distance precision is improved. And the outdoor operation is more convenient to find the real-scene calibration object and quantitatively obtain the distance between the real-scene calibration object and the camera.
The following describes the calculation of the longitudinal distance and the lateral distance between the object and the vehicle, the distance between any two objects, and the relative height between the object and the vehicle in the embodiment of the present invention, respectively.
In a first embodiment, a longitudinal distance between a target object and a vehicle is calculated.
The distance information of the target object associated with the pixel points at each position of the image and the vehicle comprises the longitudinal distance between the target object associated with the pixel points at each position of the image and the vehicle;
at least two objects or marks which are arranged at intervals on the road surface can be selected as reference points, and the reference points which are arranged in parallel and at equal intervals on the road surface, such as zebra stripes or speed reduction zones in specific areas, roadblocks which are arranged at intervals and the like, are included. Taking the spaced-arranged roadblocks as an example, the actual distance from each roadblock to the camera, the camera height and other information can be calibrated, and a configuration file can be generated.
With FIG. 3 illustrating a calculation according to one embodiment of the inventionSchematic representation of a mathematical model of the longitudinal distance. Wherein the O 'point is the camera position, and the O point is the photographer's foothold. A. Points B and X correspond to actual points in the physical world, i.e., points on an actual marker such as a zebra line or spaced-apart roadblocks. A ', B ', and X ' correspond to points in the photographed picture, i.e., image coordinate points. Plane surfaceTaking an X point as an object as an example, the imaging plane of the camera is taken as an imaging plane of the camera.
The longitudinal distance between the target object associated with the pixel points at each position of the image and the vehicle is obtained through the following steps:
acquiring an image comprising at least two reference points which are arranged at intervals on a road surface, wherein the arrangement direction of the reference points is the longitudinal direction of a vehicle; determining the position of any first pixel point along the arrangement direction of the reference points on the road surface in the image of the reference points; determining a proportional relation which is met by a reference point in an image of any first pixel point and the reference point based on the position of any first pixel point; and obtaining the longitudinal distance between the target object associated with any first pixel point and the vehicle according to the proportional relation satisfied by the reference point in the image of the any first pixel point and the reference point and the actual distance between the reference point and the vehicle.
In particular, as shown in FIG. 3,
step 1, for trianglesBecause it is a right triangle, it can be calculated according to the Pythagorean theorem:
and OA is known to be, for example, 2740 mm, -/->For the lens height also known as 1570 mm, for example, therefore calculate +.>3157.92653 mm.
Step 2: in a triangle shapeCan be calculated according to the Pythagorean theorem:
and OB is known to be 4110 mm, < + >, for example>For the lens height also known as 1570 mm, for example, therefore calculate +.>4399.65907 mm.
Step 3: inspection triangleIt is also right triangle based on camera presentation principle, thus sideParallel to the side->. From the property of equal stagger angles in parallel lines, the following relationship is known:
；
；
；
the following relation is defined according to tangent:
；
；
in the triangle shapeIn (1), know->And->The corresponding pixel distance is 1859-1576=283;
can not be provided withAnd->Namely, the following relation exists:
；
step 4:
；
；
thus the pixel unit distance corresponding to h can be solved as 849;
thenA pixel unit;
and is known toAnd->Can solve for the pixel coordinates of +.>；
Then；
Step 5:
；
；
；
the method can obtain:substituting each value to obtain the calculated value of OX:
millimeter.
For OX, the absolute error between calculated and actual measured values is 3540-3503.39 = 36.61 mm, less than the required 5 cm, i.e. 50 mm; the relative error was (3540-3503.39)/5340=0.685%.
Therefore, the longitudinal distance OX from the road surface reference point corresponding to the X 'point on the image to the camera can be obtained, the distance of AX or the distance from the X to other set reference points can be obtained, the position of the X' on the image can be determined, the corresponding relation between the image and the road surface reference point can be established, and based on the corresponding relation set of the road surface reference point corresponding to each point on the image to the distance of the camera can be established, and the corresponding relation set is used as calibration information of the longitudinal distance. When an image is obtained in the driving process, the distance between the relevant point and the vehicle can be directly obtained according to the relevant point.
In the second embodiment, the calculation of the lateral distance between the target object and the vehicle, specifically, for example, an in-vehicle sensor is performed.
The distance information between the object associated with the pixel point at each position of the image and the vehicle further comprises the lateral distance between the object associated with the pixel point at each position of the image and the vehicle.
As described above in the calibration process, in some embodiments, the image of the reference point has a perpendicular bisector perpendicular to the reference point, the perpendicular bisector corresponding being determined according to the distance of the perpendicular bisector of the line connecting the two front wheels of the vehicle, and the vehicle-mounted sensor is disposed on the perpendicular bisector.
The transverse distance between the target object and the vehicle can be obtained through the following steps: determining the position of any second pixel point on two sides of the perpendicular bisector in the image of the reference point; determining the proportional relation between any second pixel point and a reference point in the image of the reference point based on the position of any second pixel point; and obtaining the transverse distance between the target object associated with any second pixel point and the vehicle according to the proportional relation between the any second pixel point and the reference point in the image of the reference point and the actual distance between the reference point and the perpendicular bisector.
Specifically, FIG. 4 is a schematic diagram of a mathematical model for calculating lateral distance according to one embodiment of the invention.
As shown in fig. 4, wherein the bottom black line in the figure is the bottom side of the camera field of view. Wherein the method comprises the steps ofIs an auxiliary line parallel to the bottom black line, < >>And the same is true. Similarly, as many horizontal auxiliary lines as possible can be added in the real world, and then photographed and recorded in the photograph.
The black line perpendicular to the bottom of the figure is the vertical dividing line of the camera field of view, whose abscissa is equal to the width of the screen resolution, e.g. defined as. Definitions of do not jeopardize>And +.>The intersection point with the vertical line is +.>And +.>(X02' in the figure).
It will be appreciated that the object solving problem is described using a mathematical language as: is known to beForming a rectangle. />Distance measurement of +.>I.e. +.>. Rectangular->Inner one point->To->Distance (center vertical).
The lateral distance complete calculation steps are listed as follows:
step 1: sampling by drawing software to obtain pointsAnd->The specific pixel coordinate values of (a) may be respectively set asAnd->。
Step 2: determining a straight line according to two points of the plane to obtain the straight lineThe diagonal equations of (2) are:
step 3: now, the target object pixel pointOrdinate->Substituting the equation can solve the corresponding abscissa as:
；
step 4: then in the image coordinate system, it can be calculated by the following formulaAnd->Is proportional to:
wherein->And->Are known amounts.
Step 5: finally combineThe target distance is +.>。
Further, the actual application may have intersection pointsAnd +.>Difficulty in positioning or +.>And the measurement is difficult. Can be considered according to->The coordinate edge is obtained by the above calculation step to right straight line +.>Intercept->Corresponding inverse function. Target point->Ordinate->Substitution of the inverse function +.>The right straight line can be calculated>Go up the corresponding abscissa +.>. Now know +.>And->At the same time, their corresponding physical world distances can also be obtained by in-situ measurement, which is not just +.>Thus, it can be calculated +.>. Then the target distance is calculated as。
From this, X on the image can be known t And based on the transverse distance from the point corresponding road surface reference point to the camera, establishing a corresponding relation between the image and the road surface reference point, and based on the transverse distance, establishing a corresponding relation set of the distance from the road surface reference point corresponding to each point on the image to the camera, wherein the corresponding relation set is used as calibration information of the transverse direction. When an image is obtained in the driving process, the transverse position of the point can be directly obtained according to the related point position, such as the distance of shifting the current lane.
Embodiment three, for the calculation of the relative distance between any two objects.
Namely, the target objects comprise a first target object and a second target object, which correspond to any two points in the screen respectively.
The calculation of the relative distance between any two objects includes: obtaining a first longitudinal distance from a first object in the vehicle environment image to the vehicle and a second longitudinal distance from a second object in the vehicle environment image to the vehicle; obtaining a first lateral distance of the first object to the vehicle and a second lateral distance of the second object to the vehicle; the relative distance between the first object and the second object is obtained from the first longitudinal distance, the second longitudinal distance, the first lateral distance and the second lateral distance. The actual longitudinal distance and the actual transverse distance may refer to the specific calculation procedures in the first embodiment and the second embodiment.
Specifically, it is assumed that two points A and B in the screen coordinate system correspond to the coordinates ofAnd->. Then the method described above for the calculation of the longitudinal distance and the lateral distance is combined:
step 1: distance of point A to reference point in the longitudinal axis directionThe actual distance +.A. of point B to the reference point in the direction of the longitudinal axis>. Wherein the actual distance of the pixel point in the longitudinal direction is +.>Is constant non-negative because the observed object cannot appear inside the screen.
Step 2: actual distance of point A to reference point in the transverse axis directionThe actual distance +.A. of point B to the reference point in the direction of the horizontal axis>. Wherein, in calculating->And->At this time, since the points A and B may appear on the left and right sides of the center vertical line, < +.>And->It is possible to take positive and negative values, so that the distance between the point A and the point B in the transverse axis direction is the final value +.>。
Step 3: according to the Pythagorean theorem, the absolute distance between the two points A and B in the real world coordinate system can be calculated as follows:
。
FIG. 5 is a flow chart of an in-vehicle sensor-based distance calculation process, as shown in FIG. 5, according to one embodiment of the invention, including:
s111, receiving pixel coordinates of an object in the camera view from outside.
And S112, searching a corresponding target grid in the distance grid according to the pixel coordinates.
S113, calculating the distance from the target grid to the camera according to the ordinate of the target object.
S114, calculating the distance from the target grid to the vertical line according to the abscissa of the target object.
S115, if the two points are spaced apart, calculating the distance from the other point to the vertical line.
And S116, combining the two distances to calculate the distance between the two points in the horizontal direction.
S117, calculating the absolute distance between the two points in the real world by combining the distances in the horizontal and vertical directions.
In the fourth embodiment, the relative height of the object and the vehicle is calculated.
In an embodiment, after a vehicle environment image is acquired, pattern recognition is performed on the vehicle environment image, and when it is recognized that a target object in the vehicle environment image is a target object in a height direction, such as a traffic light, the relative height between the target object and the vehicle is determined based on calibration information.
That is, the distance information of the object associated with the pixel point at each position of the image and the vehicle includes the relative height of the object associated with the pixel point at each position of the image and the vehicle.
In an embodiment, the relative heights of the target objects associated with the pixels at each position of the image and the vehicle are obtained through the following steps: the reference points which are arranged in parallel and at intervals on the road surface are vertical to the road surface, and the actual relative height between the reference points and the vehicle-mounted sensor is measured; determining the position of any third pixel point in the image of the reference point; determining the relative height of any one third pixel point and the road surface based on the position of any one third pixel point; determining the ratio relation between the relative height of any third pixel point and the road surface and the relative height of the reference point and the road surface; and obtaining the relative height from the target object associated with any third pixel point to the vehicle according to the proportional relation satisfied by the relative height and the actual relative height between the reference point and the vehicle-mounted sensor.
Specifically, fig. 6 is a schematic diagram of a mathematical model for a target height less than the camera height, and fig. 7 is a schematic diagram of a mathematical model for a target height greater than the camera height.
Referring to fig. 6 and 7, step 1: ranging the scene along the original longitudinal axis, inThe point position is provided with a marking object which is perpendicular to the road surface and has a height h; in the same way->Point (s)/or(s)>The same set is made for the points and the subsequent mark points.
Step 2: the contact point between the marker perpendicular to the road surface and the road surface is not limited to、/>、/>Let it be +.>And->In the interval there is an object of a height to be determined +.>The contact point with the road surface is +.>Because of->，/>，/>，/>Are all perpendicular to the road surface, then +.>To->Distance of->；
Step 3: at the position ofIn (1) can be obtained->. Also because of->Parallel->Therefore, it is->There is->I.e. +.>；
Step 4: at the position ofIn (1) can be obtained->. Also because of->Parallel->Therefore, it is->There is->I.e. +.>；
Step 5: combining step 3 and step 4, can obtainThe method comprises the steps of carrying out a first treatment on the surface of the Also because of->And->Correspond to->And->Is thus aware of the screen coordinates of +.>The method comprises the steps of carrying out a first treatment on the surface of the Thereby can be used forTo calculate +.>；
Step 6: based on step 5Length value and relation +.>Can calculate +.>. Similarly, can get +.>；
Step 7: the coordinates H of the upper end point of the object to be measured in the image can be read from the image, thereby obtainingThe corresponding length is +.>. The following discussion is in case:
step 8: if it isThen->Then->The method comprises the steps of carrying out a first treatment on the surface of the And because of，/>Then->。
Step 8.1:for measuring the vertical distance of the coordinates of the object mount on the road surface to the camera mount, which can be calculated based on the above, then +.>；
Step 8.2: then the target is measuredThe height value of (2) is +.>；
Step 9: if it isI.e. +.>At->On the straight line, the measurement target +.>The height value of (2) is；
Step 10: if it isThen->Then->The method comprises the steps of carrying out a first treatment on the surface of the And because of，/>I.e. +.>；
Step 10.1: can be calculated to obtainAt this time->。
Therefore, the height from the road surface reference point corresponding to the X point on the image to the vehicle-mounted sensor can be obtained, the corresponding relation between the image and the road surface reference point is established based on the height, and the geometric relation between the reference point with the height corresponding to each point on the image and the height of the vehicle-mounted sensor can be established, so that the geometric relation is used as calibration information of the height direction.
In the driving process, an image is acquired, and when an object such as a traffic light lamp with the height direction of the target object is identified, the height position of the point can be directly acquired based on calibration information, so that the method is simple and quick.
In the BEV coordinate plane, the top end corresponds to the measured object moving in the horizontal directionThe value will also change slightly due to the perspective principle, resulting in a certain deviation of the height value determined by the method. However, we consider that the deviation value generated in the field of view of the camera is small and negligible.
In general, the distance calculation method of the embodiment of the invention uses the calibration object actually existing in the physical world as a scale, and the actual distance is convenient to measure and has accurate value. For example, the roadblock area is selected as an experimental site, and the placed roadblock is used as an inter-interval distance measuring object. And, calculate the target distance on the basis of the distance that the calibration object and measurement obtained.
Compared with the method of calculating the camera internal reference matrix and the camera external reference matrix based on the camera calibration mode and performing distance estimation after the image coordinates and the world coordinates are converted, the method of the embodiment of the invention is more convenient for finding out the real-scene calibration object and quantitatively obtaining the distance between the real-scene calibration object and the camera during outdoor operation; the matrix transformation calculation is not involved, so that the calculation cost is lower, and the program response is faster; and the distance measurement is performed based on the actual object, so that the result accuracy is higher.
Based on the distance calculating method based on the vehicle-mounted sensor in the above embodiment, an embodiment of the second aspect of the present invention provides a distance calculating device based on the vehicle-mounted sensor.
Fig. 8 is a schematic diagram of an in-vehicle sensor-based distance calculation apparatus according to an embodiment of the present invention, and as shown in fig. 8, the in-vehicle sensor-based distance calculation apparatus 10 includes a processor 11 and a memory 12.
Wherein the memory 12 is communicatively connected to the processor 11; the memory 12 stores therein a computer program executed by the processor, which when executed by the processor implements the distance calculation method based on the in-vehicle sensor of the above embodiment.
According to the distance calculating device 10 based on the vehicle-mounted sensor of the embodiment of the invention, by executing the distance calculating method based on the vehicle-mounted sensor of the above embodiment, the calculation result is more accurate and the implementation is simple.
The third aspect of the present invention also proposes a computer-readable storage medium having stored thereon a computer program which, when executed, implements the on-vehicle sensor-based distance calculation method of the above embodiment.
The fourth aspect of the present invention also proposes a vehicle, fig. 9 is a block diagram of a vehicle according to an embodiment of the present invention, and as shown in fig. 9, the vehicle 1 includes an in-vehicle camera 20 and the in-vehicle sensor-based distance calculating device 10 of the above embodiment, the in-vehicle sensor-based distance calculating device 10 being connected to the in-vehicle sensor 20.
The vehicle 1 according to the embodiment of the present invention adopts the distance calculating device 10 based on the vehicle-mounted sensor according to the above embodiment, so that the distance is calculated more accurately, and the safety is improved.
In the description of the present specification, reference to the terms "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples.
While embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that: many changes, modifications, substitutions and variations may be made to the embodiments without departing from the spirit and principles of the invention, the scope of which is defined by the claims and their equivalents.
Claims (12)
1. The distance calculating method based on the vehicle-mounted sensor is characterized by comprising the following steps of:
acquiring a vehicle environment image and determining the position of a target object in the vehicle environment image;
acquiring corresponding calibration information based on the position;
and determining the distance information between the target object and the vehicle based on the calibration information.
2. The distance calculating method according to claim 1, wherein the calibration information includes at least a correspondence between pixels of each position of the image acquired by the in-vehicle sensor and a target associated with pixels of each position of the image, and distance information between the target associated with pixels of each position of the image and a vehicle.
3. The distance calculating method according to claim 2, wherein the distance information of the object associated with the pixel points at each position of the image and the vehicle is obtained based on actual distance calibration of at least two reference points arranged at intervals on the road surface and the vehicle.
4. A distance calculating method according to claim 3, wherein,
the distance information of the target object associated with the pixel points of each position of the image and the vehicle comprises the longitudinal distance between the target object associated with the pixel points of each position of the image and the vehicle;
the longitudinal distance between the target object associated with the pixel points at each position of the image and the vehicle is calibrated by the following steps:
acquiring an image comprising at least two reference points which are arranged at intervals on the road surface, wherein the arrangement direction of the reference points is the longitudinal direction of the vehicle;
determining the position of any first pixel point along the arrangement direction of the reference points on the pavement in the image of the reference points;
determining a proportional relation satisfied by a reference point in an image of the arbitrary first pixel point and the reference point based on the position of the arbitrary first pixel point;
and obtaining the longitudinal distance between the target object associated with any one first pixel point and the vehicle according to the proportional relation satisfied by the reference point in the image of the reference point and the actual distance between the reference point and the vehicle.
5. A distance calculating method according to claim 4, wherein,
the distance information of the target object associated with the pixel points of each position of the image and the vehicle also comprises the transverse distance between the target object associated with the pixel points of each position of the image and the vehicle;
the lateral distance between the target object associated with the pixel points at each position of the image and the vehicle is calibrated by the following steps:
the image of the reference point is perpendicular to the reference point and is provided with a perpendicular bisector, the perpendicular bisector is correspondingly determined according to the distance of the perpendicular bisector of the connecting line of the two front wheels of the vehicle, and the vehicle-mounted sensor is arranged on the perpendicular bisector;
determining the position of any second pixel point on two sides of the perpendicular bisector in the image of the reference point;
determining a proportional relation between any second pixel point and the reference point in the image of the reference point based on the position of the any second pixel point;
and obtaining the transverse distance from the target object associated with any second pixel point to the vehicle according to the proportional relation between the any second pixel point and the reference point in the image of the reference point and the actual distance from the reference point to the perpendicular bisector.
6. The distance calculating method according to claim 5, wherein determining the distance information of the target object from the vehicle based on the calibration information includes:
obtaining a first longitudinal distance from a first object in the vehicle environment image to a vehicle, and obtaining a second longitudinal distance from a second object in the vehicle environment image to the vehicle;
obtaining a first lateral distance of the first object to the vehicle and a second lateral distance of the second object to the vehicle;
and obtaining the relative distance between the first target object and the second target object according to the first longitudinal distance, the second longitudinal distance, the first transverse distance and the second transverse distance.
7. The distance calculating method according to claim 4, wherein determining the distance information of the target object from the vehicle based on the calibration information includes:
identifying that the target object in the vehicle environment image is a target object in the height direction;
and determining the relative height of the target object and the vehicle based on the calibration information.
8. The distance calculating method according to claim 7, wherein,
the distance information of the target object associated with the pixel points of each position of the image and the vehicle comprises the relative height of the target object associated with the pixel points of each position of the image and the vehicle;
the relative heights of the target objects associated with the pixel points at each position of the image and the vehicle are calibrated through the following steps:
the reference points which are arranged in parallel and at intervals on the road surface are provided with heights which are perpendicular to the road surface, and the actual relative heights of the reference points and the vehicle-mounted sensor are measured;
determining the position of any third pixel point in the image of the reference point;
determining the relative height of any third pixel point and the road surface based on the position of the any third pixel point;
determining the ratio relation between the relative height of any third pixel point and the road surface and the relative height of the reference point and the road surface;
and obtaining the relative height from the target object associated with any one third pixel point to the vehicle according to the proportional relation satisfied by the relative height and the actual relative height between the reference point and the vehicle-mounted sensor.
9. The distance calculation method according to any one of claims 1 to 8, characterized in that the distance calculation method further comprises:
acquiring vehicle type information of a vehicle;
obtaining a calibration configuration file according to the vehicle type information;
and acquiring the calibration information in the calibration configuration file.
10. A distance calculating device based on an in-vehicle sensor, comprising:
a processor;
a memory communicatively coupled to the processor;
wherein the memory stores a computer program to be executed by the processor, the computer program realizing the in-vehicle sensor-based distance calculation method according to any one of claims 1 to 9 when executed by the processor.
11. A computer-readable storage medium, on which a computer program is stored, characterized in that the computer program, when executed, implements the vehicle-mounted sensor-based distance calculation method of any one of claims 1 to 9.
12. A vehicle, characterized by comprising:
the vehicle-mounted sensor is used for collecting vehicle environment images;
the in-vehicle sensor-based distance calculation device of claim 10, said distance calculation device being coupled to said in-vehicle sensor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311309292.2A CN117058210A (en) | 2023-10-11 | 2023-10-11 | Distance calculation method and device based on vehicle-mounted sensor, storage medium and vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311309292.2A CN117058210A (en) | 2023-10-11 | 2023-10-11 | Distance calculation method and device based on vehicle-mounted sensor, storage medium and vehicle |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117058210A true CN117058210A (en) | 2023-11-14 |
Family
ID=88661225
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311309292.2A Pending CN117058210A (en) | 2023-10-11 | 2023-10-11 | Distance calculation method and device based on vehicle-mounted sensor, storage medium and vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117058210A (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105702093A (en) * | 2016-04-21 | 2016-06-22 | 江苏联盟信息工程有限公司 | Lane judgment method based on latitude and longitude acquisition points and positioning device thereof |
| CN111353453A (en) * | 2020-03-06 | 2020-06-30 | 北京百度网讯科技有限公司 | Obstacle detection method and apparatus for vehicle |
| CN111476999A (en) * | 2020-01-17 | 2020-07-31 | 武汉理工大学 | Intelligent network-connected automobile over-the-horizon sensing system based on vehicle-road multi-sensor cooperation |
| CN111724616A (en) * | 2020-06-11 | 2020-09-29 | 樊馨 | Method and device for acquiring and sharing data based on artificial intelligence |
| CN112241007A (en) * | 2020-07-01 | 2021-01-19 | 北京新能源汽车技术创新中心有限公司 | Calibration method and arrangement structure of automatic driving environment perception sensor and vehicle |
| CN113688653A (en) * | 2020-05-18 | 2021-11-23 | 富士通株式会社 | Road center line recognition device and method and electronic equipment |
| CN116229404A (en) * | 2023-04-06 | 2023-06-06 | 上海友道智途科技有限公司 | Image defogging optimization method based on distance sensor |
| CN116386000A (en) * | 2023-02-23 | 2023-07-04 | 江铃汽车股份有限公司 | Method and system for measuring obstacle distance based on high-precision map and monocular camera |
-
2023
- 2023-10-11 CN CN202311309292.2A patent/CN117058210A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105702093A (en) * | 2016-04-21 | 2016-06-22 | 江苏联盟信息工程有限公司 | Lane judgment method based on latitude and longitude acquisition points and positioning device thereof |
| CN111476999A (en) * | 2020-01-17 | 2020-07-31 | 武汉理工大学 | Intelligent network-connected automobile over-the-horizon sensing system based on vehicle-road multi-sensor cooperation |
| CN111353453A (en) * | 2020-03-06 | 2020-06-30 | 北京百度网讯科技有限公司 | Obstacle detection method and apparatus for vehicle |
| CN113688653A (en) * | 2020-05-18 | 2021-11-23 | 富士通株式会社 | Road center line recognition device and method and electronic equipment |
| CN111724616A (en) * | 2020-06-11 | 2020-09-29 | 樊馨 | Method and device for acquiring and sharing data based on artificial intelligence |
| CN112241007A (en) * | 2020-07-01 | 2021-01-19 | 北京新能源汽车技术创新中心有限公司 | Calibration method and arrangement structure of automatic driving environment perception sensor and vehicle |
| CN116386000A (en) * | 2023-02-23 | 2023-07-04 | 江铃汽车股份有限公司 | Method and system for measuring obstacle distance based on high-precision map and monocular camera |
| CN116229404A (en) * | 2023-04-06 | 2023-06-06 | 上海友道智途科技有限公司 | Image defogging optimization method based on distance sensor |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10127461B2 (en) | Visual odometry for low illumination conditions using fixed light sources | |
| CN107341453B (en) | Lane line extraction method and device | |
| CN109544633B (en) | Target ranging method, device and equipment | |
| CN110031829B (en) | Target accurate distance measurement method based on monocular vision | |
| CN109764858B (en) | Photogrammetry method and system based on monocular camera | |
| Osman et al. | 3D accident reconstruction using low-cost imaging technique | |
| CN103729837A (en) | Rapid calibration method of single road condition video camera | |
| Fawzy | Study the accuracy of digital close range photogrammetry technique software as a measuring tool | |
| CN110555884A (en) | calibration method and device of vehicle-mounted binocular camera and terminal | |
| CN111243029A (en) | Calibration method and device of vision sensor | |
| CN111932627B (en) | Marker drawing method and system | |
| JP2018077162A (en) | Vehicle position detection device, vehicle position detection method and computer program for vehicle position detection | |
| CN112489106A (en) | Video-based vehicle size measuring method and device, terminal and storage medium | |
| CN112446915B (en) | Picture construction method and device based on image group | |
| CN112749584B (en) | Vehicle positioning method based on image detection and vehicle-mounted terminal | |
| CN113945937A (en) | Precision detection method, device and storage medium | |
| CN116386000A (en) | Method and system for measuring obstacle distance based on high-precision map and monocular camera | |
| RU2592711C1 (en) | Method and system for calibration of complex for measurement of vehicle speed | |
| CN109345576B (en) | Vehicle running speed identification method and system | |
| CN111380503A (en) | Monocular camera ranging method adopting laser-assisted calibration | |
| CN117058210A (en) | Distance calculation method and device based on vehicle-mounted sensor, storage medium and vehicle | |
| CN115546130A (en) | Height measuring method and device for digital twins and electronic equipment | |
| CN115294211A (en) | Vehicle-mounted camera installation external parameter calibration method, system, device and storage medium | |
| WO2022133986A1 (en) | Accuracy estimation method and system | |
| CN112837365B (en) | Image-based vehicle positioning method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |