CN114972537A - Road shoulder height measuring method and system considering vehicle body pitch angle influence - Google Patents
Road shoulder height measuring method and system considering vehicle body pitch angle influence Download PDFInfo
- Publication number
- CN114972537A CN114972537A CN202210592534.2A CN202210592534A CN114972537A CN 114972537 A CN114972537 A CN 114972537A CN 202210592534 A CN202210592534 A CN 202210592534A CN 114972537 A CN114972537 A CN 114972537A
- Authority
- CN
- China
- Prior art keywords
- road shoulder
- vehicle
- height
- pitch angle
- monocular camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 32
- 238000004364 calculation method Methods 0.000 claims abstract description 38
- 238000001514 detection method Methods 0.000 claims abstract description 35
- 238000012545 processing Methods 0.000 claims abstract description 18
- 238000003708 edge detection Methods 0.000 claims description 13
- 239000000725 suspension Substances 0.000 claims description 13
- 238000000691 measurement method Methods 0.000 claims description 12
- 230000003287 optical effect Effects 0.000 claims description 12
- 230000009466 transformation Effects 0.000 claims description 8
- 230000004927 fusion Effects 0.000 claims description 7
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 230000003044 adaptive effect Effects 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 3
- 238000003384 imaging method Methods 0.000 claims description 3
- 238000005259 measurement Methods 0.000 abstract description 10
- 238000010586 diagram Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000002474 experimental method Methods 0.000 description 3
- 206010039203 Road traffic accident Diseases 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000007790 scraping Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/02—Measuring arrangements characterised by the use of optical techniques for measuring length, width or thickness
- G01B11/06—Measuring arrangements characterised by the use of optical techniques for measuring length, width or thickness for measuring thickness ; e.g. of sheet material
- G01B11/0608—Height gauges
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T90/00—Enabling technologies or technologies with a potential or indirect contribution to GHG emissions mitigation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention relates to a road shoulder height measuring method and a road shoulder height measuring system considering the influence of a pitching angle of a vehicle body. The method comprises the following steps: calibrating a camera to obtain internal and external parameters, identifying the lower edge and the upper edge of the road shoulder, acquiring the pixel difference of the road shoulder in the vertical direction on an image plane by a multi-window anchor point detection method, and calculating the distance from the camera to the road shoulder and the height of the road shoulder according to the real-time pitch angle of the vehicle and the height of the camera. The system comprises a pitch angle CAN bus information acquisition unit of a vehicle body relative to the ground, a vehicle-mounted monocular camera, a road shoulder point-to-be-measured calculation unit based on a multi-window anchor point detection method, an image recognition distance data and coordinate system calibration data and road shoulder pixel coordinate data processing unit, and a road shoulder pixel coordinate and height information storage unit, wherein the vehicle-mounted camera is installed at a middle network with an included angle gamma with the horizontal plane. The invention can improve the precision of the height measurement of the road shoulder, and achieve the purposes of preventing the vehicle from colliding with the road shoulder and judging whether the special vehicle can provide information through the road shoulder.
Description
Technical Field
The invention relates to a road shoulder height measuring method and system considering the influence of a pitching angle of a vehicle body.
Background
With the rapid development of economy, the material living conditions are greatly improved, and the quantity of automobiles in China reaches 3.07 hundred million by 3 months in 2022. The automobile brings convenience, frequent traffic accidents are brought, economic property loss and casualties which are difficult to estimate are caused, in numerous traffic accidents, scraping and collision accidents occur between an automobile body and road shoulders and disturb numerous drivers, the main reason is that the visual field of the drivers is limited, the positions of the road shoulders cannot be estimated due to visual blind areas, and secondly, the drivers are inexperienced and cannot accurately estimate the heights of the road shoulders to judge the trafficability of the automobile.
In addition, when some special vehicles work, the vehicles need to cross over the shoulders, steps and the like in front of the vehicles, in order to ensure the obstacle crossing safety of the special vehicles, the heights of the shoulders and the steps in front of the vehicles need to be accurately measured, and drivers judge whether passing conditions exist according to the measurement results.
At present, a laser radar can accurately acquire an object three-dimensional coordinate point cloud to achieve the purpose of accurately measuring the height of a road shoulder, but the laser radar has the defects of high equipment cost, long processing time and the like. The height measurement based on monocular vision is relatively low in cost and high in speed. The height of the road shoulder can be measured quickly and accurately by adopting a height measurement model considering the pitch angle influence of the vehicle body and self-adaptive weighting.
Disclosure of Invention
The technical problems to be solved by the invention are as follows: the method and the system for measuring the height of the road shoulder are used for accurately measuring the height of the road shoulder by considering the influence of the pitching angle of the vehicle body.
The invention adopts the following technical scheme for solving the technical problems:
the invention provides a road shoulder height measuring method considering the influence of the pitching angle of a vehicle body, which specifically comprises the following steps: after the vehicle-mounted monocular camera is calibrated through a vehicle-mounted monocular camera calibration method considering vehicle suspension, firstly, an upper edge and a lower edge of a road shoulder are identified through a road shoulder edge detection algorithm, a multi-window anchor point detection method is used for selecting upper edge anchor points and lower edge anchor points of the road shoulder, according to relevant information, a real-time distance measurement method considering a pitch angle is used for measuring and calculating the distance between the road shoulder and the vehicle-mounted monocular camera, the relevant information comprises camera internal parameters, camera height and a vehicle pitch angle which are obtained after the vehicle-mounted monocular camera is calibrated, and then the height of the road shoulder is measured through a height measurement model based on adaptive weighting.
The road shoulder edge detection algorithm comprises the following steps:
step1, calibrating a vehicle-mounted monocular camera installed in front of a vehicle, acquiring internal and external parameters of the camera, and adopting a vehicle-mounted monocular camera calibration method considering vehicle suspension, wherein the conversion relation from the vehicle-mounted monocular camera to a vehicle suspension coordinate system is as follows:
in the above formula: x c ,Y c ,Z c Is a coordinate system of the vehicle-mounted monocular camera; beta is the vehicle roll angle; theta is a vehicle pitch angle; alpha is a vehicle slip angle; x v ,Y v ,Z v Suspending a coordinate system for the vehicle;
step2, carrying out Canny operator edge detection on the obtained image to obtain edge points of the road shoulder, wherein the Canny edge detection algorithm firstly removes noise through Gaussian filtering and then searches the position where the edge is most likely to exist through calculating the gradient direction of the image gray level;
and Step3, acquiring the upper edge, the lower edge and the rear edge of the road shoulder by applying Hough transformation to the acquired edge detection image.
The multi-window anchor point detection method is characterized in that 6 square windows are automatically generated according to the width of the whole image, each window respectively detects intersection points of vertical lines and the upper edge and the lower edge of a road shoulder, and the width calculation formula of each square window is as follows:
m i ＝p sum /6
in the formula: m is i I-1, 2, …,6 is the width of a single square window; p is a radical of sum The unit pixel is the width of the whole image.
The real-time distance measurement method considering the pitch angle includes the steps of:
step1, establishing a geometric model of the distance between an image coordinate system and a geodetic coordinate system according to the imaging principle of the vehicle-mounted monocular camera and the relation between the vehicle-mounted monocular camera and the ground plane;
step2, obtaining a relational expression of beta according to the geometric model as follows:
ε＝β+γ+θ
the expression of the distance relation between the pixel coordinate and the world coordinate is as follows:
step3, obtaining the following result according to the conversion relation between the image coordinate system and the pixel coordinate system:
x＝(u-u 0 )dx
y＝(v-v 0 )dy
O 2 B 1 2 ＝x 2 +y 2 ＝[(u-u 0 )dx] 2 +[(v-v 0 )dy] 2 ；
step4, according to
Obtaining a distance calculation formula between the road shoulder and the vehicle-mounted monocular camera:
in the above formula: b 1 Showing the intersection point of the bottom edge of the tested road shoulder and the road surface; gamma is an initial pitch angle when the vehicle-mounted monocular camera is installed in front of the vehicle, and h is the height from the ground; f is the focal length of the on-board monocular camera, f x 、f y The components of the focal length in the x-axis direction and the y-axis direction of the phase plane respectively; theta is a real-time pitch angle when the vehicle moves; beta is an included angle between the projection of the vehicle-mounted monocular camera and an optical axis; (u) 0 ,v 0 ) The position of the origin of the image coordinate system in the pixel coordinate system; l is a measured point b 1 Horizontal distance to lens; o is the intersection point of the optical axis and the image plane and is used as the origin of the image coordinate system; o is 1 Is the center of a lens of a vehicle-mounted monocular camera; o is 2 The focal point of the optical axis and the phase plane after the vehicle pitch angle changes; b is 1 (x, y) is the projection of the measured point on the image plane; o is 2 B 1 Is the distance between the projection point of the optical axis on the image plane and the projection point of the target point on the image plane.
The invention can adopt a height measurement method based on self-adaptive weighting to calculate each square window to obtain the measured value of the road shoulder for fusion, and the height relation formula is as follows:
the road shoulder height fusion calculation method comprises the following steps:
the total mean square error is:
the minimum total mean square error is:
corresponding to a weighting factor w i Comprises the following steps:
in the above formula: b 1 b 2 The length between the lower edge and the upper edge of the road shoulder; y is 1 y 2 Pixel difference of the road shoulder in the vertical direction on the image plane; f. of y Is the component of the focal length in the y-axis direction of the phase plane; l is the distance between the road shoulder and the vehicle-mounted monocular camera; h i The shoulder height is measured for a single square window; h is the height of the fused road shoulder; w is a i I 1,2, 6 is a weight coefficient of the road shoulder height measured and calculated by the corresponding square window; h 0 The road shoulder height is measured and calculated at the previous moment and stored in a road shoulder pixel coordinate and height information storage unit;
and measuring the variance for the road shoulder height corresponding to each square window.
The invention provides a road shoulder height measuring system considering the influence of a pitch angle of a vehicle body, which comprises a pitch angle CAN bus information acquisition unit of the vehicle body relative to the earth, a vehicle-mounted monocular camera, a road shoulder point to be measured calculation unit based on a multi-window anchor point detection method, an image recognition distance data and coordinate system calibration data and road shoulder pixel coordinate data processing unit, and a road shoulder pixel coordinate and height information storage unit, wherein: the image recognition distance data, the coordinate system calibration data and the road shoulder pixel coordinate data processing unit are loaded on a running vehicle, and the road shoulder height at the current moment is measured, calculated and stored according to the real-time vehicle pitch angle acquired by the vehicle body pitch angle CAN bus information acquisition unit relative to the earth, the anchor points at the upper edge and the lower edge of the road shoulder in the picture calculated by the road shoulder point to be measured calculation unit based on the multi-window anchor point detection method, the road shoulder pixel coordinate and the road shoulder height at the previous moment stored by the height information storage unit.
The pitch angle CAN bus information acquisition unit of automobile body for the earth mainly comprises inclination sensor, CAN interface, power source, on-vehicle road shoulder height information output terminal, wherein: the tilt angle sensor is connected with the CAN interface to transmit vehicle pitch angle information.
The road shoulder point-to-be-measured point calculation unit based on the multi-window anchor point detection method mainly comprises an aviation plug, a microprocessor, a CAN interface and a power supply interface, wherein: the aviation plug is connected with the vehicle-mounted monocular camera, receives acquired image data and transmits the image data to the microprocessor, and the microprocessor transmits the information to the image recognition distance data, the coordinate system calibration data and the road shoulder pixel coordinate data processing unit through the CAN interface after calculation.
The image identification distance data, coordinate system calibration data and road shoulder pixel coordinate data processing unit mainly comprises a microprocessor, a CAN interface and a power interface, wherein: and the microprocessor receives information required by calculation through the CAN interface and outputs a calculation result.
The road shoulder pixel coordinate and height information storage unit mainly comprises an RAM memory, a CAN interface, a power interface and a USB output interface, wherein: the CAN interface receives the calculation result of the system and transmits the calculation result to the RAM memory for storage; and the USB output interface is connected with the vehicle-mounted road shoulder height information output terminal to display the road shoulder height information to a driver.
Compared with the prior art, the invention has the following main advantages:
the invention adopts the monocular vision of the camera, and the cost is greatly reduced compared with the road shoulder detection of the laser radar.
Influence factors of a vehicle in a running state are fully considered, a real-time pitch angle of the vehicle is considered in a calculation process, a multi-window anchor point detection method and a self-adaptive weighted height measurement method are provided, multi-point detection is carried out on a road shoulder in front of the vehicle, then fusion is carried out, detection precision and fault tolerance rate can be effectively improved, in the embodiment, the average relative error is only 4.32%, and the maximum relative error is 8.43%.
The system provided by the invention CAN rapidly and accurately detect the height of the road shoulder, and CAN immediately provide accurate road shoulder height information for a driver through the vehicle-mounted road shoulder height information output terminal in the pitch angle CAN bus information acquisition unit of the vehicle body relative to the ground, thereby effectively reducing the risks of scraping and colliding vehicles and providing guarantee for the safety of special vehicles.
Drawings
FIG. 1 is a flow chart of a method and system for measuring shoulder height that takes into account the effect of vehicle body pitch angle in accordance with the present invention;
FIG. 2 is a schematic diagram of a road shoulder detection result according to the present invention;
FIG. 3 is a schematic diagram of a multi-window anchor point detection method according to the present invention;
FIG. 4 is a schematic diagram of a real-time distance measurement model of the present invention taking into account pitch angle;
FIG. 5 is a schematic diagram of an adaptive weighting-based height measurement model according to the present invention;
FIG. 6 is a schematic diagram of information interaction between units in the system of the present invention;
FIG. 7 is a graph showing the results of the experiment in the example of the present invention.
Detailed Description
The invention discloses a road shoulder height measuring method and system considering the influence of a pitching angle of a vehicle body. The method comprises the following steps: calibrating a camera to obtain internal and external parameters, identifying the lower edge and the upper edge of the road shoulder, acquiring the pixel difference of the road shoulder in the vertical direction on an image plane by a multi-window anchor point detection method, and calculating the distance from the camera to the road shoulder and the height of the road shoulder according to the real-time pitch angle of the vehicle and the height of the camera. The system mainly comprises a pitch angle CAN bus information acquisition unit of a vehicle body relative to the ground, a vehicle-mounted monocular camera, a road shoulder point-to-be-measured calculation unit based on a multi-window anchor point detection method, an image recognition distance data and coordinate system calibration data and road shoulder pixel coordinate data processing unit, and a road shoulder pixel coordinate and height information storage unit, wherein: the vehicle-mounted camera is arranged at the middle net with an included angle gamma with the horizontal plane. The invention focuses on exploring the influence of the pitching angle of the vehicle body on the road shoulder height measurement, can effectively improve the precision of the road shoulder height measurement, and achieves the purposes of preventing the vehicle from colliding with the road shoulder and judging whether the information can be provided through the road shoulder for special vehicles.
The present invention will be further explained and illustrated with reference to the following examples and drawings, but the present invention is not limited thereto.
The road shoulder height measuring system considering the influence of the pitching angle of the vehicle body is structurally shown in fig. 1 and 6, and comprises a pitching angle CAN bus information acquisition unit of the vehicle body relative to the earth, a vehicle-mounted monocular camera, a road shoulder point-to-be-measured calculation unit based on a multi-window anchor point detection method, an image recognition distance data and coordinate system calibration data and road shoulder pixel coordinate data processing unit, and a road shoulder pixel coordinate and height information storage unit, wherein: the vehicle-mounted monocular camera is connected with a road shoulder point-to-be-measured calculation unit based on a multi-window anchor point detection method to transmit image information; a road shoulder point to be measured calculation unit, a vehicle body pitch angle CAN bus information acquisition unit relative to the earth, and a road shoulder pixel coordinate and height information storage unit based on a multi-window anchor point detection method respectively transmit information with an image recognition distance data and coordinate system calibration data and a road shoulder pixel coordinate data processing unit, wherein the transmitted information is road shoulder anchor point information, vehicle pitch angle information and road shoulder height information measured and calculated at the previous moment; and the image identification distance data, the coordinate system calibration data and the road shoulder pixel coordinate data processing unit calculates the road shoulder height at the current moment according to the information.
The pitch angle CAN bus information acquisition unit of the vehicle body relative to the ground is arranged below a center console of a running vehicle, mainly comprises an inclination angle sensor, a CAN interface, a power supply interface and a vehicle-mounted road shoulder height information output terminal and is used for acquiring the pitch angle of the vehicle body and displaying the road shoulder height information to a driver, wherein: the tilt angle sensor is connected with the CAN interface to transmit vehicle pitch angle information.
The road shoulder point-to-be-measured calculation unit based on the multi-window anchor point detection method is installed inside a middle net of a running vehicle, mainly comprises an aviation plug, a microprocessor, a CAN interface and a power interface, and is used for processing image data and calculating the point to be measured on a road shoulder edge line, wherein: the aviation plug is connected with the vehicle-mounted monocular camera, receives acquired image data and transmits the image data to the microprocessor, and the microprocessor transmits the information to the image recognition distance data, the coordinate system calibration data and the road shoulder pixel coordinate data processing unit through the CAN interface after calculation.
The image recognition distance data, coordinate system calibration data and road shoulder pixel coordinate data processing unit is arranged below an instrument panel of a running vehicle, mainly comprises a microprocessor, a CAN interface and a power interface and is used for calculating the internal and external parameters of the vehicle-mounted monocular camera, the distance between the vehicle-mounted monocular camera and the road shoulder and the height of the road shoulder, wherein: and the microprocessor receives information required by calculation through the CAN interface and outputs a calculation result.
The road shoulder pixel coordinate and height information storage unit is arranged below an instrument panel of a running vehicle, mainly comprises an RAM memory, a CAN interface, a power interface and a USB output interface, and is used for storing each result obtained by the system through calculation, wherein: the CAN interface receives the calculation result of the system and transmits the calculation result to the RAM memory for storage; and the USB output interface is connected with the vehicle-mounted road shoulder height information output terminal to display the road shoulder height information to a driver.
The invention provides a road shoulder height measuring system considering the influence of the pitching angle of a vehicle body, which comprises the following working processes: a road shoulder point-to-be-measured calculating unit, a pitch angle CAN bus information acquiring unit of a vehicle body relative to the earth and a road shoulder pixel coordinate and height information storing unit of the multi-window anchor point detection method transmit acquired information to an image recognition distance data and coordinate system calibration data and road shoulder pixel coordinate data processing unit, calculate the distance between a vehicle-mounted monocular camera and a road shoulder and the height of the road shoulder, and transmit the distance and the height to a vehicle-mounted road shoulder height information output terminal through a CAN bus.
The road shoulder height measuring method considering the influence of the pitching angle of the vehicle body, as shown in fig. 1, 4, 5 and 6, comprises the following steps: a multi-window anchor point detection method is provided for calculating the pixel difference between the upper edge and the lower edge of the same photo at different positions of the road shoulder; a real-time distance measurement method considering a pitch angle is provided for calculating the distance between a camera and a road shoulder in a three-dimensional world; and providing a height measurement method based on self-adaptive weighting to calculate the height of the road shoulder.
The multi-window anchor point detection method automatically generates a square window according to the width of an image; before the anchor point detection, the shoulder edge line needs to be detected firstly, and the specific steps are as follows:
step1, acquiring internal and external parameter matrixes and distortion parameters of the camera:
the calibration method is characterized in that a vehicle-mounted monocular camera arranged in front of a vehicle is calibrated, the influence of vehicle suspension needs to be considered when the calibration is carried out, the vehicle can generate a roll angle, a pitch angle and a yaw angle under the condition of external force, and the factors are fully considered in the calibration process, so that the coordinate conversion from the vehicle-mounted monocular camera to the vehicle suspension is increased, and the calibration result of the vehicle-mounted monocular camera is more accurate.
The internal and external parameters of the vehicle-mounted monocular camera can be obtained through camera calibration. The internal parameters mainly refer to optical and geometrical characteristics of the vehicle-mounted monocular camera, including focal length, principal point coordinates, lens distortion and the like; extrinsic parameters include translation and rotation of the vehicle monocular camera to the suspension system and the suspension system to the geodetic coordinates, i.e., changes in orientation with respect to three-dimensional space. According to the method, the calibration of the camera is carried out by considering a vehicle-mounted monocular camera calibration method of vehicle suspension, so that the internal and external parameter matrixes and distortion parameters of the camera are obtained, and further, the spatial information is solved from the image information.
The conversion relation from the vehicle-mounted monocular camera to the vehicle suspension coordinate system is as follows:
in the above formula (1): x c ,Y c ,Z c Is a coordinate system of the vehicle-mounted monocular camera; beta is the vehicle roll angle; theta is a vehicle pitch angle; alpha is a vehicle slip angle; x v ,Y v ,Z v A vehicle suspension coordinate system.
Step2, finding the position of the edge in the image:
and (3) carrying out Canny operator edge detection on the obtained image to obtain edge points of the road shoulder, wherein the Canny edge detection algorithm firstly removes noise through Gaussian filtering, and then calculates the gray gradient direction of the image according to the formula (2) to find the position where the edge is most likely to exist.
In the above formula (2): g x Represents the gradient in the x-direction at image pixel (x, j); g y Representing the gradient in the y-direction at image pixel (x, j).
And Canny adopts non-maximum suppression, searches along the gradient direction of the edge, and only retains the point with the maximum local gradient to obtain a clear edge with the width of only one pixel.
Step3, identifying the upper, lower and rear edge lines of the road shoulder:
the Hough transformation is used for obtaining the upper edge, the lower edge and the rear edge of the road shoulder by applying the Hough transformation to the obtained edge detection image, the Hough transformation detection is used for detecting straight lines according to voting of edge points in the whole image, so that the Hough transformation detection has stronger anti-jamming capability on incomplete straight lines or existing noise in the image, thereby effectively identifying dotted lines or partially shielded straight lines in the detection process of the road shoulder edge, and an effect graph is shown in figure 2 after the Hough transformation road shoulder detection.
Next, referring to fig. 3, a multi-window anchor point detection method is provided, which can extract multiple anchor points from the shoulder of the road in the image. After Hough road shoulder detection, the method can obtain three non-intersected long straight lines a1, a2 and a3 which are respectively the bottom edge of the road shoulder, the upper edge of the road shoulder and the rear edge of the road shoulder.And automatically generating 6 square windows according to the width of the image, wherein the width of the windows is shown as the formula (3). The intersection points of a vertical line generated at any position in the window, the lower edge a1 and the upper edge a2 of the road shoulder are b1 and b2 respectively, and the distance x from the vertical line to the left of the window is [0, m ] i ]And b1 and b2 represent pixel differences of the road shoulder in the vertical direction on the image plane. According to the number of the square windows, the height of the road shoulder at 6 different positions can be calculated by each picture.
m i ＝p sum /6 (3)
In the above formula (3): m is i I-1, 2, …,6 is the width of a single square window; p is a radical of sum The unit pixel is the width of the whole image.
The real-time distance measurement method considering the pitch angle is provided, the pitch angle generated in the vehicle motion process is considered, so that the real-time distance measurement method considering the pitch angle is provided, the influence factor of the real-time pitch angle generated by the vehicle motion is added in the calculation, and the influence of the vehicle pitch angle on the vehicle monocular camera and the road shoulder distance measurement is eliminated. As shown in fig. 4, by obtaining the mounting height of the vehicle-mounted monocular camera, the coordinates of the lower edge of the road shoulder in the image plane and the real-time pitch angle of the vehicle, the distance L between the road shoulder and the vehicle-mounted monocular camera is calculated, and according to the imaging principle, the geometric relationship is obtained as follows:
ε＝β+γ+θ (4)
according to the formulas (4), (5) and (6):
O 2 B 1 2 ＝x 2 +y 2 (8)
according to the transformation relation between the image coordinate system and the pixel coordinate system, the following results can be obtained:
x＝(u-u 0 )dx (9)
y＝(v-v 0 )dy (10)
the combination of formulae (8), (9), (10) gives:
O 2 B 1 2 ＝[(u-u 0 )dx] 2 +[(v-v 0 )dy] 2 (11)
wherein
Namely:
in combination with the formulae (7), (12):
and (4) according to the internal parameters obtained by calibrating the vehicle-mounted monocular camera and the real-time pitch angle theta of the vehicle, substituting the internal parameters into the real-time pitch angle theta, and calculating the distance L between the road shoulder and the vehicle-mounted monocular camera.
In the above formulas (4) to (13): b 1 Representing the intersection point of the bottom edge of the tested road shoulder and the road surface; gamma is an initial pitch angle when the vehicle-mounted monocular camera is installed in front of the vehicle, and h is the height from the ground; f is focal length of the vehicle-mounted monocular camera, f x 、f y The components of the focal length in the x-axis direction and the y-axis direction of the phase plane respectively; theta is a real-time pitch angle when the vehicle moves; beta is an included angle between the projection of the vehicle-mounted monocular camera and an optical axis; (u) 0 ,v 0 ) The position of the origin of the image coordinate system in the pixel coordinate system; l is a measured point b 1 Horizontal distance to lens; o is the intersection point of the optical axis and the image plane and is used as the origin of the image coordinate system; o is 1 Is the center of a lens of a vehicle-mounted monocular camera; o is 2 The focal point of the optical axis and the phase plane after the vehicle pitch angle changes; b is 1 (x, y) is the projection of the measured point on the image plane；O 2 B 1 Is the distance between the projection point of the optical axis on the image plane and the projection point of the target point on the image plane.
Compared with the existing method, the height measuring method based on the self-adaptive weighting can perform self-adaptive weighting fusion on the road shoulder heights measured and calculated by a plurality of (6 in the embodiment) square windows to obtain a final road shoulder height, so that the measuring precision of the road shoulder height is improved. The method fuses the measured values of the road shoulders obtained by calculating each square window, the height relation is shown in figure 5, and the formula is as follows:
in order to reduce system errors and accurately measure the height of a road shoulder, a self-adaptive weighting fusion algorithm is designed. Respectively calculating the measured values H of the road shoulders through 6 square measuring and calculating windows i I is 1,2, …,6, independent from each other, and H is the shoulder height of the fused three-dimensional space, and the fusion calculation method is as follows:
for calculating the total mean square error, the road shoulder pixel coordinate and the road shoulder height H calculated at the previous moment and stored in the height information storage unit are extracted 0 Therefore, the total mean square error σ is derived from the equations (15) and (16) 2 。
Solving the total mean square error sigma according to the theory of extreme value solving of multivariate function 2 The minimum condition is that the weight of the road shoulder height measured and calculated by each square window is w i . When the variance is smaller, the corresponding weight value is larger, and the minimum total mean square error is
Comprises the following steps:
corresponding to a weighting factor w i Comprises the following steps:
calculating the weight w corresponding to the road shoulder height obtained by each window through the formula (19) i And substituting the weights corresponding to the 6 calculated road shoulder heights into an equation (15), and further obtaining the road shoulder height H of the fused three-dimensional space.
In the above formulas (14) to (19): b 1 b 2 The distance between the lower edge and the upper edge of the road shoulder; y is 1 y 2 Pixel difference in the vertical direction of the road shoulder on the image plane; f. of y Is the component of the focal length in the y-axis direction of the phase plane; l is the distance between the road shoulder and the vehicle-mounted monocular camera; w is a i I 1,2, 6 is a weight coefficient of the road shoulder height measured and calculated by the corresponding square window;
and measuring the variance for the road shoulder height corresponding to each square window.
In this embodiment, a manual transmission SUV with model number of great wall M2 was selected to perform a real vehicle experiment, a test vehicle was equipped with a vehicle monocular camera with model number of PC7430 and each unit in the system of the present invention, and the vehicle monocular camera had a pixel number of 720 × 480 and was installed on a medium net 0.86M from the ground, and included an angle of 15 ° with the horizontal plane. The test site is in a campus of Wuhan and Han Dynasty university, and the experiment is carried out after 20 open scenes are selected and the height of the road shoulder is measured manually.
The experimental result is shown in fig. 7, the maximum relative error of the calculation result of the shoulder height is 8.43%, the minimum relative error is 1.22%, and the average relative error is 4.32%. The method for detecting the height of the road shoulder has the advantages of accurate detection result, stable system work and small relative error fluctuation.
The technical solutions provided by the present invention have been described above, and since the content and the embodiments of the present invention have a changeable part, the content of the present invention should not be construed as limiting the invention for those skilled in the art.
Claims (10)
1. A road shoulder height measuring method considering the influence of a vehicle body pitch angle is characterized in that after a vehicle-mounted monocular camera is calibrated through a vehicle-mounted monocular camera calibration method considering vehicle suspension, the upper edge and the lower edge of a road shoulder are firstly identified through a road shoulder edge detection algorithm, anchor points of the upper edge and the lower edge of the road shoulder are selected through a multi-window anchor point detection method, the distance between the road shoulder and the vehicle-mounted monocular camera is measured and calculated through a real-time distance measuring method considering a pitch angle according to related information, the related information comprises camera internal parameters, camera height and a vehicle pitch angle which are obtained after the vehicle-mounted monocular camera is calibrated, and the height of the road shoulder is measured through a height measuring model based on self-adaptive weighting.
2. The shoulder height measurement method of claim 1, wherein the shoulder edge detection algorithm comprises the steps of:
step1, calibrating a vehicle-mounted monocular camera installed in front of a vehicle, acquiring internal and external parameters of the camera, and adopting a vehicle-mounted monocular camera calibration method considering vehicle suspension, wherein the conversion relation from the vehicle-mounted monocular camera to a vehicle suspension coordinate system is as follows:
in the above formula: x c ,Y c ,Z c Is a coordinate system of the vehicle-mounted monocular camera; beta is the vehicle roll angle; theta is a vehicle pitch angle; alpha is a vehicle slip angle; x v ,Y v ,Z v Suspending a coordinate system for the vehicle;
step2, carrying out Canny operator edge detection on the obtained image to obtain edge points of the road shoulder, wherein the Canny edge detection algorithm firstly removes noise through Gaussian filtering and then searches the position where the edge is most likely to exist through calculating the gradient direction of the image gray level;
and Step3, acquiring the upper edge, the lower edge and the rear edge of the road shoulder by applying Hough transformation to the acquired edge detection image.
3. The road shoulder height measurement method according to claim 1, wherein the multi-window anchor point detection method is to automatically generate 6 square windows according to the width of the whole image, each window respectively detects the intersection points of the vertical line and the upper and lower edges of the road shoulder, and the width calculation formula of a single square window is as follows:
m i ＝p sum /6
in the formula: m is i I-1, 2, …,6 is the width of a single square window; p is a radical of formula sum The unit pixel is the width of the whole image.
4. The method of measuring road shoulder height according to claim 1, wherein the real-time distance measurement method considering a pitch angle comprises the steps of:
step1, establishing a geometric model of the distance between an image coordinate system and a geodetic coordinate system according to the imaging principle of the vehicle-mounted monocular camera and the relation between the vehicle-mounted monocular camera and the ground plane;
step2, obtaining a relational expression of beta according to the geometric model as follows:
ε＝β+γ+θ
the expression of the distance relation between the pixel coordinate and the world coordinate is as follows:
step3, obtaining the following result according to the conversion relation between the image coordinate system and the pixel coordinate system:
x＝(u-u 0 )dx
y＝(v-v 0 )dy
O 2 B 1 2 ＝x 2 +y 2 ＝[(u-u 0 )dx] 2 +[(v-v 0 )dy] 2 ；
step4, according to
Obtaining a distance calculation formula between the road shoulder and the vehicle-mounted monocular camera:
in the above formula: b 1 Representing the intersection point of the bottom edge of the tested road shoulder and the road surface; gamma is an initial pitch angle when the vehicle-mounted monocular camera is installed in front of the vehicle, and h is the height from the ground; f is focal length of the vehicle-mounted monocular camera, f x 、f y The components of the focal length in the x-axis direction and the y-axis direction of the phase plane respectively; theta is a real-time pitch angle when the vehicle moves; beta is an included angle between the projection of the vehicle-mounted monocular camera and an optical axis; (u) 0 ,v 0 ) The position of the origin of the image coordinate system in the pixel coordinate system; l is a measured point b 1 Horizontal distance to lens; o is the intersection point of the optical axis and the image plane and is used as the origin of the image coordinate system; o is 1 Is the center of a lens of a vehicle-mounted monocular camera; o is 2 For vehicle pitch angle change rear lightThe focal point of the axis and the phase plane; b is 1 (x, y) is the projection of the measured point on the image plane; o is 2 B 1 Is the distance between the projection point of the optical axis on the image plane and the projection point of the target point on the image plane.
5. The method for measuring the height of the road shoulder according to claim 4, wherein the height measurement method based on adaptive weighting is adopted to fuse the measured values of the road shoulder calculated by each square window, and the height relation formula is as follows:
the road shoulder height fusion calculation method comprises the following steps:
the total mean square error is:
the minimum total mean square error is:
corresponding to a weighting factor w i Comprises the following steps:
in the above formula：b 1 b 2 The length between the lower edge and the upper edge of the road shoulder; y is 1 y 2 Pixel difference of the road shoulder in the vertical direction on the image plane; f. of y Is the component of the focal length in the y-axis direction of the phase plane; l is the distance between the road shoulder and the vehicle-mounted monocular camera; h i The shoulder height is measured for a single square window; h is the height of the fused road shoulder; w is a i I 1,2, 6 is a weight coefficient of the road shoulder height measured and calculated by the corresponding square window; h 0 The road shoulder height is measured and calculated at the previous moment and stored in a road shoulder pixel coordinate and height information storage unit;
and i is 1,2, … and 6, and the measured variance of the shoulder height corresponding to each square window is obtained.
6. A road shoulder height measuring system considering the influence of the pitch angle of a vehicle body is characterized by comprising a pitch angle CAN bus information acquisition unit of the vehicle body relative to the earth, a vehicle-mounted monocular camera, a road shoulder point to be measured calculation unit based on a multi-window anchor point detection method, an image recognition distance data and coordinate system calibration data and road shoulder pixel coordinate data processing unit and a road shoulder pixel coordinate and height information storage unit, wherein the five units are all loaded on a running vehicle, the image recognition distance data and coordinate system calibration data and road shoulder pixel coordinate data processing unit calculates and stores the road shoulder height at the current moment according to the real-time vehicle pitch angle acquired by the pitch angle CAN bus information acquisition unit of the vehicle body relative to the earth, the anchor point of the upper edge and the lower edge of the road shoulder in the picture calculated by the road shoulder point to be measured calculation unit based on the multi-window anchor point detection method, the road shoulder pixel coordinate and the road shoulder height at the previous moment stored by the height information storage unit The height of the shoulder.
7. The system for measuring the road shoulder height in consideration of the influence of the pitch angle of the vehicle body as claimed in claim 6, wherein the pitch angle CAN bus information acquisition unit of the vehicle body relative to the ground is mainly composed of an inclination angle sensor, a CAN interface, a power supply interface, and a vehicle-mounted road shoulder height information output terminal, wherein: the tilt angle sensor is connected with the CAN interface to transmit vehicle pitch angle information.
8. The system for measuring the height of the road shoulder with consideration of the influence of the pitching angle of the vehicle body as claimed in claim 6, wherein the calculation unit of the road shoulder point to be measured based on the multi-window anchor point detection method mainly comprises an aviation plug, a microprocessor, a CAN interface and a power supply interface, wherein: the aviation plug is connected with the vehicle-mounted monocular camera, receives acquired image data and transmits the image data to the microprocessor, and the microprocessor transmits the information to the image recognition distance data, the coordinate system calibration data and the road shoulder pixel coordinate data processing unit through the CAN interface after calculation.
9. The system for measuring the height of the road shoulder with consideration of the influence of the pitching angle of the car body as claimed in claim 6, wherein the image recognition distance data and coordinate system calibration data and road shoulder pixel coordinate data processing unit mainly comprises a microprocessor, a CAN interface and a power supply interface, wherein: and the microprocessor receives information required by calculation through the CAN interface and outputs a calculation result.
10. The system for measuring the road shoulder height in consideration of the influence of the pitch angle of the vehicle body as claimed in claim 6, wherein the road shoulder pixel coordinate and height information storage unit is mainly composed of a RAM memory, a CAN interface, a power interface and a USB output interface, wherein: the CAN interface receives the calculation result of the system and transmits the calculation result to the RAM memory for storage; and the USB output interface is connected with the vehicle-mounted road shoulder height information output terminal to display the road shoulder height information to a driver.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210592534.2A CN114972537A (en) | 2022-05-27 | 2022-05-27 | Road shoulder height measuring method and system considering vehicle body pitch angle influence |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210592534.2A CN114972537A (en) | 2022-05-27 | 2022-05-27 | Road shoulder height measuring method and system considering vehicle body pitch angle influence |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114972537A true CN114972537A (en) | 2022-08-30 |
Family
ID=82958385
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210592534.2A Pending CN114972537A (en) | 2022-05-27 | 2022-05-27 | Road shoulder height measuring method and system considering vehicle body pitch angle influence |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114972537A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115223031A (en) * | 2022-09-20 | 2022-10-21 | 凌度(广东)智能科技发展有限公司 | Monocular frame distance measuring method and device, medium and curtain wall robot |
-
2022
- 2022-05-27 CN CN202210592534.2A patent/CN114972537A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115223031A (en) * | 2022-09-20 | 2022-10-21 | 凌度(广东)智能科技发展有限公司 | Monocular frame distance measuring method and device, medium and curtain wall robot |
| CN115223031B (en) * | 2022-09-20 | 2022-12-20 | 凌度(广东)智能科技发展有限公司 | Monocular frame ranging method and device, medium and curtain wall robot |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108759667B (en) | Front truck distance measuring method under vehicle-mounted camera based on monocular vision and image segmentation | |
| CN106240458B (en) | A kind of vehicular frontal impact method for early warning based on vehicle-mounted binocular camera | |
| CN104965202B (en) | Obstacle detection method and device | |
| CN109359409A (en) | A kind of vehicle passability detection system of view-based access control model and laser radar sensor | |
| CN112698302B (en) | Sensor fusion target detection method under bumpy road condition | |
| CN105835880B (en) | Lane following system | |
| CN107966700A (en) | A kind of front obstacle detecting system and method for pilotless automobile | |
| CN106651963B (en) | A kind of installation parameter scaling method of the vehicle-mounted camera for driving assistance system | |
| CN106289159B (en) | Vehicle distance measurement method and device based on distance measurement compensation | |
| CN109031346A (en) | A kind of periphery parking position aided detection method based on 3D laser radar | |
| EP3792660A1 (en) | Method, apparatus and system for measuring distance | |
| CN105404844A (en) | Road boundary detection method based on multi-line laser radar | |
| CN110745140A (en) | Vehicle lane change early warning method based on continuous image constraint pose estimation | |
| CN107796373B (en) | Distance measurement method based on monocular vision of front vehicle driven by lane plane geometric model | |
| CN110909705B (en) | Road side parking space sensing method and system based on vehicle-mounted camera | |
| CN109410264A (en) | A kind of front vehicles distance measurement method based on laser point cloud and image co-registration | |
| CN110728720B (en) | Method, apparatus, device and storage medium for camera calibration | |
| CN107290738A (en) | A kind of method and apparatus for measuring front vehicles distance | |
| CN109345591B (en) | Vehicle posture detection method and device | |
| CN114972537A (en) | Road shoulder height measuring method and system considering vehicle body pitch angle influence | |
| CN113866783A (en) | Vehicle distance measurement method and system | |
| CN107688174A (en) | A kind of image distance-finding method, system, storage medium and vehicle-mounted visually-perceptible equipment | |
| CN113850102A (en) | Vehicle-mounted vision detection method and system based on millimeter wave radar assistance | |
| Lion et al. | Smart speed bump detection and estimation with kinect | |
| US9098774B2 (en) | Method for detection of targets in stereoscopic images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |