CN112017249A - Vehicle-mounted camera roll angle obtaining and mounting angle correcting method and device - Google Patents
Vehicle-mounted camera roll angle obtaining and mounting angle correcting method and device Download PDFInfo
- Publication number
- CN112017249A CN112017249A CN202010834781.XA CN202010834781A CN112017249A CN 112017249 A CN112017249 A CN 112017249A CN 202010834781 A CN202010834781 A CN 202010834781A CN 112017249 A CN112017249 A CN 112017249A
- Authority
- CN
- China
- Prior art keywords
- angle
- camera
- vehicle
- mounted camera
- roll angle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
- G06T2207/30256—Lane; Road marking
Abstract
The invention discloses a method for acquiring a roll angle of a vehicle-mounted camera, which comprises the following steps: (1) acquiring a shot image with a vehicle body area by using a vehicle-mounted camera; (2) extracting all horizontal straight lines of the vehicle body area in the shot image; (3) calculating the slopes of all straight lines in the horizontal direction, and taking the slope value of the straight line with the slope in the middle as the final horizontal included angle; (4) and calculating the roll angle of the camera according to the horizontal included angle. The method can quickly and accurately correct the roll angle of the camera, and is simple and convenient to calculate. The invention also discloses a real-time dynamic correction method for the installation angle of the vehicle-mounted camera, which can dynamically correct the installation angle of the camera in real time through a roll angle, an azimuth angle and a pitch angle, wherein the roll angle is obtained through a roll angle obtaining method of the vehicle-mounted camera, and the azimuth angle and the pitch angle are obtained through a lane line vanishing point obtaining mode. The invention also discloses corresponding electronic equipment and a computer readable storage medium.
Description
Technical Field
The invention relates to a vehicle-mounted camera, in particular to real-time dynamic correction of the mounting position of the vehicle-mounted camera.
Background
In the ADAS system, visual ranging techniques provide it with distance information for objects in a road scene. The camera marks and determines the corresponding relation of an image coordinate system, a camera coordinate system and a world coordinate system, and the distance of the target is obtained by calculating and transforming a projection matrix and camera parameters and utilizing a triangulation distance measuring principle. After the focal length of the camera is fixed, the change of the internal parameter can be ignored under the general condition, so the accuracy and the adaptability of the calibration of the external parameter of the camera have larger influence on the distance measurement accuracy.
Referring to chinese patent CN201680069618.0, the vehicle-mounted camera mounting angle is a high-precision calibration performed by using an accurate checkerboard pattern drawn on a calibration board or calibration cloth before the camera is delivered from the factory or used, and the vehicle often jolts or shakes under ordinary road conditions, the camera mounting angle and the initial state will generate a deviation, and at this time, if the distance calculation is performed by using the previous calibration result, a large error will be generated.
The performance improvement of the ADAS product is restricted to a certain extent by visual ranging, so that the development of a set of visual ranging system capable of dynamically correcting the installation angle of the camera is very important content.
For this reason, chinese patent CN201910469033.3 discloses a method for correcting the camera external parameters at high frequency during the use of the vehicle, however, it mainly corrects the pitch angle and the camera height, and has limited correction for other camera external parameters.
In chinese patent CN201910510879.7, a lane line in a road image is extracted first, then constraint equations of a pitch angle, a yaw angle, and a roll angle are derived according to a preset modification formula, and the pitch angle, the yaw angle, and the roll angle are solved through the constraint equations.
In chinese patent CN201910362590.5, in the image shot by the camera, the continuously shot grayscale image is tracked and calibrated to perform three-dimensional reconstruction, and then the ground plane equation is fitted, although it does not depend on any parallel lines and other signs on the road surface, and the problem of accurate detection is solved, the calculation amount is large, and the required system is complex and consumes a long time.
Therefore, a method for correcting the roll angle and the installation position of the vehicle-mounted camera, which has high accuracy and is convenient and quick to detect, is urgently needed.
Disclosure of Invention
The invention aims to provide a method for acquiring the roll angle of a vehicle-mounted camera, corresponding electronic equipment and a computer readable storage medium, which can quickly and accurately correct the roll angle of the camera and are simple and convenient to calculate.
Another object of the present invention is to provide a method for dynamically correcting an installation angle of a vehicle-mounted camera in real time, and a corresponding electronic device and a computer readable storage medium, which can quickly and accurately obtain a roll angle of the camera and correct an installation position of the camera through the roll angle, a pitch angle and an azimuth angle.
In order to achieve the purpose, the invention discloses a method for acquiring a roll angle of a vehicle-mounted camera, which comprises the following steps: (1) acquiring a shot image with a vehicle body area by using a vehicle-mounted camera; (2) extracting all horizontal straight lines of the vehicle body area in the shot image; (3) calculating the slopes of all straight lines in the horizontal direction, and taking the slope value of the straight line with the slope in the middle as the final horizontal included angle; (4) and calculating the roll angle of the camera according to the horizontal included angle.
Compared with the prior art, the method and the device have the advantages that the roll angle of the camera is calculated by identifying the horizontal straight line on the vehicle body, on one hand, the characteristics of the vehicle body provided with the camera are identified, so that the detection result is accurate, and the time consumption is reduced. On the other hand, the roll angle is calculated by identifying the horizontal straight line and calculating the slope of the horizontal straight line, so that the method is simple and convenient. On the other hand, the slope of the straight line with the middle slopes of all the identified horizontal straight lines on the vehicle body is taken as the final horizontal included angle, the calculation result is more accurate, the interference caused by extreme conditions can be quickly and effectively eliminated, and the correction stability is ensured.
Preferably, the method further comprises (a) between the steps (2) and (3): and removing the interference straight line in the horizontal straight line. The detection accuracy is further improved.
Specifically, the step (a) specifically includes: and deleting the straight line in the horizontal direction with the length smaller than the preset value.
Preferably, in the step (2), all horizontal straight lines of the vehicle body region in the captured image are extracted by using an LSD straight line detection method. The LSD line detection method is used for line extraction, and is simple and accurate. Of course, the line extraction may be performed in other manners, and is not limited to the LSD line detection.
Specifically, the step (2) specifically includes: (21) extracting a vehicle body area image in the shot image; (22) calculating gradients of the vehicle body area image in the vertical direction and the horizontal direction; (23) selecting pixel points with gradient values meeting the preset direction threshold range of the horizontal direction gradient, and performing pseudo-sorting; (24) taking the pseudo-sorting result as a seed point to perform 4 neighborhood region growth, and calculating the circumscribed rectangle of each connected region to form a rectangular frame; (25) and judging whether the density of the alignment points in each rectangular frame meets a preset threshold value or not, outputting the rectangular frames meeting the preset threshold value as screened straight lines, wherein the alignment points are pixel points of which the gradient angles and the main shaft angles of the rectangular frames are kept consistent in a tolerance range.
The invention also discloses a real-time dynamic correction method for the installation angle of the vehicle-mounted camera, which comprises the steps of collecting shot images by the vehicle-mounted camera to calculate the roll angle, the azimuth angle and the pitch angle of the vehicle-mounted camera, and dynamically correcting the installation angle of the camera in real time according to the roll angle, the azimuth angle and the pitch angle, wherein the roll angle is calculated by the method for acquiring the roll angle of the vehicle-mounted camera.
Compared with the prior art, the method calculates the roll angle of the camera by identifying the horizontal straight line on the vehicle body, and on one hand, the characteristics on the vehicle body provided with the camera are identified to ensure that the detection result is accurate. On the other hand, the roll angle is calculated by identifying the horizontal straight line and calculating the slope of the horizontal straight line, so that the method is simple and convenient. On the other hand, the slope of the straight line with the middle slopes of all the horizontal straight lines on the recognized vehicle body is taken as the final included angle in the horizontal direction, so that the calculation is convenient and quick, the calculation result is more accurate, the interference caused by extreme conditions can be effectively eliminated, and the correction stability is ensured. In another aspect, the invention can correct three installation angles of the camera at the same time: roll angle, azimuth angle, pitch angle.
Preferably, the step of acquiring the shot image by using the vehicle-mounted camera to calculate the azimuth angle and the pitch angle of the vehicle-mounted camera comprises the following steps: performing multi-lane line detection and screening out lane lines by using the shot images; and calculating the azimuth angle and the pitch angle of the camera by using the lane lines.
Preferably, the step of detecting the multiple lane lines and screening the lane lines is carried out, and two parallel lane lines are screened out.
Specifically, the "calculating the azimuth angle and the pitch angle of the camera using the lane line" specifically includes: and calculating the intersection point of the two parallel lane lines to obtain a vanishing point, and calculating the azimuth angle and the pitch angle of the camera by using the vanishing point. The invention obtains the azimuth angle and the pitch angle by solving the way of vanishing points of the lane lines.
Specifically, the specific steps of screening out two parallel lane lines include: and selecting the longest two straight lane lines as the screened lane lines.
More specifically, the specific steps of screening out two parallel lane lines include: acquiring lane detection data according to the shot image, screening lane line data, projecting the lane line data into a top view, calculating the slopes of all lane lines in the top view, calculating the average value of the slopes of all lane lines, and deleting the lane lines of which the difference values between the slopes and the average value exceed a preset threshold value in all lane lines; and sequencing the rest lane lines after deletion, and selecting the two longest straight lane lines as the screened lane lines.
Preferably, the step (a4) specifically includes: and forming state variables by the roll angle, the azimuth angle, the pitch angle and the angular speed of the current camera to construct Kalman filtering, and correcting the installation angle of the camera according to a filtering result. A Kalman filter is established for the three angles and the angular rate, noise is filtered, the change of the angles is corrected, and the dynamic correction precision is further improved.
Specifically, the angular velocities include angular velocities of a roll angle, an azimuth angle, and a pitch angle.
The invention also discloses an electronic device, comprising: a camera; one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors, the programs comprising instructions for performing the in-vehicle camera roll angle acquisition method as described above.
The invention also discloses a computer readable storage medium, which comprises a computer program used in combination with the camera, wherein the computer program can be executed by a processor to realize the vehicle-mounted camera roll angle acquisition method.
The invention also discloses an electronic device, comprising: a camera; one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors, the programs including instructions for performing the in-vehicle camera installation angle real-time dynamic correction method as described above.
The invention also discloses a computer readable storage medium, which comprises a computer program used in combination with the camera, wherein the computer program can be executed by a processor to realize the method for dynamically correcting the installation angle of the vehicle-mounted camera in real time.
Drawings
FIG. 1 is a flow chart of a method for dynamically correcting the installation angle of a vehicle-mounted camera in real time according to the invention.
FIG. 2 is a flow chart of the method for acquiring the azimuth angle and the pitch angle of the vehicle-mounted camera according to the invention.
FIG. 3 is a flowchart of a roll angle obtaining method of a vehicle-mounted camera according to the invention.
Fig. 4 is a flowchart of extracting all horizontal straight lines of the vehicle body region image according to the present invention.
FIG. 5 is a solution pitch angle
The geometric figure of (a).
Figure 6 is a geometry for solving for the azimuth angle Ψ.
FIG. 7 is a diagram illustrating the relationship between horizontal angle and roll angle.
Detailed Description
In order to explain technical contents, structural features, and objects and effects of the present invention in detail, the following detailed description is given with reference to the accompanying drawings in conjunction with the embodiments.
Referring to fig. 1, the invention discloses a real-time dynamic correction method 100 for a vehicle-mounted camera installation angle, comprising the following steps: (A1) acquiring shot images by using the vehicle-mounted camera, (A2) calculating the roll angle, the azimuth angle and the pitch angle of the vehicle-mounted camera according to the shot images, and (A3) dynamically correcting the installation angle of the camera in real time according to the roll angle, the azimuth angle and the pitch angle.
Preferably, before the step (a2), the method further comprises the steps of: and acquiring the preset internal calibration parameters and external calibration results of the camera and the correction angle of the camera in the previous frame. In the steps (a2) and (A3), specific numerical calculations are performed by using the internal calibration parameters and the external calibration results preset by the camera and the camera correction angle of the previous frame, which are specifically described below.
The method comprises the steps of obtaining internal calibration parameters of the camera, static external reference calibration results and the roll angle, the azimuth angle and the pitch angle corrected by the camera in the previous frame by reading configuration parameters, wherein the internal calibration parameters and the static external reference calibration results comprise information such as the focal length and the optical center position of the camera, a distortion matrix, the height of the camera, the roll angle, the azimuth angle and the pitch angle of the camera. The last frame of camera correction angle is the roll angle, azimuth angle and pitch angle corrected by the last frame of camera.
Referring to fig. 2, a method 200 for acquiring a shot image by using a vehicle-mounted camera to calculate an azimuth angle and a pitch angle of the vehicle-mounted camera includes the steps of: (B1) and (B2) carrying out multi-lane line detection according to the shot image and screening out two parallel lane lines, and calculating the intersection point of the two parallel lane lines to obtain a vanishing point. (B3) The azimuth angle and the pitch angle of the camera are calculated using the vanishing point.
Preferably, the step (B1) of screening out two parallel lane lines includes: and selecting the longest two straight lane lines as the screened lane lines.
Specifically, the specific steps of screening out two parallel lane lines include: (1) and acquiring lane detection data and screening lane line data. (2) Projecting the lane line data into a top view. (3) Calculating slopes of all lane lines in the top view. (4) The average of all lane line slopes is calculated. (5) And deleting the lane lines of which the difference values between the slopes and the average value exceed a preset threshold value in all the lane lines, namely, rejecting the lane lines with far deviation from the average value. (6) And sequencing the rest lane lines after deletion, and selecting the two longest straight lane lines as the screened lane lines.
Certainly, different from the above embodiment, the azimuth angle and the pitch angle may also be calculated by other methods, for example, after all lane lines are screened out, the slope of the lane line is calculated, and the azimuth angle and the pitch angle are calculated by using the slope of the lane line. The slope of the lane line may be the average of the slopes of all the lane lines, or the intermediate value of the slopes of the lane lines may be used as the final lane line. And screening the lane lines before calculating the slope of the lane lines, and removing the lane lines with the length less than the preset value.
In step B1, the multi-lane line detection is performed based on the detection data acquired by the ADAS preamble. In step B2, the point of intersection of the two parallel lane lines is calculated as the point of intersection where the two lane lines extend to infinity on the original image. The original image is included in the detection data acquired in advance according to the ADAS, and is a captured image captured by the front camera.
The specific step of projecting the lane line data into a top view includes:
(1) projecting all lane lines onto a top view according to a perspective transformation matrix:
wherein u and v are image coordinates, Xw, Yw and Zw are world coordinates, k is a scale factor, dx and dy are pixel sizes in x and y directions, f is a focal length of the camera, and u0 and v0 are optical center positions.
The rotation matrix R is:
theta is the roll angle of the dynamic correction of the last frame,
the pitch angle corrected for the previous frame, and the azimuth angle corrected for the previous frame Ψ are the included angle between the camera optical center and the lane line direction.
Wherein, step (1) also includes carrying on Gaussian to reduce and sample to the detected data before.
(2) Will world coordinate YwZw is mapped to top view coordinates x, y in scale k:
in step B2, a vanishing point is obtained by screening according to the obtained multi-lane detection result and the previous frame of camera calibration result. Of course, other methods for preventing lane line screening, such as selecting a straight line of a vertical gradient in the area outside the vehicle to identify a lane line, may be used, and the present invention is not limited to this embodiment.
In step B3, the vanishing point P (xt, yt) is used to calculate the azimuth Ψ and the pitch angle of the camera
Using formulas
Computing
I.e. calculating tvkThen the pitch angle can be calculated through an arctangent function
Specifically, see fig. 5, where 0z is the z-axis of the world coordinate system, 0y is the y-axis of the image coordinate system, and f is the image captureThe machine focal length, the image coordinates of the vanishing point P (xt, yt) (i.e., the intersection of the two lane lines at the image plane), Pyt is the projection of the vanishing point P (xt, yt) onto the y-axis of the camera coordinate system. Cyo 'is the optical axis direction and cy is the y coordinate of the camera's optical center. t is tvkThe pitch angle is calculated as the difference between Pyt and cy by means of an arctan function
Solving is performed to calculate the azimuth angle Ψ according to the geometry in fig. 6, where xoy is the camera coordinate system, 0 is the optical center coordinate, 00' is the optical axis, and f is the focal length. yt 0' is the road direction. By geometric solving, the value of the azimuth angle can be determined.
Referring to fig. 3, a method 300 of calculating a roll angle of an in-vehicle camera from a photographed image includes the steps of: (S1) acquiring a photographed image having a vehicle body area with the vehicle-mounted camera. (S2) straight lines in all horizontal directions of the vehicle body region in the captured image are extracted. (S3) the slope of all the horizontal lines is calculated, and the slope value at the middle of the slope of the line is taken as the final horizontal angle. (S4) calculating a roll angle of the camera according to the horizontal included angle. Wherein, the step (S1) may be performed in the step a 1.
In step S1, the captured image having the vehicle body area is captured by the leading camera at the vehicle tail. In this embodiment, the images of the head and tail regions are captured by the cameras at the head and tail of the vehicle to perform the line detection. Of course, the camera at the head or the tail of the vehicle can be used for shooting the image of the area at the head or the tail of the vehicle to perform the straight line detection. The specifically adopted camera is a camera for correcting the installation angle, and is not limited in the range of the front-mounted camera at the tail of the vehicle head.
In step S2, all the horizontal straight lines of the vehicle body region in the captured image are extracted using the LSD straight line detection method.
Referring to fig. 4, the step S2 specifically includes: (S21) the vehicle body area image in the captured image is extracted. (S22) the gradients in the vertical direction and the horizontal direction in the vehicle body area image are calculated. (S23) screening out the gradients in the horizontal direction according to a preset direction threshold value, and performing pseudo-sorting. (S24) performing 4-neighborhood region growing using the result of the pseudo-sorting as a seed point, and calculating a bounding rectangle of each connected region to form a rectangular box. (S25) determining whether the density of the alignment points in each rectangular frame meets a predetermined threshold. (S26) the rectangular box that meets the preset threshold is output as the straight line that is filtered out. The alignment points are pixel points of which the gradient angle and the main shaft angle of the rectangular frame are consistent in a tolerance range. The present invention extracts a horizontal straight line of a vehicle body part in a photographed image by a simplified LSD straight line detection method.
Of course, other algorithms may be adopted to perform the line detection, for example, Hough line detection, Freeman line detection algorithm, neural network feature extraction and line acquisition, and the like.
Wherein, step S22 specifically includes: (1) and calculating the image gradient by using a sobel edge operator. (2) And (3) convolving the vehicle body area image I by using a sobel vertical operator and a horizontal operator respectively to obtain gradients in the vertical direction and the horizontal direction.
The direction of the line is then calculated by the following equation:
gradient amplitude:
of course, other methods may be used to perform the image gradient calculation, and the method is not limited to the embodiment.
Step S23 specifically includes: selecting the pixel points with gradient values within a preset threshold from the horizontal gradient (the horizontal gradient obtained by calculation in the step S22), and performing pseudo-sorting. Specifically, a linear direction F is used as a reference, a gradient of which the linear direction is within a preset threshold range from the linear direction F is selected as a selected horizontal direction gradient, and the pixel points of which the gradient values are the selected horizontal direction gradient are subjected to pseudo-sorting.
Step S24 specifically includes: and taking the pixel points with gradient values meeting a preset threshold value as seed points to increase the 4-neighborhood region to form a rectangular frame.
Step S25 specifically includes: the straight line is obtained by rectangular estimation in the LSD algorithm (the divided straight line can be represented by a circumscribed rectangle). The area is configured to be rectangular, the size of the rectangle is set to be 10 pixels (of course, other values can be adopted, and are set by a technician as required), and pixels with gradient angles in the rectangular area and main axis angles of the rectangle within a tolerance range are screened out as alignment points, in this embodiment, pixels with tolerances set to be 20 ° (of course, other values can be set as required) which are consistent within a range become alignment points. And outputting the straight line when the density of the alignment points is greater than the threshold value.
Preferably, the steps S2 and S3 further include (a): and removing the interference straight line in the horizontal direction, and further improving the correction accuracy. Specifically, a straight line in the horizontal direction having a length smaller than a preset value is deleted, that is, a shorter straight line is deleted. Of course, different from the above embodiment, all the horizontal straight lines are sorted, and the first N straight lines are selected, where N is set by a technician as required, and is preferably greater than or equal to 3. Based on the present embodiment, it is also possible to delete the horizontal direction straight line having a length greater than a preset value (vehicle body width).
The step S3 includes: (S3a) the slopes of all the horizontal straight lines are obtained. (S3b) taking the value of the slope with the straight line slope in the middle as the final horizontal included angle. All the horizontal straight lines in the step (S3a) are the straight lines screened in the step (a).
The step (S3a) specifically includes:
(1) calculate the center of the rectangle:
g (j) is the gradient value of pixel j, where j represents each pixel in the region.
(2) The angle of the rectangle (i.e. the horizontal straight line) is found from the angle of the characteristic vector:
the angle of the eigenvector refers to the angle of the eigenvector corresponding to the minimum eigenvalue set as the main direction of the rectangle by solving the eigenvalue of the M matrix; wherein the content of the first and second substances,
the step (S3b) specifically includes: and (3) calculating the slopes of all the horizontal direction straight lines in the step (S3a), sorting the slopes, and selecting the slope with the intermediate value as the final horizontal direction included angle. And if the number of the straight lines in the horizontal direction is even, selecting the slope value of the straight line with the slope value in the middle of the two straight lines and approaching the average value of the slopes of all the straight lines in the horizontal direction as the final horizontal included angle.
In step S4, if the roll angle is θ, the slope corresponding to the final horizontal angle is tan θ.
Referring to fig. 7, xoy is a camera coordinate system, M1, M2, M3 are three points in a straight line with a slope at an intermediate value, oc is the center of a rectangular frame, ox1 is the final horizontal direction, and oy1 is the vertical direction after roll angle correction.
The step a3 specifically includes: and forming state variables by the roll angle, the azimuth angle, the pitch angle and the angular speed of the current camera to construct Kalman filtering, and correcting the installation angle of the camera according to a filtering result. Specifically, the angular velocities are angular velocities of a roll angle, an azimuth angle, and a pitch angle, respectively.
The invention also discloses an electronic device, comprising: a camera; one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors, the programs comprising instructions for performing the in-vehicle camera roll angle acquisition method as described above.
The invention also discloses a computer readable storage medium, which comprises a computer program used in combination with the camera, wherein the computer program can be executed by a processor to realize the vehicle-mounted camera roll angle acquisition method.
The invention also discloses an electronic device, comprising: a camera; one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors, the programs including instructions for performing the in-vehicle camera installation angle real-time dynamic correction method as described above.
The invention also discloses a computer readable storage medium, which comprises a computer program used in combination with the camera, wherein the computer program can be executed by a processor to realize the method for dynamically correcting the installation angle of the vehicle-mounted camera in real time.
The above disclosure is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the scope of the present invention, therefore, the present invention is not limited by the appended claims.
Claims (16)
1. A method for acquiring a roll angle of a vehicle-mounted camera is characterized by comprising the following steps: the method comprises the following steps:
(1) acquiring a shot image with a vehicle body area by using a vehicle-mounted camera;
(2) extracting all horizontal straight lines of the vehicle body area in the shot image;
(3) calculating the slopes of all straight lines in the horizontal direction, and taking the slope value of the straight line with the slope in the middle as the final horizontal included angle;
(4) and calculating the roll angle of the camera according to the horizontal included angle.
2. The vehicle-mounted camera roll angle acquisition method according to claim 1, characterized in that: the method also comprises the following steps (a) between the steps (2) and (3): and removing the interference straight line in the horizontal straight line.
3. The vehicle-mounted camera roll angle acquisition method according to claim 2, characterized in that: the step (a) specifically comprises: and deleting the straight line in the horizontal direction with the length smaller than the preset value.
4. The vehicle-mounted camera roll angle acquisition method according to claim 1, characterized in that: and (3) extracting all horizontal straight lines of the vehicle body region in the shot image by using an LSD straight line detection method in the step (2).
5. The on-vehicle camera roll angle acquisition method according to claim 4, characterized in that: the step (2) specifically comprises:
(21) extracting a vehicle body area image in the shot image;
(22) calculating gradients of the vehicle body area image in the vertical direction and the horizontal direction;
(23) selecting pixel points with gradient values meeting the preset direction threshold range of the horizontal direction gradient, and performing pseudo-sorting;
(24) taking the pseudo-sorting result as a seed point to perform 4 neighborhood region growth, and calculating the circumscribed rectangle of each connected region to form a rectangular frame;
(25) and judging whether the density of the alignment points in each rectangular frame meets a preset threshold value or not, outputting the rectangular frames meeting the preset threshold value as screened straight lines, wherein the alignment points are pixel points of which the gradient angles and the main shaft angles of the rectangular frames are kept consistent in a tolerance range.
6. A real-time dynamic correction method for the installation angle of a vehicle-mounted camera is characterized in that a vehicle-mounted camera is used for collecting shot images to calculate the roll angle, the azimuth angle and the pitch angle of the vehicle-mounted camera, and the installation angle of the camera is dynamically corrected in real time according to the roll angle, the azimuth angle and the pitch angle, and the method is characterized in that: the roll angle is calculated using the on-vehicle camera roll angle acquisition method of any one of claims 1-5.
7. The method for dynamically correcting the mounting angle of the vehicle-mounted camera in real time according to claim 6, characterized in that: the method for acquiring the shot image by utilizing the vehicle-mounted camera to calculate the azimuth angle and the pitch angle of the vehicle-mounted camera comprises the following steps:
performing multi-lane line detection and screening out lane lines by using the shot images;
and calculating the azimuth angle and the pitch angle of the camera by using the lane lines.
8. The method for dynamically correcting the mounting angle of the vehicle-mounted camera in real time according to claim 7, characterized in that: the method comprises the following steps of carrying out multi-lane line detection and screening lane lines, and screening out two parallel lane lines.
9. The method for dynamically correcting the mounting angle of the vehicle-mounted camera in real time according to claim 8, characterized in that: the "calculating the azimuth angle and the pitch angle of the camera by using the lane line" is specifically as follows: and calculating the intersection point of the two parallel lane lines to obtain a vanishing point, and calculating the azimuth angle and the pitch angle of the camera by using the vanishing point.
10. The method for dynamically correcting the mounting angle of the vehicle-mounted camera in real time according to claim 8, characterized in that: the specific steps of screening out two parallel lane lines comprise: and selecting the longest two straight lane lines as the screened lane lines.
11. The method for dynamically correcting the mounting angle of the vehicle-mounted camera in real time according to claim 10, characterized in that: the specific steps of screening out two parallel lane lines comprise: acquiring lane detection data according to the shot image, screening lane line data, projecting the lane line data into a top view, calculating the slopes of all lane lines in the top view, calculating the average value of the slopes of all lane lines, and deleting the lane lines of which the difference values between the slopes and the average value exceed a preset threshold value in all lane lines; and sequencing the rest lane lines after deletion, and selecting the two longest straight lane lines as the screened lane lines.
12. The method for dynamically correcting the mounting angle of the vehicle-mounted camera in real time according to claim 6, characterized in that: the 'correcting the installation angle of the camera according to the roll angle, the azimuth angle and the pitch angle' comprises the following steps: and forming a state variable by the roll angle, the azimuth angle, the pitch angle and the angular velocity of the current camera to construct Kalman filtering, and correcting the installation angle of the camera according to a filtering result, wherein the angular velocity comprises the angular velocities of the roll angle, the azimuth angle and the pitch angle.
13. An electronic device, characterized in that: the method comprises the following steps:
a camera;
one or more processors;
a memory; and
one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors, the programs comprising instructions for performing the in-vehicle camera roll angle acquisition method of any of claims 1-5.
14. A computer-readable storage medium comprising a computer program for use with a camera, characterized in that: the computer program is executable by a processor to implement the in-vehicle camera roll angle acquisition method as set forth in any one of claims 1-5.
15. An electronic device, characterized in that: the method comprises the following steps:
a camera;
one or more processors;
a memory; and
one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors, the programs comprising instructions for performing the in-vehicle camera mounting angle real-time dynamic correction method of any one of claims 6-12.
16. A computer-readable storage medium comprising a computer program for use with a camera, characterized in that: the computer program is executable by a processor to implement the method for real-time dynamic correction of the installation angle of an in-vehicle camera according to any one of claims 6 to 12.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010834781.XA CN112017249A (en) | 2020-08-18 | 2020-08-18 | Vehicle-mounted camera roll angle obtaining and mounting angle correcting method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010834781.XA CN112017249A (en) | 2020-08-18 | 2020-08-18 | Vehicle-mounted camera roll angle obtaining and mounting angle correcting method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112017249A true CN112017249A (en) | 2020-12-01 |
Family
ID=73506089
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010834781.XA Pending CN112017249A (en) | 2020-08-18 | 2020-08-18 | Vehicle-mounted camera roll angle obtaining and mounting angle correcting method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112017249A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112215214A (en) * | 2020-12-11 | 2021-01-12 | 智道网联科技(北京)有限公司 | Method and system for adjusting camera offset of intelligent vehicle-mounted terminal |
| CN112580516A (en) * | 2020-12-21 | 2021-03-30 | 上海眼控科技股份有限公司 | Road scene recognition method, device, equipment and storage medium |
| CN112862899A (en) * | 2021-02-07 | 2021-05-28 | 黑芝麻智能科技(重庆)有限公司 | External parameter calibration method, device and system for image acquisition equipment |
| CN113658252A (en) * | 2021-05-17 | 2021-11-16 | 毫末智行科技有限公司 | Method, medium, apparatus for estimating elevation angle of camera, and camera |
| WO2023014246A1 (en) | 2021-08-06 | 2023-02-09 | Общество с ограниченной ответственностью "ЭвоКарго" | Method of calibrating extrinsic video camera parameters |
| WO2023185272A1 (en) * | 2022-03-28 | 2023-10-05 | 腾讯科技（深圳）有限公司 | Method and apparatus for calibrating roll angle of on-board camera, and device and storage medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101894271A (en) * | 2010-07-28 | 2010-11-24 | 重庆大学 | Visual computing and prewarning method of deviation angle and distance of automobile from lane line |
| JP2013129264A (en) * | 2011-12-20 | 2013-07-04 | Honda Motor Co Ltd | Calibration method for on-vehicle camera and calibration device for on-vehicle camera |
| CN109141347A (en) * | 2017-06-28 | 2019-01-04 | 京东方科技集团股份有限公司 | Vehicle-mounted vidicon distance measuring method and device, storage medium and electronic equipment |
| CN109931944A (en) * | 2019-04-02 | 2019-06-25 | 百度在线网络技术（北京）有限公司 | A kind of AR air navigation aid, device, vehicle end equipment, server-side and medium |
| CN110221328A (en) * | 2019-07-23 | 2019-09-10 | 广州小鹏汽车科技有限公司 | A kind of Combinated navigation method and device |
| CN110264525A (en) * | 2019-06-13 | 2019-09-20 | 惠州市德赛西威智能交通技术研究院有限公司 | A kind of camera calibration method based on lane line and target vehicle |
| CN110349219A (en) * | 2018-04-04 | 2019-10-18 | 杭州海康威视数字技术股份有限公司 | A kind of Camera extrinsic scaling method and device |
| CN110718068A (en) * | 2019-09-27 | 2020-01-21 | 华中科技大学 | Road monitoring camera installation angle estimation method |
| CN111126306A (en) * | 2019-12-26 | 2020-05-08 | 江苏罗思韦尔电气有限公司 | Lane line detection method based on edge features and sliding window |
-
2020
- 2020-08-18 CN CN202010834781.XA patent/CN112017249A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101894271A (en) * | 2010-07-28 | 2010-11-24 | 重庆大学 | Visual computing and prewarning method of deviation angle and distance of automobile from lane line |
| JP2013129264A (en) * | 2011-12-20 | 2013-07-04 | Honda Motor Co Ltd | Calibration method for on-vehicle camera and calibration device for on-vehicle camera |
| CN109141347A (en) * | 2017-06-28 | 2019-01-04 | 京东方科技集团股份有限公司 | Vehicle-mounted vidicon distance measuring method and device, storage medium and electronic equipment |
| CN110349219A (en) * | 2018-04-04 | 2019-10-18 | 杭州海康威视数字技术股份有限公司 | A kind of Camera extrinsic scaling method and device |
| CN109931944A (en) * | 2019-04-02 | 2019-06-25 | 百度在线网络技术（北京）有限公司 | A kind of AR air navigation aid, device, vehicle end equipment, server-side and medium |
| CN110264525A (en) * | 2019-06-13 | 2019-09-20 | 惠州市德赛西威智能交通技术研究院有限公司 | A kind of camera calibration method based on lane line and target vehicle |
| CN110221328A (en) * | 2019-07-23 | 2019-09-10 | 广州小鹏汽车科技有限公司 | A kind of Combinated navigation method and device |
| CN110718068A (en) * | 2019-09-27 | 2020-01-21 | 华中科技大学 | Road monitoring camera installation angle estimation method |
| CN111126306A (en) * | 2019-12-26 | 2020-05-08 | 江苏罗思韦尔电气有限公司 | Lane line detection method based on edge features and sliding window |
Non-Patent Citations (3)
| Title |
|---|
| GROMPONE VON GIOI ET AL: "LSD: A Fast Line Segment Detector with a False Detection Control", 《IEEE》, vol. 30, no. 4, pages 722 - 732, XP011280594, DOI: 10.1109/TPAMI.2008.300 * |
| 余厚云 等: "基于摄像机模型的运动车辆车道偏离检测", 《东南大学学报(自然科学版)》, vol. 39, no. 05, pages 933 - 936 * |
| 陈军 等: "基于道路特征的车载相机标定动态补偿算法", 《机械工程学报》, vol. 40, no. 20, pages 112 - 117 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112215214A (en) * | 2020-12-11 | 2021-01-12 | 智道网联科技(北京)有限公司 | Method and system for adjusting camera offset of intelligent vehicle-mounted terminal |
| CN112580516A (en) * | 2020-12-21 | 2021-03-30 | 上海眼控科技股份有限公司 | Road scene recognition method, device, equipment and storage medium |
| CN112862899A (en) * | 2021-02-07 | 2021-05-28 | 黑芝麻智能科技(重庆)有限公司 | External parameter calibration method, device and system for image acquisition equipment |
| CN112862899B (en) * | 2021-02-07 | 2023-02-28 | 黑芝麻智能科技(重庆)有限公司 | External parameter calibration method, device and system for image acquisition equipment |
| CN113658252A (en) * | 2021-05-17 | 2021-11-16 | 毫末智行科技有限公司 | Method, medium, apparatus for estimating elevation angle of camera, and camera |
| WO2023014246A1 (en) | 2021-08-06 | 2023-02-09 | Общество с ограниченной ответственностью "ЭвоКарго" | Method of calibrating extrinsic video camera parameters |
| WO2023185272A1 (en) * | 2022-03-28 | 2023-10-05 | 腾讯科技（深圳）有限公司 | Method and apparatus for calibrating roll angle of on-board camera, and device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112017249A (en) | Vehicle-mounted camera roll angle obtaining and mounting angle correcting method and device | |
| CN110930459B (en) | Vanishing point extraction method, camera calibration method and storage medium | |
| CN108805934B (en) | External parameter calibration method and device for vehicle-mounted camera | |
| CN107463918B (en) | Lane line extraction method based on fusion of laser point cloud and image data | |
| US7889887B2 (en) | Lane recognition apparatus | |
| US6985619B1 (en) | Distance correcting apparatus of surroundings monitoring system and vanishing point correcting apparatus thereof | |
| CN107463890B (en) | A kind of Foregut fermenters and tracking based on monocular forward sight camera | |
| CN109752701A (en) | A kind of road edge detection method based on laser point cloud | |
| CN111336951B (en) | Method and apparatus for calibrating external parameters of image sensor | |
| CN112270713A (en) | Calibration method and device, storage medium and electronic device | |
| JP5310027B2 (en) | Lane recognition device and lane recognition method | |
| US20200285913A1 (en) | Method for training and using a neural network to detect ego part position | |
| CN109544635B (en) | Camera automatic calibration method based on enumeration heuristic | |
| CN102483881B (en) | Pedestrian-crossing marking detecting method and pedestrian-crossing marking detecting device | |
| KR20180098945A (en) | Method and apparatus for measuring speed of vehicle by using fixed single camera | |
| CN115578470A (en) | Monocular vision positioning method and device, storage medium and electronic equipment | |
| CN112489106A (en) | Video-based vehicle size measuring method and device, terminal and storage medium | |
| Lion et al. | Smart speed bump detection and estimation with kinect | |
| Cerri et al. | Free space detection on highways using time correlation between stabilized sub-pixel precision IPM images | |
| CN111881878A (en) | Lane line identification method for look-around multiplexing | |
| KR102003387B1 (en) | Method for detecting and locating traffic participants using bird's-eye view image, computer-readerble recording medium storing traffic participants detecting and locating program | |
| CN111626180B (en) | Lane line detection method and device based on polarization imaging | |
| CN111738035A (en) | Method, device and equipment for calculating yaw angle of vehicle | |
| CN115294211A (en) | Vehicle-mounted camera installation external parameter calibration method, system, device and storage medium | |
| JP2004062519A (en) | Lane mark detector |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: Room 101, No. 1, East Ring 3rd Street, Jitiagang, Huangjiang Town, Dongguan City, Guangdong Province, 523000 Applicant after: Guangdong Zhengyang Sensor Technology Co.,Ltd. Address before: 523000 Jitigang Village, Huangjiang Town, Dongguan City, Guangdong Province Applicant before: DONGGUAN ZHENGYANG ELECTRONIC MECHANICAL Co.,Ltd. |
|
| CB02 | Change of applicant information |