CN115526940A - Pitch angle calibration method and device of image acquisition device, driving device and medium - Google Patents
Pitch angle calibration method and device of image acquisition device, driving device and medium Download PDFInfo
- Publication number
- CN115526940A CN115526940A CN202211208507.7A CN202211208507A CN115526940A CN 115526940 A CN115526940 A CN 115526940A CN 202211208507 A CN202211208507 A CN 202211208507A CN 115526940 A CN115526940 A CN 115526940A
- Authority
- CN
- China
- Prior art keywords
- lane lines
- lane
- vanishing point
- determining
- pitch angle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/11—Complex mathematical operations for solving equations, e.g. nonlinear equations, general mathematical optimization problems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30244—Camera pose
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
- G06T2207/30256—Lane; Road marking
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Theoretical Computer Science (AREA)
- Pure & Applied Mathematics (AREA)
- Mathematical Optimization (AREA)
- Mathematical Analysis (AREA)
- Data Mining & Analysis (AREA)
- Computational Mathematics (AREA)
- Operations Research (AREA)
- Algebra (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the application discloses a pitch angle calibration method, a pitch angle calibration device, a running device and a medium of an image acquisition device, which are used for solving the technical problem that the accuracy of vehicle-mounted camera pitch angle calibration is not high, and the method can be applied to service scenes including automatic driving of vehicles, such as ports, port, road freight transportation, city delivery, mines, airports and the like, and comprises the following steps: acquiring a lane image acquired by an image acquisition device; the lane image comprises a plurality of lane lines; detecting the plurality of lane lines from the lane image; judging whether a lane line which is not parallel to at least two other lane lines exists in the plurality of lane lines; if so, finding out and eliminating the non-parallel lane lines from the plurality of lane lines, and determining the actual vanishing points of the lane lines by using the remaining lane lines; and determining the pitch angle of the image acquisition device by using the actual vanishing point and the camera internal reference of the image acquisition device.
Description
Technical Field
The embodiment of the application relates to the technical field of automatic driving, in particular to a pitch angle calibration method and device of an image acquisition device, a driving device and a medium.
Background
In the automatic driving or driving-assistant visual perception scheme, some elements perceived on a pixel plane need to be projected to a vehicle body coordinate system for three-dimensional reconstruction so as to be provided for a subsequent decision plan or a module.
In the prior art, the commonly used 2D-to-3D reconstruction scheme based on the ground plane assumption strongly depends on the external reference precision of the camera. However, the vehicle is affected by the road during driving, such as the road surface is uneven, or the ground has foreign matters, and the actual external parameters of the vehicle and the static calibration result are inconsistent due to the change of the tire pressure of the vehicle. Especially, the change of the pitch angle caused by the vehicle bump seriously affects the monocular distance measurement precision.
In order to solve the above problems, a currently commonly adopted solution is to perform lane line detection through forward vision, use a calculated intersection point of the lane lines as a vanishing point, calibrate a camera pitch angle through a position of the vanishing point, and further estimate a pitch angle of the vehicle. However, all lane lines detected by default in the scheme are parallel, and the actual situation does not always meet the situation, for example, at a junction of a lane and a bifurcation, a plurality of non-parallel lane lines exist, which causes that a calculated intersection point cannot represent the position of a true vanishing point, and further causes that the dynamic pitch angle calibration result is inaccurate.
In view of this, how to improve the calibration accuracy of the vehicle-mounted camera pitch angle becomes a technical problem to be solved urgently.
Disclosure of Invention
The embodiment of the application provides a pitch angle calibration method and device of an image acquisition device, a driving device and a medium, and aims to solve the technical problem that the accuracy of vehicle-mounted camera pitch angle calibration is not high.
In a first aspect, to solve the above technical problem, an embodiment of the present invention provides a method for calibrating a pitch angle of an image capturing device, including:
acquiring a lane image acquired by an image acquisition device; the lane image comprises a plurality of lane lines;
judging whether the lane lines are not parallel to at least two other lane lines or not; if so, finding out and eliminating the non-parallel lane lines from the plurality of lane lines, and determining the actual vanishing points of the lane lines by using the remaining lane lines;
and determining the pitch angle of the image acquisition device by using the actual vanishing point and the camera internal reference of the image acquisition device.
Acquiring a lane image acquired by an image acquisition device; further judging whether the plurality of lane lines have lane lines which are not parallel to at least two other lane lines; if so, finding out and eliminating the non-parallel lane lines from the plurality of lane lines, and determining the actual vanishing points of the lane lines by using the remaining lane lines; the unparallel lane lines can be eliminated, so that more accurate actual vanishing points are obtained, and the accuracy of the determined actual vanishing points is improved, so that the accuracy of determining the pitch angle of the image acquisition device is improved by using the actual vanishing points and the camera internal parameters of the image acquisition device.
One possible embodiment, the determining whether there is a lane line that is not parallel to at least two other lane lines in the plurality of lane lines includes:
determining a first vanishing point of the plurality of lane lines;
calculating an average distance between the first vanishing point and the plurality of lane lines;
if the average distance is larger than a preset threshold value, determining that the non-parallel lane lines exist in the plurality of lane lines; otherwise, determining that the non-parallel lane lines do not exist in the plurality of lane lines.
Whether the plurality of lane lines are not parallel or not can be quickly determined by calculating the average distance between the first vanishing point of the plurality of lane lines and further according to the size relation between the sub-average distance and the preset threshold value, so that the judgment efficiency can be effectively improved.
One possible embodiment, determining a first vanishing point for the plurality of lane lines, comprises:
determining a linear equation of each lane line;
determining a first intersection point of the plurality of lane lines by using a least square method according to a linear equation of each lane line;
the first intersection point is taken as the first vanishing point.
The first vanishing point of the plurality of lane lines can be quickly determined by calculating the linear equation of the plurality of lane lines by the least square method.
One possible embodiment, calculating an average distance between the first vanishing point and the plurality of lane lines, comprises:
calculating the distance between the first vanishing point and each lane line;
performing sum operation on the distances between all lane lines and the first vanishing point to obtain a sum operation result;
and carrying out quotient operation on the sum operation result and the total number of the plurality of lane lines to obtain the average distance.
By calculating the distance between the first vanishing point and each lane line and further calculating the average value of the distances, the average distance between the first vanishing point and the lane lines can be quickly calculated.
One possible embodiment, finding and removing the non-parallel lane lines from the plurality of lane lines, and determining the actual vanishing point of the lane lines by using the remaining lane lines, comprises:
setting the total number of a plurality of lane lines as M, the total number of the non-parallel lane lines as N, wherein N is more than or equal to 1 and less than or equal to M-2, starting from the value of N as 1, removing the N lane lines from the plurality of lane lines in the local wheel removing operation, using second vanishing points corresponding to the remaining lane lines and the average distance from the second vanishing points to the remaining lane lines until each lane line is removed for N times to complete the local wheel removing operation, and judging whether the minimum average distance obtained in the local wheel removing operation is smaller than the preset threshold value or not; if the minimum average distance is smaller than the preset threshold, determining a second vanishing point corresponding to the minimum average distance as the actual vanishing point; if the minimum average distance is larger than or equal to the preset threshold, N accumulation 1 repeatedly executes the rejecting operation of the current round until M-N is larger than 2 and the minimum average distance is smaller than the preset threshold, or N is evaluated as the rejecting operation of the corresponding round of M-2.
The method comprises the steps that the number of non-parallel lane lines is 1, the number of the non-parallel lane lines corresponding to a plurality of lane lines is removed in each round of removing operation, second vanishing points corresponding to the remaining lane lines and the average distance from the second vanishing points to the remaining lane lines are calculated until the number of the non-parallel lane lines corresponding to each round of removing operation is removed, the round of removing operation is completed, whether the second vanishing points determined by the round of removing operation are actual vanishing points is determined by judging whether the minimum average distance obtained in the round of removing operation is smaller than a preset threshold value, when the second vanishing points determined by the round of removing operation are determined to be the actual vanishing points, removing operation is not continued, the actual vanishing points can be determined quickly, and the operation amount can be reduced.
In one possible embodiment, the method further comprises:
and when M-N =2, setting a second vanishing point which is the nearest to the actual vanishing point determined in the previous frame of lane image in the removing operation of the M-2 corresponding wheel in the vehicle advancing direction as the actual vanishing point.
When the number of the parallel lane lines in the plurality of lane lines is 2, the second vanishing point which is the nearest to the actual vanishing point determined in the previous frame lane image in the vehicle advancing direction in the second vanishing point determined in the current round of removing operation is taken as the actual vanishing point, so that the accuracy of determining the actual vanishing point can be improved.
In a second aspect, an embodiment of the present application provides a pitch angle calibration apparatus for an image capturing device, including:
the acquisition unit is used for acquiring the lane image acquired by the image acquisition device; the lane image comprises a plurality of lane lines;
the processing unit is used for judging whether the plurality of lane lines have lane lines which are not parallel to at least two other lane lines; if so, finding out and eliminating the non-parallel lane lines from the plurality of lane lines, and determining the actual vanishing points of the lane lines by using the remaining lane lines;
and the determining unit is used for determining the pitch angle of the image acquisition device by using the actual vanishing point and the camera internal reference of the image acquisition device.
In one possible embodiment, the processing unit is further configured to:
determining a first vanishing point of the plurality of lane lines;
calculating an average distance between the first vanishing point and the plurality of lane lines;
if the average distance is larger than a preset threshold value, determining that the non-parallel lane lines exist in the plurality of lane lines; otherwise, determining that the non-parallel lane lines do not exist in the plurality of lane lines.
In one possible embodiment, the processing unit is further configured to:
determining a linear equation of each lane line;
determining a first intersection point of the plurality of lane lines by using a least square method according to a linear equation of each lane line;
the first intersection point is taken as the first vanishing point.
In one possible embodiment, the processing unit is further configured to:
calculating the distance between the first vanishing point and each lane line;
performing sum operation on the distances between all lane lines and the first vanishing point to obtain a sum operation result;
and carrying out quotient operation on the sum operation result and the total number of the plurality of lane lines to obtain the average distance.
In one possible embodiment, the processing unit is further configured to:
setting the total number of a plurality of lane lines as M, the total number of the non-parallel lane lines as N, wherein N is more than or equal to 1 and less than or equal to M-2, starting from the value of N as 1, removing the N lane lines from the plurality of lane lines in the current round of removing operation, using second vanishing points corresponding to the remaining lane lines and the average distance from the second vanishing points to the remaining lane lines until each lane line is removed for N times to complete the current round of removing operation, and judging whether the minimum average distance obtained in the current round of removing operation is smaller than the preset threshold value or not; if the minimum average distance is smaller than the preset threshold, determining a second vanishing point corresponding to the minimum average distance as the actual vanishing point; if the minimum average distance is larger than or equal to the preset threshold, N accumulation 1 repeatedly executes the removing operation of the current round until M-N is larger than 2 and the minimum average distance is smaller than the preset threshold, or N value is taken as the removing operation of the corresponding round of M-2.
In one possible embodiment, the processing unit is further configured to:
and when M-N =2, setting a second vanishing point which is the nearest to the actual vanishing point determined in the previous frame of lane image in the removing operation of the M-2 corresponding wheel in the vehicle advancing direction as the actual vanishing point.
In a third aspect, an embodiment of the present application further provides a running device, including:
moving the carrier;
the image acquisition device is arranged on the movable carrier and used for acquiring lane images in the advancing direction of the movable carrier;
a control device disposed on the mobile carrier or a cloud device in communication with the mobile carrier, the control device being configured to perform the method according to the first aspect.
In a fourth aspect, an embodiment of the present application further provides a pitch angle calibration apparatus for an image capturing device, including:
at least one processor, and
a memory coupled to the at least one processor;
wherein the memory stores instructions executable by the at least one processor, and the at least one processor performs the pitch angle calibration method according to the first aspect by executing the instructions stored by the memory.
In a fifth aspect, an embodiment of the present application further provides a computer-readable storage medium, including:
a memory for storing a plurality of data to be transmitted,
the memory is configured to store instructions that, when executed by the processor, cause the apparatus comprising the computer-readable storage medium to perform the pitch angle calibration method as described in the first aspect above.
In a sixth aspect, the present application further provides a computer program product, where the computer program product includes computer program code, and the computer program code is used to implement the pitch angle calibration method according to the first aspect.
Through the technical solutions in one or more of the above embodiments of the present application, the embodiments of the present application have at least the following technical effects:
in the embodiment provided by the application, the lane image acquired by the image acquisition device is acquired; further judging whether the plurality of lane lines have lane lines which are not parallel to at least two other lane lines; if so, finding out and eliminating the non-parallel lane lines from the plurality of lane lines, and determining the actual vanishing points of the lane lines by using the rest lane lines; the unparallel lane lines can be eliminated, so that more accurate actual vanishing points are obtained, and the accuracy of the determined actual vanishing points is improved, so that the accuracy of determining the pitch angle of the image acquisition device is improved by using the actual vanishing points and the camera internal parameters of the image acquisition device.
Drawings
Fig. 1 is a flowchart of a pitch angle calibration method for an image acquisition device according to an embodiment of the present disclosure;
FIG. 2 is a schematic view of a lane image provided by an embodiment of the present application;
FIG. 3 is a schematic diagram of a first intersection provided by an embodiment of the present invention;
fig. 4 is a schematic diagram of a distance between a first vanishing point and each lane line according to the embodiment of the present application;
fig. 5 to 8 are schematic diagrams illustrating a round of removing operation in a plurality of lane lines according to an embodiment of the present disclosure;
fig. 9 is a schematic structural diagram of a pitch angle calibration device of an image acquisition device according to an embodiment of the present application;
fig. 10 is a schematic structural diagram of a running device according to an embodiment of the present application.
Detailed Description
The application provides a method and a device for calibrating a pitch angle of an image acquisition device, a running device and a medium, and aims to solve the technical problem that the accuracy of calibration of the pitch angle of a vehicle-mounted camera is not high.
In order to better understand the technical solutions, the technical solutions of the present application are described in detail below with reference to the drawings and specific embodiments, and it should be understood that the specific features in the embodiments and examples of the present application are detailed descriptions of the technical solutions of the present application, and are not limitations of the technical solutions of the present application, and the technical features in the embodiments and examples of the present application may be combined with each other without conflict.
Referring to fig. 1, an embodiment of the present application provides a method for calibrating a pitch angle of an image capturing device, where the method includes the following steps.
Step 101: acquiring a lane image acquired by an image acquisition device; the lane image comprises a plurality of lane lines; the image acquisition device can be a camera, such as a vehicle-mounted camera, a camera, or an image sensor.
In some embodiments, after the lane image acquired by the image acquisition device is acquired, a plurality of lane lines can be detected from the lane image;
please refer to fig. 2, which is a schematic diagram of a lane image according to an embodiment of the present disclosure. The lane image shown in fig. 2 is captured by an image capturing device installed on an autonomous vehicle, and the vehicle-mounted image may be an image currently captured by the image capturing device, a current frame image in a video frame recorded by the image capturing device in real time, an image captured by the image capturing device after receiving a control command, or the like.
The lane line detection is performed on the lane image shown in fig. 2, and it is possible to detect 4 lane lines in total in the lane image shown in fig. 2.
After all lane lines in the lane image are detected, step 102 may be performed.
Step 102: judging whether a plurality of lane lines have lane lines which are not parallel to at least two other lane lines; if so, finding out and eliminating non-parallel lane lines from the multiple lane lines, and determining the actual vanishing points of the lane lines by using the rest lane lines.
Judging whether the plurality of lane lines have lane lines which are not parallel to at least two other lane lines can be realized by the following modes:
a first vanishing point for a plurality of lane lines is determined.
The linear equation of each lane line can be determined when all lane lines are detected, and the linear equation L can be used for 4 lane lines in the lane image shown in FIG. 2 1 、L 2 、L 3 、L 4 Representation (these line equations can be expressed as equations in the image coordinate system). Then, according to the equation of the straight line of each lane line, a first intersection (as shown in fig. 3, which is a schematic diagram of the first intersection provided in the embodiment of the present application) of the lane lines (the lane lines shown in fig. 2 are 4) is determined by using a least square method, and the first intersection is used as a first vanishing point of the lane lines.
Then, the average distance between the first vanishing point and the plurality of lane lines can be calculated; specifically, the distance between the first vanishing point and each lane line may be calculated according to the coordinate of the first vanishing point in the image coordinate system and a linear equation of each lane line in the image coordinate system (see fig. 4 for a schematic diagram of the distance between the first vanishing point and each lane line provided in the embodiment of the present application, and the distances between the first vanishing point and the four lane lines may be sequentially recorded as d 1 、d 2 、d 3 、d 4 ) And performing sum operation on the distances between all lane lines and the first vanishing point to obtain a sum operation result (d) 1 +d 2 +d 3 +d 4 ) Carrying out quotient operation on the sum operation result and the total number (4) of the plurality of lane lines to obtain the average distance (d) between the first vanishing point and the plurality of lane lines 1 +d 2 +d 3 +d 4 )/4。
If the average distance is larger than a preset threshold value, determining that the plurality of lane lines are not parallel; otherwise, determining that no non-parallel lane lines exist in the plurality of lane lines. The preset threshold may be an average error maximum of the average distance in units of pixel distances, and may be set to be 2 pixel distances, for example.
After determining whether the plurality of lane lines have the non-parallel lane lines by the method, the non-parallel lane lines can be found out and removed, and the actual vanishing points of the lane lines are determined by using the remaining lane lines, which can be specifically realized by the following modes:
setting the total number of a plurality of lane lines as M, the total number of non-parallel lane lines as N, wherein N is more than or equal to 1 and less than or equal to M-2, starting from the value of N as 1, removing the N lane lines from the plurality of lane lines in the local wheel removing operation, using second vanishing points corresponding to the remaining lane lines and the average distance from the second vanishing points to the remaining lane lines until each lane line is removed for N times to complete the local wheel removing operation, and judging whether the minimum average distance obtained in the local wheel removing operation is smaller than a preset threshold value or not; if the minimum average distance is smaller than a preset threshold value, determining a second vanishing point corresponding to the minimum average distance as an actual vanishing point; and if the minimum average distance is larger than or equal to the preset threshold, the N accumulation 1 repeatedly executes the rejecting operation of the current round until M-N is larger than 2 and the minimum average distance is smaller than the preset threshold, or rejecting operation of a corresponding round with the value of N being M-2 is completed.
Taking fig. 3 as an example, M =4, when N is 1 (i.e. there are 1 unparallel lane lines), please refer to fig. 5-8, which are schematic diagrams of a round of removing operation among a plurality of lane lines provided in the embodiment of the present application, for the four lane lines in fig. 3, in the first round of removing operation, 1 (i.e. N = 1) lane line is removed each time, the remaining 3 (i.e. M-N) lines are used to determine the corresponding second vanishing point, and the average distance from the second vanishing point to the corresponding three lane lines is calculated, as shown in fig. 5, the lane line L is removed 1 Using lane line L 2 ～L 4 Determining the corresponding second vanishing point O 1 And the second vanishing point O 1 And a lane line L 2 ～L 4 Average distance AV of 1 (ii) a As shown in fig. 6, is removedLane line L 2 Using lane line L 1 Lane line L 3 Lane line L 4 Determining a corresponding second vanishing point O 2 And a second vanishing point O 2 And a lane line L 1 Lane line L 3 Lane line L 4 Average distance AV of 2 (ii) a As shown in fig. 7, the lane line L is removed 3 By lane line L 1 Lane line L 2 Lane line L 4 Determining a corresponding second vanishing point O 3 And a second vanishing point O 3 And a lane line L 1 Lane line L 2 Lane line L 4 Average distance AV of 3 (ii) a As shown in fig. 8, the lane line L is removed 4 Using lane line L 1 -lane line L 3 Determining a corresponding second vanishing point O 4 And a second vanishing point O 4 And a lane line L 1 Lane line L 3 Average distance AV of 4 From the second vanishing point O1 to the second vanishing point O 4 Corresponding average distance AV 1 Average distance AV 4 The average distance with the smallest value is determined as the minimum average distance (AV) 4 = 0), and determines whether the minimum average distance is smaller than a preset threshold value (assumed to be 0.2), the minimum average distance AV may be determined 4 Less than a predetermined threshold, and M-N =4-1=3>2, its corresponding second vanishing point O 4 As the actual vanishing points of the plurality of lane lines, lane line L 4 Are not parallel lane lines.
In some embodiments, when M-N =2, the minimum average distance determined in the current round of the culling operation is greater than a preset threshold, and a vanishing point that is closest to an actual vanishing point determined in the previous frame of the lane image in the vehicle traveling direction among all the second vanishing points in the current round of the culling operation is taken as the actual vanishing point.
As an example of the first round of elimination, if the minimum average distance AV is 4 =0.25, the minimum average distance AV is determined 4 When the N accumulation is larger than the preset threshold value, the N accumulation is 1 to 2, then the second round rejecting operation is carried out in a mode similar to the first round rejecting operation, and because the distance from each second vanishing point to the corresponding lane line is zero at the moment, the distance from each second vanishing point to the corresponding lane line cannot be selected because the distance from each second vanishing point to the corresponding lane line is zero at the moment M-2=4-2=NThe minimum average distance, at which the vanishing point that is the closest in the vehicle traveling direction from the actual vanishing point determined in the previous frame of lane image among all the second vanishing points in the second round of operation can be taken as the actual vanishing point.
Step 103: and determining the pitch angle of the image acquisition device by using the actual vanishing point and the camera internal reference of the image acquisition device.
And obtaining an included angle between the image acquisition device and the ground according to the actual vanishing point and internal parameters of the image acquisition device, namely determining the pitch angle of the image acquisition device.
In the embodiment provided by the application, the lane image acquired by the image acquisition device is acquired; further judging whether the lane lines are not parallel to at least two other lane lines or not; if so, finding out and eliminating the non-parallel lane lines from the plurality of lane lines, and determining the actual vanishing points of the lane lines by using the remaining lane lines; the unparallel lane lines can be eliminated, so that more accurate actual vanishing points are obtained, and the accuracy of the determined actual vanishing points is improved, so that the accuracy of determining the pitch angle of the image acquisition device is improved by using the actual vanishing points and the camera internal parameters of the image acquisition device.
The pitch angle calibration method provided by the embodiment of the application can be realized at a vehicle end, can also be realized at a cloud server, and can also be realized by combining the vehicle end and the cloud server, and the embodiment of the application does not limit the method. During combined implementation, the vehicle end and the cloud end correspondingly execute relevant steps, for example, the vehicle end executes step 101, the cloud end server executes step 102 and step 103, images are collected through the vehicle end sensor and processed in the cloud end server, computing power of the cloud end server is fully utilized, vehicle end computing power resources are saved, and processing efficiency is greatly improved.
In an embodiment of the present application, a device for calibrating a pitch angle of an image acquisition device is further provided, where the specific implementation of the method for calibrating a pitch angle of an image acquisition device of the device may refer to the description in the embodiment of the method, and repeated descriptions thereof are omitted, please refer to fig. 9, and the device includes:
an acquiring unit 901, configured to acquire a lane image acquired by an image acquisition device; the lane image comprises a plurality of lane lines;
a processing unit 902, configured to determine whether there is a lane line that is not parallel to at least two other lane lines in the multiple lane lines; if so, finding out and eliminating the non-parallel lane lines from the plurality of lane lines, and determining the actual vanishing points of the lane lines by using the remaining lane lines;
a determining unit 903, configured to determine a pitch angle of the image capturing device by using the actual vanishing point and a camera internal parameter of the image capturing device.
In a possible implementation, the processing unit 902 is further configured to:
determining a first vanishing point of the plurality of lane lines;
calculating an average distance between the first vanishing point and the plurality of lane lines;
if the average distance is larger than a preset threshold value, determining that the non-parallel lane lines exist in the plurality of lane lines; otherwise, determining that the non-parallel lane lines do not exist in the plurality of lane lines.
In a possible implementation, the processing unit 902 is further configured to:
determining a linear equation of each lane line;
determining a first intersection point of the plurality of lane lines by using a least square method according to a linear equation of each lane line;
taking the first intersection as the first vanishing point.
In a possible implementation, the processing unit 902 is further configured to:
calculating the distance between the first vanishing point and each lane line;
performing sum operation on the distances between all lane lines and the first vanishing point to obtain a sum operation result;
and carrying out quotient operation on the sum operation result and the total number of the plurality of lane lines to obtain the average distance.
In a possible implementation, the processing unit 902 is further configured to:
setting the total number of a plurality of lane lines as M, the total number of the non-parallel lane lines as N, wherein N is more than or equal to 1 and less than or equal to M-2, starting from the value of N as 1, removing the N lane lines from the plurality of lane lines in the local wheel removing operation, using second vanishing points corresponding to the remaining lane lines and the average distance from the second vanishing points to the remaining lane lines until each lane line is removed for N times to complete the local wheel removing operation, and judging whether the minimum average distance obtained in the local wheel removing operation is smaller than the preset threshold value or not; if the minimum average distance is smaller than the preset threshold, determining a second vanishing point corresponding to the minimum average distance as the actual vanishing point; if the minimum average distance is larger than or equal to the preset threshold, N accumulation 1 repeatedly executes the removing operation of the current round until M-N is larger than 2 and the minimum average distance is smaller than the preset threshold, or N value is taken as the removing operation of the corresponding round of M-2.
In a possible implementation, the processing unit 902 is further configured to:
and when M-N =2, setting a second vanishing point which is the nearest to the actual vanishing point determined in the previous frame of lane image in the removing operation of the M-2 corresponding wheel in the vehicle advancing direction as the actual vanishing point.
It should be noted that, in the embodiment of the present application, the division of the unit is schematic, and is only one logic function division, and when the actual implementation is realized, another division manner may be provided. In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit may be implemented in the form of hardware, or may also be implemented in the form of a software functional unit.
The integrated unit, if implemented as a software functional unit and sold or used as a stand-alone product, may be stored in a processor readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, a network device, or the like) or a processor (processor) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
It should be noted that the apparatus provided in the embodiment of the present application can implement all the method steps implemented by the method embodiment and achieve the same technical effect, and detailed descriptions of the same parts and beneficial effects as the method embodiment in this embodiment are omitted here.
Fig. 10 is a schematic structural diagram of a driving device according to an embodiment of the present application, where the driving device includes:
a mobile carrier 1001; the mobile carrier 1001 may be an autonomous vehicle, a smart wearable device, or the like.
An image capturing device 1002 provided on the moving carrier 1001 for capturing a lane image in a traveling direction of the moving carrier 1001; the image capturing device 1002 may be a camera, such as a vehicle-mounted camera, a camera, or an image sensor, and may be changed according to the mobile carrier 1001.
The control device 1003 is disposed on the mobile carrier 1001 or the image acquisition device 1002 or a cloud device in communication with the mobile carrier 1001, and the control device 1003 is configured to execute the method for calibrating the vehicle-mounted camera pitch angle as described above, and for a specific implementation, reference may be made to the description of the vehicle-mounted camera pitch angle calibration method, and repeated details are not repeated. The control device 1003 may be a processor, a microprocessor, a server, or the like, and if the control device 1003 is disposed on the autonomous vehicle or the smart wearable device or the image capture device 1002, the control device 1003 may be the microprocessor or the processor, and if the control device 1003 is disposed on a cloud device communicating with the autonomous vehicle or the smart wearable device, the control device 1003 may also be the server.
The embodiment of the application provides a device that image acquisition device pitch angle was markd, includes: at least one processor, and
a memory coupled to the at least one processor;
the memory stores instructions executable by the at least one processor, and the at least one processor executes the instructions stored by the memory to perform the method for calibrating the pitch angle of the image acquisition device.
An embodiment of the present application further provides a readable storage medium, including:
a memory for storing a plurality of data files to be transmitted,
the memory is configured to store instructions that, when executed by the processor, cause the apparatus comprising the readable storage medium to perform the method for image capturing apparatus pitch angle calibration as described above.
The readable storage medium may be any available medium or data storage device that can be accessed by a processor, including volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. By way of example and not limitation, nonvolatile Memory may include Read-Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash Memory, solid State Disk (SSD), magnetic Memory (e.g., floppy Disk, hard Disk, magnetic tape, magneto-Optical Disk (MO), etc.), optical Memory (e.g., CD, DVD, BD, HVD, etc.). Volatile Memory can include Random Access Memory (RAM), which can act as external cache Memory. By way of example and not limitation, RAM is available in a variety of forms, such as Dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM). The storage devices of the disclosed aspects are intended to comprise, without being limited to, these and other suitable types of memory.
An embodiment of the present application further provides a computer program product, which includes computer program code for implementing the pitch angle calibration method as described above.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or program product. Accordingly, embodiments of the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present application may take the form of a computer program product embodied on one or more readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer/processor-usable program code embodied therein.
Embodiments of the present application are described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These program instructions may also be stored in a readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer/processor implemented process such that the instructions which execute on the computer/processor or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various changes and modifications may be made in the embodiments of the present application without departing from the scope of the embodiments of the present application. Thus, if such modifications and variations of the embodiments of the present application fall within the scope of the claims of the present application and their equivalents, the embodiments of the present application are intended to include such modifications and variations as well.
Claims (10)
1. A pitch angle calibration method of an image acquisition device is characterized by comprising the following steps:
acquiring a lane image acquired by an image acquisition device; the lane image comprises a plurality of lane lines;
judging whether the lane lines are not parallel to at least two other lane lines or not; if so, finding out and eliminating the non-parallel lane lines from the plurality of lane lines, and determining the actual vanishing points of the lane lines by using the remaining lane lines;
and determining the pitch angle of the image acquisition device by using the actual vanishing point and the camera internal reference of the image acquisition device.
2. The method for calibrating the pitch angle according to claim 1, wherein the step of determining whether there is a lane line that is not parallel to at least two other lane lines in the plurality of lane lines comprises:
determining a first vanishing point of the plurality of lane lines;
calculating an average distance between the first vanishing point and the plurality of lane lines;
if the average distance is larger than a preset threshold value, determining that the non-parallel lane lines exist in the plurality of lane lines; otherwise, determining that the non-parallel lane lines do not exist in the plurality of lane lines.
3. The pitch angle calibration method of claim 2, wherein determining a first vanishing point for the plurality of lane lines comprises:
determining a linear equation of each lane line;
determining a first intersection point of the plurality of lane lines by using a least square method according to a linear equation of each lane line;
taking the first intersection as the first vanishing point.
4. The method of claim 2, wherein calculating the average distance between the first vanishing point and the plurality of lane lines comprises:
calculating the distance between the first vanishing point and each lane line;
performing sum operation on the distances between all lane lines and the first vanishing point to obtain a sum operation result;
and carrying out quotient operation on the sum operation result and the total number of the plurality of lane lines to obtain the average distance.
5. The method for calibrating a pitch angle according to any one of claims 2 to 4, wherein the step of finding and removing the non-parallel lane lines from the plurality of lane lines and determining an actual vanishing point of the lane lines using the remaining lane lines comprises:
setting the total number of a plurality of lane lines as M, the total number of the non-parallel lane lines as N, wherein N is more than or equal to 1 and less than or equal to M-2, starting from the value of N as 1, removing the N lane lines from the plurality of lane lines in the local wheel removing operation, using second vanishing points corresponding to the remaining lane lines and the average distance from the second vanishing points to the remaining lane lines until each lane line is removed for N times to complete the local wheel removing operation, and judging whether the minimum average distance obtained in the local wheel removing operation is smaller than the preset threshold value or not; if the minimum average distance is smaller than the preset threshold value, determining a second vanishing point corresponding to the minimum average distance as the actual vanishing point; if the minimum average distance is larger than or equal to the preset threshold, N accumulation 1 repeatedly executes the removing operation of the current round until M-N is larger than 2 and the minimum average distance is smaller than the preset threshold, or N value is taken as the removing operation of the corresponding round of M-2.
6. The pitch angle calibration method as set forth in claim 5, further comprising:
and when M-N =2, setting a second vanishing point which is the nearest to the actual vanishing point determined in the previous frame of lane image in the removing operation of the M-2 corresponding wheel in the vehicle advancing direction as the actual vanishing point.
7. The utility model provides an image acquisition device's angle of pitch calibration device which characterized in that includes:
the acquisition unit is used for acquiring the lane image acquired by the image acquisition device; the lane image comprises a plurality of lane lines;
the processing unit is used for judging whether the plurality of lane lines are not parallel to at least two other lane lines or not; if so, finding out and eliminating the non-parallel lane lines from the plurality of lane lines, and determining the actual vanishing points of the lane lines by using the remaining lane lines;
and the determining unit is used for determining the pitch angle of the image acquisition device by using the actual vanishing point and the camera internal parameter of the image acquisition device.
8. A running device, characterized by comprising:
moving the carrier;
the image acquisition device is arranged on the movable carrier and used for acquiring lane images in the advancing direction of the movable carrier;
a control device disposed on the mobile carrier or a cloud device in communication with the mobile carrier, the control device being configured to execute the pitch angle calibration method according to any one of claims 1 to 6.
9. The utility model provides an image acquisition device's angle of pitch calibration device which characterized in that includes:
at least one processor, and
a memory coupled to the at least one processor;
wherein the memory stores instructions executable by the at least one processor, and the at least one processor performs the pitch angle calibration method as claimed in any one of claims 1-6 by executing the instructions stored by the memory.
10. A computer-readable storage medium, comprising a memory,
the memory is configured to store instructions that, when executed by the processor, cause an apparatus comprising the computer-readable storage medium to implement the pitch angle calibration method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211208507.7A CN115526940A (en) | 2022-09-30 | 2022-09-30 | Pitch angle calibration method and device of image acquisition device, driving device and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211208507.7A CN115526940A (en) | 2022-09-30 | 2022-09-30 | Pitch angle calibration method and device of image acquisition device, driving device and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115526940A true CN115526940A (en) | 2022-12-27 |
Family
ID=84698757
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211208507.7A Pending CN115526940A (en) | 2022-09-30 | 2022-09-30 | Pitch angle calibration method and device of image acquisition device, driving device and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115526940A (en) |
-
2022
- 2022-09-30 CN CN202211208507.7A patent/CN115526940A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20180307922A1 (en) | Method of detecting obstacle around vehicle | |
| CN111612728B (en) | 3D point cloud densification method and device based on binocular RGB image | |
| EP3671643A1 (en) | Method and apparatus for calibrating the extrinsic parameter of an image sensor | |
| EP2759959A2 (en) | Method and system for detecting multi-lanes | |
| US10457283B2 (en) | Vehicle driving assist apparatus | |
| US20160305785A1 (en) | Road surface detection device and road surface detection system | |
| CN111627001B (en) | Image detection method and device | |
| WO2018030010A1 (en) | Road surface estimation device, vehicle control device, road surface estimation method, and program | |
| JP6038422B1 (en) | Vehicle determination device, vehicle determination method, and vehicle determination program | |
| JP2014009975A (en) | Stereo camera | |
| CN110853366A (en) | Method and device for detecting parking position of vehicle | |
| CN111950504B (en) | Vehicle detection method and device and electronic equipment | |
| CN111386530B (en) | Vehicle detection method and apparatus | |
| CN113269766A (en) | Hob abrasion detection method, hob abrasion detection device, hob abrasion detection equipment and readable storage medium | |
| CN110751040A (en) | Three-dimensional object detection method and device, electronic equipment and storage medium | |
| CN114863388A (en) | Method, device, system, equipment, medium and product for determining obstacle orientation | |
| CN112614174A (en) | Point cloud complementing and point cloud dividing method and device, electronic equipment and storage medium | |
| US20200193184A1 (en) | Image processing device and image processing method | |
| CN114120266A (en) | Vehicle lane change detection method and device, electronic equipment and storage medium | |
| JP2014046710A (en) | Neutral point correction unit and neutral point correction method for steering angle sensor | |
| CN115526940A (en) | Pitch angle calibration method and device of image acquisition device, driving device and medium | |
| CN114049394B (en) | Monocular distance measuring method, device, equipment and storage medium | |
| US20220067396A1 (en) | Vehicle external environment recognition apparatus | |
| US20220319195A1 (en) | Method for identifying road markings and monocular camera | |
| CN114037977A (en) | Road vanishing point detection method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |