CN112419423A

CN112419423A - Calibration method, calibration device, electronic equipment and storage medium

Info

Publication number: CN112419423A
Application number: CN202011189288.3A
Authority: CN
Inventors: 马政; 黄瑞; 刘春晓; 石建萍
Original assignee: Shanghai Sensetime Lingang Intelligent Technology Co Ltd
Current assignee: Shanghai Sensetime Lingang Intelligent Technology Co Ltd
Priority date: 2020-10-30
Filing date: 2020-10-30
Publication date: 2021-02-26

Abstract

The disclosure provides a calibration method, a calibration device, an electronic device and a storage medium, wherein the calibration method comprises the following steps: acquiring a road surface image which is acquired by image acquisition equipment and comprises a plurality of lane lines and world coordinates of a plurality of position points on each lane line under a world coordinate system; and determining the current homography matrix of the image acquisition equipment based on the position information of each lane line in the road surface image, the world coordinates of the plurality of position points and the initial homography matrix of the image acquisition equipment.

Description

Calibration method, calibration device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of computer vision technologies, and in particular, to a calibration method, an apparatus, an electronic device, and a storage medium.

Background

Along with the rapid development of artificial intelligence technology, the traditional industry is combined with information technology, convenience is brought to the life of people, for example, the automobile industry is combined with the information technology, an intelligent automobile capable of automatically driving can be produced, and distance measurement is an important link of the intelligent automobile in the automatic driving process. In the distance measuring sensor adopted by the intelligent automobile for driving assistance, the vision sensor can obtain richer road structure environment information, and the price is also lower.

In the visual ranging, the monocular visual ranging technology has the characteristics of low cost, simple system installation, good stability and the like compared with the monocular visual ranging technology, so that the monocular visual ranging technology is widely adopted. In monocular visual ranging, a homography matrix (homography matrix) is needed, a world coordinate of a target object in a world coordinate system can be obtained based on a pixel coordinate of the shot target object in an image coordinate system and the homography matrix, and distance information between the target object and a preset position point can be obtained based on the world coordinate. Therefore, the accuracy of the homography matrix directly affects the accuracy of the ranging result.

The homography matrix is obtained by calibration in advance, during calibration, a reference object is placed manually, a reference object picture is obtained according to the vehicle-mounted camera, the homography matrix of the vehicle-mounted camera is determined through pixels corresponding to the reference object in the reference object picture and coordinates of the reference object in a world coordinate system, calibration efficiency is low, the position of the vehicle-mounted camera is changed due to mechanical vibration of a vehicle in the driving process, and the homography matrix obtained by calibration of the vehicle-mounted camera before is not accurate when the distance of a target object is determined.

Disclosure of Invention

The disclosed embodiment provides at least one calibration scheme.

In a first aspect, an embodiment of the present disclosure provides a calibration method, including:

acquiring a road surface image which is acquired by image acquisition equipment and comprises a plurality of lane lines and world coordinates of a plurality of position points on each lane line under a world coordinate system;

and determining the current homography matrix of the image acquisition equipment based on the position information of each lane line in the road surface image, the world coordinates of the plurality of position points and the initial homography matrix of the image acquisition equipment.

After the world coordinates which are acquired by the image acquisition equipment and comprise a plurality of lane line road surface images and a plurality of position points on each lane line under a world coordinate system are acquired, the adjustment of the initial homography matrix can be completed through the position information of each lane line in the road surface images and the world coordinates of the position points, and the current homography matrix of the image acquisition equipment is determined; for example, the image acquisition device may be disposed on a running vehicle, so that as the vehicle runs, the road surface image is always the road surface image acquired by the image acquisition device while maintaining the current relative position relationship with the vehicle, and when the road surface image includes a plurality of lane lines, the initial homography matrix of the image acquisition device may be continuously corrected, thereby alleviating a problem that the homography matrix is not accurate any more due to mechanical vibration or unevenness of the road surface on which the vehicle runs during the running of the vehicle.

In one possible implementation, the world coordinates of a plurality of location points on each lane line are obtained as follows:

determining world coordinates of intersection points of the target line segments and each lane line in the road surface image under the world coordinate system;

and acquiring the world coordinates of a plurality of position points on each lane line under the world coordinate system based on the world coordinates of each intersection point.

In one possible embodiment, the determining world coordinates of an intersection of each lane line and a target line segment in the road surface image in the world coordinate system includes:

selecting a plurality of initial position points under the world coordinate system, wherein the distance between each initial position point and the origin of the world coordinate system is less than a preset distance threshold;

and determining the target line segment formed by connecting the initial pixel coordinates corresponding to the plurality of initial position points in the road surface image and the world coordinates of the intersection point of the target line segment and each lane line in the world coordinate system based on the initial homography matrix.

According to the method and the device, the initial position point with the smaller distance from the origin point in the world coordinate system is selected, so that the error of the physical distance corresponding to the initial pixel coordinate of the initial position point in the image coordinate system based on the initial homography matrix is smaller, and the error of the world coordinate corresponding to the intersection point in the world coordinate system based on the initial pixel coordinate is smaller.

In a possible implementation manner, the determining, based on the initial homography matrix, the target line segment formed by connecting the initial pixel coordinates corresponding to the plurality of initial position points in the road surface image, and the world coordinates of the intersection point of the target line segment and each lane line in the world coordinate system includes:

determining initial pixel coordinates corresponding to each initial position point in the road surface image based on the initial homography matrix and the initial position coordinates of each initial position point in the world coordinate system;

performing straight line fitting on the initial pixel coordinates corresponding to the plurality of initial position points to obtain the target line segment;

acquiring intersection pixel coordinates of the intersection point of the target line segment and each lane line in the road surface image;

and determining the world coordinates of the intersection point under the world coordinate system based on the intersection point pixel coordinates of the intersection point and the initial homography matrix.

In a possible implementation manner, the obtaining, based on the world coordinates of each intersection point, the world coordinates of a plurality of position points on each lane line in a world coordinate system includes:

and acquiring the world coordinates of a plurality of position points which are positioned on the same lane line with each intersection point under a world coordinate system based on the world coordinates of each intersection point and a preset position interval.

In one possible embodiment, the determining the current homography matrix of the image capturing device based on the position information of each lane line in the road surface image, the world coordinates of the plurality of position points, and the initial homography matrix of the image capturing device includes:

determining initial pixel coordinates of the plurality of position points in the road surface image based on the world coordinates of the plurality of position points and the initial homography matrix;

based on the position information of each lane line in the road surface image, correcting the initial pixel coordinates corresponding to a plurality of position points on the lane line to obtain corrected pixel coordinates of the plurality of position points on the lane line in the road surface image;

and obtaining the current homography matrix of the image acquisition equipment based on the corrected pixel coordinates and the world coordinates of a plurality of position points on each lane line.

In the embodiment of the disclosure, the initial pixel coordinates of the multiple position points in the road surface image are determined based on the world coordinates and the initial homography matrix of the multiple position points, and when the initial homography matrix is no longer accurate, the initial pixel coordinates can be quickly corrected based on the initial pixel coordinates of the multiple position points and the position information of the lane line in the road surface image, so that the current homography matrix of the image acquisition device is quickly determined based on the corrected pixel coordinates.

In a possible implementation manner, the correcting initial pixel coordinates corresponding to a plurality of position points on each lane line based on the position information of each lane line in the road surface image to obtain corrected pixel coordinates of the plurality of position points on the lane line in the road surface image includes:

determining a fitted straight line of a plurality of initial pixel coordinates and a deviation angle between the fitted straight line of each lane line and the corresponding fitted straight line of the lane line in the road surface image based on the position information of each lane line in the road surface image and the determined initial pixel coordinates corresponding to the plurality of position points on the lane line;

and correcting the initial pixel coordinates corresponding to the plurality of position points on the lane line based on the determined deviation angle to obtain corrected pixel coordinates of the plurality of position points on the lane line in the road surface image.

In the embodiment of the disclosure, when the initial homography matrix is no longer accurate, the initial pixel coordinates of the plurality of position points determined based on the initial homography matrix in the image coordinate system may deviate from the corresponding lane line, and by determining the deviation angle between the initial pixel coordinates of the plurality of position points on each lane line in the image coordinate system and the lane line, the initial pixel coordinates of the plurality of position points on the lane line may be corrected based on the deviation angle, so that the pixel coordinates of the plurality of position points on the lane line may be more accurately determined to be corrected in the image coordinate system.

In a possible embodiment, the determining, based on the position information of each lane line in the road surface image and the determined initial pixel coordinates corresponding to the plurality of position points on the lane line, a fitted straight line of the plurality of initial pixel coordinates, and a deviation angle between the fitted straight line of the lane line in the road surface image includes:

generating a first lane line equation under an image coordinate system corresponding to each lane line in the road surface image based on the position information of each lane line in the road surface image;

performing straight line fitting on the initial pixel coordinates corresponding to the plurality of position points on each lane line to generate a second lane line equation corresponding to each lane line;

and determining the deviation angle of the straight line represented by the second lane line equation relative to the straight line represented by the first lane line equation based on the first lane line equation and the second lane line equation corresponding to each lane line.

In a possible embodiment, after determining the current homography matrix of the image capturing device, the method further includes:

acquiring a target image obtained after the image acquisition component shoots a target object;

determining pixel coordinates of the target object in the target image based on the target image;

and determining world coordinates of the target object under the world coordinate system based on the pixel coordinates and the current homography matrix.

In a possible implementation, after determining the world coordinates of the target object in the world coordinate system, the method further includes:

and determining the distance between the target object and the preset position point based on the world coordinate of the target object in the world coordinate system and the coordinate of the preset position point in the world coordinate system.

After the current homography matrix of the image acquisition equipment is obtained, the world coordinate of the target object in the world coordinate system can be accurately determined by using the current homography matrix, and further the distance between the target object and the target object is determined.

In a second aspect, an embodiment of the present disclosure provides a calibration apparatus, including:

the system comprises an acquisition module, a display module and a display module, wherein the acquisition module is used for acquiring a road surface image which is acquired by image acquisition equipment and comprises a plurality of lane lines and world coordinates of a plurality of position points on each lane line under a world coordinate system;

and the determining module is used for determining the current homography matrix of the image acquisition equipment based on the position information of each lane line in the road surface image, the world coordinates of the plurality of position points and the initial homography matrix of the image acquisition equipment.

In a third aspect, an embodiment of the present disclosure provides an electronic device, including: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating via the bus when the electronic device is running, the machine-readable instructions when executed by the processor performing the steps of the calibration method according to the first aspect.

In a fourth aspect, the disclosed embodiments provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, performs the steps of the calibration method according to the first aspect.

In order to make the aforementioned objects, features and advantages of the present disclosure more comprehensible, preferred embodiments accompanied with figures are described in detail below.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly described below, and the drawings herein incorporated in and forming a part of the specification illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the technical solutions of the present disclosure. It is appreciated that the following drawings depict only certain embodiments of the disclosure and are therefore not to be considered limiting of its scope, for those skilled in the art will be able to derive additional related drawings therefrom without the benefit of the inventive faculty.

FIG. 1 is a flow chart illustrating a calibration method provided by an embodiment of the present disclosure;

FIG. 2 is a flowchart illustrating a method for obtaining world coordinates of a plurality of location points on each lane line according to an embodiment of the present disclosure;

FIG. 3 is a flowchart illustrating a method for determining a current homography matrix of an image capture device according to an embodiment of the present disclosure;

FIG. 4a shows one of the flowcharts for determining the current homography matrix of the image capturing device provided by the embodiments of the present disclosure;

fig. 4b illustrates a second flowchart of determining a current homography matrix of an image capturing device according to an embodiment of the present disclosure;

FIG. 4c is a third flowchart illustrating a method for determining a current homography matrix of an image capturing device according to an embodiment of the disclosure;

FIG. 5 is a flowchart illustrating a method for performing ranging based on a current homography matrix of an image capturing device according to an embodiment of the disclosure;

fig. 6 is a schematic structural diagram illustrating a calibration apparatus provided in an embodiment of the present disclosure;

fig. 7 shows a schematic structural diagram of an electronic device provided in an embodiment of the present disclosure.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be described clearly and completely with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are only a part of the embodiments of the present disclosure, not all of the embodiments. The components of the embodiments of the present disclosure, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present disclosure, presented in the figures, is not intended to limit the scope of the claimed disclosure, but is merely representative of selected embodiments of the disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the disclosure without making creative efforts, shall fall within the protection scope of the disclosure.

In the fields of automatic driving and robots, visual ranging is often required to be carried out by means of image acquisition equipment, the principle of visual ranging by means of image acquisition equipment is to determine pixel coordinates of a target object shot by the image acquisition equipment in an image coordinate system, then world coordinates of the target object in a world coordinate system are determined based on a homography matrix of the image acquisition equipment, and then the distance between a preset position point and the target object is determined according to the world coordinates of the preset position point and the world coordinates of the target object, wherein the preset position point can be the origin of the set world coordinate system, so that the accuracy of the homography matrix directly influences the accuracy of ranging results. The homography matrix of the image acquisition equipment on the vehicle can be obtained by pre-calibrating the position relation between the image acquisition equipment and the vehicle, and after the image acquisition equipment on the vehicle is calibrated, the distance of the target object can be determined according to the homography matrix.

Based on the research, the disclosure provides a calibration method, after acquiring a world coordinate which is acquired by an image acquisition device and comprises a plurality of lane line road surface images and a plurality of position points on each lane line under a world coordinate system, adjusting an initial homography matrix through position information of each lane line in the road surface images and the world coordinates of the position points, and determining a current homography matrix of the image acquisition device; for example, the image acquisition device may be disposed on a running vehicle, so that as the vehicle runs, the road surface image is always the road surface image acquired by the image acquisition device while maintaining the current relative position relationship with the vehicle, and when the road surface image includes a plurality of lane lines, the initial homography matrix of the image acquisition device may be continuously corrected, thereby alleviating a problem that the homography matrix is not accurate any more due to mechanical vibration or unevenness of the road surface on which the vehicle runs during the running of the vehicle.

The technical solutions in the present disclosure will be described clearly and completely with reference to the accompanying drawings in the present disclosure, and it is to be understood that the described embodiments are only a part of the embodiments of the present disclosure, and not all of the embodiments. The components of the present disclosure, as generally described and illustrated in the figures herein, may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present disclosure, presented in the figures, is not intended to limit the scope of the claimed disclosure, but is merely representative of selected embodiments of the disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the disclosure without making creative efforts, shall fall within the protection scope of the disclosure.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.

To facilitate understanding of the present embodiment, first, a detailed description is given to a calibration method disclosed in an embodiment of the present disclosure, where an execution subject of the calibration method provided in the embodiment of the present disclosure is generally a computer device with certain computing capability, and the computer device includes, for example: a terminal device, which may be a User Equipment (UE), a mobile device, a User terminal, a Personal Digital Assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, or a server or other processing devices. In some possible implementations, the calibration method may be implemented by a processor calling computer readable instructions stored in a memory.

The following describes a calibration method provided by the embodiment of the present disclosure by taking an execution subject as a terminal device.

Referring to fig. 1, a flowchart of a calibration method provided by the embodiment of the present disclosure, the calibration method may be applied to a processor in an on-board device, and the calibration method includes steps S101 to S102, where:

s101, acquiring a road surface image which is acquired by image acquisition equipment and comprises a plurality of lane lines and world coordinates of a plurality of position points on each lane line under a world coordinate system.

Illustratively, the image capturing device may include a monocular camera, or a monocular camera, and may include a camera for capturing RGB images, grayscale images, or depth images, the image capturing device may be disposed on a vehicle, and when the vehicle is driving on a road surface, the image capturing device may capture a road surface image of the current road surface, and then transmit the road surface image to a processor in the vehicle-mounted device, and the processor detects a lane line in the road surface image, and if the road surface image includes a plurality of lane lines and at least includes two lane lines that do not overlap, the current homography matrix of the image capturing device may be determined according to a plurality of position points on each lane line in the road surface image.

Here, the plurality of location points on each lane line are included location points of the lane line on the current road surface, and these location points may be location points existing on the lane line of the road surface, that is, location points actually existing in the geographic environment.

Illustratively, the world coordinate system may be pre-established in the following manner:

a world coordinate system is established with a front axle center point of a vehicle or a mapping point of a vehicle body center on the ground as an origin, a forward direction of the vehicle as an X-axis, a direction perpendicular to the forward direction of the vehicle as a Y-axis, and a direction pointing to the sky as a Z-axis, and after obtaining the world coordinate system, how to specifically obtain world coordinates of a plurality of position points on each lane line under the world coordinate system will be described in detail later.

S102, determining a current homography matrix of the image acquisition equipment based on the position information of each lane line in the road surface image, the world coordinates of the plurality of position points and the initial homography matrix of the image acquisition equipment.

For example, the position information of each lane line in the road surface image may be represented by the position of the pixel points of the lane line formed in the road surface image, for example, the pixel points of the lane line formed in the road surface image may be screened out by an image recognition technology, the pixel coordinates of the pixel points in the road surface image may be further obtained, then a linear equation of the lane line in the road surface image may be determined by the pixel coordinates corresponding to the pixel points, and the position information of the lane line in the road surface image may be represented by the linear equation.

For example, the initial homography matrix may be a homography matrix calibrated before the vehicle runs, where the homography matrix calibrated before the vehicle runs may be calibrated by a currently known calibration method, such as placing a reference object on a stationary road surface, determining the homography matrix of the image acquisition device on the stationary vehicle by the geographic position of the reference object in a world coordinate system and the pixel coordinates of the reference object in the image acquired by the image acquisition device.

The method comprises the following steps that the relative position relation between the image acquisition equipment and the vehicle is likely to change along with the running of the vehicle, correspondingly, the homography matrix obtained by calibrating the image acquisition equipment before the vehicle runs is no longer accurate, the image acquisition equipment on the vehicle can be calibrated by continuously acquiring a road surface image comprising a plurality of lane lines along with the running of the vehicle, and the process of calibrating the image acquisition equipment each time can be called a correction process of the homography matrix obtained by last calibration of the image acquisition equipment, so that the initial homography matrix can also be the homography matrix obtained after last correction.

For example, based on an initial homography matrix of the image capturing device, world coordinates of a plurality of position points may be converted to obtain corresponding positions of the plurality of position points in the road surface image, if the initial homography matrix is accurate, straight lines corresponding to the obtained corresponding positions of the plurality of position points in the road surface image should coincide with position information of the lane line in the road surface image, otherwise, the initial homography matrix is not accurate, and at this time, the initial homography matrix may be adjusted based on the corresponding positions of the plurality of position points in the road surface image and the position information of the lane line in the road surface image to obtain a current homography matrix of the image capturing device.

According to the calibration method provided by S101-S102, after the world coordinates which are acquired by the image acquisition equipment and comprise the road surface images of the multiple lane lines and the multiple position points on each lane line in the world coordinate system are acquired, the adjustment of the initial homography matrix can be completed through the position information of each lane line in the road surface images and the world coordinates of the multiple position points, and the current homography matrix of the image acquisition equipment is determined; for example, the image acquisition device may be disposed on a running vehicle, so that as the vehicle runs, the road surface image is always the road surface image acquired by the image acquisition device while maintaining the current relative position relationship with the vehicle, and when the road surface image includes a plurality of lane lines, the initial homography matrix of the image acquisition device may be continuously corrected, thereby alleviating a problem that the homography matrix is not accurate any more due to mechanical vibration or unevenness of the road surface on which the vehicle runs during the running of the vehicle.

The processes of S101 to S102 described above will be analyzed below with reference to specific examples.

For the above S101, as shown in fig. 2, the world coordinates of a plurality of position points on each lane line may be obtained in the following manner, specifically including S201 to S202:

s201, determining world coordinates of intersection points of target line segments and each lane line in the road surface image under a world coordinate system;

s202, acquiring the world coordinates of a plurality of position points on each lane line under the world coordinate system based on the world coordinates of each intersection point.

A large number of test results show that when a position point which is closer to an origin point in a world coordinate system is projected to a road surface image according to a homography matrix, along with the change of the homography matrix, when the position point is projected to the world coordinate system according to the changed homography matrix, the deviation in the world coordinate system is smaller, so that a target line segment can be generated based on the pixel coordinate, the distance between the end point of the target line segment and the origin point in the world coordinate system is smaller than a preset distance threshold value, the distance between an intersection point which is determined based on the target line segment and each lane line in the road surface image and the origin point in the world coordinate system is also smaller than the preset distance threshold value, and therefore, when the intersection point in the road surface image is converted to the world coordinate system based on the initial homography matrix, the deviation of the obtained intersection point in the world coordinate system is also small, and the intersection point can be considered to be positioned on the lane line in the world coordinate system, based on this, the world coordinates of a plurality of position points on each lane line under the world coordinate system can be acquired based on the world coordinates of each intersection point.

Specifically, with respect to the above S201, when determining the world coordinate of the intersection point of the target line segment in the road surface image and each lane line in the world coordinate system, the following S2011 to S2012 are included:

and S2011, selecting a plurality of initial position points in a world coordinate system, wherein the distance between each initial position point and the origin of the world coordinate system is smaller than a preset distance threshold.

As described above, when the pixel coordinates obtained by projecting the position point closer to the origin of the world coordinate system onto the road surface image according to the homography matrix are projected back to the world coordinate system according to the changed homography matrix as the change of the homography matrix, the deviation in the world coordinate system is smaller, and therefore, the position point where the distance from the origin of the world coordinate system is smaller than the preset distance threshold is selected as the initial position point.

For example, the explanation here may be that, as the pose of the image capture device changes, the deviation of the corresponding physical distance when the initial pixel coordinate of the initial position point in the road surface image is projected under the world coordinate system is small:

in the road surface image, with the change of the pose of the image acquisition equipment, the offset of the physical distance corresponding to the corresponding pixel point of the near position point (the position point with the distance from the origin point under the world coordinate system being less than the preset distance threshold) in the road surface image under the world coordinate system is smaller than the offset of the physical distance corresponding to the corresponding pixel point of the far position point (the position point with the distance from the origin point under the world coordinate system being greater than or equal to the preset distance threshold) in the road surface image under the world coordinate system, wherein the physical distance may specifically include the distance between the horizontal axis and the vertical axis under the world coordinate system, for example, the physical distance between the pixel point corresponding to the near position point under the world coordinate system and the vertical axis is 5m or less, and the physical distance between the pixel point corresponding to the far position point under the world coordinate system and the vertical axis is 20 m, and even larger, after the homography matrix is changed, according to a plurality of tests, the offset of the physical distance of the pixel point corresponding to the near position point in the world coordinate system is smaller than the physical offset of the pixel point corresponding to the far position point in the world coordinate system.

Taking a specific test result as an example, when the position of the image capturing device changes, the pixel coordinate of the position point M in the world coordinate system in the road surface image is (400, 500), and the pixel coordinate of the position point M in the world coordinate system in the road surface image may change to (400, 510), and at this time, if the pixel coordinate (400, 500) determined based on the image capturing device before the change of the position is still used as the pixel coordinate of the position point M, although the pixel coordinate (400, 510) of the position point M after the change of the homography matrix is 10 pixels different from the pixel coordinate (400, 500) of the position point M before the change of the homography matrix in the vertical coordinate direction, the corresponding physical distance is only 50cm, so that the pixel coordinate of the position point closer to the origin point in the road surface image in the world coordinate system, along with the change of the pose of the image acquisition equipment, the deviation of the corresponding physical distance in a world coordinate system is small.

S2012, based on the initial homography matrix, determining a target line segment formed by connecting the initial pixel coordinates corresponding to the plurality of initial position points in the road surface image, and world coordinates of the intersection point of the target line segment and each lane line in a world coordinate system.

For example, initial pixel coordinates corresponding to a plurality of initial position points in the road surface image may be determined based on the initial homography matrix, and further, a straight line fitting may be performed based on the plurality of initial pixel coordinates to obtain the target line segment, and an intersection point of the target line segment and each lane line in the road surface image may be detected by an image recognition technique, or may be determined by straight line equations corresponding to the target line segment and the lane lines, respectively.

For example, after obtaining the plurality of initial position points, initial pixel coordinates of the plurality of initial position points in the road surface image may be obtained according to the initial homography matrix.

Then, a target line segment is obtained based on the initial pixel coordinate connection, after the target line segment is obtained, intersection point pixel coordinates of intersection points of the target line segment and each lane line under an image coordinate system corresponding to the road image can be further determined, the intersection point pixel coordinates can be used for representing position information of the intersection points in the road image, and the selected initial position points are position points which are close to the origin of the world coordinate system under the world coordinate system, so that the obtained world coordinates of the intersection points in the world coordinate system can also be approximately used for representing intersection points of line segments formed by the initial position points on the road and the lane lines through the initial homography matrix, namely the intersection points of the target line segment and each lane line in the road image under the world coordinate system can be approximately considered to be located on the lane lines under the world coordinate system.

Specifically, for the above S2012, when determining, based on the initial homography matrix, a target line segment formed by connecting initial pixel coordinates corresponding to a plurality of initial position points in the road surface image and a world coordinate of a world coordinate system of an intersection of the target line segment and each lane line, the following S20121 to S20124 are included

S20121, based on the initial homography matrix and the initial position coordinates of each initial position point in the world coordinate system, determining the corresponding initial pixel coordinates of each initial position point in the road surface image.

Specifically, here, a world coordinate matrix formed by world coordinates of the plurality of initial position points in the world coordinate system may be used as a known quantity, an initial homography matrix of the image capturing device may be used as the known quantity, and then the known quantity is substituted into a conversion equation of the pixel coordinates and the world coordinates, so as to obtain initial pixel coordinates corresponding to the plurality of initial position points.

S20122, performing straight line fitting on the initial pixel coordinates corresponding to the multiple initial position points to obtain a target line segment.

Illustratively, the plurality of initial position points include two initial position points, and the target line segment is a line segment formed by connecting initial pixel coordinates corresponding to the two initial position points.

S20123, intersection pixel coordinates of the intersection point of the target line segment and each lane line in the road surface image are obtained.

For example, when determining the intersection pixel coordinate, the intersection pixel coordinate of the intersection in the image coordinate system corresponding to the road surface image may be obtained based on a linear equation of the target line segment in the road surface image and a linear equation of each lane line in the road surface image, or the road surface image may be directly input into a pre-established pixel coordinate recognition model to determine the intersection pixel coordinate corresponding to each intersection.

Here, the pixel coordinate determination model may perform image recognition based on the road surface image, determine the positions of the intersection points in the road surface image, and then determine the intersection point pixel coordinates of each intersection point in the image coordinate system based on the determined position of the intersection point in the road surface image.

S20124, determining world coordinates of the intersection points in a world coordinate system based on the intersection point pixel coordinates of the intersection points and the initial homography matrix.

Specifically, a pixel coordinate matrix may be configured as a known quantity according to intersection pixel coordinates of the plurality of intersections in the image coordinate system, an initial homography matrix of the image capturing device may be used as the known quantity, and then the known quantity is substituted into a conversion equation of the pixel coordinates and the world coordinates, so as to obtain the world coordinates of the plurality of intersections in the world coordinate system.

Further, with respect to the above S202, when obtaining world coordinates of a plurality of position points on each lane line in the world coordinate system based on the world coordinates of each intersection point, the method may include:

and determining the world coordinates of a plurality of position points which are positioned on the same lane line with each intersection point under the world coordinate system based on the world coordinates of each intersection point and a preset position interval.

The world coordinates of the intersection points are located on the straight line equations corresponding to the lane lines in the world coordinate system, and the straight line equations corresponding to the lane lines in the world coordinate system can be determined, for example, a plurality of lane lines are parallel lane lines, when the vehicle travels between the parallel lane lines, the straight line equations corresponding to the lane lines in the world coordinate system can be straight lines parallel to the vehicle traveling direction, if the vehicle traveling direction is the X-axis direction of the world coordinate system, the straight line equations corresponding to the lane lines in the world coordinate system can be determined according to the world coordinates of the intersection points in the world coordinate system, for example, the world coordinate system (a, b) of the intersection point corresponding to a certain lane line L, and the straight line equations corresponding to the lane line L in the world coordinate system can be denoted as y-b, and further, the predetermined position interval, for example, the interval is 5m, that is, the world coordinates of a plurality of position points located on the same lane line with each intersection point can be obtained in the same manner based on the world coordinates of the position points located on the same lane line with the intersection point and located 5 meters, 10 meters, 20 meters, 25 meters away from the world coordinates of the intersection point.

After obtaining the world coordinate system of the plurality of location points on each lane line, the current homography matrix of the image capturing device may be further determined, and specifically, for the above S102, when determining the current homography matrix of the image capturing device based on the location information of each lane line in the road surface image, the world coordinates of the plurality of location points, and the initial homography matrix of the image capturing device, as shown in fig. 3, the following S301 to S303 may be included:

s301, based on the world coordinates of the position points and the initial homography matrix, determining initial pixel coordinates of the position points in the road surface image.

For example, the initial pixel coordinates of the multiple position points on each lane line in the image coordinate system where the road surface image is located may be determined according to the world coordinates of the multiple position points and the initial homography matrix, and specifically, the world coordinate matrix formed by the world coordinates of the multiple position points in the world coordinate system may be used as a known quantity, the initial homography matrix of the image capturing device may be used as a known quantity, and then the known quantity is substituted into a conversion equation of the pixel coordinates and the world coordinates, so as to obtain the initial pixel coordinates of the multiple position points in the image coordinate system where the road surface image is located.

S302, based on the position information of each lane line in the road surface image, correcting the initial pixel coordinates corresponding to the multiple position points on the lane line to obtain the corrected pixel coordinates of the multiple position points on the lane line in the road surface image.

For example, if the initial homography matrix is no longer accurate, the determined initial pixel coordinates of the plurality of position points may deviate from the position information of the lane line in the road surface image, and based on this idea, the calibration method provided by the embodiment of the disclosure may correct the initial homography matrix by using the deviation.

For example, the position information of each lane line in the road surface image may be represented by a straight line equation of the lane line in the image coordinate system corresponding to the road surface image.

For example, after determining that the initial pixel coordinates of the plurality of position points on each lane line deviate from the lane line, the initial pixel coordinates of the plurality of position points on the lane line may be corrected to obtain corrected pixel coordinates of the plurality of position points on the lane line in the road surface image.

By correcting the initial pixel coordinates, the corrected pixel coordinates corresponding to the multiple position points on each lane line obtained here can be made to be consistent with the position relationship of the lane line in the road surface image and the position relationship of the world coordinates of the multiple position points on the lane line and the world coordinates of the lane line in the world coordinate system, so that the current homography matrix of the image acquisition device can be further determined according to the corrected pixel coordinates of the multiple position points on the lane line and the world coordinates of the multiple position points.

And S303, obtaining the current homography matrix of the image acquisition equipment based on the corrected pixel coordinates and the world coordinates of the plurality of position points on each lane line.

Specifically, a pixel coordinate matrix may be formed based on the corrected pixel coordinates of the plurality of position points on each lane line, a world coordinate matrix may be formed based on the world coordinates of the plurality of position points on each lane line in the world coordinate system, and then the current homography matrix of the image capturing apparatus may be determined by substituting the pixel coordinate matrix and the world coordinate matrix as known quantities and the current homography matrix of the image capturing apparatus as unknown quantities into a conversion equation of the pixel coordinates and the world coordinates, which will be described in detail later.

Specifically, in S302, when the initial pixel coordinates corresponding to the plurality of position points on each lane line are corrected based on the position information of each lane line in the road surface image to obtain the corrected pixel coordinates of the plurality of position points on the lane line in the road surface image, the method may include the following steps S3021 to S3022:

and S3021, determining a fitted straight line of the initial pixel coordinates and a deviation angle between the fitted straight line of the initial pixel coordinates and the fitted straight line of the lane line in the road surface image based on the position information of each lane line in the road surface image and the determined initial pixel coordinates corresponding to the position points on the lane line.

Here, as the vehicle travels, when the relative position between the image capturing device and the vehicle changes, the initial homography matrix of the image capturing device may not be accurate any more, and the initial pixel coordinates of the plurality of position points on the lane line determined based on the initial homography matrix may not be accurate any more, so that the fitting straight lines of the initial pixel coordinates may not coincide with the corresponding fitting straight lines of the lane line in the road surface image, and thus, there is a deviation angle between the fitting straight lines corresponding to the plurality of initial pixel coordinates and the corresponding fitting straight lines of the lane line in the road surface image.

Specifically, the fitted straight line corresponding to the lane line in the road surface image is a perceived lane line in the road surface image, that is, the lane line detection is performed on the road surface image to obtain a plurality of pixel points constituting the lane line, and then the straight line fitting is performed based on pixel coordinates corresponding to the pixel points to obtain the fitted straight line corresponding to the lane line in the road surface image.

Specifically, when determining a fitted straight line of a plurality of initial pixel coordinates based on the position information of each lane line in the road surface image and the determined initial pixel coordinates corresponding to a plurality of position points on the lane line, and a deviation angle between the fitted straight line corresponding to the lane line in the road surface image, the following S30211 to S30213 may be included:

s30211, generating a first lane line equation under an image coordinate system corresponding to each lane line in the road surface image based on the position information of each lane line in the road surface image;

s30212, performing straight line fitting on the initial pixel coordinates corresponding to the plurality of position points on each lane line to generate a second lane line equation corresponding to each lane line;

s30213, based on the first lane line equation and the second lane line equation corresponding to each lane line, determining a deviation angle of a straight line represented by the second lane line equation relative to a straight line represented by the first lane line equation.

When determining the first lane line equation in the image coordinate system corresponding to each lane line, a plurality of pixel points of the lane line in the image coordinate system may be identified first, and then the first lane line equation in the image coordinate system corresponding to the lane line may be determined by a least square method based on the plurality of pixel points.

Similarly, when performing straight line fitting on the initial pixel coordinates of the plurality of position points on each lane line in the image coordinate system, the straight line fitting may also be performed according to a least square method, and specifically, the second lane line equation corresponding to the lane line may be obtained according to the following formulas (1), (2), and (3):

wherein (x)_i,y_i) Representing the initial pixel coordinates of the ith position point belonging to the same lane line; n represents the number of a plurality of position points belonging to the same lane line;

an average value of abscissa values in the initial pixel coordinates of a plurality of position points on the lane line;

an average value of vertical coordinate values in initial pixel coordinates of a plurality of position points on the lane line is represented; b. b₀And b₁Representing the unknown parameters in the second linear equation.

Substituting the initial pixel coordinates of the plurality of position points on the lane line into the above formulas (1) to (3) to obtain the unknown parameter b in the second linear equation corresponding to the lane line₀And b₁Then, a second linear equation corresponding to the lane line can be obtained: y is b₁x-b₀。

Similarly, in the same manner, a second linear equation corresponding to each lane line can be obtained.

Further, after the first linear equation and the second linear equation corresponding to each lane line are obtained, the deviation angle of the straight line represented by the second lane line equation relative to the straight line represented by the first lane line equation can be determined based on the slope of the linear equation.

And S3022, based on the determined deviation angle, correcting the initial pixel coordinates corresponding to the plurality of position points on the lane line to obtain corrected pixel coordinates of the plurality of position points on the lane line in the road surface image.

After the deviation angle between the straight line where the initial pixel coordinates of the plurality of position points on each lane line are located and the straight line where the lane line is located is obtained, the initial pixel coordinates of the plurality of position points may be further corrected according to the deviation angle, and specifically, the initial pixel coordinates of the plurality of position points may be corrected by the following formulas (4) and (5):

wherein (x)_i,y_i) Initial pixel coordinates (x) of the ith position point among the plurality of position points on any one lane line_i',y_i') represents the corrected pixel coordinates of the ith position point; n represents the number of a plurality of position points on the lane line; theta represents the deviation angle of a straight line represented by the second lane line equation corresponding to any lane line relative to a straight line represented by the first lane line equation; and M (theta) represents a corresponding rotation matrix when the straight line represented by the first lane line equation corresponding to any lane line rotates to the straight line represented by the second lane line equation.

In the same manner, the corrected pixel coordinates of the plurality of position points on each lane line in the road surface image can be obtained.

In the embodiment of the present disclosure, the initial pixel coordinates of the plurality of position points on each lane line in the image coordinate system are determined according to the initial homography matrix, when the initial homography matrix is no longer accurate, the initial pixel coordinates of the plurality of position points determined based on the initial homography matrix in the image coordinate system may deviate from the corresponding lane line, and by determining the deviation angle between the initial pixel coordinates of the plurality of position points on each lane line in the image coordinate system and the lane line, the initial pixel coordinates of the plurality of position points on the lane line may be corrected based on the deviation angle, so that the corrected pixel coordinates of the plurality of position points on the lane line in the image coordinate system may be determined more accurately.

For the above step S303, after obtaining the corrected pixel coordinates of the plurality of position points on each lane line in the road surface image, the current homography matrix of the image capturing device may be determined based on the corrected pixel coordinates and the world coordinates of the plurality of position points on each lane line, and specifically, the current homography matrix of the image capturing device may be determined according to the following processes:

recording world coordinates of a plurality of position points on each lane line in a world coordinate system as follows: (X)₁,Y₁)～(X_N,Y_N) Where N denotes a plurality of position points on the lane line in the road surface imageAnd recording a world coordinate matrix A, a correction pixel coordinate matrix C and a current homography matrix of the image acquisition equipment B, wherein the number is specifically expressed as follows:

then, substituting the world coordinate matrix A, the corrected pixel coordinate matrix C and the current homography matrix B into a conversion equation of the pixel coordinate and the world coordinate, wherein the conversion equation is expressed by the following formula (6):

A＝B×C (6)；

solving the conversion equation to obtain the current homography matrix B ═ AA (AA) of the image acquisition equipment^T)*(CA^T)^-1。

Taking the example of two lane lines included in the road surface image, a specific embodiment of determining the current homography matrix of the image capturing device is given below with reference to fig. 4a to 4 c:

as shown in fig. 4a, for a road surface image obtained by an image acquisition device arranged on a running vehicle, identifying a lane line in the road surface image to obtain two lane lines, namely a lane line L1 and a lane line L2, then selecting two position points close to the origin of a world coordinate system, obtaining initial pixel coordinates of the two position points in an image coordinate system where the road surface image is located according to an initial homography matrix, and obtaining initial pixel coordinates of a1 and a2 in the image coordinate system as shown in fig. 4 a; then, an intersection B1 of the target line segment made up of a1 and a2 with the lane line L1, and an intersection C1 with the lane line L2, respectively, are determined.

Converting the initial pixel coordinates of the intersection points B1 and C1 according to the initial homography matrix to obtain world coordinates corresponding to the intersection points B1 and C1 in a world coordinate system, considering that A1 and A2 are obtained by selecting position points close to the origin of the world coordinate system and the initial homography matrix, the world coordinates corresponding to the intersection points B1 and C1 in the world coordinate system can still be considered to be located on a straight line equation corresponding to lane lines L1 and L2 of the road surface, and then according to the straight line equation of the lane line L1 and the lane line L2 in the world coordinate system, the world coordinates of the intersection points B1 and C1 in the world coordinate system, the other position points located on the lane line L1 on the road surface and the other position points located on the lane line L2 on the road surface can be obtained continuously.

Then, based on the initial homography matrix, initial pixel coordinates of a plurality of position points on the lane line L1 in the image coordinate system of the road surface image are obtained, that is, initial pixel coordinates of B2, B3, B4, and B5 in the image coordinate system of fig. 4B are obtained, and correspondingly, initial pixel coordinates of a plurality of position points on the lane line L2 in the image coordinate system of the road surface image are obtained, that is, initial pixel coordinates of C2, C3, C4, and C5 in the image coordinate system of fig. 4B are obtained.

Then, a straight line fitting is performed based on B1, B2, B3, B4 and B5 to obtain a straight line L1 ', and a straight line fitting is performed based on C1, C2, C3, C4 and C5 to obtain a straight line L2', and then, a deviation angle θ 1 of the straight line L1 'with respect to the straight line L1 and a deviation angle θ 2 of the straight line L2' with respect to the straight line L2 in the image coordinate system are further determined, and then, the initial pixel coordinates of B1, B2, B3, B4 and B5 are corrected according to the above-mentioned formula (4) and formula (5) to obtain corrected pixel coordinates of B1 ', B2', B3 ', B4' and B5 'as shown in fig. 4C, and the initial pixel coordinates of C1', C2 'and C2' as shown in fig. 4C 2.

Then, the world coordinates of the plurality of position points on the lane line L1 and the lane line L2 in the world coordinate system and the corrected pixel coordinates in the image coordinate system are substituted into the conversion equation of the pixel coordinates and the world coordinates, so that the current homography matrix of the image acquisition device can be obtained.

Further, after obtaining the current homography matrix of the image acquisition device, the distance measurement may be performed on the target object based on the current homography matrix, as shown in fig. 5, which is a flowchart of the distance measurement method provided in the embodiment of the present disclosure, specifically including the following steps S501 to S504:

s501, acquiring a target image obtained after the image acquisition equipment shoots a target object;

s502, determining the pixel coordinates of the target object in the target image based on the target image;

s503, determining world coordinates of the target object in a world coordinate system based on the pixel coordinates and the current homography matrix;

s504, determining the distance between the target object and the preset position point based on the world coordinate of the target object in the world coordinate system and the coordinate of the preset position point in the world coordinate system.

Taking a vehicle as an example, the preset position point may be a projection of a center point of a front axle of the vehicle on the ground, or a projection of a center point of a vehicle body on the ground, and when the preset position point is used as an origin of a world coordinate system, coordinates of the origin in the world coordinate system are known, and the preset position point may be used as a vehicle distance measuring point corresponding to a distance between a target object and the vehicle when the distance is measured.

The whole process from S501 to S504 is a process of performing distance measurement through the current homography matrix after obtaining the current homography matrix of the image acquisition device, because the target object in the target object image has an area, after obtaining the target object image, determining a distance measurement point of the target object according to the target object image, and then determining the distance between the target object and the vehicle based on the world coordinates of the distance measurement point and the preset position point in the world coordinate system.

Specifically, after the target image of the target object is obtained, based on an image recognition technology, the labeling frame of the target object is obtained, for example, a central position point of a tangent line between the labeling frame and the ground may be used as a distance measuring point, and then a pixel coordinate of the distance measuring point is used as a pixel coordinate of the target object in the image coordinate system.

After the pixel coordinates of the target object in the image coordinate system are obtained, the pixel coordinates of the target object in the image coordinate system and the current homography matrix are input into a conversion equation of the pixel coordinates and the world coordinates, so that the world coordinates of the target object in the world coordinate system can be obtained, and further the Euclidean distance between the world coordinates and the world coordinates of the preset position point is calculated according to the world coordinates of the target object in the world coordinate system and the world coordinates of the preset position point, so that the distance between the target object and the vehicle can be determined.

It will be understood by those skilled in the art that in the method of the present invention, the order of writing the steps does not imply a strict order of execution and any limitations on the implementation, and the specific order of execution of the steps should be determined by their function and possible inherent logic.

Based on the same technical concept, a calibration device corresponding to the calibration method is also provided in the embodiments of the present disclosure, and because the principle of solving the problem of the device in the embodiments of the present disclosure is similar to that of the calibration method in the embodiments of the present disclosure, the implementation of the device can refer to the implementation of the method, and repeated details are not repeated.

Referring to fig. 6, a schematic diagram of a calibration apparatus 600 provided in an embodiment of the present disclosure is shown, where the calibration apparatus 600 includes:

the acquisition module 601 is configured to acquire a road surface image including a plurality of lane lines acquired by an image acquisition device, and world coordinates of a plurality of position points on each lane line in a world coordinate system;

the determining module 602 is configured to determine a current homography matrix of the image capturing device based on the position information of each lane line in the road surface image, the world coordinates of the plurality of position points, and the initial homography matrix of the image capturing device.

In one possible implementation, the obtaining module 601 is configured to obtain the world coordinates of a plurality of location points on each lane line according to the following manner:

determining world coordinates of intersection points of the target line segments and each lane line in the road surface image under a world coordinate system;

In one possible implementation, the obtaining module 601, when configured to determine a world coordinate of an intersection of each lane line and a target line segment in the road surface image under a world coordinate system, includes:

selecting a plurality of initial position points under a world coordinate system, wherein the distance between each initial position point and the origin of the world coordinate system is less than a preset distance threshold;

and determining a target line segment formed by connecting the initial pixel coordinates corresponding to the plurality of initial position points in the road surface image and world coordinates of the intersection point of the target line segment and each lane line in a world coordinate system based on the initial homography matrix.

In one possible implementation, the obtaining module 601, when configured to obtain the world coordinates of a plurality of location points on each lane line in the world coordinate system based on the world coordinates of each intersection point, includes:

and acquiring the world coordinates of a plurality of position points which are positioned on the same lane line with each intersection point under the world coordinate system based on the world coordinates of each intersection point and a preset position interval.

In one possible implementation, the obtaining module 601, when configured to determine, based on the initial homography matrix, a target line segment formed by connecting initial pixel coordinates corresponding to a plurality of initial position points in the road surface image, and a world coordinate of an intersection of the target line segment and each lane line in a world coordinate system, includes:

determining initial pixel coordinates corresponding to each initial position point in the road surface image based on the initial homography matrix and the initial position coordinates of each initial position point in a world coordinate system;

performing linear fitting on the initial pixel coordinates corresponding to the plurality of initial position points to obtain a target line segment;

acquiring intersection pixel coordinates of intersections of the target line segment and each lane line in the road surface image;

and determining the world coordinates of the intersection points under the world coordinate system based on the intersection point pixel coordinates of the intersection points and the initial homography matrix.

In one possible implementation, the determining module 602, when configured to determine the current homography matrix of the image capturing device based on the position information of each lane line in the road surface image, the world coordinates of the plurality of position points, and the initial homography matrix of the image capturing device, includes:

based on the position information of each lane line in the road surface image, correcting the initial pixel coordinates corresponding to the multiple position points on the lane line to obtain the corrected pixel coordinates of the multiple position points on the lane line in the road surface image;

and obtaining the current homography matrix of the image acquisition equipment based on the corrected pixel coordinates and the world coordinates of the plurality of position points on each lane line.

In one possible implementation, the determining module 602, when configured to correct the initial pixel coordinates corresponding to the multiple position points on each lane line based on the position information of each lane line in the road surface image, to obtain corrected pixel coordinates of the multiple position points on the lane line in the road surface image, includes:

determining a fitting straight line of a plurality of initial pixel coordinates and a deviation angle between the fitting straight line and the corresponding fitting straight line of each lane line in the road surface image based on the position information of each lane line in the road surface image and the determined initial pixel coordinates corresponding to the plurality of position points on the lane line;

In one possible implementation, the determining module 602, when configured to determine a fitted straight line of the plurality of initial pixel coordinates based on the position information of each lane line in the road surface image and the determined initial pixel coordinates corresponding to the plurality of position points on the lane line, and a deviation angle between the fitted straight line of the plurality of initial pixel coordinates and the corresponding fitted straight line of the lane line in the road surface image, includes:

In a possible implementation, the determining module 602, after determining the current homography matrix of the image capturing device, is further configured to:

and determining world coordinates of the target object in a world coordinate system based on the pixel coordinates and the current homography matrix.

In one possible implementation, the determining module 602, after determining the world coordinates of the target object in the world coordinate system, is further configured to:

The description of the processing flow of each module in the device and the interaction flow between the modules may refer to the related description in the above method embodiments, and will not be described in detail here.

Corresponding to the calibration method in fig. 1, an embodiment of the present disclosure further provides an electronic device 700, as shown in fig. 7, which is a schematic structural diagram of the electronic device 700 provided in the embodiment of the present disclosure, and includes:

a processor 701, a memory 702, and a bus 703; the memory 702 is used for storing execution instructions and includes a memory 7021 and an external memory 7022; the memory 7021 is also referred to as an internal memory, and is used to temporarily store operation data in the processor 701 and data exchanged with an external memory 7022 such as a hard disk, the processor 701 exchanges data with the external memory 7022 through the memory 7021, and when the electronic device 700 is operated, the processor 701 and the memory 702 communicate with each other through the bus 703, so that the processor 701 executes the following instructions: acquiring a road surface image which is acquired by image acquisition equipment and comprises a plurality of lane lines and world coordinates of a plurality of position points on each lane line under a world coordinate system; and determining the current homography matrix of the image acquisition equipment based on the position information of each lane line in the road surface image, the world coordinates of the plurality of position points and the initial homography matrix of the image acquisition equipment.

The embodiment of the present disclosure further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the calibration method in the above method embodiment are executed. The storage medium may be a volatile or non-volatile computer-readable storage medium.

The embodiments of the present disclosure also provide a computer program product, where the computer program product carries a program code, and instructions included in the program code may be used to execute the steps of the calibration method described in the foregoing method embodiments, which may be referred to specifically for the foregoing method embodiments, and are not described herein again.

The computer program product may be implemented by hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied in a computer storage medium, and in another alternative embodiment, the computer program product is embodied in a Software product, such as a Software Development Kit (SDK), or the like.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the apparatus described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, apparatus, and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present disclosure may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present disclosure. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

Finally, it should be noted that: the above-mentioned embodiments are merely specific embodiments of the present disclosure, which are used for illustrating the technical solutions of the present disclosure and not for limiting the same, and the scope of the present disclosure is not limited thereto, and although the present disclosure is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive of the technical solutions described in the foregoing embodiments or equivalent technical features thereof within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present disclosure, and should be construed as being included therein. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.

Claims

1. A calibration method, comprising:

2. The calibration method according to claim 1, wherein the world coordinates of the plurality of location points on each lane line are acquired as follows:

3. The calibration method according to claim 2, wherein the determining the world coordinates of the intersection point of the target line segment in the road surface image and each lane line in the world coordinate system comprises:

4. The calibration method according to claim 3, wherein the determining, based on the initial homography matrix, the target line segment formed by connecting the initial pixel coordinates corresponding to the plurality of initial position points in the road surface image, and the world coordinates of the intersection point of the target line segment and each lane line in the world coordinate system comprises:

5. The calibration method according to any one of claims 2 to 4, wherein the obtaining the world coordinates of the plurality of position points on each lane line in the world coordinate system based on the world coordinates of each intersection point comprises:

6. The calibration method according to any one of claims 1 to 5, wherein the determining a current homography matrix of the image acquisition device based on the position information of each lane line in the road surface image, the world coordinates of the plurality of position points, and the initial homography matrix of the image acquisition device comprises:

7. The calibration method according to claim 6, wherein the correcting initial pixel coordinates corresponding to a plurality of position points on each lane line based on the position information of each lane line in the road surface image to obtain corrected pixel coordinates of the plurality of position points on the lane line in the road surface image comprises:

8. The calibration method according to claim 7, wherein determining a fitted straight line of a plurality of the initial pixel coordinates based on the position information of each lane line in the road surface image and the determined initial pixel coordinates corresponding to a plurality of position points on the lane line, and a deviation angle between the fitted straight line of the lane line in the road surface image comprises:

9. The calibration method according to any one of claims 1 to 8, wherein after determining the current homography matrix of the image acquisition device, further comprising:

10. The calibration method according to claim 9, further comprising, after determining the world coordinates of the target object in the world coordinate system:

11. A calibration device, comprising:

12. An electronic device, comprising: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating via the bus when the electronic device is running, the machine-readable instructions when executed by the processor performing the steps of the calibration method as claimed in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a computer program which, when being executed by a processor, carries out the steps of the calibration method according to any one of claims 1 to 10.