CN115439550A

CN115439550A - Camera calibration method, distance measurement method, equipment and storage medium

Info

Publication number: CN115439550A
Application number: CN202111283270.4A
Authority: CN
Inventors: 侯玉璐; 史利民; 张磊; 宋玉强; 刘虹昕
Original assignee: Tianjin Wuba Driving Test Information Technology Co ltd
Current assignee: Tianjin Wuba Driving Test Information Technology Co ltd
Priority date: 2021-11-01
Filing date: 2021-11-01
Publication date: 2022-12-06

Abstract

The embodiment of the application provides a camera calibration method, a distance measurement method, equipment and a storage medium. And calculating an affine matrix corresponding to the grids under each grid contained in the checkerboard according to the coordinate change state of the grids between the world coordinate system and the pixel coordinate system corresponding to the calibration image, wherein the affine matrix is used for calibrating the monocular camera. Therefore, different imaging areas of the monocular camera can be finely calibrated, and the calibration precision is effectively improved. On the basis, a calibrated monocular camera can be used for distance measurement, the world coordinates of the objects are calculated by using the affine matrix of the corresponding grids by determining which grid of the checkerboard the objects to be measured fall into respectively, and then the distance between the objects is calculated. Because parameters such as the relative position of the grids, the world coordinates and the like are not influenced by image distortion, errors caused by image distortion and the like can be effectively reduced in the distance measurement process, and the distance measurement precision is improved.

Description

Camera calibration method, distance measurement method, equipment and storage medium

Technical Field

The present disclosure relates to the field of camera calibration technologies, and in particular, to a camera calibration method, a distance measurement method, a device, and a storage medium.

Background

In a driving school, a student needs to perform GPS positioning on a vehicle in a process of examination or driving practice so as to prompt whether the vehicle correctly arrives at a specified position.

However, the vehicle often has the problem of losing the lock in the GPS positioning due to the factors such as the ceiling or the tree shielding, which causes the vehicle positioning to be wrong or even impossible.

Disclosure of Invention

Aspects of the present disclosure provide a camera calibration method, a distance measurement method, a device, and a storage medium, which are used to improve camera calibration accuracy and/or distance measurement accuracy.

The embodiment of the application provides a distance measuring method, which comprises the following steps:

shooting a live-action image containing a first object and a second object by using a monocular camera, wherein the monocular camera is fixedly assembled on the first object, and calibration parameters of the monocular camera comprise an affine matrix under each grid in a checkerboard used in a calibration process;

respectively calculating pixel coordinates of key points on the first object and the second object in a pixel coordinate system corresponding to the live-action image;

determining a first grid into which a key point on the first object falls and a second grid into which a key point on the second object falls according to a pixel coordinate range corresponding to each grid in the checkerboard in the pixel coordinate system and pixel coordinates corresponding to the key points on the first object and the second object respectively;

calculating world coordinates of key points on the first object in a world coordinate system according to the affine matrix under the first grid and pixel coordinates corresponding to the key points on the first object, wherein the world coordinate system is established according to the size of the checkerboard;

calculating world coordinates of key points on the second object in a world coordinate system according to the affine matrix in the second grid and pixel coordinates corresponding to the key points on the second object;

and calculating the distance between the first object and the second object based on the world coordinates corresponding to the key points on the first object and the second object respectively.

The embodiment of the present application further provides a camera calibration method, including:

after the monocular camera is fixed, photographing a checkerboard placed in the visual field of the camera by using the monocular camera to obtain a calibration image;

establishing a world coordinate system according to the size of the checkerboard;

under each grid contained in the checkerboard, calculating an affine matrix corresponding to the grid according to the coordinate change state of the grid between the world coordinate system and the pixel coordinate system corresponding to the calibration image;

and taking the affine matrix corresponding to each grid contained in the checkerboard as the calibration parameters of the monocular camera.

The embodiment of the application also provides a computing device, which comprises a memory and a processor;

the memory is to store one or more computer instructions;

the processor is coupled with the memory for executing the one or more computer instructions for:

respectively calculating the pixel coordinates of key points on the first object and the second object in a pixel coordinate system corresponding to the live-action image;

calculating the distance between the first object and the second object based on the world coordinates corresponding to the key points on the first object and the second object respectively

the memory is to store one or more computer instructions;

and taking the affine matrix corresponding to each grid contained in the checkerboard as the calibration of the monocular camera.

Embodiments of the present application also provide a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, cause the one or more processors to perform the foregoing ranging method or camera calibration method.

In the embodiment of the application, a brand-new monocular camera calibration scheme is provided, a world coordinate system can be established according to the size of a checkerboard for calibration, and an affine matrix corresponding to a grid is calculated according to the coordinate change state of the grid between the world coordinate system and a pixel coordinate system corresponding to a calibration image under each grid contained in the checkerboard; and taking the affine matrix corresponding to each grid contained in the checkerboard as the calibration of the monocular camera. Therefore, different imaging areas of the monocular camera can be finely calibrated, and the calibration precision is effectively improved. On the basis, a calibrated monocular camera can be used for distance measurement, and in the distance measurement process, the distance measurement method can determine into which grid of the checkerboard two objects to be measured respectively fall, so that the world coordinates of the objects are calculated by using the affine matrix of the corresponding grids, and further the distance between the objects is calculated. Because parameters such as the relative position of the grid and the world coordinate are not affected by image distortion, errors caused by image distortion and the like can be effectively reduced and the distance measurement precision is improved in the distance measurement process based on the camera calibration scheme provided by the embodiment. In addition, based on the ranging function, the specified object can be accurately positioned under the condition of not depending on GPS positioning equipment.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:

fig. 1 is a schematic flowchart of a ranging method according to an exemplary embodiment of the present disclosure;

FIG. 2 is a schematic diagram of a live-action image according to an exemplary embodiment of the present application;

FIG. 3 is a schematic diagram of a construction scheme of a world coordinate system according to an exemplary embodiment of the present application;

fig. 4 is a schematic flowchart of a camera calibration method according to another exemplary embodiment of the present application;

fig. 5 is a schematic structural diagram of a computing device according to another exemplary embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail and completely with reference to the following specific embodiments of the present application and the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

At present, the problem of GPS positioning unlocking of a vehicle often occurs due to the fact that the vehicle is shielded by a ceiling or trees and the like, so that the vehicle positioning is wrong or even cannot be positioned. To this end, in some embodiments of the present application: a brand-new monocular camera calibration scheme is provided, a world coordinate system can be established according to the size of a checkerboard for calibration, and an affine matrix corresponding to a grid is calculated under each grid contained in the checkerboard according to the coordinate change state of the grid between the world coordinate system and a pixel coordinate system corresponding to a calibration image; and taking the affine matrix corresponding to each grid contained in the checkerboard as the calibration of the monocular camera. Therefore, different imaging areas of the monocular camera can be finely calibrated, and the calibration precision is effectively improved. On the basis, a calibrated monocular camera can be used for ranging, and in the ranging process, the distance between two objects to be measured can be determined to fall into one of the grids, so that the world coordinate is calculated by using the affine matrix of the corresponding grid, and the distance between the objects is further calculated. Because parameters such as the relative position of the grid and the world coordinate are not affected by image distortion, errors caused by image distortion and the like can be effectively reduced and the distance measurement precision is improved in the distance measurement process based on the camera calibration scheme provided by the embodiment. In addition, based on the distance measuring function, the specified object can be accurately positioned under the condition of not depending on GPS positioning equipment.

The technical solutions provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings.

Fig. 1 is a flowchart illustrating a ranging method according to an exemplary embodiment of the present application, where the ranging method may be executed by a data processing device, the data processing device may be implemented as a combination of software and/or hardware, and the data processing device may be integrated in a computing device. Referring to fig. 1, the method includes:

step 100, shooting a live-action image containing a first object and a second object by using a monocular camera, wherein the monocular camera is fixedly assembled on the first object, and calibration parameters of the monocular camera comprise an affine matrix under each grid in a checkerboard used in a calibration process;

step 101, respectively calculating pixel coordinates of key points on a first object and a second object in a pixel coordinate system corresponding to a live-action image;

step 102, determining a second grid into which key points on the first grid and the second grid fall, wherein the key points on the first object fall, according to a pixel coordinate range corresponding to each grid in a pixel coordinate system in the checkerboard grids and pixel coordinates corresponding to the key points on the first object and the second object respectively;

103, calculating world coordinates of key points on the first object in a world coordinate system according to the affine matrix in the first grid and pixel coordinates corresponding to the key points on the first object, wherein the world coordinate system is established according to the size of the checkerboard;

104, calculating world coordinates of key points on the second object in a world coordinate system according to the affine matrix in the second grid and pixel coordinates corresponding to the key points on the second object;

and 105, calculating the distance between the first object and the second object based on the world coordinates corresponding to the key points on the first object and the second object respectively.

The ranging method provided by the embodiment can be applied to various scenes needing ranging. For example, automatic driving, driving school test, etc., the application scenario is not specifically limited in this embodiment. In different application scenarios, the objects to be measured may be diverse, for example, in an autonomous driving scenario, the objects to be measured may be a host vehicle and an adjacent vehicle; for another example, in a driving school test scenario, the objects to be measured may be the host vehicle and white lines on the ground. Of course, this is merely exemplary, and the present embodiment does not limit the attribute of the object to be measured.

In this embodiment, the monocular camera may be fixedly mounted on the first object, and the camera calibration may be performed, and after the camera calibration is completed, the monocular camera may not move any more, so as to ensure validity of the calibration parameters of the monocular camera. The embodiment can adopt a checkerboard for camera calibration. Alternatively, after the monocular camera is fixed, the field of view of the monocular camera may be measured, and a checkerboard which can just cover the field of view of the camera is selected, and the size and the number of the checkerboard may be set as required, which is merely exemplary, and the present embodiment is not limited thereto.

On the basis, the checkerboard can be photographed by using the monocular camera to obtain a calibration image, and then the monocular camera can be calibrated according to the calibration object and the checkerboard, and the specific calibration process will be detailed later. In this embodiment, in the camera calibration process, an affine matrix may be calculated for each grid in the checkerboard, and the affine matrix under each grid in the checkerboard may be used as a calibration parameter of the monocular camera. The affine matrix is used for representing an affine relationship between a pixel coordinate system and a world coordinate system, wherein the pixel coordinate system can be established according to the size of an image shot by the monocular camera, and the world coordinate system can be established according to the size of a checkerboard.

After the camera calibration is completed, the monocular camera may move synchronously with the first object, and the monocular camera may capture a live-action image including the first object and the second object. The second object may be any object in the real scene that needs to be measured from the first object, for example, in an automatic driving scene, the first object may be a host vehicle, and the second object may be a leading vehicle; for another example, in a driving school test scenario, the first object may be the host vehicle and the second object may be a white line on the ground. Fig. 2 is a schematic diagram of a live-action image according to an exemplary embodiment of the present application. Referring to fig. 2, in the exemplary live-action image, the first object is a vehicle and the second object is a white line on the ground.

In this embodiment, key points may be selected on the first object and the second object, and the key points may be used to represent the objects where the key points are located, for example, the locations of the key points may be used to represent the locations of the objects where the key points are located, so that the distance between the key points on the first object and the second object may be calculated to perform the distance measurement on the first object and the second object. In this embodiment, the selection position of the key point may be customized as needed, for example, a central point on the object may be selected as the key point, or a designated point or a random point on the object may also be selected as the key point, which is not limited in this embodiment.

Referring to fig. 1, in step 101, pixel coordinates of key points of a first object and a second object in a pixel coordinate system corresponding to a live view image may be calculated, respectively. In this embodiment, pixel coordinates may be obtained in the live-action image by using opencv and the like, which is only exemplary, and in this embodiment, pixel coordinates of the keypoints of the first object and the second object in the pixel coordinate system corresponding to the live-action image may be calculated by using any technical scheme that is proposed now or in the future and that obtains pixel coordinates in the image, and detailed description is not further provided herein.

For the key points on the first object, it is preferable that the pixel coordinates corresponding to the key points on the first object are calculated and stored after the monocular camera is fixedly assembled. The pre-calculation process may include: shooting key points on a first object by using a monocular camera to obtain a test image; in the test image, pixel coordinates of the keypoints on the first object in a pixel coordinate system are calculated and stored. In the distance measuring process, the relative position of the monocular camera and the first object is kept unchanged, so that the pixel coordinates corresponding to the key points on the first object in any real-scene image shot by the monocular camera are kept unchanged, and the prestored pixel coordinates of the key points on the first object in the pixel coordinate system can be directly obtained and used in the distance measuring process.

For the key points on the second object, the pixel coordinates of the key points on the second object in the pixel coordinate system are obtained in a real-time calculation mode because the second object is flexible and variable. In order to further improve the accuracy of the pixel coordinates of the key points on the second object, in this embodiment, target detection may be performed on the live-action image to frame a position area of the second object in the live-action image; in the location area, pixel coordinates of key points on the second object are calculated. In the process of performing target detection on the live-action image, a deep learning model may be used for execution, and various target detection algorithms proposed now or in the future may be used in the deep learning model, which is not limited in this embodiment. For example, if the second object is a white line on the ground, the deep learning model may be used to frame a white line region in the live-action image, and then the above opencv method is used to calculate the pixel coordinates of the key points on the white line in the white line region, and after the target detection processing is added, the accuracy of the pixel coordinates of the key points may be effectively improved.

As mentioned above, during the camera calibration process, a calibration image may be obtained, and in the calibration image, the corresponding pixel coordinate range of each grid in the checkerboard in the pixel coordinate system may be determined according to the above-mentioned manner of obtaining the pixel coordinates. For example, for one of the grids, the pixel coordinates corresponding to the four corner points of the grid can be calculated respectively, and the pixel coordinate range covered by the grid can be determined based on the pixel coordinates corresponding to the four corner points.

Based on this, referring to fig. 1, in step 102, a second grid into which the keypoint on the first object and the keypoint on the second object fall may be determined according to the pixel coordinate range corresponding to each grid in the checkerboard in the pixel coordinate system and the pixel coordinates corresponding to the keypoints on the first object and the second object, respectively. In practical application, a unique identifier can be configured for each grid in the checkerboard, and the relative position relationship between the grids can be recorded, so that the first grid into which the key points on the first object fall and the second grid into which the key points on the second object fall can be accurately determined in the checkerboard.

Referring to fig. 1, in determining a first lattice in which a key point on a first object falls and a second lattice in which a key point on a second object falls, affine matrices corresponding to the first lattice and the second lattice may be obtained from calibration parameters of a monocular camera.

In

steps

103 and 104, world coordinates of the key point on the first object in a world coordinate system may be calculated according to the affine matrix under the first grid and the pixel coordinates corresponding to the key point on the first object; and calculating world coordinates of key points on the second object in a world coordinate system according to the affine matrix under the second grid and pixel coordinates corresponding to the key points on the second object. In this way, world coordinates corresponding to the keypoints on the first object and the second object, respectively, can be obtained. In the process of calculating the world coordinates, an affine matrix under the grid where the key points fall is used, so that the world coordinates can be calculated by using the adaptive affine matrix under the condition that the key points are located in different areas of the real image, and the calculated world coordinates are more accurate.

Referring to fig. 1, in step 105, a distance between a first object and a second object may be calculated based on world coordinates corresponding to key points on the first object and the second object, respectively. Here, a distance between two key points on the first object and the second object may be calculated as a distance between the first object and the second object based on the world coordinates corresponding to the key points on the first object and the second object, respectively.

In an alternative implementation, the distance between the key point on the first object and the X-axis and the Y-axis in the world coordinate system can be calculated according to the world coordinates corresponding to the key point on the first object; calculating the distance between the key point on the second object and the X axis and the Y axis in the world coordinate system according to the world coordinate corresponding to the key point on the second object; calculating Euclidean distance between key points on the first object and the second object as the distance between the first object and the second object based on the distance between the key points on the first object and the X axis and the Y axis in the world coordinate system and the distance between the key points on the second object and the X axis and the Y axis in the world coordinate system.

In this implementation, in calculating the euclidean distance between two keypoints, the distances from the two keypoints to the X-axis and the Y-axis in the world coordinate system are calculated, taking the keypoints on the second object as an example, in an exemplary scheme, the X-axis in the world coordinate system may be set to coincide with the specified edge of the specified grid in the checkerboard, so that in calculating the distance from the keypoints to the X-axis in the world coordinate system: a first distance between a keypoint on the second object and a first edge closest to the X-axis in the world coordinate system in the second grid in which it falls may be calculated; acquiring a second distance between the first edge and an X axis in a world coordinate system; and taking the sum of the first distance and the second distance as the distance between the key point on the second object and the X axis in the world coordinate system. In this exemplary scheme, the second distance between the first edge and the X-axis in the world coordinate system may be obtained by multiplying the number of grids between the first edge and the X-axis by the length of a single grid, and since the two parameter values of the number of grids and the length of the grid side do not change due to image distortion, the second distance between the first edge and the X-axis in the world coordinate system will not have an error regardless of whether there is distortion in the live-action image, so that, of the distances from the key point on the second object to the X-axis in the world coordinate system, only the first distance may cause an error, and the second distance may not have an error, which greatly reduces the overall calculation error, and particularly, in the case of distortion in the live-action image, may effectively improve the range error caused by distortion. The technical problem that the existing Zhang calibration and other camera calibration schemes cannot cope with the inclination angle and the image distortion can be effectively solved. In addition, in this exemplary scheme, the distance from the key point on the second object to the Y axis in the world coordinate system and the distance from the key point on the first object to the X axis and the Y axis in the world coordinate system may also be calculated in the same manner, and details are not repeated here. Under the condition that the distance from the coordinate axis is more accurate, the accuracy of the calculated Euclidean distance can be effectively improved, and the distance measurement accuracy is further improved.

Accordingly, in the present embodiment, the affine matrix under each of the grids in the checkerboard for calibration may be used as the calibration parameter of the monocular camera, based on which, the calibrated monocular camera may be used to perform the ranging operation, and during the ranging process, it may be determined into which grid of the checkerboard two objects to be measured fall, so as to use the affine matrix of the corresponding grid to calculate the world coordinate, and further calculate the distance between the objects. Because parameters such as the relative position of the grid and the world coordinate are not influenced by image distortion, the distance measurement scheme provided by the embodiment can effectively reduce errors caused by image distortion and the like and improve the distance measurement precision.

In the above or following embodiments, based on the above ranging scheme, GPS data corresponding to the key point on the second object may also be acquired; and calculating the GPS data corresponding to the key points on the first object according to the GPS data corresponding to the key points on the second object and the distance between the first object and the second object so as to position the first object.

The present embodiment proposes a positioning scheme based on the above-described ranging scheme. In this embodiment, after the distance between the first object and the second object is determined, the GPS data of the second object can be acquired, and in the case where the GPS data of one of the objects and the distance between the two objects are known, the GPS data of the other object can be calculated quickly and accurately. For example, the first object may be a vehicle, and the second object may be a white line on the ground, wherein the GPS data of the white line may be known in advance, and the GPS data of the vehicle may be calculated in a case where the base of the GPS data of the white line and the distance between the vehicle and the white line are known, so as to realize vehicle positioning.

In order to increase the positioning accuracy, it is further proposed in this embodiment that at least two monocular cameras are fixedly mounted on the first object. Wherein the at least two monocular cameras may be respectively mounted at different positions of the first object. For example, the first object is a vehicle and the at least two monocular cameras are distributed in front of, behind and/or to the sides of the vehicle.

On the basis, GPS data corresponding to key points on a second object shot by at least two monocular cameras can be respectively obtained, and the second objects shot by different monocular cameras are not identical; and calculating the GPS data corresponding to the key points on the first object according to the GPS data corresponding to the key points on the second object shot by the at least two monocular cameras and the distance between the first object and the second object shot by the at least two monocular cameras.

In this regard, the distance between the first object and the second object captured by the monocular camera may be calculated under each monocular camera by using the ranging scheme proposed in the above embodiments, and it should be noted that the second objects captured by different monocular cameras may not be identical. For example, a monocular camera may be installed in front of and behind the vehicle and calibrated, and on the basis, the monocular camera installed in front of the vehicle may be used to photograph the front white line and the front white line, and the monocular camera installed in back of the vehicle may be used to photograph the rear white line and the back white line, so that the distance between the front white line and the rear white line and the distance between the rear white line and the front white line may be calculated under the two monocular cameras, respectively. The GPS data of the front white line and the rear white line are known, so that the GPS data of the vehicle head and the vehicle tail can be rapidly and accurately calculated, the GPS data of the vehicle head and the vehicle tail are mutually corrected and fused, and the GPS data of the vehicle can be accurately determined.

In this embodiment, based on the ranging function provided in the above embodiments, accurate positioning of a specific object can be achieved without depending on GPS positioning equipment.

In the above or the following embodiments, the monocular camera may be calibrated in advance, and an optional calibration process may be:

after the monocular camera is fixed, the monocular camera is utilized to photograph the checkerboard in the visual field of the camera to obtain a calibration image;

under each grid contained in the checkerboard, calculating an affine matrix corresponding to the grid according to the coordinate change state of the grid between a world coordinate system and a pixel coordinate system corresponding to the calibration image;

and taking affine matrixes corresponding to all grids contained in the checkerboards as calibration parameters of the monocular camera.

Wherein, in the process of establishing the world coordinate system, the X axis in the world coordinate system can be set to coincide with the appointed side of the appointed grid in the checkerboard. Thus, the coordinate axes of the world coordinate system will fall on the sides of the relevant grid in the checkerboard. Fig. 3 is a schematic diagram of a world coordinate system according to an exemplary embodiment of the present application. As illustrated in FIG. 3, the world coordinate system may be constructed with the corner point in the upper left corner of the grid in the upper left corner of the checkerboard as the origin, with the X axis coinciding with the top edge of the grid and the Y axis coinciding with the left edge of the grid. In the world coordinate system, according to the size of each grid, the world coordinate of the corner point in each grid, the number of grids spanned between the coordinate axes, the covered world coordinate range, and other parameters that may be used in the ranging scheme may be determined, and these parameters may be recorded in the present embodiment and directly called in the foregoing ranging scheme.

In this embodiment, a pixel coordinate system may also be established according to the size of the captured calibration image. The image is composed of pixels, and the pixel coordinates can be used to indicate the location of the pixel in the image. Based on the pixel coordinate system, the aforementioned target detection and opencv techniques can be used to determine the pixel coordinates of the corner points in each grid in the checkerboard, the covered pixel coordinate range, and other parameters that may be used in the ranging scheme, and these parameters can be recorded in this embodiment and directly called in the aforementioned ranging scheme.

On the basis, the grid can be used as a unit, the coordinate change state of each grid between the world coordinate system and the pixel coordinate system corresponding to the calibration image can be represented according to various parameters of each grid in the checkerboard in the world coordinate system and various parameters in the pixel coordinate system, and then the affine matrix corresponding to the grid can be calculated. Taking one target grid as an example, the world coordinates of a corner point a of the target grid are (M1, N1), the target grid is turned over in the calibration image, and the pixel coordinates of the corner point a are (x 1, y 1), so that the following matrix relationship necessarily exists:

under the condition that the world coordinates and the pixel coordinates of the corner point A are known, the affine matrix of the target grid can be accurately calculated.

Of course, the target lattice may be rotated not only in a two-dimensional space but also in a three-dimensional space on the marker image, in which case the dimension may be increased in the affine matrix. The matrix relationship between the world coordinates and the pixel coordinates of the corner point a becomes accordingly:

the third element in the pixel coordinate matrix [ x1, y1, x1 x y1,1] is not limited to x1 x y1, and may adopt any value that can satisfy the matrix irreversible requirement (for example, any value that is not 0, is not equal to x1 or y1, and has no other multiple relation with x1 or y 1). In addition, in practical application, the affine matrix of the target grid can be calculated as follows by turning the corner point a for a plurality of times or adding the turning data of other corner points of the target grid.

In the same way, affine matrixes under other grids in the chessboard can be calculated, and the affine matrixes of each grid are used as calibration parameters of the monocular camera.

Accordingly, in the embodiment, a brand new monocular camera calibration scheme is provided, a world coordinate system can be established according to the size of a checkerboard for calibration, and an affine matrix corresponding to a grid is calculated according to the coordinate change state of the grid between the world coordinate system and a pixel coordinate system corresponding to a calibration image under each grid included in the checkerboard; and taking the affine matrix corresponding to each grid contained in the checkerboard as the calibration of the monocular camera. Therefore, different imaging areas of the monocular camera can be finely calibrated, and the calibration precision is effectively improved.

Fig. 4 is a schematic flowchart of a camera calibration method according to another exemplary embodiment of the present application. The method may be performed by a data processing apparatus, which may be implemented as a combination of software and/or hardware, which may be integrated in a computing device. Referring to fig. 4, the method includes:

step 400, after the monocular camera is fixed, photographing the checkerboard in the camera view by using the monocular camera to obtain a calibration image;

step 401, establishing a world coordinate system according to the size of the checkerboard;

step 402, under each grid included in the checkerboard, calculating an affine matrix corresponding to the grid according to the coordinate change state of the grid between a world coordinate system and a pixel coordinate system corresponding to the calibration image;

and step 403, taking the affine matrix corresponding to each grid contained in the checkerboard grid as a calibration parameter of the monocular camera.

The camera calibration scheme provided by the embodiment can be applied to a scene in which a monocular camera needs to be calibrated, and the application scene is not limited by the embodiment.

The camera calibration scheme provided by this embodiment may be used as a basis for the monocular camera to participate in other application works, where the application works that the monocular camera may participate include, but are not limited to, the distance measurement work, the positioning work, and the like mentioned in the above embodiments, and this embodiment does not limit the application field after the camera calibration scheme.

In an alternative embodiment, the process of calculating the affine matrix may comprise:

under the target grid, calculating pixel coordinates of at least one target corner point of the target grid in a pixel coordinate system corresponding to the calibration image;

determining world coordinates corresponding to at least one target corner point respectively in a world coordinate system;

and calculating an affine matrix corresponding to the target grid based on the world coordinates and the pixel coordinates corresponding to at least one target corner grid.

In an alternative embodiment, the coordinate axes of the world coordinate system may each be set to coincide with a specified edge of a specified grid in the checkerboard.

It should be noted that, for the technical details in the embodiments of the camera calibration scheme, reference may be made to the related description of the camera calibration process in the foregoing embodiments of the ranging scheme, and for the sake of brevity, detailed description is not repeated herein, but this should not cause a loss of the scope of the present application.

Accordingly, in the embodiment, a brand new monocular camera calibration scheme is provided, which may be configured to establish a world coordinate system according to the size of a checkerboard used for calibration, and calculate an affine matrix corresponding to a grid under each grid included in the checkerboard according to a coordinate change state of the grid between the world coordinate system and a pixel coordinate system corresponding to a calibration image; and (4) taking the affine matrix corresponding to each grid contained in the checkerboard as the calibration of the monocular camera. Therefore, different imaging areas of the monocular camera can be finely calibrated, and the calibration precision is effectively improved.

It should be noted that, the executing subjects of the steps of the method provided in the foregoing embodiments may be the same device, or different devices may also be used as the executing subjects of the method. For example, the execution subject of steps 100 to 105 may be device a; for another example, the execution subject of

steps

101 and 102 may be device a, and the execution subject of step 103 may be device B; and so on.

In addition, in some of the flows described in the above embodiments and the drawings, a plurality of operations are included in a specific order, but it should be clearly understood that the operations may be executed out of the order presented herein or in parallel, and the sequence numbers of the operations, such as 101, 102, etc., are merely used for distinguishing different operations, and the sequence numbers do not represent any execution order per se. Additionally, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel. It should be noted that, the descriptions of "first", "second", etc. in this document are used to distinguish different key points, lattices, etc., and do not represent the order of precedence, nor limit "first" and "second" to be different types.

Fig. 5 is a schematic structural diagram of a computing device according to another exemplary embodiment of the present application. As shown in fig. 5, the computing device includes: a memory 50 and a processor 51.

A processor 51, coupled to the memory 50, for executing the computer program in the memory 50 for:

determining a second grid into which key points on the first object and the second object fall, wherein the key points on the first object and the second object fall, according to the pixel coordinate range of each grid in the checkerboard in the pixel coordinate system and the pixel coordinates corresponding to the key points on the first object and the second object respectively;

In an alternative embodiment, the processor 51 may be further configured to:

acquiring GPS data corresponding to the key points on the second object;

and calculating the GPS data corresponding to the key points on the first object according to the GPS data corresponding to the key points on the second object and the distance between the first object and the second object so as to position the first object.

In an alternative embodiment, at least two monocular cameras are fixedly mounted on the first object, and the processor 51 is operable in calculating the GPS data corresponding to the keypoints on the first object based on the GPS data corresponding to the keypoints on the second object and the distance between the first object and the second object to:

respectively acquiring GPS data corresponding to key points on second objects shot by at least two monocular cameras, wherein the second objects shot by different monocular cameras are not identical;

and calculating the GPS data corresponding to the key points on the first object according to the GPS data corresponding to the key points on the second object shot by the at least two monocular cameras and the distance between the first object and the second object shot by the at least two monocular cameras.

In an alternative embodiment, the first object is a vehicle and the at least two monocular cameras are distributed in front of, behind and/or to the sides of the vehicle.

In an alternative embodiment, the processor 51, in calculating the pixel coordinates of the keypoint on the second object in the pixel coordinate system, is operable to:

carrying out target detection on the live-action image to frame a position area of the second object in the live-action image;

in the location area, pixel coordinates of key points on the second object are calculated.

In an alternative embodiment, the processor 51, in calculating the pixel coordinates of the keypoint on the first object in the pixel coordinate system, is operable to:

acquiring pixel coordinates of a key point on a first object stored in advance in a pixel coordinate system;

the process of predetermining the pixel coordinates of the keypoint on the first object in the pixel coordinate system comprises:

selecting key points on a first object;

shooting key points on a first object by using a monocular camera to obtain a test image;

in the test image, pixel coordinates of the keypoints on the first object in a pixel coordinate system are calculated and stored.

In an alternative embodiment, the processor 51, during calibration of the monocular camera, may be configured to:

after the monocular camera is fixed, photographing the checkerboard in the camera view by using the monocular camera to obtain a calibration image;

and taking the affine matrix corresponding to each grid contained in the checkerboards as the calibration parameters of the monocular camera.

In an alternative embodiment, the processor 51 may be configured to, in establishing the world coordinate system according to the dimensions of the checkerboard:

coordinate axes in the world coordinate system are respectively set to coincide with the specified sides of the specified grids in the checkerboard.

In an alternative embodiment, the processor 51, in calculating the distance between the first object and the second object based on the world coordinates corresponding to the key points on the first object and the second object, may be configured to:

calculating the distance between the key point on the first object and an X axis and a Y axis in a world coordinate system according to the world coordinate corresponding to the key point on the first object;

calculating the distance between the key point on the second object and the X axis and the Y axis in the world coordinate system according to the world coordinate corresponding to the key point on the second object;

calculating Euclidean distance between key points on the first object and the second object as the distance between the first object and the second object based on the distance between the key points on the first object and the X axis and the Y axis in the world coordinate system and the distance between the key points on the second object and the X axis and the Y axis in the world coordinate system.

In an alternative embodiment, the X-axis of the world coordinate system is arranged to coincide with a specified edge of a specified grid of the checkerboard, and the processor 51, in calculating the distance between the key point on the second object and the X-axis of the world coordinate system, is operable to:

calculating a first distance between a key point on the second object and a first edge, closest to the X-axis in the world coordinate system, in the second grid in which the key point falls;

acquiring a second distance between the first edge and an X axis in a world coordinate system;

the sum of the first distance and the second distance is taken as the distance between the key point on the second object and the X-axis in the world coordinate system.

Further, as shown in fig. 5, the computing device further includes: communication components 52, display 53, power components 54, audio components 55, and the like. Only some of the components are schematically shown in fig. 5, and the computing device is not meant to include only the components shown in fig. 5.

It should be noted that, for the technical details in the embodiments of the computing device, reference may be made to the related description in the foregoing embodiments of the ranging scheme, and for the sake of brevity, detailed description is not provided herein, but this should not cause a loss of the scope of the present application.

Based on the structure of the computing device provided in fig. 5, the foregoing camera calibration scheme may also be implemented based on the computing device in still another exemplary embodiment of the present application.

In this embodiment, the processor 51 is operable to:

Optionally, the processor 51, in calculating the affine matrix, may be configured to:

and calculating an affine matrix corresponding to the target grid based on the world coordinates and the pixel coordinates corresponding to the at least one target corner grid.

Optionally, the processor 51, in establishing the world coordinate system according to the size of the checkerboard, may be configured to:

coordinate axes in the world coordinate system are respectively set to coincide with the designated sides of the designated grids in the checkerboard.

It should be noted that, for the technical details in the embodiments of the computing device, reference may be made to the related description in the embodiments of the camera calibration scheme, and for the sake of brevity, detailed description is not repeated herein, but this should not cause a loss of the protection scope of the present application.

Accordingly, the present application further provides a computer-readable storage medium storing a computer program, where the computer program can implement the steps that can be executed by a computing device in the foregoing method embodiments when executed.

The memory of FIG. 5, described above, is used to store computer programs and may be configured to store various other data to support operations on the computing platform. Examples of such data include instructions for any application or method operating on the computing platform, contact data, phonebook data, messages, pictures, videos, and so forth. The memory may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

The communication component in fig. 5 is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as a WiFi, a 2G, 3G, 4G/LTE, 5G and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies.

The display of fig. 5 includes a screen, which may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

The power supply assembly of fig. 5 provides power to the various components of the device in which the power supply assembly is located. The power components may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device in which the power component is located.

The audio component of fig. 5 described above may be configured to output and/or input an audio signal. For example, the audio component includes a Microphone (MIC) configured to receive an external audio signal when the device in which the audio component is located is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal may further be stored in a memory or transmitted via a communication component. In some embodiments, the audio assembly further comprises a speaker for outputting audio signals.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include forms of volatile memory in a computer readable medium, random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer-readable media, including both permanent and non-permanent, removable and non-removable media, may implement the information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising one of 8230; \8230;" 8230; "does not exclude the presence of additional like elements in a process, method, article, or apparatus that comprises that element.

The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art to which the present application pertains. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method of ranging, comprising:

2. The method of claim 1, further comprising:

acquiring GPS data corresponding to the key points on the second object;

3. The method of claim 2, wherein the first object is fixedly equipped with at least two monocular cameras, and the calculating of the GPS data corresponding to the keypoints on the first object from the GPS data corresponding to the keypoints on the second object and the distance between the first object and the second object comprises:

respectively acquiring GPS data corresponding to key points on second objects shot by the at least two monocular cameras, wherein the second objects shot by different monocular cameras are not identical;

4. The method according to claim 3, wherein the first object is a vehicle and the at least two monocular cameras are distributed in front of, behind and/or to the sides of the vehicle.

5. The method of claim 1, wherein the calculating pixel coordinates of keypoints on the second object in a pixel coordinate system comprises:

performing target detection on the live-action image to frame a position area of the second object in the live-action image;

6. The method of claim 1, wherein said calculating pixel coordinates of keypoints on the first object in a pixel coordinate system comprises:

acquiring pixel coordinates of a key point on the first object, which are stored in advance, in a pixel coordinate system;

wherein the process of predetermining the pixel coordinates of the keypoint on the first object in a pixel coordinate system comprises:

selecting key points on the first object;

shooting key points on the first object by using the monocular camera to obtain a test image;

in the test image, pixel coordinates of keypoints on the first object in the pixel coordinate system are calculated and stored.

7. The method according to claim 1, wherein the process of calibrating the monocular camera comprises:

after the monocular camera is fixed, photographing a checkerboard placed in the field of view of the camera by using the monocular camera to obtain a calibration image;

establishing the world coordinate system according to the size of the checkerboard;

and taking the affine matrix corresponding to each grid contained in the checkerboard as the calibration parameter of the monocular camera.

8. The method of claim 7, wherein said establishing said world coordinate system according to said checkerboard dimensions comprises:

and setting the coordinate axes in the world coordinate system to be respectively superposed with the appointed sides of the appointed grids in the checkerboard.

9. The method of claim 1, wherein calculating the distance between the first object and the second object based on the world coordinates corresponding to the keypoints on the first object and the second object respectively comprises:

calculating the distance between the key point on the first object and an X axis and a Y axis in the world coordinate system according to the world coordinate corresponding to the key point on the first object;

calculating the distance between the key point on the second object and an X axis and a Y axis in the world coordinate system according to the world coordinate corresponding to the key point on the second object;

calculating Euclidean distances between key points on the first object and the second object as the distances between the first object and the second object based on the distances between the key points on the first object and the X-axis and the Y-axis in the world coordinate system and the distances between the key points on the second object and the X-axis and the Y-axis in the world coordinate system.

10. The method of claim 9, wherein the X-axis of the world coordinate system is set to coincide with a specified edge of a specified grid of the checkerboard, and wherein the calculating of the distance between the key point on the second object and the X-axis of the world coordinate system comprises:

calculating a first distance between a keypoint on the second object and a first edge of the second grid in which the keypoint falls, the first edge being closest to the X axis in the world coordinate system;

acquiring a second distance between the first edge and an X axis in the world coordinate system;

and taking the sum of the first distance and the second distance as the distance between the key point on the second object and the X axis in the world coordinate system.

11. A camera calibration method is characterized by comprising the following steps:

12. The method of claim 11, wherein the process of computing an affine matrix comprises:

under a target grid, calculating pixel coordinates of at least one target corner point of the target grid in a pixel coordinate system corresponding to the calibration image;

determining world coordinates corresponding to the at least one target corner point respectively in the world coordinate system;

13. The method of claim 11, wherein said establishing a world coordinate system based on dimensions of said checkerboard comprises:

and setting the coordinate axes in the world coordinate system to be superposed with the specified sides of the specified grids in the checkerboard respectively.

14. A computing device comprising a memory and a processor;

the memory is to store one or more computer instructions;

15. A computing device comprising a memory and a processor;

the memory is to store one or more computer instructions;

16. A computer-readable storage medium storing computer instructions which, when executed by one or more processors, cause the one or more processors to perform the ranging method of any one of claims 1-10 or the camera calibration method of any one of claims 11-13.