CN115471563A - Calibration method and device of vehicle-mounted all-round looking system - Google Patents

Calibration method and device of vehicle-mounted all-round looking system Download PDF

Info

Publication number
CN115471563A
CN115471563A CN202111387379.2A CN202111387379A CN115471563A CN 115471563 A CN115471563 A CN 115471563A CN 202111387379 A CN202111387379 A CN 202111387379A CN 115471563 A CN115471563 A CN 115471563A
Authority
CN
China
Prior art keywords
pattern
calibration
dimensional coordinates
identification information
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111387379.2A
Other languages
Chinese (zh)
Inventor
刘锋
夏晓
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Co Wheels Technology Co Ltd
Original Assignee
Beijing Co Wheels Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Co Wheels Technology Co Ltd filed Critical Beijing Co Wheels Technology Co Ltd
Priority to CN202111387379.2A priority Critical patent/CN115471563A/en
Publication of CN115471563A publication Critical patent/CN115471563A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/80—Geometric correction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Processing (AREA)

Abstract

The application provides a calibration method and a device of a vehicle-mounted all-round system, which relate to the technical field of intelligent vehicles, and the method comprises the following steps: the method comprises the steps of obtaining an image of a calibration object collected by a vehicle-mounted camera, wherein the calibration object comprises a plurality of areas, at least one area comprises a pattern containing identification information, the identification information of the pattern and two-dimensional coordinates of corner points of the pattern are identified, and calibration external parameters of the camera are determined according to the identification information of the pattern and the two-dimensional coordinates of the corner points. Therefore, the angular points of the pattern of the calibration object are identified, so that the external reference calibration of the camera is carried out, the distortion generated during the identification of the edge area of the image is reduced, the requirements on surrounding light rays and the arrangement difficulty of the calibration object are reduced, and the identification rate of the angular points and the calibration efficiency and precision are improved.

Description

Calibration method and device of vehicle-mounted all-round looking system
Technical Field
The application relates to the technical field of intelligent vehicles, in particular to a calibration method and a calibration device for a vehicle-mounted all-round-looking system.
Background
At present, the calibration method of image acquisition devices in most automobile panoramic systems is checkerboard calibration, that is, checkerboard patterns are arranged on the front, back, left and right sides of a vehicle stopped at a fixed position, parameters of the checkerboard patterns are recorded, a checkerboard opposite to the image acquisition device is found, and external parameters are automatically calibrated for a camera by combining parameters such as physical coordinates of angular points of the checkerboard patterns, but the checkerboard patterns positioned at the edge of the image of the camera are large in distortion during identification, requirements for surrounding light rays are high, the angular point identification is difficult, so that the efficiency and the precision of external parameter calibration are low, and the difficulty in arrangement of the checkerboard patterns is relatively high.
Disclosure of Invention
The present application is directed to solving, at least to some extent, one of the technical problems in the related art.
Therefore, an object of the present application is to provide a calibration method for a vehicle-mounted panoramic system.
A second objective of the present application is to provide a calibration apparatus for a vehicle-mounted looking-around system.
A third object of the present application is to propose a vehicle.
A fourth object of the present application is to propose a calibration object.
A fifth object of the present application is to propose a vehicle.
A sixth object of the present application is to provide an electronic apparatus.
A seventh object of the present application is to propose a vehicle.
An eighth object of the present application is to provide a computer-readable storage medium.
In order to achieve the above object, an embodiment of a first aspect of the present application provides a calibration method for a vehicle-mounted looking-around system, including: acquiring an image of a calibration object acquired by a vehicle-mounted camera, wherein the calibration object comprises a plurality of areas, and at least one area comprises a code pattern containing identification information; identifying identification information of the pattern and two-dimensional coordinates of corner points of the pattern; and determining the calibration external parameters of the camera according to the identification information of the pattern and the two-dimensional coordinates of the corner points.
According to one embodiment of the present application, the plurality of regions includes: a plurality of first areas corresponding to the image edges and a plurality of second areas arranged at intervals with the first areas in the calibration object, wherein the first areas and the second areas comprise at least one pattern.
According to one embodiment of the application, the density of the pattern of the first area is different from the density of the pattern of the second area.
According to one embodiment of the application, the size of the pattern of the first area is different from the size of the pattern of the second area.
According to an embodiment of the application, said identifying two-dimensional coordinates of corner points of said pattern comprises: identifying the pattern in the image; and when the recognition rate of the pattern in the image is lower than a preset recognition rate threshold, re-acquiring the image or overlapping recognition results of the pattern of multiple frames of the image for recognition until the recognition rate reaches the preset recognition rate threshold.
According to an embodiment of the present application, the calibration method of the vehicle-mounted looking-around system according to the embodiment of the present application further includes: identifying the pattern of no less than three of the regions in the image, the three of the regions including at least one of the first regions and at least one of the second regions.
According to an embodiment of the present application, before determining the calibration external reference of the camera according to the identification information of the pattern and the two-dimensional coordinates of the corner point, the method further includes: and carrying out distortion removal processing on the two-dimensional coordinates of the corner points.
According to an embodiment of the present application, the determining a calibration external parameter of the camera according to the identification information of the pattern and the two-dimensional coordinates of the corner point includes: and determining the calibration external parameters of the camera according to the identification information of the pattern, the two-dimensional coordinates of the corner points and the three-dimensional coordinates of corresponding points in the scale pattern.
According to an embodiment of the present application, the determining a calibration external parameter of the camera according to the identification information of the pattern and the two-dimensional coordinates of the corner point includes: determining candidate calibration external parameters of the camera corresponding to a single area in the image according to the identification information of the pattern and the two-dimensional coordinates of the corner points; determining the projection error of the corner point according to the candidate calibration external parameters, the identification information of the pattern and the two-dimensional coordinates of the corner point; and if the projection error is smaller than a preset projection error threshold value, determining the calibration external parameters of the camera according to the identification information of the pattern and the two-dimensional coordinates of the corner points.
According to an embodiment of the present application, the calibration method of the vehicle-mounted looking-around system according to the embodiment of the present application further includes: and if the projection error is equal to or larger than the projection error threshold value, outputting reminding information, wherein the reminding information is used for reminding a user to replace the calibration site or re-lay the calibration object.
In order to achieve the above object, a second aspect of the present application provides a calibration apparatus for a vehicle-mounted looking-around system, including: the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring an image of a calibration object acquired by a vehicle-mounted camera, the calibration object comprises a plurality of areas, and at least one area comprises a pattern containing identification information; the identification module is used for identifying the identification information of the pattern and the two-dimensional coordinates of the corner points of the pattern; and the determining module is used for determining the calibration external parameters of the camera according to the identification information of the pattern and the two-dimensional coordinates of the corner points.
According to an embodiment of the present application, the identification module is specifically configured to: identifying the pattern in the image; and when the recognition rate of the pattern in the image is lower than a preset recognition rate threshold, re-acquiring the image or overlapping recognition results of the pattern of multiple frames of the image for recognition until the recognition rate reaches the preset recognition rate threshold.
According to an embodiment of the present application, the determining module is further configured to: and before determining the calibration external reference of the camera according to the two-dimensional coordinates of the corner points, carrying out distortion removal processing on the two-dimensional coordinates of the corner points.
According to an embodiment of the present application, the determining module is specifically configured to: and determining the calibration external parameters of the camera according to the identification information of the pattern, the two-dimensional coordinates of the corner points and the three-dimensional coordinates of the corresponding points in the scale pattern.
According to an embodiment of the present application, the determining module is specifically configured to: determining candidate calibration external parameters of the camera corresponding to a single area in the image according to the identification information of the pattern and the two-dimensional coordinates of the corner points; determining the projection error of the corner point according to the candidate calibration external parameters, the identification information of the pattern and the two-dimensional coordinates of the corner point; and if the projection error is smaller than a preset projection error threshold value, determining the calibration external parameters of the camera according to the identification information of the pattern and the two-dimensional coordinates of the corner points.
According to an embodiment of the present application, the determining module is further configured to: and if the projection error is equal to or larger than the projection error threshold value, outputting reminding information, wherein the reminding information is used for reminding a user to replace the calibration site or re-lay the calibration object.
To achieve the above object, an embodiment of a third aspect of the present application proposes a vehicle including: the calibration device of the vehicle-mounted looking-around system according to the embodiment of the second aspect of the application.
To achieve the above object, a fourth aspect of the present application provides a calibration object for calibrating an on-vehicle looking-around system, including: the calibration object comprises a plurality of first areas and a plurality of second areas, wherein the first areas are located in the calibration object and correspond to the edges of the image, the second areas are arranged at intervals with the first areas, and the first areas and the second areas comprise at least one pattern containing identification information.
According to one embodiment of the application, the density of the pattern of the first area is different from the density of the pattern of the second area. According to one embodiment of the application, the size of the pattern of the first area is different from the size of the pattern of the second area.
In order to achieve the above object, an embodiment of a fifth aspect of the present application provides a vehicle, and an on-board looking-around system of the vehicle performs calibration by acquiring an image of a calibration object according to an embodiment of a fourth aspect of the present application.
To achieve the above object, a sixth aspect of the present application provides an electronic device, including: the calibration method comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein when the processor executes the program, the calibration method of the vehicle-mounted looking-around system is realized according to the embodiment of the first aspect of the application. To achieve the above object, an embodiment of the seventh aspect of the present application provides a vehicle including the electronic device according to the embodiment of the sixth aspect of the present application.
To achieve the above object, an eighth aspect of the present application provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements a calibration method of an on-board looking-around system according to an embodiment of the first aspect of the present application.
Drawings
FIG. 1 is a flow chart illustrating a method for calibrating a vehicle mounted look-around system in accordance with an exemplary embodiment of the present application;
FIG. 2 is a schematic illustration of a method of calibrating an on-board look-around system according to an exemplary embodiment of the present application;
FIG. 3 is a flow chart illustrating another method for calibrating an on-board vehicle look-around system in accordance with an exemplary embodiment of the present application;
FIG. 4 is a flow chart illustrating another method for calibrating a vehicle mounted look-around system in accordance with an exemplary embodiment of the present application;
FIG. 5 is a block diagram illustrating a calibration arrangement for an on-board vehicle look-around system in accordance with an exemplary embodiment of the present application;
FIG. 6 is a schematic illustration of a vehicle according to an exemplary embodiment of the present application;
FIG. 7 is a schematic diagram of an electronic device according to an exemplary embodiment of the present application;
FIG. 8 is a schematic illustration of another vehicle according to an exemplary embodiment of the present application.
Detailed Description
Reference will now be made in detail to the embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the drawings are exemplary and intended to be used for explaining the present application and should not be construed as limiting the present application.
Fig. 1 is a flowchart illustrating a calibration method of a vehicle-mounted looking-around system according to an exemplary embodiment of the present application, where the calibration method of the vehicle-mounted looking-around system, as shown in fig. 1, includes the following steps:
s101, acquiring an image of a calibration object acquired by a vehicle-mounted camera, wherein the calibration object comprises a plurality of areas, and at least one area comprises a pattern containing identification information.
Specifically, the calibration method of the vehicle-mounted looking-around system according to the embodiment of the present application may be executed by the calibration device of the vehicle-mounted looking-around system provided by the embodiment of the present application, and the calibration device of the vehicle-mounted looking-around system may be arranged in the vehicle-mounted looking-around system to provide a calibration service for the system.
The method comprises the steps of acquiring images of a calibration object acquired by a vehicle-mounted camera, wherein the vehicle-mounted camera is positioned in a vehicle-mounted all-around system, the vehicle-mounted all-around system is a panoramic all-around system formed by splicing images acquired by cameras (such as fisheye cameras or wide-angle cameras) arranged in four directions of the front, the back, the left and the right of a vehicle body, the calibration object is an object used for calibrating the vehicle-mounted all-around system, the calibration object comprises a plurality of areas, the calibration object comprises a plurality of first areas corresponding to the edges of images in the calibration object and a plurality of second areas arranged at intervals with the first areas, and the first areas and the second areas comprise at least one pattern. As shown in fig. 2, for example, eight regions may be respectively disposed in four directions, i.e., up, down, left, and right, of the calibration object, wherein four regions corresponding to the edges of the image, i.e., four corners, are respectively used as the first regions 21, and four regions corresponding to the middle of the four sides and spaced from the first regions are respectively used as the second regions 22, for convenience of description, the region where the first region 21 is located is referred to as an edge region of the calibration object, and the region where the second region 22 is located is referred to as a non-edge region of the calibration object. The areas include patterns containing identification information, the patterns can be AprilTag, one-dimensional codes, two-dimensional codes and any patterns with information recording functions, and the patterns in the areas can be one, a plurality or a combination of a plurality of patterns. The patterns of different areas have different densities, and the edge area of the calibration object is different from the non-edge area of the calibration object, namely the density of the patterns of the edge first area is less than or greater than that of the patterns of the non-edge second area, and the sizes of the patterns of different areas are also different, and the edge area of the calibration object is different from the non-edge area of the calibration object, namely the sizes of the patterns of the edge first area are less than or greater than that of the patterns of the non-edge second area.
The identification information recorded in the pattern can be number information, character information, position information or other information, such as 1, 2, 3, 4 \8230 \ 8230or one, two, three, four \8230 \ 8230, etc., the pattern can also be a pattern with certain identification, such as an animal image, a character image, a plant image, a geometric figure, etc., all the patterns on the calibration object are arranged according to a preset sequence, so that each pattern or pattern combination has a corresponding position, the vehicle stores or temporarily stores the preset sequence information of all the patterns, and the identification pattern or pattern combination is combined through the image collected by the identification camera, so that the coordinate position of all the patterns can be obtained.
The calibration object may also have a central region for parking a vehicle to be calibrated.
The pattern is AprilTag as an example, each camera of the vehicle-mounted all-round looking system can acquire three groups of AprilTags in different areas, because the coverage range of each camera comprises a left area, a middle area and a right area, and the three groups of AprilTags respectively represent the three areas covered by the camera, so that the three groups of AprilTags are acquired to ensure the comprehensiveness and integrity of the image acquired by each camera, the left part, the middle part and the right part of the image are covered, the accuracy of the acquired data is more reliable, the calibration stability is better, and the calibration effect is further improved. For example, in fig. 2, three apriltags collected by the camera located at the left side are located in the three areas of the upper left, the left and the lower left, respectively, in the density of the AprilTag, the density of the AprilTag in the edge area of the calibration object is smaller than that in the non-edge area of the calibration object, that is, the density of the aprilat in the edge first area is smaller than that in the non-edge second area, and in the size of the grid corresponding to the AprilTag, the size of the grid corresponding to the edge area of the calibration object is larger than that in the non-edge area of the calibration object, that is, the size of the grid corresponding to the aprilat in the edge first area is larger than that in the non-edge second area, and in addition, the ID (Identity) of the AprilTag is also unique.
S102, identifying the identification information of the pattern and the two-dimensional coordinates of the corner points of the pattern.
Specifically, the April Tag is a rectangular pattern and has four corners, each corner corresponds to one corner point, the adjacent April tags may share the corner points, and the image of the calibration object acquired by the vehicle-mounted camera acquired in step S101 is identified, so that the identification information recorded by each April Tag in the image and the two-dimensional coordinates of each corner point of each April Tag are obtained. The angular points are four vertexes and a central point of each April Tag, the serial numbers of the four vertexes are known based on a real serial number preset in a physical space, and the coordinates of the angular points are coordinates of the angular points on a plane where the image of the calibration object is located.
It should be noted that the requirement of angular point identification of April Tag on light around the vehicle environment is not high, and all angular points do not need to be detected by one frame, but coordinates of the angular points can be extracted based on the accumulated result of multi-frame identification, so as to calibrate the vehicle-mounted all-around system.
S103, determining calibration external parameters of the camera according to the identification information of the pattern and the two-dimensional coordinates of the corner points.
Specifically, the calibration external parameters of the corresponding vehicle-mounted camera are determined according to the identification information of the pattern identified in step S102 and the two-dimensional coordinates of the corner points of the pattern.
The external reference refers to a rotation matrix and a translation matrix corresponding to the camera, and is used for describing a conversion relation between a camera coordinate system and other coordinate systems, for example, a conversion relation between the camera coordinate system and a vehicle body coordinate system.
Taking an example of solving the external reference of the panoramic camera coordinate system and the world coordinate system, the external camera reference calibration is to establish a world coordinate system by means of a calibration object and fix a transformation relation between the vehicle body coordinate system and the world coordinate system, wherein the vehicle body coordinate system needs to fix the camera on the vehicle body and keep relatively static with the vehicle body, and the world coordinate system can be in a state of keeping the relation between the position and the posture of the vehicle body and the calibration object fixed all the time when the vehicle is parked, so that the external parameters of the camera coordinate system and the world coordinate system are determined by utilizing the imaging information of the calibration plate on an image plane and the inherent size information of the calibration object.
In the embodiment of the application, an image of a calibration object which is acquired by a vehicle-mounted camera and comprises a plurality of areas is acquired, at least one area comprises a pattern containing identification information, the identification information of the pattern and two-dimensional coordinates of corner points in the pattern are identified, and calibration external parameters of the camera are determined according to the identification information of the pattern and the two-dimensional coordinates of the corner points. Therefore, the angular points of the pattern of the calibration object are identified, so that the external reference calibration of the camera is carried out, the distortion generated during the identification of the edge area of the image is reduced, the requirements on surrounding light rays and the arrangement difficulty of the calibration object are reduced, and the identification rate of the angular points and the calibration efficiency and precision are improved.
Fig. 3 is a schematic flowchart of a calibration method of a vehicle-mounted looking-around system according to a second embodiment of the present application.
On the basis of the foregoing embodiment, as shown in fig. 3, the calibration method of the vehicle-mounted all-round system according to the embodiment of the present application may specifically include the following steps:
s301, acquiring an image of a calibration object acquired by a vehicle-mounted camera, wherein the calibration object comprises a plurality of areas, and at least one area comprises a pattern containing identification information.
Specifically, step S301 in this embodiment is the same as step S101 in the above embodiment, and is not described herein again.
The step S102 "identifying the identification information of the pattern and the two-dimensional coordinates of the corner point of the pattern" in the above embodiment may specifically include the following steps S302 and S303:
s302, identifying the pattern in the image.
Specifically, the images of the calibration object acquired by the vehicle-mounted camera in step S301 are recognized, so as to obtain the patterns in the corresponding images, that is, after the vehicle stops at the fixed pose in the calibration object, the images acquired by the cameras are respectively acquired, and the patterns in the images are detected and recognized.
As a possible embodiment, a pattern of not less than three areas in the image may be recognized, the three areas including at least one first area and at least one second area.
And S303, when the recognition rate of the pattern in the image is lower than a preset recognition rate threshold, re-acquiring the image or overlapping the recognition results of the pattern of the multi-frame image for recognition until the recognition rate reaches the preset recognition rate threshold.
Specifically, an identification rate threshold is preset, the identification rate of the pattern in the image of the calibration object identified in step S302 is compared with the preset identification rate threshold, and when the identification rate of the pattern is lower than the preset identification rate threshold, the pattern in the image of the calibration object is reacquired or the identification results of the patterns in the images of multiple frames are subjected to deduplication processing and then are superimposed until the identification rate of the pattern is higher than the preset identification rate threshold.
It should be noted that, in order to maintain the calibration accuracy, it is necessary to satisfy that the recognition rate of the patterns of the three regions on each image is greater than a preset recognition rate threshold, where the preset threshold may be set to 70% -80%, for example, 75%.
S304, distortion removal processing is carried out on the two-dimensional coordinates of the corner points.
Specifically, distortion removal processing may be performed on the two-dimensional coordinates of the corner points of the pattern in the image identified in step S302 according to a distortion table or a calibrated camera internal parameter and the like, so as to obtain the two-dimensional coordinates of the corner points from which distortion is removed.
The step S103 of determining the calibration external reference of the camera according to the identification information of the pattern and the two-dimensional coordinates of the corner points in the embodiment may specifically include the following step S305:
s305, determining candidate calibration external parameters of the camera corresponding to the single area in the image according to the identification information of the pattern and the two-dimensional coordinates of the corner points.
Specifically, the external parameters of the camera corresponding to a single region in the image may be calibrated according to the two-dimensional coordinates of the distortion-removed corner points acquired in step S304, the identification information of the pattern identified in step S302, and the three-dimensional coordinates of the corresponding points in the scale pattern, and the obtained external parameters are candidate calibration external parameters, for example, three groups of candidate calibration external parameters of a certain position camera may be marked and respectively named as T1-T3, that is, the corner point candidate calibration external parameter T1 of the middle region is taken, the corner point candidate calibration external parameter T2 of the left region is taken, the corner point candidate calibration external parameter T3 of the right region is taken, and the candidate calibration external parameters of the other three cameras may be continuously calibrated by using this method.
It should be noted that the two-dimensional coordinates of the corner points from which distortion is removed correspond one-to-one to the known three-dimensional coordinates of the points on the established scale pattern, and the extrinsic reference can be calibrated by using the opencvpnp function for a camera without distortion.
S306, determining the projection error of the corner point according to the candidate calibration external parameters, the identification information of the pattern and the two-dimensional coordinates of the corner point.
Specifically, based on the candidate calibration external parameters of the camera corresponding to the single region in the image determined in step S305, the identification information of the pattern identified in step S302, and the two-dimensional coordinates of the distortion-removed corner points acquired in step S304, the projection errors corresponding to the corner points, that is, the three-dimensional world coordinates of the four vertices and the center point on the three-dimensional coordinate points, that is, the calibration pattern are further determined, and the projection points of these coordinate points on the image are calculated according to the internal parameters and the external parameters of the camera, so as to obtain the projection errors, that is, the differences between the projection points and the coordinates of the points extracted from the real image, for example, the projection errors of the corner points of the single region are solved by using the three-dimensional points on the calibration pattern of the three regions with respect to the three candidate calibration external parameters T1, T2, and T3, respectively.
And S307, if the projection error is smaller than a preset projection error threshold, determining a calibration external parameter of the camera according to the two-dimensional coordinates of the corner points.
Specifically, a projection error threshold is preset, the projection error is compared with the preset projection error threshold according to the candidate calibration external parameter, the identification information of the pattern and the projection error of the corner determined by the two-dimensional coordinates of the corner, and if the projection error is smaller than the preset projection error threshold, the calibration external parameter of the camera is determined according to the two-dimensional coordinates of the corner. The projection error threshold value can be set according to needs, and the specific numerical value of the preset projection error threshold value is not limited too much.
It should be noted that when the projection errors are all smaller than the set threshold, the external reference calibration precision of each region is considered to be high, and the three-dimensional coordinates of the whole calibration object and the extracted angle errors are both very small, so that the whole calibration precision requirement can be met, and at this time, the calibration external references T4 of the three regions are selected as the final calibration external reference result.
And S308, if the projection error is equal to or greater than the projection error threshold, outputting reminding information, wherein the reminding information is used for reminding a user to replace the calibration site or re-lay the calibration object.
Specifically, the projection error is compared with a preset projection error threshold value according to the candidate calibration external parameter, the identification information of the pattern and the projection error of the corner point determined by the two-dimensional coordinates of the corner point, and if the projection error is not less than, namely equal to or greater than the preset projection error threshold value, the prompting information that the calibration site needs to be replaced or the calibration object needs to be laid again is output to a user for subsequent processing.
It should be noted that, when the projection error is equal to or greater than the set threshold, the projection error may be caused by a size error of the calibration object itself, or may be caused by that the entire calibration object is not on a plane, or the relative positions of the calibration objects in the three areas are inconsistent with a preset value, and the like, which requires a corresponding process of replacing a site or re-laying the calibration object until corresponding requirements are met.
In the embodiment of the application, the method includes the steps of acquiring an image of a calibration object which is acquired by a vehicle-mounted camera and comprises a plurality of areas, wherein at least one area comprises a pattern containing identification information, identifying the pattern in the image, when the identification rate of the pattern is lower than a preset identification rate threshold, re-acquiring the image or overlapping the identification results of the patterns of a plurality of frames of images for identification until the identification rate reaches the preset identification rate threshold, then performing distortion removal processing on two-dimensional coordinates of corner points, determining candidate calibration external parameters of a camera corresponding to a single area in the image according to the identification information of the pattern and the two-dimensional coordinates of the corner points, determining a projection error of the corner points according to the candidate calibration external parameters, the identification information of the pattern and the two-dimensional coordinates of the corner points, if the projection error is smaller than the preset projection error threshold, determining the calibration external parameters of the camera according to the two-dimensional coordinates of the corner points, and outputting information for reminding a user of replacing a calibration field or re-laying the calibration object if the projection error is equal to or larger than the projection error threshold. Therefore, the angular points of the pattern of the calibration object are identified, so that the external reference calibration of the camera is carried out, the distortion generated during the identification of the edge area of the image is reduced, the requirements on surrounding light rays and the arrangement difficulty of the calibration object are reduced, and the identification rate of the angular points and the calibration efficiency and precision are improved. Meanwhile, the pattern is judged and correspondingly processed through a preset identification rate threshold value, the identification rate of the angular point is further improved, the problem of a calibration object or a calibration field can be found through calculating a projection error, and the calibration precision is further improved from an algorithm and a feedback mechanism.
For clearly describing the calibration method of the vehicle-mounted looking-around system in the embodiment of the present application, a detailed implementation process of the calibration method of the vehicle-mounted looking-around system in the embodiment of the present application is described in detail below with reference to fig. 4. As shown in fig. 4, the method may specifically include the following steps:
s401, acquiring an image of the calibration object acquired by the vehicle-mounted camera.
S402, identifying the pattern in the image.
And S403, judging whether the recognition rate of the pattern in the image is lower than a preset recognition rate threshold value. If yes, go to step S404, otherwise go to step S405.
S404, re-acquiring the image or overlapping the recognition results of the pattern of the multi-frame image for recognition until the recognition rate reaches a preset recognition rate threshold value. Step S402 is performed.
And S405, carrying out distortion removal processing on the two-dimensional coordinates of the corner points.
S406, determining candidate calibration external parameters of the camera corresponding to a single area in the image according to the identification information of the pattern, the two-dimensional coordinates of the corner points and the three-dimensional coordinates of the corresponding points in the scale pattern.
S407, determining the projection error of the corner point according to the candidate calibration external parameters, the identification information of the pattern and the two-dimensional coordinates of the corner point.
S408, judging whether the projection error is smaller than a preset projection error threshold value. If so, step S409 is executed, otherwise, step S410 is executed.
And S409, determining calibration external parameters of the camera according to the two-dimensional coordinates of the corner points.
And S410, outputting reminding information, wherein the reminding information is used for reminding a user to replace a calibration site or re-lay a calibration object.
Fig. 5 is a block diagram illustrating a calibration apparatus of an on-board looking-around system according to an exemplary embodiment of the present application, and as shown in fig. 5, the calibration apparatus 500 of the on-board looking-around system includes: an acquisition module 501, an identification module 502 and a determination module 503.
The acquiring module 501 is configured to acquire an image of a calibration object acquired by a vehicle-mounted camera, where the calibration object includes a plurality of regions, and at least one region includes a pattern including identification information.
The identification module 502 is configured to identify the identification information of the pattern and two-dimensional coordinates of a corner of the pattern.
The determining module 503 is configured to determine a calibration external parameter of the camera according to the identification information of the pattern and the two-dimensional coordinates of the corner point.
According to an embodiment of the present application, the identifying module 502 is specifically configured to: identifying a pattern in the image; and when the recognition rate of the patterns in the image is lower than a preset recognition rate threshold, re-acquiring the image or overlapping the recognition results of the patterns of the multi-frame image for recognition until the recognition rate reaches the preset recognition rate threshold.
According to an embodiment of the present application, the determining module 503 is further configured to: and before determining the calibration external reference of the camera according to the identification information of the pattern and the two-dimensional coordinates of the corner points, carrying out distortion removal processing on the two-dimensional coordinates of the corner points.
According to an embodiment of the present application, the determining module 503 is specifically configured to: and determining calibration external parameters of the camera according to the identification information of the pattern, the two-dimensional coordinates of the corner points and the three-dimensional coordinates of the corresponding points in the scale pattern.
According to an embodiment of the present application, the determining module 503 is specifically configured to: determining candidate calibration external parameters of a camera corresponding to a single area in the image according to the identification information of the pattern and the two-dimensional coordinates of the corner points; determining the projection error of the corner points according to the candidate calibration external parameters, the identification information of the pattern and the two-dimensional coordinates of the corner points; and if the projection error is smaller than a preset projection error threshold value, determining the calibration external parameters of the camera according to the identification information of the pattern and the two-dimensional coordinates of the corner points.
According to an embodiment of the present application, the determining module 503 is further configured to: and if the projection error is equal to or larger than the projection error threshold value, outputting reminding information, wherein the reminding information is used for reminding a user to replace the calibration site or re-lay the calibration object.
It should be noted that the explanation of the embodiment of the calibration method for the vehicle-mounted looking-around system is also applicable to the calibration device for the vehicle-mounted looking-around system in the embodiment of the present application, and the specific process is not described herein again.
In the embodiment of the application, the method includes the steps of acquiring an image of a calibration object which is acquired by a vehicle-mounted camera and comprises a plurality of areas, wherein at least one area comprises a pattern containing identification information, identifying the pattern in the image, when the identification rate of the pattern is lower than a preset identification rate threshold, re-acquiring the image or overlapping the identification results of the patterns of a plurality of frames of images for identification until the identification rate reaches the preset identification rate threshold, then performing distortion removal processing on two-dimensional coordinates of corner points, determining candidate calibration external parameters of a camera corresponding to a single area in the image according to the identification information of the pattern and the two-dimensional coordinates of the corner points, determining a projection error of the corner points according to the candidate calibration external parameters, the identification information of the pattern and the two-dimensional coordinates of the corner points, if the projection error is smaller than the preset projection error threshold, determining the calibration external parameters of the camera according to the two-dimensional coordinates of the corner points, and outputting information for reminding a user of replacing a calibration field or re-laying the calibration object if the projection error is equal to or larger than the projection error threshold. Therefore, the angular points of the pattern of the calibration object are identified, so that the external reference calibration of the camera is carried out, the distortion generated during the identification of the edge area of the image is reduced, the requirements on surrounding light rays and the arrangement difficulty of the calibration object are reduced, and the identification rate of the angular points and the calibration efficiency and precision are improved. Meanwhile, the pattern is judged and correspondingly processed through a preset identification rate threshold value, the identification rate of the angular point is further improved, the problem of a calibration object or a calibration site can be found through calculating a projection error, and the calibration precision is further improved from an algorithm and a feedback mechanism.
In order to implement the foregoing embodiment, the embodiment of the present application further provides a vehicle 600, as shown in fig. 6, where the vehicle 600 specifically includes: the calibration apparatus 500 of the vehicle-mounted looking-around system as shown in the above embodiment.
In order to implement the foregoing embodiment, an embodiment of the present application further provides a calibration object, which is used for calibrating a vehicle-mounted around-view system, where the calibration object specifically includes: the image calibration device comprises a plurality of first areas and a plurality of second areas, wherein the first areas are positioned in the calibration object and correspond to the edges of the image, the second areas are arranged at intervals with the first areas, and the first areas and the second areas comprise at least one pattern containing identification information.
According to one embodiment of the application, the density of the pattern of the first area is different from the density of the pattern of the second area.
According to one embodiment of the application, the size of the pattern of the first area is different from the size of the pattern of the second area.
In order to implement the foregoing embodiment, the embodiment of the present application further provides a vehicle, in which an on-vehicle looking-around system of the vehicle performs calibration by acquiring an image of a calibration object as shown in the foregoing embodiment.
In order to implement the foregoing embodiment, an embodiment of the present application further provides an electronic device 700, as shown in fig. 7, where the electronic device 700 specifically includes: the calibration method of the vehicle-mounted looking-around system comprises a memory 701, a processor 702 and a computer program which is stored on the memory 701 and can run on the processor 702, wherein when the processor 702 executes the program, the calibration method of the vehicle-mounted looking-around system as shown in the embodiment is realized.
In order to implement the foregoing embodiment, the embodiment of the present application further provides a vehicle 800, as shown in fig. 8, where the vehicle 800 may specifically include: the electronic device 700 as shown in the above embodiment.
In order to implement the foregoing embodiments, the present application further provides a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to implement the calibration method of the vehicle-mounted looking-around system as shown in the foregoing embodiments.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the present application, "a plurality" means two or more unless specifically limited otherwise.
In the description of the present specification, reference to the description of "one embodiment," "some embodiments," "an example," "a specific example," or "some examples" or the like means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
While embodiments of the present application have been shown and described above, it will be understood that the above embodiments are exemplary and should not be construed as limiting the present application and that changes, modifications, substitutions and alterations in the above embodiments may be made by those of ordinary skill in the art within the scope of the present application.

Claims (24)

1. A calibration method of a vehicle-mounted all-round system is characterized by comprising the following steps:
acquiring an image of a calibration object acquired by a vehicle-mounted camera, wherein the calibration object comprises a plurality of areas, and at least one area comprises a pattern containing identification information;
identifying identification information of the pattern and two-dimensional coordinates of corner points of the pattern;
and determining the calibration external parameters of the camera according to the identification information of the pattern and the two-dimensional coordinates of the corner points.
2. The calibration method according to claim 1, wherein the plurality of regions comprise: a plurality of first areas corresponding to the image edges and a plurality of second areas arranged at intervals with the first areas in the calibration object, wherein the first areas and the second areas comprise at least one pattern.
3. Calibration method according to claim 2, characterized in that the density of the pattern of the first area is different from the density of the pattern of the second area.
4. Calibration method according to claim 2, characterized in that the size of the pattern of the first area is different from the size of the pattern of the second area.
5. The calibration method according to claim 1, wherein the identification information of the identification pattern and the two-dimensional coordinates of the corner points of the pattern comprise:
identifying the pattern in the image;
and when the recognition rate of the pattern in the image is lower than a preset recognition rate threshold, re-acquiring the image or overlapping recognition results of the pattern of multiple frames of the image for recognition until the recognition rate reaches the preset recognition rate threshold.
6. The calibration method according to claim 2, further comprising:
identifying the pattern of no less than three of the regions in the image, the three of the regions including at least one of the first region and at least one of the second region.
7. The calibration method according to claim 1, wherein before determining the calibration external parameters of the camera according to the identification information of the pattern and the two-dimensional coordinates of the corner points, the method further comprises:
and carrying out distortion removal processing on the two-dimensional coordinates of the corner points.
8. The calibration method according to claim 1, wherein said determining calibration external parameters of said camera according to said pattern identification information and said two-dimensional coordinates of said corner points comprises:
and determining the calibration external parameters of the camera according to the identification information of the pattern, the two-dimensional coordinates of the corner points and the three-dimensional coordinates of the corresponding points in the scale pattern.
9. The calibration method according to claim 1, wherein said determining calibration external parameters of the camera according to the identification information of the pattern and the two-dimensional coordinates of the corner points comprises:
determining candidate calibration external parameters of the camera corresponding to a single area in the image according to the identification information of the pattern and the two-dimensional coordinates of the corner points;
determining the projection error of the corner point according to the candidate calibration external parameters, the identification information of the pattern and the two-dimensional coordinates of the corner point;
and if the projection error is smaller than a preset projection error threshold value, determining the calibration external parameters of the camera according to the identification information of the pattern and the two-dimensional coordinates of the corner points.
10. The calibration method according to claim 9, further comprising:
and if the projection error is equal to or larger than the projection error threshold value, outputting reminding information, wherein the reminding information is used for reminding a user to replace the calibration site or re-lay the calibration object.
11. A calibration device of a vehicle-mounted all-round system is characterized by comprising:
the system comprises an acquisition module, a processing module and a display module, wherein the acquisition module is used for acquiring an image of a calibration object acquired by a vehicle-mounted camera, the calibration object comprises a plurality of areas, and at least one area comprises a pattern containing identification information;
the identification module is used for identifying the identification information of the pattern and the two-dimensional coordinates of the corner points of the pattern;
and the determining module is used for determining the calibration external parameters of the camera according to the identification information of the pattern and the two-dimensional coordinates of the corner points.
12. The calibration device according to claim 11, wherein the identification module is specifically configured to:
identifying the pattern in the image;
and when the recognition rate of the pattern in the image is lower than a preset recognition rate threshold, re-acquiring the image or overlapping recognition results of the pattern of multiple frames of the image for recognition until the recognition rate reaches the preset recognition rate threshold.
13. The calibration apparatus according to claim 11, wherein the determination module is further configured to:
and before determining the calibration external reference of the camera according to the two-dimensional coordinates of the corner points, carrying out distortion removal processing on the two-dimensional coordinates of the corner points.
14. The calibration device according to claim 11, wherein the determination module is specifically configured to:
and determining the calibration external parameters of the camera according to the identification information of the pattern, the two-dimensional coordinates of the corner points and the three-dimensional coordinates of corresponding points in the scale pattern.
15. The calibration device according to claim 11, wherein the determination module is specifically configured to:
determining candidate calibration external parameters of the camera corresponding to a single area in the image according to the identification information of the pattern and the two-dimensional coordinates of the corner points;
determining the projection error of the corner point according to the candidate calibration external parameters, the identification information of the pattern and the two-dimensional coordinates of the corner point;
and if the projection error is smaller than a preset projection error threshold value, determining the calibration external parameters of the camera according to the identification information of the pattern and the two-dimensional coordinates of the corner points.
16. The calibration device of claim 15, wherein the determining module is further configured to:
and if the projection error is equal to or larger than the projection error threshold value, outputting reminding information, wherein the reminding information is used for reminding a user to replace the calibration site or re-lay the calibration object.
17. A vehicle, characterized by comprising: calibration arrangement for a vehicle mounted surround view system according to any of the claims 11-16.
18. A calibration object for calibrating a vehicle-mounted all-round system is characterized by comprising a plurality of first areas and a plurality of second areas, wherein the first areas are located in the calibration object and correspond to the edges of an image, the second areas are arranged at intervals with the first areas, and the first areas and the second areas comprise at least one pattern containing identification information.
19. The target of claim 18, wherein the pattern of the first area has a density that is different from a density of the pattern of the second area.
20. The target of claim 18, wherein the pattern of the first area is a different size than the pattern of the second area.
21. A vehicle characterized in that its onboard vision system is calibrated by acquiring images of a calibration object according to any one of claims 18-20.
22. An electronic device, comprising: memory, processor and computer program stored on the memory and executable on the processor, which when executed by the processor implements a method of calibrating an in-vehicle surround view system according to any of claims 1 to 10.
23. A vehicle characterized by comprising the electronic apparatus of claim 22.
24. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out a method for calibrating a vehicle mounted surround view system according to any one of claims 1 to 10.
CN202111387379.2A 2021-11-22 2021-11-22 Calibration method and device of vehicle-mounted all-round looking system Pending CN115471563A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111387379.2A CN115471563A (en) 2021-11-22 2021-11-22 Calibration method and device of vehicle-mounted all-round looking system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111387379.2A CN115471563A (en) 2021-11-22 2021-11-22 Calibration method and device of vehicle-mounted all-round looking system

Publications (1)

Publication Number Publication Date
CN115471563A true CN115471563A (en) 2022-12-13

Family

ID=84363433

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111387379.2A Pending CN115471563A (en) 2021-11-22 2021-11-22 Calibration method and device of vehicle-mounted all-round looking system

Country Status (1)

Country Link
CN (1) CN115471563A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116202424A (en) * 2023-04-28 2023-06-02 深圳一清创新科技有限公司 Vehicle body area detection method, tractor and tractor obstacle avoidance system

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116202424A (en) * 2023-04-28 2023-06-02 深圳一清创新科技有限公司 Vehicle body area detection method, tractor and tractor obstacle avoidance system
CN116202424B (en) * 2023-04-28 2023-08-04 深圳一清创新科技有限公司 Vehicle body area detection method, tractor and tractor obstacle avoidance system

Similar Documents

Publication Publication Date Title
CN112907676B (en) Calibration method, device and system of sensor, vehicle, equipment and storage medium
CN110264520B (en) Vehicle-mounted sensor and vehicle pose relation calibration method, device, equipment and medium
CN110163930B (en) Lane line generation method, device, equipment, system and readable storage medium
CN112270713B (en) Calibration method and device, storage medium and electronic device
CN108805934B (en) External parameter calibration method and device for vehicle-mounted camera
CN113592950B (en) Multi-camera calibration method and related equipment in large space environment based on optical dynamic capturing
CN112785656B (en) Calibration method and device of dual-stereoscopic camera, electronic equipment and storage medium
US20220276339A1 (en) Calibration method and apparatus for sensor, and calibration system
CN111539484B (en) Method and device for training neural network
CN112907675B (en) Calibration method, device, system, equipment and storage medium of image acquisition equipment
CN112465970B (en) Navigation map construction method, device, system, electronic device and storage medium
CN111815707A (en) Point cloud determining method, point cloud screening device and computer equipment
CN111383279A (en) External parameter calibration method and device and electronic equipment
CN109918977A (en) Determine the method, device and equipment of free time parking stall
CN110962844A (en) Vehicle course angle correction method and system, storage medium and terminal
CN111932627B (en) Marker drawing method and system
CN104167109A (en) Detection method and detection apparatus for vehicle position
CN107145828B (en) Vehicle panoramic image processing method and device
CN113658262A (en) Camera external parameter calibration method, device, system and storage medium
CN111383264A (en) Positioning method, positioning device, terminal and computer storage medium
CN112465831A (en) Curve scene perception method, system and device based on binocular stereo camera
CN115471563A (en) Calibration method and device of vehicle-mounted all-round looking system
CN117315046A (en) Method and device for calibrating looking-around camera, electronic equipment and storage medium
CN114445415B (en) Method for dividing drivable region and related device
CN115082565A (en) Camera calibration method, device, server and medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination