CN107886472B - Image splicing calibration method and image splicing calibration device of panoramic parking system - Google Patents

Image splicing calibration method and image splicing calibration device of panoramic parking system Download PDF

Info

Publication number
CN107886472B
CN107886472B CN201610873420.XA CN201610873420A CN107886472B CN 107886472 B CN107886472 B CN 107886472B CN 201610873420 A CN201610873420 A CN 201610873420A CN 107886472 B CN107886472 B CN 107886472B
Authority
CN
China
Prior art keywords
calibration
acquisition device
image acquisition
cloth
calibration cloth
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201610873420.XA
Other languages
Chinese (zh)
Other versions
CN107886472A (en
Inventor
周绍辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
SHENZHEN ROADROVER TECHNOLOGY CO LTD
Original Assignee
SHENZHEN ROADROVER TECHNOLOGY CO LTD
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by SHENZHEN ROADROVER TECHNOLOGY CO LTD filed Critical SHENZHEN ROADROVER TECHNOLOGY CO LTD
Priority to CN201610873420.XA priority Critical patent/CN107886472B/en
Publication of CN107886472A publication Critical patent/CN107886472A/en
Application granted granted Critical
Publication of CN107886472B publication Critical patent/CN107886472B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformation in the plane of the image
    • G06T3/40—Scaling the whole image or part thereof
    • G06T3/4038—Scaling the whole image or part thereof for image mosaicing, i.e. plane images composed of plane sub-images
    • G06T5/80—

Abstract

The invention discloses an image splicing calibration method and device of a panoramic parking system, which determine the positions of a third calibration cloth and a fourth calibration cloth in a world coordinate system by using the width of a vehicle body, determine the position of the first calibration cloth in the world coordinate system by using images acquired by a left image acquisition device and a right image acquisition device when the vehicle is at a first position, determine the position of the second calibration cloth in the world coordinate system by using images acquired by the left image acquisition device and the right image acquisition device when the vehicle is at a second position, determine the mounting information of each image acquisition device by using the positions of the four calibration cloths in the world coordinate system, determine the corresponding pixel points of each pixel point in the panoramic parking image in the images acquired by each image acquisition device based on the mounting information, establish and store a mapping relationship, and complete the image splicing calibration. Based on the scheme disclosed by the invention, the operation difficulty of a user and the probability of operation deviation can be reduced, and the accuracy of image splicing calibration is improved.

Description

Image splicing calibration method and image splicing calibration device of panoramic parking system
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to an image stitching calibration method and an image stitching calibration device for a panoramic parking system.
Background
The panoramic parking system is a novel vehicle-mounted auxiliary device, and is characterized in that 4 cameras (such as ultra-wide angle fisheye cameras) arranged at the front, the rear, the left and the right of a vehicle simultaneously collect real-time images around the vehicle, and a 360-degree panoramic view of the vehicle is finally formed by carrying out distortion reduction, visual angle conversion, image splicing and image enhancement on the images. In the parking process, a driver can know the sight blind areas around the vehicle through the panoramic parking system, so that the parking process is simpler.
Before the panoramic parking system is put into use, images shot by 4 cameras need to be spliced and calibrated, so that the images shot by the 4 cameras can be accurately combined into a 360-degree panoramic image. At present, two image splicing calibration modes for a panoramic parking system are mainly provided.
The first method comprises the following steps: the method comprises the steps of placing calibration cloths around a vehicle, wherein the calibration cloths placed around the vehicle are strictly parallel or perpendicular to the longitudinal center line of the vehicle, accurately measuring the relative distance between each calibration cloth and the vehicle, and carrying out image splicing calibration by a panoramic parking system according to the relative distance.
And the second method comprises the following steps: the method comprises the steps that calibration cloth is placed around a vehicle, wherein the calibration cloth placed around the vehicle is strictly parallel to or perpendicular to a longitudinal center line (which is a center line connecting a head and a tail of the vehicle) of the vehicle, so that the calibration cloth can be covered by a visual angle of a camera, calibration points are manually selected for images collected by the cameras on a display interface of the panoramic parking system respectively, and if the selected calibration points are accurate, the panoramic parking system can perform image splicing calibration on the images collected by the 4 cameras.
The image splicing calibration mode has strict position requirements on the placement calibration cloth of a user, the user is required to accurately measure the relative distance between the calibration cloth and a vehicle, or the user is required to manually select a calibration point, when any operation has deviation, the image splicing is caused to have problems, the operation of the image splicing calibration process is complicated, and the difficulty is high.
Disclosure of Invention
In view of the above, the present invention provides an image stitching calibration method and an image stitching calibration apparatus for a panoramic parking system, so as to reduce the operation difficulty of a user in the image stitching calibration process.
In order to achieve the purpose, the invention provides the following technical scheme:
the invention provides an image splicing calibration method of a panoramic parking system, wherein the panoramic parking system comprises a forward image acquisition device, a backward image acquisition device, a left image acquisition device and a right image acquisition device, and the image splicing calibration method comprises the following steps:
under the condition that the vehicle is located at an initial position, placing a first calibration cloth, a second calibration cloth, a third calibration cloth and a fourth calibration cloth on the front side, the rear side, the left side and the right side of the vehicle respectively, wherein a midpoint of a right side of the third calibration cloth and a midpoint of a left side of the fourth calibration cloth are aligned with a midpoint of a wheel track of the vehicle, the first calibration cloth and the second calibration cloth respectively comprise a middle area, a left area and a right area which are divided in advance, the forward image acquisition device can acquire an image of the first calibration cloth, the backward image acquisition device can acquire an image of the second calibration cloth, the left image acquisition device can acquire an image of the third calibration cloth, and the right image acquisition device can acquire an image of the fourth calibration cloth;
determining the positions of the third calibration cloth and the fourth calibration cloth in a pre-constructed world coordinate system according to the width of the vehicle body of the vehicle, wherein the origin of the world coordinate system is the central point of a rectangle formed by the central points of four wheels of the vehicle;
controlling the vehicle to straightly move forwards to a first position, acquiring images of left areas of the third calibration cloth and the first calibration cloth currently acquired by the left image acquisition device, and acquiring images of right areas of the fourth calibration cloth and the first calibration cloth currently acquired by the right image acquisition device;
analyzing the images currently acquired by the left image acquisition device and the right image acquisition device, and determining the position relationship of the first calibration cloth relative to the third calibration cloth and the fourth calibration cloth;
determining the position of the first calibration cloth in the world coordinate system according to the positions of the third calibration cloth and the fourth calibration cloth in the world coordinate system and the position relation of the first calibration cloth relative to the third calibration cloth and the fourth calibration cloth;
controlling the vehicle to move to a second position straightly and backwards, acquiring images of left areas of the third calibration cloth and the second calibration cloth currently acquired by the left image acquisition device, and acquiring images of right areas of the fourth calibration cloth and the second calibration cloth currently acquired by the right image acquisition device;
analyzing the images currently acquired by the left image acquisition device and the right image acquisition device, and determining the position relationship of the second calibration cloth relative to the third calibration cloth and the fourth calibration cloth;
determining the position of the second calibration cloth in the world coordinate system according to the positions of the third calibration cloth and the fourth calibration cloth in the world coordinate system and the position relation of the second calibration cloth relative to the third calibration cloth and the fourth calibration cloth;
determining installation information of the forward image acquisition device, the backward image acquisition device, the left image acquisition device and the right image acquisition device on a vehicle body according to the positions of the first calibration cloth, the second calibration cloth, the third calibration cloth and the fourth calibration cloth in the world coordinate system, wherein the installation information comprises installation positions and installation angles;
and determining corresponding pixel points of all pixel points in the panoramic parking image in the images acquired by the forward image acquisition device, the backward image acquisition device, the left image acquisition device and the right image acquisition device by using the installation information of the forward image acquisition device, the backward image acquisition device, the left image acquisition device and the right image acquisition device, and establishing and storing the mapping relation of the corresponding pixel points.
Preferably, in the image stitching calibration method, the pre-established world coordinate system has an origin which is a central point of a rectangle formed by central points of four wheels of the vehicle, a positive direction of an X axis points to the right side of the vehicle, a positive direction of a Y axis points to the rear side of the vehicle, and a positive direction of a Z axis is vertically upward;
the determining the positions of the third calibration cloth and the fourth calibration cloth in a pre-constructed world coordinate system according to the width of the vehicle body comprises the following steps:
determining the coordinates of the middle point of the right side of the third calibration cloth in the world coordinate system according to the width of the vehicle body, wherein the coordinates of the middle point of the right side of the third calibration cloth in the world coordinate system are the positions of the third calibration cloth in the world coordinate system;
and determining the coordinates of the middle point of the left side of the fourth calibration cloth in the world coordinate system according to the width of the vehicle body, wherein the coordinates of the middle point of the left side of the fourth calibration cloth in the world coordinate system are the positions of the fourth calibration cloth in the world coordinate system.
Preferably, in the image stitching calibration method, the analyzing images acquired by the left image acquisition device and the right image acquisition device to determine the position relationship of the first calibration cloth with respect to the third calibration cloth and the fourth calibration cloth includes:
analyzing the images of the left areas of the third calibration cloth and the first calibration cloth acquired by the left image acquisition device, determining the distance from a first calibration point of the third calibration cloth to a second calibration point of the left area of the first calibration cloth, analyzing the images of the right areas of the fourth calibration cloth and the first calibration cloth acquired by the right image acquisition device, and determining the distance from a third calibration point of the fourth calibration cloth to a fourth calibration point of the right area of the first calibration cloth;
the determining the position of the first calibration cloth in the world coordinate system according to the positions of the third calibration cloth and the fourth calibration cloth in the world coordinate system and the position relation of the first calibration cloth relative to the third calibration cloth and the fourth calibration cloth comprises:
and determining the position of the first calibration cloth in the world coordinate system according to the positions of the third calibration cloth and the fourth calibration cloth in the world coordinate system, the distance from the first calibration point to the second calibration point, the distance from the third calibration point to the fourth calibration point and the distance from the second calibration point to the fourth calibration point.
Preferably, in the image stitching calibration method, the analyzing images acquired by the left image acquisition device and the right image acquisition device and determining the position relationship of the second calibration cloth with respect to the third calibration cloth and the fourth calibration cloth includes:
analyzing the images of the left areas of the third calibration cloth and the second calibration cloth acquired by the left image acquisition device, determining the distance from a fifth calibration point of the third calibration cloth to a sixth calibration point of the left area of the second calibration cloth, analyzing the images of the right areas of the fourth calibration cloth and the second calibration cloth acquired by the right image acquisition device, and determining the distance from a seventh calibration point of the fourth calibration cloth to an eighth calibration point of the right area of the second calibration cloth;
the determining the position of the second calibration cloth in the world coordinate system according to the positions of the third calibration cloth and the fourth calibration cloth in the world coordinate system and the position relationship of the second calibration cloth relative to the third calibration cloth and the fourth calibration cloth comprises:
and determining the position of the second calibration cloth in the world coordinate system according to the positions of the third calibration cloth and the fourth calibration cloth in the world coordinate system, the distance from the fifth calibration point to the sixth calibration point, the distance from the seventh calibration point to the eighth calibration point and the distance from the sixth calibration point to the eighth calibration point.
Preferably, in the image stitching calibration method, the first calibration point and the fifth calibration point of the third calibration cloth are central points of the third calibration cloth; and the third calibration point and the seventh calibration point of the fourth calibration cloth are central points of the fourth calibration cloth.
Preferably, in the image stitching calibration method, the determining, according to the positions of the first calibration cloth, the second calibration cloth, the third calibration cloth, and the fourth calibration cloth in the world coordinate system, the installation information of the forward image capturing device, the backward image capturing device, the left image capturing device, and the right image capturing device on the vehicle body includes:
determining a first actual position of a preset corner point of the first calibration cloth in a first calibration cloth image acquired by the forward image acquisition device, a second actual position of a preset corner point of the second calibration cloth in a second calibration cloth image acquired by the backward image acquisition device, a third actual position of a preset corner point of the third calibration cloth in a third calibration cloth image acquired by the leftward image acquisition device, and a fourth actual position of a preset corner point of the fourth calibration cloth in a fourth calibration cloth image acquired by the rightward image acquisition device under the condition that the vehicle is at the preset position;
setting installation information of the forward image acquisition device; calculating a first virtual position of a preset corner point of a first calibration cloth in a virtual image of the first calibration cloth generated by the forward image acquisition device in an installation state according to currently set installation information under the condition that a vehicle is at the preset position according to the position of the forward image acquisition device in the world coordinate system; if the difference value between the first actual position and the first virtual position is smaller than a forward minimum value, the currently set installation information of the forward image acquisition device is the actual installation information of the forward image acquisition device; if the difference value between the first actual position and the first virtual position is larger than or equal to the forward minimum value, resetting the installation information of the forward image acquisition device, and executing subsequent steps until the actual installation information of the forward image acquisition device is determined;
setting installation information of the backward image acquisition device; calculating a second virtual position of a preset corner point of a second calibration cloth in a virtual image of the second calibration cloth generated by the backward image acquisition device in an installation state according to currently set installation information under the condition that the vehicle is at the preset position according to the position of the backward image acquisition device in the world coordinate system; if the difference value between the second actual position and the second virtual position is smaller than the backward minimum value, the currently set installation information of the backward image acquisition device is the actual installation information of the backward image acquisition device; if the difference value between the second actual position and the second virtual position is larger than or equal to the backward minimum value, resetting the installation information of the backward image acquisition device, and executing subsequent steps until the actual installation information of the backward image acquisition device is determined;
setting installation information of the left image acquisition device; calculating a third virtual position of a preset corner point of a third calibration cloth in a virtual image of the third calibration cloth generated by the left image acquisition device in an installation state according to currently set installation information under the condition that the vehicle is at the preset position according to the position of the left image acquisition device in the world coordinate system; if the difference value between the third actual position and the third virtual position is smaller than a minimum value in the left direction, the currently set installation information of the left image acquisition device is the actual installation information of the left image acquisition device; if the difference value between the third actual position and the third virtual position is larger than or equal to the minimum value in the left direction, resetting the installation information of the left image acquisition device, and executing subsequent steps until the actual installation information of the left image acquisition device is determined;
setting installation information of the right image acquisition device; according to the position of the right image acquisition device in the world coordinate system, under the condition that the vehicle is at the preset position, calculating a fourth virtual position of a preset corner point of a fourth calibration cloth in a virtual image of the fourth calibration cloth generated by the right image acquisition device in the installation state according to the currently set installation information; if the difference value between the fourth actual position and the fourth virtual position is smaller than the minimum right value, the currently set installation information of the right image acquisition device is the actual installation information of the right image acquisition device; if the difference value between the fourth actual position and the fourth virtual position is larger than or equal to the right minimum numerical value, resetting the installation information of the right image acquisition device, and executing subsequent steps until the actual installation information of the right image acquisition device is determined.
The invention also provides an image stitching calibration device of the panoramic parking system, the panoramic parking system comprises a forward image acquisition device, a backward image acquisition device, a left image acquisition device and a right image acquisition device, and the image stitching calibration device comprises:
a first position determination unit, which determines the positions of the third calibration cloth and the fourth calibration cloth in a pre-constructed world coordinate system according to the width of the vehicle body when the vehicle is at an initial position, wherein the origin of the world coordinate system is the central point of a rectangle formed by the central points of four wheels of the vehicle; under the condition that the vehicle is located at an initial position, placing a first calibration cloth, a second calibration cloth, a third calibration cloth and a fourth calibration cloth on the front side, the rear side, the left side and the right side of the vehicle respectively, wherein a midpoint of a right side of the third calibration cloth and a midpoint of a left side of the fourth calibration cloth are aligned with a midpoint of a wheel track of the vehicle, the first calibration cloth and the second calibration cloth respectively comprise a middle area, a left area and a right area which are divided in advance, the forward image acquisition device can acquire an image of the first calibration cloth, the backward image acquisition device can acquire an image of the second calibration cloth, the left image acquisition device can acquire an image of the third calibration cloth, and the right image acquisition device can acquire an image of the fourth calibration cloth;
a first position relation determining unit, configured to, when the vehicle travels straight forward to a first position, acquire images of left areas of the third calibration cloth and the first calibration cloth currently acquired by the left image acquisition device, acquire images of right areas of the fourth calibration cloth and the first calibration cloth currently acquired by the right image acquisition device, analyze the images currently acquired by the left image acquisition device and the right image acquisition device, and determine a position relation of the first calibration cloth with respect to the third calibration cloth and the fourth calibration cloth;
the second position determining unit is used for determining the position of the first calibration cloth in the world coordinate system according to the positions of the third calibration cloth and the fourth calibration cloth in the world coordinate system and the position relation of the first calibration cloth relative to the third calibration cloth and the fourth calibration cloth;
a second position relation determining unit, configured to, when the vehicle travels straight backward to a second position, acquire images of left areas of the third calibration cloth and the second calibration cloth currently acquired by the left image acquisition device, acquire images of right areas of the fourth calibration cloth and the second calibration cloth currently acquired by the right image acquisition device, analyze the images currently acquired by the left image acquisition device and the right image acquisition device, and determine a position relation of the second calibration cloth with respect to the third calibration cloth and the fourth calibration cloth;
a third position determining unit, configured to determine a position of the second calibration cloth in the world coordinate system according to positions of the third calibration cloth and the fourth calibration cloth in the world coordinate system and a positional relationship of the second calibration cloth with respect to the third calibration cloth and the fourth calibration cloth;
the installation information determining unit is used for determining the installation information of the forward image acquisition device, the backward image acquisition device, the left image acquisition device and the right image acquisition device on a vehicle body according to the positions of the first calibration cloth, the second calibration cloth, the third calibration cloth and the fourth calibration cloth in the world coordinate system, wherein the installation information comprises installation positions and installation angles;
and the calibration unit is used for determining corresponding pixel points of all pixel points in the panoramic parking image in the images acquired by the forward image acquisition device, the backward image acquisition device, the left image acquisition device and the right image acquisition device by using the installation information of the forward image acquisition device, the backward image acquisition device, the left image acquisition device and the right image acquisition device, and establishing and storing the mapping relation of the corresponding pixel points.
Preferably, in the image stitching calibration apparatus, the pre-established world coordinate system has an origin that is a center point of a rectangle formed by center points of four wheels of the vehicle, a positive direction of an X axis points to a right side of the vehicle, a positive direction of a Y axis points to a rear side of the vehicle, and a positive direction of a Z axis is vertically upward;
the first position determination unit is specifically configured to: determining the coordinates of the middle point of the right side of the third calibration cloth in the world coordinate system according to the width of the vehicle body, wherein the coordinates of the middle point of the right side of the third calibration cloth in the world coordinate system are the positions of the third calibration cloth in the world coordinate system; and determining the coordinates of the middle point of the left side of the fourth calibration cloth in the world coordinate system according to the width of the vehicle body, wherein the coordinates of the middle point of the left side of the fourth calibration cloth in the world coordinate system are the positions of the fourth calibration cloth in the world coordinate system.
Preferably, in the image stitching calibration apparatus, the first position relationship determining unit is specifically configured to: analyzing images of left areas of the third calibration cloth and the first calibration cloth collected by the left image acquisition device, determining the distance from a first calibration point of the third calibration cloth to a second calibration point of the left area of the first calibration cloth, analyzing images of right areas of the fourth calibration cloth and the first calibration cloth collected by the right image acquisition device, and determining the distance from a third calibration point of the fourth calibration cloth to a fourth calibration point of the right area of the first calibration cloth;
the second position determination unit is specifically configured to: and determining the position of the first calibration cloth in the world coordinate system according to the positions of the third calibration cloth and the fourth calibration cloth in the world coordinate system, the distance from the first calibration point to the second calibration point, the distance from the third calibration point to the fourth calibration point and the distance from the second calibration point to the fourth calibration point.
Preferably, in the image stitching calibration apparatus, the second positional relationship determination unit is specifically configured to: analyzing images of left areas of the third calibration cloth and the second calibration cloth collected by the left image acquisition device, determining the distance from a fifth calibration point of the third calibration cloth to a sixth calibration point of the left area of the second calibration cloth, analyzing images of right areas of the fourth calibration cloth and the second calibration cloth collected by the right image acquisition device, and determining the distance from a seventh calibration point of the fourth calibration cloth to an eighth calibration point of the right area of the second calibration cloth;
the third position determination unit is specifically configured to: and determining the position of the second calibration cloth in the world coordinate system according to the positions of the third calibration cloth and the fourth calibration cloth in the world coordinate system, the distance from the fifth calibration point to the sixth calibration point, the distance from the seventh calibration point to the eighth calibration point and the distance from the sixth calibration point to the eighth calibration point.
Preferably, in the image stitching calibration device, the installation information determining unit includes:
the corner actual position determining module is used for determining a first actual position of a preset corner of the first calibration cloth in a first calibration cloth image acquired by the forward image acquisition device, a second actual position of a preset corner of the second calibration cloth in a second calibration cloth image acquired by the backward image acquisition device, a third actual position of a preset corner of the third calibration cloth in a third calibration cloth image acquired by the leftward image acquisition device, and a fourth actual position of a preset corner of the fourth calibration cloth in a fourth calibration cloth image acquired by the rightward image acquisition device under the condition that the vehicle is at the preset position;
the first processing module is used for setting the installation information of the forward image acquisition device; calculating a first virtual position of a preset corner point of a first calibration cloth in a virtual image of the first calibration cloth generated by the forward image acquisition device in an installation state according to currently set installation information under the condition that a vehicle is at the preset position according to the position of the forward image acquisition device in the world coordinate system; if the difference value between the first actual position and the first virtual position is smaller than a forward minimum value, the currently set installation information of the forward image acquisition device is the actual installation information of the forward image acquisition device; if the difference value between the first actual position and the first virtual position is larger than or equal to the forward minimum value, resetting the installation information of the forward image acquisition device, and executing subsequent operation until the actual installation information of the forward image acquisition device is determined;
the second processing module is used for setting the installation information of the backward image acquisition device; calculating a second virtual position of a preset corner point of a second calibration cloth in a virtual image of the second calibration cloth generated by the backward image acquisition device in an installation state according to currently set installation information under the condition that the vehicle is at the preset position according to the position of the backward image acquisition device in the world coordinate system; if the difference value between the second actual position and the second virtual position is smaller than the backward minimum value, the currently set installation information of the backward image acquisition device is the actual installation information of the backward image acquisition device; if the difference value between the second actual position and the second virtual position is larger than or equal to the backward minimum value, resetting the installation information of the backward image acquisition device, and executing subsequent operation until the actual installation information of the backward image acquisition device is determined;
the third processing module is used for setting the installation information of the left image acquisition device; calculating a third virtual position of a preset corner point of a third calibration cloth in a virtual image of the third calibration cloth generated by the left image acquisition device in an installation state according to currently set installation information under the condition that the vehicle is at the preset position according to the position of the left image acquisition device in the world coordinate system; if the difference value between the third actual position and the third virtual position is smaller than a minimum value in the left direction, the currently set installation information of the left image acquisition device is the actual installation information of the left image acquisition device; if the difference value between the third actual position and the third virtual position is larger than or equal to the minimum value in the left direction, resetting the installation information of the left image acquisition device, and executing subsequent operations until the actual installation information of the left image acquisition device is determined;
the fourth processing module is used for setting the installation information of the right image acquisition device; calculating a fourth virtual position of a preset corner point of a fourth calibration cloth in a virtual image of the fourth calibration cloth generated by the right image acquisition device in an installation state according to currently set installation information under the condition that the vehicle is at the preset position according to the position of the right image acquisition device in the world coordinate system; if the difference value between the fourth actual position and the fourth virtual position is smaller than the minimum right value, the currently set installation information of the right image acquisition device is the actual installation information of the right image acquisition device; and if the difference value between the fourth actual position and the fourth virtual position is larger than or equal to the right minimum numerical value, resetting the installation information of the right image acquisition device, and executing subsequent operations until the actual installation information of the right image acquisition device is determined.
Therefore, the beneficial effects of the invention are as follows:
based on the image splicing method of the panoramic parking system disclosed by the invention, a user needs to respectively place a first calibration cloth, a second calibration cloth, a third calibration cloth and a fourth calibration cloth on the front, back, left and right sides of a vehicle when the vehicle is positioned at an initial position, so that a forward image acquisition device, a backward image acquisition device, a left image acquisition device and a right image acquisition device in the panoramic parking system can respectively acquire images of the first calibration cloth, the second calibration cloth, the third calibration cloth and the fourth calibration cloth, a midpoint of a right side of the third calibration cloth and a midpoint of a left side of the fourth calibration cloth are aligned with a midpoint of a wheel distance of the vehicle, the panoramic parking system determines positions of the third calibration cloth and the fourth calibration cloth in a pre-constructed world coordinate system by using a vehicle body width, the panoramic parking system determines images acquired by the left image acquisition device and the right image acquisition device when the vehicle is positioned at the first position, determines positions of the first calibration cloth in the world coordinate system by using images acquired by the left image acquisition device and the right image acquisition device, and determines corresponding pixel points of the panoramic image acquisition device in the panoramic parking system based on the corresponding mounting information of the panoramic image acquisition device, and the panoramic parking system.
For a user, before the panoramic parking system performs image splicing calibration, only a first calibration cloth, a second calibration cloth, a third calibration cloth and a fourth calibration cloth need to be respectively placed on the front side, the rear side, the left side and the right side of a vehicle, and in the process of placing the calibration cloths, only the middle point of the right side of the third calibration cloth and the middle point of the left side of the fourth calibration cloth need to be aligned with the middle point of the wheel track of the vehicle, but the calibration cloths on the periphery of the vehicle do not need to be strictly parallel or perpendicular to the longitudinal center line of the vehicle; in the process of image splicing and calibration of the panoramic parking system, a user only needs to control the vehicle to move to the first position and the second position, and does not need to manually select a calibration point for images acquired by the image acquisition devices. The image splicing calibration method based on the panoramic parking system disclosed by the invention can be seen to reduce the operation difficulty of a user, and correspondingly, the probability of deviation of the user operation can be reduced, so that the accuracy of image splicing calibration is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the prior art descriptions will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a flowchart of an image stitching calibration method for a panoramic parking system according to the present invention;
FIG. 2 is a schematic view of the placement of the calibration cloth according to the present invention;
FIG. 3 is a schematic illustration of the present invention with the vehicle in a first position;
FIG. 4 is a schematic view of the present invention with the vehicle in a second position;
FIG. 5 is a schematic view of the distance between the calibration points with the vehicle in the first position according to the present invention;
FIG. 6 is a schematic structural diagram of an image stitching calibration apparatus of a panoramic parking system according to the present invention;
fig. 7 is a schematic structural diagram of an installation information determining unit according to the present disclosure.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention discloses an image splicing calibration method and an image splicing calibration device of a panoramic parking system, which are used for reducing the operation difficulty of a user in the image splicing calibration process. The panoramic parking system comprises a forward image acquisition device, a backward image acquisition device, a left image acquisition device and a right image acquisition device.
Referring to fig. 1, fig. 1 is a flowchart of an image stitching calibration method of a panoramic parking system according to the present invention. The image stitching calibration method comprises the following steps:
step S1: in the case where the vehicle is located at the initial position, calibration cloths are respectively laid on the front side, the rear side, the left side, and the right side of the vehicle.
Specifically, the calibration cloth placed on the front side of the vehicle is referred to as a first calibration cloth 101, the calibration cloth placed on the rear side of the vehicle is referred to as a second calibration cloth 102, the calibration cloth placed on the left side of the vehicle is referred to as a third calibration cloth 103, and the calibration cloth placed on the right side of the vehicle is referred to as a fourth calibration cloth 104, as shown in fig. 2.
Wherein, the middle point of the right side of the third calibration cloth 103 and the middle point of the left side of the fourth calibration cloth 104 are aligned with the middle point of the wheel track of the vehicle. The first calibration cloth 101 includes a pre-divided middle area 101a, a left area 101b, and a right area 101c, respectively, and the second calibration cloth includes a pre-divided middle area 102a, a left area 102b, and a right area 102c, respectively.
Under the condition that the vehicle is located initial position, the image of first calibration cloth 101 can be gathered to the preceding image acquisition device, and the image of second calibration cloth 102 can be gathered to the image acquisition device that is followed to the back, and the image acquisition device that is followed to the left can gather the image of third calibration cloth 103, and the image acquisition device that is followed to the right can gather the image of fourth calibration cloth 104.
Step S2: the positions of the third calibration cloth 103 and the fourth calibration cloth 104 in the pre-constructed world coordinate system are determined according to the width of the vehicle body. Wherein the origin of the pre-constructed world coordinate system is: the center points of the four wheels of the vehicle form the center point of a rectangle.
The width of the vehicle body is known, and the midpoint of the right side of the third calibration cloth 103 and the midpoint of the left side of the fourth calibration cloth 104 are aligned with the midpoint of the track of the vehicle, so that the positions of the third calibration cloth 103 and the fourth calibration cloth 104 in the world coordinate system constructed in advance can be determined according to the width of the vehicle body.
And step S3: and controlling the vehicle to straightly move forwards to a first position, acquiring images of the left area 101a of the third calibration cloth 103 and the first calibration cloth 101 currently acquired by the left image acquisition device, and acquiring images of the right area 101c of the fourth calibration cloth 104 and the first calibration cloth 101 currently acquired by the right image acquisition device.
When the vehicle is located at the first position, the left image capturing device can simultaneously capture images of the left regions 101a of the third calibration cloth 103 and the first calibration cloth 101, and the right image capturing device can simultaneously capture images of the right regions 101c of the fourth calibration cloth 104 and the first calibration cloth 101, as shown in fig. 3.
And step S4: the images currently acquired by the left image acquisition device and the right image acquisition device are analyzed, and the position relation of the first calibration cloth 101 relative to the third calibration cloth 103 and the fourth calibration cloth 104 is determined.
After the vehicle is controlled to straightly move forwards to the first position, the image collected by the left image collecting device comprises: an image of the third calibration cloth 103, and an image of the left area 101a of the first calibration cloth 101; the image collected by the right image collecting device comprises: an image of the fourth calibration cloth 104, and an image of the right area 101c of the first calibration cloth 101.
The images of the left side area 101a of the third calibration cloth 103 and the first calibration cloth 101 acquired by the left image acquisition device and the images of the right side area 101c of the fourth calibration cloth 104 and the first calibration cloth 101 acquired by the right image acquisition device are analyzed to determine the position relationship of the first calibration cloth 101 with respect to the third calibration cloth 103 and the fourth calibration cloth 104.
Step S5: and determining the position of the first calibration cloth 101 in the world coordinate system according to the positions of the third calibration cloth 103 and the fourth calibration cloth 104 in the world coordinate system and the position relation of the first calibration cloth 101 relative to the third calibration cloth 103 and the fourth calibration cloth 104.
The positions of the third calibration cloth 103 and the fourth calibration cloth 104 in the world coordinate system are determined, and the positional relationship of the first calibration cloth 101 with respect to the third calibration cloth 103 and the fourth calibration cloth 104 is determined in step S4, whereby the position of the first calibration cloth 101 in the world coordinate system can be determined.
Step S6: and controlling the vehicle to straightly move backwards to a second position, acquiring images of the left areas 102a of the third calibration cloth 103 and the second calibration cloth 102 currently acquired by the left image acquisition device, and acquiring images of the right areas 102c of the fourth calibration cloth 104 and the second calibration cloth 102 currently acquired by the right image acquisition device.
When the vehicle is located at the second position, the left image capturing device can simultaneously capture images of the left areas 102a of the third calibration cloth 103 and the second calibration cloth 102, and the right image capturing device can simultaneously capture images of the right areas 102c of the fourth calibration cloth 104 and the second calibration cloth 102, as shown in fig. 4.
Step S7: and analyzing the images currently acquired by the left image acquisition device and the right image acquisition device, and determining the position relation of the second calibration cloth 102 relative to the third calibration cloth 103 and the fourth calibration cloth 104.
After controlling the vehicle to straightly move backwards to the second position, the image collected by the left image collecting device comprises: an image of the third calibration cloth 103, and an image of the left area 102a of the second calibration cloth 102; the image collected by the right image collecting device comprises: an image of the fourth calibration cloth 104, and an image of the right area 102c of the second calibration cloth 102.
The images of the left side areas 102a of the third calibration cloth 103 and the second calibration cloth 102 acquired by the left image acquisition device and the images of the right side areas 102c of the fourth calibration cloth 104 and the second calibration cloth 102 acquired by the right image acquisition device are analyzed to determine the position relationship of the second calibration cloth 102 relative to the third calibration cloth 103 and the fourth calibration cloth 104.
Step S8: and determining the position of the second calibration cloth in the world coordinate system according to the positions of the third calibration cloth and the fourth calibration cloth in the world coordinate system and the position relation of the second calibration cloth relative to the third calibration cloth and the fourth calibration cloth.
The positions of the third calibration cloth 103 and the fourth calibration cloth 104 in the world coordinate system are determined, and the positional relationship of the second calibration cloth 102 with respect to the third calibration cloth 103 and the fourth calibration cloth 104 is determined in step S7, whereby the position of the second calibration cloth 102 in the world coordinate system can be determined.
Here, it should be noted that, in the implementation process, step S6 to step S8 may be executed first, and then step S3 to step S5 may be executed.
Step S9: according to the positions of the first calibration cloth 101, the second calibration cloth 102, the third calibration cloth 103 and the fourth calibration cloth 104 in a world coordinate system, the installation information of the forward image acquisition device, the backward image acquisition device, the left image acquisition device and the right image acquisition device on the vehicle body is determined. Wherein the installation information includes an installation position and an installation angle.
Step S10: and determining corresponding pixel points of all pixel points in the panoramic parking image in the images acquired by the forward image acquisition device, the backward image acquisition device, the left image acquisition device and the right image acquisition device by utilizing the installation information of the forward image acquisition device, the backward image acquisition device, the left image acquisition device and the right image acquisition device, and establishing and storing the mapping relation of the corresponding pixel points.
After the installation information of the forward image acquisition device, the backward image acquisition device, the left image acquisition device and the right image acquisition device is determined, the coordinates of pixel points in the images acquired by the image acquisition devices in a pre-constructed world coordinate system can be determined, namely the positions of the pixel points in the images acquired by the image acquisition devices in the panoramic parking image are determined, so that the corresponding pixel points of the pixel points in the panoramic parking image in the images acquired by the forward image acquisition device, the backward image acquisition device, the left image acquisition device and the right image acquisition device are determined, the mapping relation between the pixel points in the panoramic parking image and the corresponding pixel points in the images acquired by the forward image acquisition device, the backward image acquisition device, the left image acquisition device and the right image acquisition device is established, and the mapping relation is stored.
In implementation, a first three-dimensional coordinate system may be constructed from a central point of a sensor in a forward image acquisition device, a second three-dimensional coordinate system may be constructed from a central point of a sensor in a backward image acquisition device, a third three-dimensional coordinate system may be constructed from a central point of a sensor in a leftward image acquisition device, and a fourth three-dimensional coordinate system may be constructed from a central point of a sensor in a rightward image acquisition device. By performing coordinate conversion, the coordinates of coordinate points in the first three-dimensional coordinate system, the second three-dimensional coordinate system, the third three-dimensional coordinate system and the fourth three-dimensional coordinate system in a pre-constructed world coordinate system can be determined, namely the coordinates of pixel points in the images acquired by the image acquisition devices in the pre-constructed world coordinate system are determined, and the positions of the pixel points in the images acquired by the image acquisition devices in the panoramic parking image are also determined.
After the image splicing calibration is completed, in the operation process of the panoramic parking system, according to the stored mapping relation, all pixel points forming the panoramic parking image are searched from the images collected by the front image collecting device, the rear image collecting device, the left image collecting device and the right image collecting device, and the searched pixel points form the panoramic parking image.
According to the image splicing method of the panoramic parking system disclosed by the invention, when a user locates at an initial position of a vehicle, the user needs to respectively place a first calibration cloth, a second calibration cloth, a third calibration cloth and a fourth calibration cloth on the front, back, left and right sides of the vehicle, so that a forward image acquisition device, a backward image acquisition device, a left image acquisition device and a right image acquisition device in the panoramic parking system can respectively acquire images of the first calibration cloth, the second calibration cloth, the third calibration cloth and the fourth calibration cloth are symmetrical relative to a longitudinal center line of the vehicle, a midpoint of a right side of the third calibration cloth and a midpoint of a left side of the fourth calibration cloth are aligned with a midpoint of a wheel distance of the vehicle, then the panoramic parking system determines positions of the third calibration cloth and the fourth calibration cloth in a pre-constructed world coordinate system by using a vehicle body width, the panoramic parking system determines images acquired by using the left image acquisition device and the right image acquisition device when the vehicle is located at the first position, determines positions of the first calibration cloth in the panoramic parking system, determines positions of the panoramic parking system by using corresponding to acquire images of the images acquired by using the left calibration cloth and the left image acquisition device, determines the panoramic parking device, and determines the corresponding to acquire pixels of the images of the panoramic parking system, and determines the corresponding to acquire the positions of the parking device in the corresponding to acquire the corresponding to the image acquisition devices in the corresponding to the panoramic parking system, and the calibration image acquisition device, and determines the corresponding to acquire the calibration image acquisition points of the corresponding to acquire the image acquisition device in the image, and the corresponding to acquire the image acquisition device in the corresponding to the panoramic parking device, and the panoramic parking system, and determines the corresponding to acquire the image acquisition device.
For a user, before the panoramic parking system performs image splicing calibration, only a first calibration cloth, a second calibration cloth, a third calibration cloth and a fourth calibration cloth need to be placed respectively at the front, back, left and right sides of a vehicle, and in the process of placing the calibration cloths, only the third calibration cloth and the fourth calibration cloth need to be ensured to be symmetrical relative to the longitudinal center line of the vehicle, the middle point of the right side edge of the third calibration cloth and the middle point of the left side edge of the fourth calibration cloth are aligned with the middle point of the wheel track of the vehicle, and the calibration cloths around the vehicle do not need to be strictly parallel or perpendicular to the longitudinal center line of the vehicle; in the process of image splicing and calibration of the panoramic parking system, a user only needs to control the vehicle to move to the first position and the second position, and does not need to manually select a calibration point for images acquired by all the image acquisition devices. Therefore, the image splicing calibration method based on the panoramic parking system disclosed by the invention reduces the operation difficulty of the user, and correspondingly can reduce the probability of deviation of the user operation, thereby improving the accuracy of image splicing calibration.
In the image stitching calibration method disclosed in the above description of the present invention, as an embodiment, the pre-constructed world coordinate system has an origin that is a central point of a rectangle formed by central points of four wheels of the vehicle, a positive direction of an X axis is directed to a right side of the vehicle, a positive direction of a Y axis is directed to a rear side of the vehicle, and a positive direction of a Z axis is directed vertically upward.
Accordingly, the step S2 determines the positions of the third calibration cloth 103 and the fourth calibration cloth 104 in the pre-constructed world coordinate system according to the width of the vehicle body, and may be implemented as follows:
determining the coordinate of the middle point of the right side of the third calibration cloth in the world coordinate system according to the width of the vehicle body of the vehicle, wherein the coordinate of the middle point of the right side of the third calibration cloth in the world coordinate system is the position of the third calibration cloth in the world coordinate system;
and determining the coordinates of the middle point of the left side of the fourth calibration cloth in the world coordinate system according to the vehicle body width of the vehicle, wherein the coordinates of the middle point of the left side of the fourth calibration cloth in the world coordinate system are the positions of the fourth calibration cloth in the world coordinate system.
The pre-constructed world coordinate system has the origin of a rectangular central point formed by central points of four wheels of the vehicle, the positive direction of an X axis points to the right side of the vehicle, the positive direction of a Y axis points to the rear side of the vehicle, and the positive direction of a Z axis points vertically upwards. And the midpoint of the right side of the third calibration cloth 103 and the midpoint of the left side of the fourth calibration cloth 104 are aligned with the midpoint of the track of the vehicle, the coordinate of the midpoint of the right side of the third calibration cloth 103 is (-a) 1 /2-B1, 0), the coordinate of the midpoint of the left side of the fourth calibration cloth 104 is (A) 1 /2+B 1 ,0,0). Wherein A is 1 Is the body width of the vehicle, B 1 For the third calibration cloth 103 distance between the midpoint of the right side of the body and the left edge of the body, B 2 Is the distance between the midpoint of the left side of the fourth calibration cloth 104 and the right edge of the vehicle body.
As a preferred embodiment, during the process of placing the third calibration cloth 103 and the fourth calibration cloth 104, the third calibration cloth 103 and the fourth calibration cloth 104 are ensured to be symmetrical with respect to the longitudinal center line of the vehicle, the middle point of the right side of the third calibration cloth 103 and the middle point of the left side of the fourth calibration cloth 104 are both aligned with the middle point of the wheel base of the vehicle, in addition, the right side of the third calibration cloth 103 is aligned with the left edge of the vehicle body, and the right side of the fourth calibration cloth 104 is aligned with the right edge of the vehicle body.
Accordingly, the middle point of the right side of the third calibration cloth 103 has a coordinate of (-A) 1 /2, 0), the coordinate of the midpoint of the left side of the fourth calibration cloth 104 is (A) 1 /2，0,0)。
In addition, in the image stitching calibration method disclosed in the above description of the present invention, the images of the left side area 101a of the third calibration cloth 103 and the first calibration cloth 101 acquired by the left image acquisition device and the images of the right side area 101c of the fourth calibration cloth 104 and the first calibration cloth 101 acquired by the right image acquisition device are analyzed to determine the position relationship of the first calibration cloth 101 with respect to the third calibration cloth 103 and the fourth calibration cloth 104, and the following method may be adopted:
analyzing images of the left areas 101a of the third calibration cloth 103 and the first calibration cloth 101 collected by the left image collecting device, and determining the distance from the first calibration point of the third calibration cloth 103 to the second calibration point of the left area 101a of the first calibration cloth 101;
images of the fourth calibration cloth 104 and the right area 101c of the first calibration cloth 101 collected by the right-direction image collecting device are analyzed, and the distance from the third calibration point of the fourth calibration cloth 104 to the fourth calibration point of the right area 101c of the first calibration cloth 101 is determined.
Correspondingly, the position of the first calibration cloth 101 in the world coordinate system is determined according to the positions of the third calibration cloth 103 and the fourth calibration cloth 104 in the world coordinate system and the position relationship of the first calibration cloth 101 relative to the third calibration cloth 103 and the fourth calibration cloth 104, which may be implemented as follows:
and determining the position of the first calibration cloth in the world coordinate system according to the positions of the third calibration cloth and the fourth calibration cloth in the world coordinate system, the distance from the first calibration point to the second calibration point, the distance from the third calibration point to the fourth calibration point and the distance from the second calibration point to the fourth calibration point.
Specifically, the method comprises the following steps:
a first index point is selected in the third calibration cloth 103, and a second index point is selected in the left area 101a of the first calibration cloth 101. By analyzing the image of the third calibration cloth 103 acquired by the left image acquisition device and the image of the left area 101a of the first calibration cloth 101, the distance between the first calibration point and the second calibration point is determined, which is the actual distance between the first calibration point and the second calibration point, not the distance in the image.
The third index point is selected in the fourth calibration cloth 104, and the fourth index point is selected in the right area 101c of the first calibration cloth 101. By analyzing the image of the fourth calibration cloth 104 acquired by the right-hand image acquisition device and the image of the right-hand area 101c of the first calibration cloth 101, the distance between the third calibration point and the fourth calibration point is determined, which is the actual distance between the third calibration point and the fourth calibration point, not the distance in the image.
And then, determining the distance between the second calibration point and the fourth calibration point, and determining the position of the first calibration cloth in the world coordinate system according to the positions of the third calibration cloth and the fourth calibration cloth in the world coordinate system, the distance from the first calibration point to the second calibration point, the distance from the third calibration point to the fourth calibration point and the distance from the second calibration point to the fourth calibration point.
Taking fig. 5 as an example, the positions of the third calibration cloth 103 in the world coordinate system are: the coordinates of the middle point of the right side of the third calibration cloth 103 in the world coordinate system, and the position of the fourth calibration cloth 104 in the world coordinate system are: the coordinates of the midpoint of the left side of the fourth calibration cloth 104 in the world coordinate system.
The position of the first calibration point in the third calibration cloth 103 is known, and the coordinates of the first calibration point in the world coordinate system can be determined from the coordinates of the midpoint of the right side of the third calibration cloth 103 in the world coordinate system. Likewise, the position of the third calibration point in the fourth calibration cloth 104 is known, and the coordinates of the third calibration point in the world coordinate system can be determined according to the coordinates of the midpoint of the left side of the fourth calibration cloth 104 in the world coordinate system.
The distance between the first calibration point and the second calibration point is a, the distance between the third calibration point and the fourth calibration point is b, and the distance between the second calibration point and the fourth calibration point is c. The coordinates of the second and fourth index points in the world coordinate system can be determined based on the coordinates of the first and third index points in the world coordinate system and the aforementioned distance values.
In this embodiment, the coordinates of the second calibration point and the fourth calibration point in the world coordinate system may be used as the positions of the first calibration cloth 101 in the world coordinate system, or the coordinates of another point (for example, the center point of the first calibration cloth 101) in the world coordinate system may be calculated from the coordinates of the second calibration point and the fourth calibration point in the world coordinate system, and the coordinates of the point in the world coordinate system may be used as the positions of the first calibration cloth 101 in the world coordinate system.
In the image stitching calibration method disclosed in the above description of the present invention, the images of the left areas 102a of the third calibration cloth 103 and the second calibration cloth 102 collected by the left image collecting device and the images of the right areas 102c of the fourth calibration cloth 104 and the second calibration cloth 102 collected by the right image collecting device are analyzed to determine the position relationship of the second calibration cloth 102 with respect to the third calibration cloth 103 and the fourth calibration cloth 104, and the following method may be adopted:
analyzing images of the left areas 102a of the third calibration cloth 103 and the second calibration cloth 102 acquired by the left image acquisition device, and determining the distance from a fifth calibration point of the third calibration cloth 103 to a sixth calibration point of the left area 102a of the second calibration cloth 102;
the images of the fourth calibration cloth 104 and the right region 102c of the second calibration cloth 102 acquired by the right-direction image acquisition device are analyzed to determine the distance from the seventh calibration point of the fourth calibration cloth 104 to the eighth calibration point of the right region 102c of the second calibration cloth 102.
Correspondingly, determining the position of the second calibration cloth 102 in the world coordinate system according to the positions of the third calibration cloth 103 and the fourth calibration cloth 104 in the world coordinate system and the position relationship of the second calibration cloth 102 relative to the third calibration cloth 103 and the fourth calibration cloth 104 comprises:
and determining the position of the second calibration cloth 102 in the world coordinate system according to the positions of the third calibration cloth 103 and the fourth calibration cloth 104 in the world coordinate system, the distance from the fifth calibration point to the sixth calibration point, the distance from the seventh calibration point to the eighth calibration point, and the distance from the sixth calibration point to the eighth calibration point.
Specifically, the method comprises the following steps:
a fifth index point is selected in the third calibration cloth 103, and a sixth index point is selected in the left area 102a of the second calibration cloth 102. By analyzing the image of the third calibration cloth 103 acquired by the left image acquisition device and the image of the left area 102a of the second calibration cloth 102, the distance between the fifth calibration point and the sixth calibration point is determined, which is the actual distance between the fifth calibration point and the sixth calibration point, and not the distance in the image.
A seventh index point is selected in the fourth calibration cloth 104 and an eighth index point is selected in the right area 102c of the second calibration cloth 102. By analyzing the image of the fourth calibration cloth 104 acquired by the right-hand image acquisition device and the image of the right-hand area 102c of the second calibration cloth 102, the distance between the seventh calibration point and the eighth calibration point is determined, which is the actual distance between the seventh calibration point and the eighth calibration point, and not the distance in the image.
And then, determining the distance between the sixth calibration point and the eighth calibration point, and determining the position of the second calibration cloth 102 in the world coordinate system according to the positions of the third calibration cloth 103 and the fourth calibration cloth 104 in the world coordinate system, the distance from the fifth calibration point to the sixth calibration point, the distance from the seventh calibration point to the eighth calibration point, and the distance from the sixth calibration point to the eighth calibration point.
An example is illustrated:
the position of the third calibration cloth 103 in the world coordinate system is: the coordinates of the middle point of the right side of the third calibration cloth 103 in the world coordinate system, and the position of the fourth calibration cloth 104 in the world coordinate system are: coordinates of the midpoint of the left side of the fourth calibration cloth 104 in the world coordinate system.
The position of the fifth calibration point in the third calibration cloth 103 is known, and the coordinates of the fifth calibration point in the world coordinate system can be determined according to the coordinates of the midpoint of the right side of the third calibration cloth 103 in the world coordinate system. Likewise, the position of the seventh calibration point in the fourth calibration cloth 104 is known, and the coordinates of the seventh calibration point in the world coordinate system can be determined according to the coordinates of the midpoint of the left side of the fourth calibration cloth 104 in the world coordinate system.
The distance between the fifth calibration point and the sixth calibration point is d, the distance between the seventh calibration point and the eighth calibration point is e, and the distance between the sixth calibration point and the eighth calibration point is f. The coordinates of the sixth and eighth index points in the world coordinate system can be determined based on the coordinates of the fifth and seventh index points in the world coordinate system and the aforementioned distance values.
In this embodiment, the coordinates of the sixth calibration cloth 102 and the eighth calibration cloth in the world coordinate system may be used as the positions of the second calibration cloth 102 in the world coordinate system, or the coordinates of another point (for example, the center point of the second calibration cloth 102) in the world coordinate system may be calculated from the coordinates of the sixth calibration cloth and the eighth calibration cloth in the world coordinate system, and the coordinates of the point in the world coordinate system may be used as the positions of the second calibration cloth 102 in the world coordinate system.
In a preferred embodiment, the first calibration point and the fifth calibration point of the third calibration cloth 103 are selected as the same point in the third calibration cloth 103, and the third calibration point and the seventh calibration point of the fourth calibration cloth 104 are selected as the same point in the fourth calibration cloth 104. When the first calibration point and the fifth calibration point are selected as the same point, and the third calibration point and the seventh calibration point are selected as the same point, the operation amount in the image stitching calibration process can be reduced.
For example, the first and fifth calibration points of the third calibration cloth 103 are selected as the center point of the third calibration cloth 103, or as the center point of the right side of the third calibration cloth 103. The third and seventh calibration points of the fourth calibration cloth 104 are selected as the center point of the fourth calibration cloth 104 or as the center point of the left side of the fourth calibration cloth 104.
In the image stitching calibration method disclosed by the invention, in step S10, the installation information of the forward image capturing device, the backward image capturing device, the left image capturing device and the right image capturing device on the vehicle body is determined according to the positions of the first calibration cloth, the second calibration cloth, the third calibration cloth and the fourth calibration cloth in the world coordinate system, and the following method can be adopted:
under the condition that the vehicle is at the preset position, the preset angular point of the first calibration cloth is at a first actual position in a first calibration cloth image acquired by a front image acquisition device, the preset angular point of the second calibration cloth is at a second actual position in a second calibration cloth image acquired by a rear image acquisition device, the preset angular point of the third calibration cloth is at a third actual position in a third calibration cloth image acquired by a left image acquisition device, and the preset angular point of the fourth calibration cloth is at a fourth actual position in a fourth calibration cloth image acquired by a right image acquisition device;
setting installation information of a forward image acquisition device; calculating a first virtual position of a preset angular point of a first calibration cloth in a virtual image of the first calibration cloth generated by the forward image acquisition device in a mounting state according to currently set mounting information under the condition that the vehicle is at the preset position according to the position of the forward image acquisition device in a world coordinate system; if the difference value between the first actual position and the first virtual position is smaller than the forward minimum value, the currently set installation information of the forward image acquisition device is the actual installation information of the forward image acquisition device; if the difference value between the first actual position and the first virtual position is larger than or equal to the forward minimum value, resetting the installation information of the forward image acquisition device, and executing subsequent steps until the actual installation information of the forward image acquisition device is determined;
setting installation information of a backward image acquisition device; calculating a second virtual position of a preset corner point of a second calibration cloth in a virtual image of the second calibration cloth generated by the backward image acquisition device in the installation state according to the currently set installation information under the condition that the vehicle is at the preset position according to the position of the backward image acquisition device in the world coordinate system; if the difference value between the second actual position and the second virtual position is smaller than the backward minimum value, the currently set installation information of the backward image acquisition device is the actual installation information of the backward image acquisition device; if the difference value between the second actual position and the second virtual position is larger than or equal to the backward minimum value, resetting the installation information of the backward image acquisition device, and executing the subsequent steps until the actual installation information of the backward image acquisition device is determined;
setting installation information of a left image acquisition device; calculating a third virtual position of a preset corner point of a third calibration cloth in a virtual image of the third calibration cloth generated by the left image acquisition device in the installation state according to the currently set installation information under the condition that the vehicle is at the preset position according to the position of the left image acquisition device in the world coordinate system; if the difference value between the third actual position and the third virtual position is smaller than the minimum value in the left direction, the currently set installation information of the left image acquisition device is the actual installation information of the left image acquisition device; if the difference value between the third actual position and the third virtual position is larger than or equal to the minimum value in the left direction, resetting the installation information of the left image acquisition device, and executing the subsequent steps until the actual installation information of the left image acquisition device is determined;
setting installation information of a right-direction image acquisition device; calculating a fourth virtual position of a preset corner point of a fourth calibration cloth in a virtual image of the fourth calibration cloth generated by the right image acquisition device in the installation state according to the currently set installation information under the condition that the vehicle is at the preset position according to the position of the right image acquisition device in the world coordinate system; if the difference value between the fourth actual position and the fourth virtual position is smaller than the right minimum value, the currently set installation information of the right image acquisition device is the actual installation information of the right image acquisition device; and if the difference value between the fourth actual position and the fourth virtual position is larger than or equal to the right minimum value, resetting the installation information of the right image acquisition device, and executing subsequent steps until the actual installation information of the right image acquisition device is determined.
Here, the installation information of the forward image capturing device is determined as an example:
firstly, when the vehicle is at the preset position, an image of the first calibration cloth actually acquired by the forward image acquisition device is acquired, and the position of the preset angular point of the first calibration cloth in the image is determined, wherein the position is the actual position of the preset angular point of the first calibration cloth.
Then, setting installation information of the forward image acquisition device, calculating a virtual position of a preset angular point of the first calibration cloth in a virtual image of the first calibration cloth generated by the forward image acquisition device under the condition that the forward image acquisition device is installed by adopting the installation information, and if the deviation of the virtual position and the actual position of the preset angular point of the first calibration cloth meets a preset requirement, indicating that the currently set installation information of the forward image acquisition device is very close to the actual installation information, wherein the currently set installation information can be used as the actual installation information of the forward image acquisition device. If the deviation between the virtual position and the actual position of the preset corner point of the first calibration cloth is large, the deviation between the currently set installation information of the forward image acquisition device and the actual installation information is large, at the moment, the installation information of the forward image acquisition device is adjusted according to the preset step length, and then follow-up operation is executed until the actual installation information of the forward image acquisition device is obtained.
Here, it should be noted that the installation information of each image capturing device includes an installation position and an installation angle. And the installation position of the image acquisition device comprises the coordinates of the image acquisition device on each coordinate axis in the three-dimensional coordinate system, and the installation angle of the image acquisition device comprises the angle of the image acquisition device on each coordinate axis in the three-dimensional coordinate system. That is, the installation information of each image pickup device includes 6 variables.
In implementation, in the process of adjusting the installation information of each image acquisition device, 6 variables are adjusted one by one, and after one variable is determined, the next variable is adjusted until all 6 variables are determined.
The invention discloses an image splicing calibration method of a panoramic parking system, and correspondingly, the invention also discloses an image splicing calibration device of the panoramic parking system. The following description of the image stitching calibration apparatus and the above description of the image stitching calibration method may be referred to each other.
Referring to fig. 6, fig. 6 is a schematic structural diagram of an image stitching calibration apparatus of a panoramic parking system according to the present invention. The image stitching calibration apparatus includes a first position determination unit 100, a first positional relationship determination unit 200, a second position determination unit 300, a second positional relationship determination unit 400, a third position determination unit 500, a mounting information determination unit 600, and a calibration unit 700.
The first position determination unit 100 determines the positions of the third calibration cloth and the fourth calibration cloth in a pre-constructed world coordinate system according to the width of the vehicle body when the vehicle is at the initial position, wherein the origin of the world coordinate system is the center point of a rectangle formed by the center points of four wheels of the vehicle.
The vehicle comprises a vehicle body, a front side, a rear side, a left side and a right side, wherein the front side, the rear side, the left side and the right side of the vehicle body are respectively provided with a first calibration cloth, a second calibration cloth, a third calibration cloth and a fourth calibration cloth, the middle point of the right side of the third calibration cloth and the middle point of the left side of the fourth calibration cloth are aligned with the middle point of the wheel track of the vehicle body, the first calibration cloth and the second calibration cloth respectively comprise a middle area, a left area and a right area which are divided in advance, a front image acquisition device can acquire an image of the first calibration cloth, a rear image acquisition device can acquire an image of the second calibration cloth, a left image acquisition device can acquire an image of the third calibration cloth, and a right image acquisition device can acquire an image of the fourth calibration cloth.
The first position relation determining unit 200 is configured to, when the vehicle travels straight forward to the first position, acquire images of left areas of the first calibration cloth and the third calibration cloth currently acquired by the left image acquisition device, acquire images of right areas of the first calibration cloth and the fourth calibration cloth currently acquired by the right image acquisition device, analyze images currently acquired by the left image acquisition device and the right image acquisition device, and determine a position relation of the first calibration cloth with respect to the third calibration cloth and the fourth calibration cloth.
And a second position determining unit 300, configured to determine the position of the first calibration cloth in the world coordinate system according to the positions of the third calibration cloth and the fourth calibration cloth in the world coordinate system and the position relationship of the first calibration cloth relative to the third calibration cloth and the fourth calibration cloth.
The second position relation determining unit 400 is configured to, when the vehicle travels straight backward to a second position, acquire images of left areas of a third calibration cloth and a second calibration cloth currently acquired by the left image acquiring device, acquire images of right areas of a fourth calibration cloth and a second calibration cloth currently acquired by the right image acquiring device, analyze the images currently acquired by the left image acquiring device and the right image acquiring device, and determine a position relation of the second calibration cloth with respect to the third calibration cloth and the fourth calibration cloth.
And a third position determining unit 500, configured to determine the position of the second calibration cloth in the world coordinate system according to the positions of the third calibration cloth and the fourth calibration cloth in the world coordinate system and the position relationship of the second calibration cloth with respect to the third calibration cloth and the fourth calibration cloth.
And the installation information determining unit 600 is configured to determine installation information of the forward image capturing device, the backward image capturing device, the left image capturing device and the right image capturing device on the vehicle body according to positions of the first calibration cloth, the second calibration cloth, the third calibration cloth and the fourth calibration cloth in a world coordinate system, where the installation information includes an installation position and an installation angle.
The calibration unit 700 is configured to determine, by using the installation information of the forward image acquisition device, the backward image acquisition device, the left image acquisition device, and the right image acquisition device, corresponding pixels of each pixel in the panoramic parking image in the images acquired by the forward image acquisition device, the backward image acquisition device, the left image acquisition device, and the right image acquisition device, and establish and store a mapping relationship between the corresponding pixels.
By using the image splicing calibration device disclosed by the invention, a user only needs to place the first calibration cloth, the second calibration cloth, the third calibration cloth and the fourth calibration cloth respectively at the front, back, left and right sides of a vehicle before the image splicing calibration of a panoramic parking system, and only needs to ensure that the midpoint of the right side edge of the third calibration cloth and the midpoint of the left side edge of the fourth calibration cloth are aligned with the midpoint of the wheel track of the vehicle in the process of placing the calibration cloth, but does not need to ensure that the calibration cloth around the vehicle is strictly parallel or vertical to the longitudinal center line of the vehicle; in the process of image splicing and calibration of the panoramic parking system, a user only needs to control the vehicle to move to the first position and the second position, and does not need to manually select a calibration point for images acquired by the image acquisition devices. The image splicing calibration method based on the panoramic parking system disclosed by the invention can be seen to reduce the operation difficulty of a user, and correspondingly, the probability of deviation of the user operation can be reduced, so that the accuracy of image splicing calibration is improved.
In the image stitching calibration apparatus disclosed in the above description of the present invention, as an embodiment, the origin of the world coordinate system constructed in advance is a central point of a rectangle formed by central points of four wheels of the vehicle, a positive direction of an X axis is directed to a right side of the vehicle, a positive direction of a Y axis is directed to a rear side of the vehicle, and a positive direction of a Z axis is directed vertically upward.
Correspondingly, the first position determination unit 100 is specifically configured to: determining the coordinate of the middle point of the right side of the third calibration cloth in a world coordinate system according to the vehicle body width of the vehicle, wherein the coordinate of the middle point of the right side of the third calibration cloth in the world coordinate system is the position of the third calibration cloth in the world coordinate system; and determining the coordinates of the middle point of the left side of the fourth calibration cloth in the world coordinate system according to the vehicle body width of the vehicle, wherein the coordinates of the middle point of the left side of the fourth calibration cloth in the world coordinate system are the positions of the fourth calibration cloth in the world coordinate system.
As an example, in the image stitching calibration apparatus disclosed above in the present invention, the first position relation determining unit 200 is specifically configured to: the method comprises the steps of analyzing images of a left area of a first calibration cloth and a left area of a third calibration cloth collected by a left image acquisition device, determining the distance from a first calibration point of the third calibration cloth to a second calibration point of the left area of the first calibration cloth, analyzing images of a right area of the first calibration cloth and a right area of a fourth calibration cloth collected by a right image acquisition device, and determining the distance from a third calibration point of the fourth calibration cloth to a fourth calibration point of the right area of the first calibration cloth.
The second position determination unit 300 is specifically configured to: and determining the position of the first calibration cloth in the world coordinate system according to the positions of the third calibration cloth and the fourth calibration cloth in the world coordinate system, the distance from the first calibration point to the second calibration point, the distance from the third calibration point to the fourth calibration point and the distance from the second calibration point to the fourth calibration point.
As an example, in the image stitching calibration apparatus disclosed above in the present invention, the second position relation determining unit 400 is specifically configured to: analyzing images of left areas of the third calibration cloth and the second calibration cloth acquired by the left image acquisition device, determining the distance from a fifth calibration point of the third calibration cloth to a sixth calibration point of the left area of the second calibration cloth, analyzing images of right areas of the fourth calibration cloth and the second calibration cloth acquired by the right image acquisition device, and determining the distance from a seventh calibration point of the fourth calibration cloth to an eighth calibration point of the right area of the second calibration cloth.
The third position determination unit 500 is specifically configured to: and determining the position of the second calibration cloth in the world coordinate system according to the positions of the third calibration cloth and the fourth calibration cloth in the world coordinate system, the distance from the fifth calibration point to the sixth calibration point, the distance from the seventh calibration point to the eighth calibration point and the distance from the sixth calibration point to the eighth calibration point.
As an example, in the image stitching calibration apparatus disclosed in the above description of the present invention, the installation information determining unit 600, as shown in fig. 7, includes a corner actual position determining module 601, a first processing module 602, a second processing module 603, a third processing module 604, and a fourth processing module 605.
Wherein:
the corner actual position determining module 601 is configured to determine, when the vehicle is at the preset position, a first actual position of a preset corner of the first calibration cloth in a first calibration cloth image acquired by the front image acquisition device, a second actual position of a preset corner of the second calibration cloth in a second calibration cloth image acquired by the rear image acquisition device, a third actual position of a preset corner of the third calibration cloth in a third calibration cloth image acquired by the left image acquisition device, and a fourth actual position of a preset corner of the fourth calibration cloth in a fourth calibration cloth image acquired by the right image acquisition device.
A first processing module 602, configured to set installation information of a forward image capturing device; calculating a first virtual position of a preset angular point of a first calibration cloth in a virtual image of the first calibration cloth generated by the forward image acquisition device in an installation state according to currently set installation information under the condition that the vehicle is at a preset position according to the position of the forward image acquisition device in a world coordinate system; if the difference value between the first actual position and the first virtual position is smaller than the forward minimum value, the currently set installation information of the forward image acquisition device is the actual installation information of the forward image acquisition device; and if the difference value between the first actual position and the first virtual position is larger than or equal to the forward minimum value, resetting the installation information of the forward image acquisition device, and executing subsequent operations until the actual installation information of the forward image acquisition device is determined.
A second processing module 603, configured to set installation information of the backward image acquisition device; calculating a second virtual position of a preset corner point of a second calibration cloth in a virtual image of the second calibration cloth generated by the backward image acquisition device in the installation state according to the currently set installation information under the condition that the vehicle is at the preset position according to the position of the backward image acquisition device in the world coordinate system; if the difference value between the second actual position and the second virtual position is smaller than the backward minimum value, the currently set installation information of the backward image acquisition device is the actual installation information of the backward image acquisition device; and if the difference value between the second actual position and the second virtual position is larger than or equal to the backward minimum value, resetting the installation information of the backward image acquisition device, and executing subsequent operation until the actual installation information of the backward image acquisition device is determined.
A third processing module 604, configured to set installation information of the left image capturing device; calculating a third virtual position of a preset corner point of a third calibration cloth in a virtual image of the third calibration cloth generated by the left image acquisition device in the installation state according to the currently set installation information under the condition that the vehicle is at the preset position according to the position of the left image acquisition device in the world coordinate system; if the difference value between the third actual position and the third virtual position is smaller than the minimum value in the left direction, the currently set installation information of the left image acquisition device is the actual installation information of the left image acquisition device; and if the difference value between the third actual position and the third virtual position is greater than or equal to the minimum value in the left direction, resetting the installation information of the left image acquisition device, and executing subsequent operations until the actual installation information of the left image acquisition device is determined.
A fourth processing module 605, configured to set installation information of the right image capturing device; calculating a fourth virtual position of a preset corner point of a fourth calibration cloth in a virtual image of the fourth calibration cloth generated by the right image acquisition device in the installation state according to the currently set installation information under the condition that the vehicle is at the preset position according to the position of the right image acquisition device in the world coordinate system; if the difference value between the fourth actual position and the fourth virtual position is smaller than the right minimum value, the currently set installation information of the right image acquisition device is the actual installation information of the right image acquisition device; and if the difference value between the fourth actual position and the fourth virtual position is greater than or equal to the right minimum numerical value, resetting the installation information of the right image acquisition device, and executing subsequent operations until the actual installation information of the right image acquisition device is determined.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising one of 8230; \8230;" 8230; "does not exclude the presence of additional like elements in a process, method, article, or apparatus that comprises the element.
In the present specification, the embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.

Claims (11)

1. An image stitching calibration method of a panoramic parking system, wherein the panoramic parking system comprises a forward image acquisition device, a backward image acquisition device, a left image acquisition device and a right image acquisition device, and is characterized by comprising the following steps:
under the condition that a vehicle is located at an initial position, placing a first calibration cloth, a second calibration cloth, a third calibration cloth and a fourth calibration cloth on the front side, the rear side, the left side and the right side of the vehicle respectively, wherein a midpoint of the right side of the third calibration cloth and a midpoint of the left side of the fourth calibration cloth are aligned with a midpoint of a wheel track of the vehicle, the first calibration cloth and the second calibration cloth respectively comprise a middle area, a left area and a right area which are divided in advance, the forward image acquisition device can acquire an image of the first calibration cloth, the backward image acquisition device can acquire an image of the second calibration cloth, the left image acquisition device can acquire an image of the third calibration cloth, and the right image acquisition device can acquire an image of the fourth calibration cloth;
determining the positions of the third calibration cloth and the fourth calibration cloth in a pre-constructed world coordinate system according to the width of the vehicle body of the vehicle, wherein the origin of the world coordinate system is the central point of a rectangle formed by the central points of four wheels of the vehicle;
controlling the vehicle to straightly move forwards to a first position, acquiring images of left areas of the third calibration cloth and the first calibration cloth currently acquired by the left image acquisition device, and acquiring images of right areas of the fourth calibration cloth and the first calibration cloth currently acquired by the right image acquisition device;
analyzing the images currently acquired by the left image acquisition device and the right image acquisition device, and determining the position relationship of the first calibration cloth relative to the third calibration cloth and the fourth calibration cloth;
determining the position of the first calibration cloth in the world coordinate system according to the positions of the third calibration cloth and the fourth calibration cloth in the world coordinate system and the position relation of the first calibration cloth relative to the third calibration cloth and the fourth calibration cloth;
controlling the vehicle to move to a second position straightly and backwards, acquiring images of left areas of the third calibration cloth and the second calibration cloth currently acquired by the left image acquisition device, and acquiring images of right areas of the fourth calibration cloth and the second calibration cloth currently acquired by the right image acquisition device;
analyzing the images currently acquired by the left image acquisition device and the right image acquisition device, and determining the position relationship of the second calibration cloth relative to the third calibration cloth and the fourth calibration cloth;
determining the position of the second calibration cloth in the world coordinate system according to the positions of the third calibration cloth and the fourth calibration cloth in the world coordinate system and the position relation of the second calibration cloth relative to the third calibration cloth and the fourth calibration cloth;
determining the installation information of the forward image acquisition device, the backward image acquisition device, the left image acquisition device and the right image acquisition device on a vehicle body according to the positions of the first calibration cloth, the second calibration cloth, the third calibration cloth and the fourth calibration cloth in the world coordinate system, wherein the installation information comprises installation positions and installation angles;
and determining corresponding pixel points of all pixel points in the panoramic parking image in the images acquired by the forward image acquisition device, the backward image acquisition device, the left image acquisition device and the right image acquisition device by using the installation information of the forward image acquisition device, the backward image acquisition device, the left image acquisition device and the right image acquisition device, and establishing and storing the mapping relation of the corresponding pixel points.
2. The image stitching calibration method according to claim 1, wherein the pre-constructed world coordinate system has an origin point which is a central point of a rectangle formed by central points of four wheels of the vehicle, a positive direction of an X axis points to the right side of the vehicle, a positive direction of a Y axis points to the rear side of the vehicle, and a positive direction of a Z axis points vertically upwards;
the determining the positions of the third calibration cloth and the fourth calibration cloth in a pre-constructed world coordinate system according to the width of the vehicle body comprises the following steps:
determining the coordinates of the middle point of the right side of the third calibration cloth in the world coordinate system according to the width of the vehicle body, wherein the coordinates of the middle point of the right side of the third calibration cloth in the world coordinate system are the positions of the third calibration cloth in the world coordinate system;
and determining the coordinates of the middle point of the left side of the fourth calibration cloth in the world coordinate system according to the width of the vehicle body, wherein the coordinates of the middle point of the left side of the fourth calibration cloth in the world coordinate system are the positions of the fourth calibration cloth in the world coordinate system.
3. The image stitching calibration method according to claim 1 or 2,
the analyzing the images acquired by the left image acquisition device and the right image acquisition device to determine the position relationship of the first calibration cloth relative to the third calibration cloth and the fourth calibration cloth comprises:
analyzing the images of the left areas of the third calibration cloth and the first calibration cloth acquired by the left image acquisition device, determining the distance from a first calibration point of the third calibration cloth to a second calibration point of the left area of the first calibration cloth, analyzing the images of the right areas of the fourth calibration cloth and the first calibration cloth acquired by the right image acquisition device, and determining the distance from a third calibration point of the fourth calibration cloth to a fourth calibration point of the right area of the first calibration cloth;
the determining the position of the first calibration cloth in the world coordinate system according to the positions of the third calibration cloth and the fourth calibration cloth in the world coordinate system and the position relationship of the first calibration cloth relative to the third calibration cloth and the fourth calibration cloth comprises:
and determining the position of the first calibration cloth in the world coordinate system according to the positions of the third calibration cloth and the fourth calibration cloth in the world coordinate system, the distance from the first calibration point to the second calibration point, the distance from the third calibration point to the fourth calibration point and the distance from the second calibration point to the fourth calibration point.
4. The image stitching calibration method of claim 3,
the analyzing the images acquired by the left image acquisition device and the right image acquisition device to determine the position relationship of the second calibration cloth relative to the third calibration cloth and the fourth calibration cloth comprises:
analyzing the images of the left areas of the third calibration cloth and the second calibration cloth acquired by the left image acquisition device, determining the distance from a fifth calibration point of the third calibration cloth to a sixth calibration point of the left area of the second calibration cloth, analyzing the images of the right areas of the fourth calibration cloth and the second calibration cloth acquired by the right image acquisition device, and determining the distance from a seventh calibration point of the fourth calibration cloth to an eighth calibration point of the right area of the second calibration cloth;
the determining the position of the second calibration cloth in the world coordinate system according to the positions of the third calibration cloth and the fourth calibration cloth in the world coordinate system and the position relationship of the second calibration cloth relative to the third calibration cloth and the fourth calibration cloth comprises:
and determining the position of the second calibration cloth in the world coordinate system according to the positions of the third calibration cloth and the fourth calibration cloth in the world coordinate system, the distance from the fifth calibration point to the sixth calibration point, the distance from the seventh calibration point to the eighth calibration point and the distance from the sixth calibration point to the eighth calibration point.
5. The image stitching calibration method according to claim 4, wherein the first and fifth calibration points of the third calibration cloth are center points of the third calibration cloth; and the third calibration point and the seventh calibration point of the fourth calibration cloth are the central points of the fourth calibration cloth.
6. The image stitching calibration method according to any one of claims 1 to 2, wherein the determining of the installation information of the forward image capturing device, the backward image capturing device, the left image capturing device and the right image capturing device on the vehicle body according to the positions of the first calibration cloth, the second calibration cloth, the third calibration cloth and the fourth calibration cloth in the world coordinate system comprises:
determining a first actual position of a preset corner point of the first calibration cloth in a first calibration cloth image acquired by the forward image acquisition device, a second actual position of a preset corner point of the second calibration cloth in a second calibration cloth image acquired by the backward image acquisition device, a third actual position of a preset corner point of the third calibration cloth in a third calibration cloth image acquired by the leftward image acquisition device, and a fourth actual position of a preset corner point of the fourth calibration cloth in a fourth calibration cloth image acquired by the rightward image acquisition device under the condition that the vehicle is at the preset position;
setting installation information of the forward image acquisition device; calculating a first virtual position of a preset corner point of a first calibration cloth in a virtual image of the first calibration cloth generated by the forward image acquisition device in an installation state according to currently set installation information under the condition that a vehicle is at the preset position according to the position of the forward image acquisition device in the world coordinate system; if the difference value between the first actual position and the first virtual position is smaller than a forward minimum value, the currently set installation information of the forward image acquisition device is the actual installation information of the forward image acquisition device; if the difference value between the first actual position and the first virtual position is larger than or equal to the forward minimum value, resetting the installation information of the forward image acquisition device, and executing subsequent steps until the actual installation information of the forward image acquisition device is determined;
setting installation information of the backward image acquisition device; calculating a second virtual position of a preset corner point of a second calibration cloth in a virtual image of the second calibration cloth generated by the backward image acquisition device in an installation state according to currently set installation information under the condition that the vehicle is at the preset position according to the position of the backward image acquisition device in the world coordinate system; if the difference value between the second actual position and the second virtual position is smaller than the backward minimum value, the currently set installation information of the backward image acquisition device is the actual installation information of the backward image acquisition device; if the difference value between the second actual position and the second virtual position is larger than or equal to the backward minimum value, resetting the installation information of the backward image acquisition device, and executing subsequent steps until the actual installation information of the backward image acquisition device is determined;
setting installation information of the left image acquisition device; calculating a third virtual position of a preset corner point of a third calibration cloth in a virtual image of the third calibration cloth generated by the left image acquisition device in an installation state according to currently set installation information under the condition that the vehicle is at the preset position according to the position of the left image acquisition device in the world coordinate system; if the difference value between the third actual position and the third virtual position is smaller than a minimum value in the left direction, the currently set installation information of the left image acquisition device is the actual installation information of the left image acquisition device; if the difference value between the third actual position and the third virtual position is larger than or equal to the minimum value in the left direction, resetting the installation information of the left image acquisition device, and executing subsequent steps until the actual installation information of the left image acquisition device is determined;
setting installation information of the right image acquisition device; calculating a fourth virtual position of a preset corner point of a fourth calibration cloth in a virtual image of the fourth calibration cloth generated by the right image acquisition device in an installation state according to currently set installation information under the condition that the vehicle is at the preset position according to the position of the right image acquisition device in the world coordinate system; if the difference value between the fourth actual position and the fourth virtual position is smaller than the minimum right value, the currently set installation information of the right image acquisition device is the actual installation information of the right image acquisition device; and if the difference value between the fourth actual position and the fourth virtual position is larger than or equal to the right minimum numerical value, resetting the installation information of the right image acquisition device, and executing subsequent steps until the actual installation information of the right image acquisition device is determined.
7. An image splicing calibration device of a panoramic parking system, which comprises a forward image acquisition device, a backward image acquisition device, a left image acquisition device and a right image acquisition device, and is characterized in that under the condition that a vehicle is positioned at an initial position, a first calibration cloth, a second calibration cloth, a third calibration cloth and a fourth calibration cloth are respectively placed on the front side, the rear side, the left side and the right side of the vehicle, the middle point of the right side of the third calibration cloth and the middle point of the left side of the fourth calibration cloth are aligned with the middle point of the wheel track of the vehicle, the first calibration cloth and the second calibration cloth respectively comprise a middle area, a left area and a right area which are divided in advance, the forward image acquisition device can acquire an image of the first calibration cloth, the backward image acquisition device can acquire an image of the second calibration cloth, the left image acquisition device can acquire an image of the third calibration cloth, and the right image acquisition device can acquire an image of the fourth calibration cloth; the image stitching calibration device comprises:
a first position determination unit, which determines the positions of the third calibration cloth and the fourth calibration cloth in a pre-constructed world coordinate system according to the width of the vehicle body when the vehicle is at the initial position, wherein the origin of the world coordinate system is the central point of a rectangle formed by the central points of four wheels of the vehicle;
a first position relation determination unit, configured to, when the vehicle travels straight forward to a first position, acquire images of left areas of the third calibration cloth and the first calibration cloth currently acquired by the left image acquisition device, acquire images of right areas of the fourth calibration cloth and the first calibration cloth currently acquired by the right image acquisition device, analyze the images currently acquired by the left image acquisition device and the right image acquisition device, and determine a position relation of the first calibration cloth with respect to the third calibration cloth and the fourth calibration cloth;
the second position determining unit is used for determining the position of the first calibration cloth in the world coordinate system according to the positions of the third calibration cloth and the fourth calibration cloth in the world coordinate system and the position relation of the first calibration cloth relative to the third calibration cloth and the fourth calibration cloth;
a second position relation determining unit, configured to, when the vehicle travels straight backward to a second position, acquire images of left areas of the third calibration cloth and the second calibration cloth currently acquired by the left image acquisition device, acquire images of right areas of the fourth calibration cloth and the second calibration cloth currently acquired by the right image acquisition device, analyze the images currently acquired by the left image acquisition device and the right image acquisition device, and determine a position relation of the second calibration cloth with respect to the third calibration cloth and the fourth calibration cloth;
a third position determining unit, configured to determine a position of the second calibration cloth in the world coordinate system according to positions of the third calibration cloth and the fourth calibration cloth in the world coordinate system and a positional relationship of the second calibration cloth with respect to the third calibration cloth and the fourth calibration cloth;
the installation information determining unit is used for determining the installation information of the forward image acquisition device, the backward image acquisition device, the left image acquisition device and the right image acquisition device on a vehicle body according to the positions of the first calibration cloth, the second calibration cloth, the third calibration cloth and the fourth calibration cloth in the world coordinate system, wherein the installation information comprises installation positions and installation angles;
and the calibration unit is used for determining corresponding pixel points of all pixel points in the panoramic parking image in the images acquired by the forward image acquisition device, the backward image acquisition device, the left image acquisition device and the right image acquisition device by utilizing the installation information of the forward image acquisition device, the backward image acquisition device, the left image acquisition device and the right image acquisition device, and establishing and storing the mapping relation of the corresponding pixel points.
8. The image stitching calibration device according to claim 7, wherein the pre-constructed world coordinate system has an origin which is a central point of a rectangle formed by central points of four wheels of the vehicle, a positive direction of an X axis is directed to the right side of the vehicle, a positive direction of a Y axis is directed to the rear side of the vehicle, and a positive direction of a Z axis is vertically upward;
the first position determination unit is specifically configured to: determining the coordinates of the middle point of the right side of the third calibration cloth in the world coordinate system according to the width of the vehicle body, wherein the coordinates of the middle point of the right side of the third calibration cloth in the world coordinate system are the positions of the third calibration cloth in the world coordinate system; and determining the coordinates of the middle point of the left side of the fourth calibration cloth in the world coordinate system according to the vehicle body width of the vehicle, wherein the coordinates of the middle point of the left side of the fourth calibration cloth in the world coordinate system are the positions of the fourth calibration cloth in the world coordinate system.
9. The image stitching calibration device of claim 7 or 8,
the first position relationship determination unit is specifically configured to: analyzing images of left areas of the third calibration cloth and the first calibration cloth collected by the left image acquisition device, determining the distance from a first calibration point of the third calibration cloth to a second calibration point of the left area of the first calibration cloth, analyzing images of right areas of the fourth calibration cloth and the first calibration cloth collected by the right image acquisition device, and determining the distance from a third calibration point of the fourth calibration cloth to a fourth calibration point of the right area of the first calibration cloth;
the second position determination unit is specifically configured to: and determining the position of the first calibration cloth in the world coordinate system according to the positions of the third calibration cloth and the fourth calibration cloth in the world coordinate system, the distance from the first calibration point to the second calibration point, the distance from the third calibration point to the fourth calibration point and the distance from the second calibration point to the fourth calibration point.
10. The image stitching calibration device of claim 9,
the second positional relationship determination unit is specifically configured to: analyzing the images of the left areas of the third calibration cloth and the second calibration cloth acquired by the left image acquisition device, determining the distance from a fifth calibration point of the third calibration cloth to a sixth calibration point of the left area of the second calibration cloth, analyzing the images of the right areas of the fourth calibration cloth and the second calibration cloth acquired by the right image acquisition device, and determining the distance from a seventh calibration point of the fourth calibration cloth to an eighth calibration point of the right area of the second calibration cloth;
the third position determination unit is specifically configured to: and determining the position of the second calibration cloth in the world coordinate system according to the positions of the third calibration cloth and the fourth calibration cloth in the world coordinate system, the distance from the fifth calibration point to the sixth calibration point, the distance from the seventh calibration point to the eighth calibration point and the distance from the sixth calibration point to the eighth calibration point.
11. The image stitching calibration device according to any one of claims 7 to 8, wherein the mounting information determination unit includes:
the corner actual position determining module is used for determining a first actual position of a preset corner of the first calibration cloth in a first calibration cloth image acquired by the forward image acquisition device, a second actual position of a preset corner of the second calibration cloth in a second calibration cloth image acquired by the backward image acquisition device, a third actual position of a preset corner of the third calibration cloth in a third calibration cloth image acquired by the leftward image acquisition device, and a fourth actual position of a preset corner of the fourth calibration cloth in a fourth calibration cloth image acquired by the rightward image acquisition device under the condition that the vehicle is at the preset position;
the first processing module is used for setting the installation information of the forward image acquisition device; calculating a first virtual position of a preset corner point of a first calibration cloth in a virtual image of the first calibration cloth generated by the forward image acquisition device in an installation state according to currently set installation information under the condition that a vehicle is at the preset position according to the position of the forward image acquisition device in the world coordinate system; if the difference value between the first actual position and the first virtual position is smaller than a forward minimum value, the currently set installation information of the forward image acquisition device is the actual installation information of the forward image acquisition device; if the difference value between the first actual position and the first virtual position is larger than or equal to the forward minimum value, resetting the installation information of the forward image acquisition device, and executing subsequent operation until the actual installation information of the forward image acquisition device is determined;
the second processing module is used for setting the installation information of the backward image acquisition device; calculating a second virtual position of a preset corner point of a second calibration cloth in a virtual image of the second calibration cloth generated by the backward image acquisition device in an installation state according to currently set installation information under the condition that the vehicle is at the preset position according to the position of the backward image acquisition device in the world coordinate system; if the difference value between the second actual position and the second virtual position is smaller than the backward minimum value, the currently set installation information of the backward image acquisition device is the actual installation information of the backward image acquisition device; if the difference value between the second actual position and the second virtual position is larger than or equal to the backward minimum value, resetting the installation information of the backward image acquisition device, and executing subsequent operation until the actual installation information of the backward image acquisition device is determined;
the third processing module is used for setting the installation information of the left image acquisition device; calculating a third virtual position of a preset corner point of a third calibration cloth in a virtual image of the third calibration cloth generated by the left image acquisition device in an installation state according to currently set installation information under the condition that the vehicle is at the preset position according to the position of the left image acquisition device in the world coordinate system; if the difference value between the third actual position and the third virtual position is smaller than a minimum value in the left direction, the currently set installation information of the left image acquisition device is the actual installation information of the left image acquisition device; if the difference value between the third actual position and the third virtual position is larger than or equal to the minimum value in the left direction, resetting the installation information of the left image acquisition device, and executing subsequent operations until the actual installation information of the left image acquisition device is determined;
the fourth processing module is used for setting the installation information of the right image acquisition device; according to the position of the right image acquisition device in the world coordinate system, under the condition that the vehicle is at the preset position, calculating a fourth virtual position of a preset corner point of a fourth calibration cloth in a virtual image of the fourth calibration cloth generated by the right image acquisition device in the installation state according to the currently set installation information; if the difference value between the fourth actual position and the fourth virtual position is smaller than the minimum right value, the currently set installation information of the right image acquisition device is the actual installation information of the right image acquisition device; if the difference value between the fourth actual position and the fourth virtual position is larger than or equal to the right minimum numerical value, resetting the installation information of the right image acquisition device, and executing subsequent operations until the actual installation information of the right image acquisition device is determined.
CN201610873420.XA 2016-09-30 2016-09-30 Image splicing calibration method and image splicing calibration device of panoramic parking system Active CN107886472B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610873420.XA CN107886472B (en) 2016-09-30 2016-09-30 Image splicing calibration method and image splicing calibration device of panoramic parking system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610873420.XA CN107886472B (en) 2016-09-30 2016-09-30 Image splicing calibration method and image splicing calibration device of panoramic parking system

Publications (2)

Publication Number Publication Date
CN107886472A CN107886472A (en) 2018-04-06
CN107886472B true CN107886472B (en) 2023-03-03

Family

ID=61769283

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610873420.XA Active CN107886472B (en) 2016-09-30 2016-09-30 Image splicing calibration method and image splicing calibration device of panoramic parking system

Country Status (1)

Country Link
CN (1) CN107886472B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109509232B (en) * 2018-10-09 2023-09-12 珠海大轩信息科技有限公司 Automatic calibration system for off-line of panoramic image system
CN109767473B (en) * 2018-12-30 2022-10-28 惠州华阳通用电子有限公司 Panoramic parking device calibration method and device

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102736634A (en) * 2012-07-10 2012-10-17 浙江捷尚视觉科技有限公司 Camera angle regulation method for vehicle panorama
CN103871071A (en) * 2014-04-08 2014-06-18 北京经纬恒润科技有限公司 Method for camera external reference calibration for panoramic parking system
CN104036275A (en) * 2014-05-22 2014-09-10 东软集团股份有限公司 Method and device for detecting target objects in vehicle blind areas
CN104282010A (en) * 2013-07-06 2015-01-14 南京车联网研究院有限公司 Vehicle multi-fisheye-camera 360-degree overhead-view image splicing curve calibration method

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5870486A (en) * 1991-12-11 1999-02-09 Texas Instruments Incorporated Method of inferring sensor attitude through multi-feature tracking
US10444406B2 (en) * 2014-04-17 2019-10-15 Siemens Aktiengesellschaft Short term cloud coverage prediction using ground-based all sky imaging
US20150302575A1 (en) * 2014-04-17 2015-10-22 Siemens Aktiengesellschaft Sun location prediction in image space with astronomical almanac-based calibration using ground based camera

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102736634A (en) * 2012-07-10 2012-10-17 浙江捷尚视觉科技有限公司 Camera angle regulation method for vehicle panorama
CN104282010A (en) * 2013-07-06 2015-01-14 南京车联网研究院有限公司 Vehicle multi-fisheye-camera 360-degree overhead-view image splicing curve calibration method
CN103871071A (en) * 2014-04-08 2014-06-18 北京经纬恒润科技有限公司 Method for camera external reference calibration for panoramic parking system
CN104036275A (en) * 2014-05-22 2014-09-10 东软集团股份有限公司 Method and device for detecting target objects in vehicle blind areas

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
卢官明 等;全景视图泊车辅助系统中的多视点视频拼接;《南京邮电大学学报》;20160629;全文 *

Also Published As

Publication number Publication date
CN107886472A (en) 2018-04-06

Similar Documents

Publication Publication Date Title
CN107856667B (en) Parking assist system and method
CN110567469B (en) Visual positioning method and device, electronic equipment and system
US8373751B2 (en) Apparatus and method for measuring location and distance of object by using camera
EP3678096B1 (en) Method for calculating a tow hitch position
CN109766757B (en) Parking space high-precision positioning method and system integrating vehicle and visual information
US20130002861A1 (en) Camera distance measurement device
US20080140286A1 (en) Parking Trace Recognition Apparatus and Automatic Parking System
JP2014146182A (en) In-vehicle image processing apparatus
US20200286244A1 (en) Image processing method and apparatus
JP2008131177A (en) Correcting device for on-board camera, correcting method, and production method for vehicle using same correcting method
JP6779365B2 (en) Object detection device and vehicle
CN112614192B (en) On-line calibration method of vehicle-mounted camera and vehicle-mounted information entertainment system
US9162621B2 (en) Parking support apparatus
CN111678518B (en) Visual positioning method for correcting automatic parking path
US20240109490A1 (en) Imaging system and method
JP6035620B2 (en) On-vehicle stereo camera system and calibration method thereof
US11880993B2 (en) Image processing device, driving assistance system, image processing method, and program
CN112927309A (en) Vehicle-mounted camera calibration method and device, vehicle-mounted camera and storage medium
CN107895344B (en) Video splicing device and method
CN107886472B (en) Image splicing calibration method and image splicing calibration device of panoramic parking system
JP2012198857A (en) Approaching object detector and approaching object detection method
KR20170057684A (en) Method for assistance parking vehicle using front camera
US9827906B2 (en) Image processing apparatus
JP6802999B2 (en) Compartment line detection system
CN112819711A (en) Monocular vision-based vehicle reverse positioning method utilizing road lane line

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant