CN113436275A - Method and system for determining berth size based on calibration plate - Google Patents
Method and system for determining berth size based on calibration plate Download PDFInfo
- Publication number
- CN113436275A CN113436275A CN202110783139.8A CN202110783139A CN113436275A CN 113436275 A CN113436275 A CN 113436275A CN 202110783139 A CN202110783139 A CN 202110783139A CN 113436275 A CN113436275 A CN 113436275A
- Authority
- CN
- China
- Prior art keywords
- coordinates
- calibration
- berth
- modes
- mode
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 33
- 238000001514 detection method Methods 0.000 claims abstract description 33
- 238000012163 sequencing technique Methods 0.000 claims description 6
- 238000010276 construction Methods 0.000 abstract description 4
- 230000004927 fusion Effects 0.000 abstract description 4
- 239000002352 surface water Substances 0.000 abstract description 4
- 238000012216 screening Methods 0.000 abstract description 2
- 230000006870 function Effects 0.000 description 15
- 238000013461 design Methods 0.000 description 3
- 238000005259 measurement Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000011426 transformation method Methods 0.000 description 1
- 239000013598 vector Substances 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
- G06T2207/30256—Lane; Road marking
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Length Measuring Devices By Optical Means (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method and a system for determining a parking space size based on a calibration plate, which relate to the field of intelligent parking management and comprise the following steps: utilizing an existing calibration plate detection model, screening out calibration plate picture data containing different placing modes according to the difference of detection model results, utilizing three modes to calibrate the plate picture data, carrying out camera parameter estimation through inner corner coordinate detection and corresponding preset world coordinate system coordinates, and utilizing data fitting to obtain a depth estimation function of a surface water plane under the same coordinate system; and finally, projecting the four vertex coordinates of the acquired parking position wire frame into a world coordinate system of a corresponding mode, and finally acquiring parking position size information in a mode of mode fusion. The invention can realize automatic, efficient, accurate and rapid berth size estimation without increasing excessive manual operation steps, and provides important data for subsequent three-dimensional scene construction.
Description
Technical Field
The invention relates to the field of intelligent parking management, in particular to a method and a system for determining a parking space size based on a calibration plate.
Background
Under a static traffic scene, a parking position line is used as an important indicator for judging whether vehicles run/park violation, accurate estimation of various parameters such as position, size and the like is facilitated, more accurate estimation and judgment of matching relation of the vehicles and the parking positions under a complex scene are facilitated, for example, under the condition that a parking lot is busy, the front and back shielding of the vehicles is obvious, and when the ground key point information of the vehicles cannot be accurately acquired, whether the vehicles are located in a certain specific parking position or not and whether a parking position crossing condition exists or not is difficult to judge only through two-dimensional image information of a single visual angle. However, the common color image lacks depth information and the image is not equivalent to scale, so that it is difficult to estimate the size of its internal target, such as relative size and absolute size.
However, the physical size information of the parking line is currently used as an important parameter for three-dimensional parking lot scene reconstruction, a unified and concise method for estimating is not provided at present under the condition of a non-fixed camera view angle, and the parameter is generally obtained by a field measurement mode; however, the parking spaces of a parking lot are not of uniform size, and the sizes of the no-parking area and the normal parking area are inconsistent in a large probability, so that the operation complexity is brought to field manual measurement and data storage, and later field measurement data and camera parameter matching, and the standardized and accurate operation process is not easy to make.
Disclosure of Invention
In order to solve the technical problems, the invention provides a method and a system for determining the size of a berth based on a calibration plate, which realize the automatic determination of the size of the berth and can solve the problems of higher operation difficulty, lower accuracy and efficiency of determining the size of the berth caused by manually determining the size of the berth in the prior art.
In order to achieve the above object, in one aspect, the present invention provides a method for determining a berth size based on a calibration plate, the method including: acquiring images of calibration plates with different placing modes and coordinates Q _ i of inner corners of the calibration plates in the different placing modes in a world coordinate system, wherein the calibration plates are placed in a parking area through the different placing modes;
acquiring coordinates P _ i of inner corner points of the calibration plate in the image according to the image of the calibration plate containing different putting modes, and determining camera parameters transp _ i and potential berth area coordinates relative to the coordinates of the calibration plate in different putting modes according to the P _ i and the Q _ i;
carrying out equidistant x-y plane grid division on the potential berth area coordinates to obtain a grid point coordinate set Qs;
acquiring a depth estimation function f _ depth _ i from the two-dimensional projection coordinates of the berth area to the corresponding camera coordinate system according to the transp _ i and the Qs;
and determining the size of the berth according to the transp _ i, the f _ depth _ i and the position information of the four vertex coordinates of the berth in the pictures acquired by the camera corresponding to the transp _ i under the world coordinate system.
Further, the step of acquiring coordinates Q _ i of the corner point in the fixed board under the world coordinate system under different placing modes includes:
configuring the coordinates of the inner corner point of the left lower corner of the calibration board as the coordinates of the origin of a world coordinate system in the current mode, and calculating the coordinates of any inner corner point in the calibration board according to the actual physical size and the placing mode of the calibration board;
and carrying out preset mode sequencing on any inner corner point coordinate in the calibration plate to obtain the Q _ i.
Further, the step of obtaining the coordinates P _ i of the inner corner point of the calibration board in the image according to the image of the calibration board including different placing modes includes:
carrying out data contrast enhancement on local pictures in the images of the calibration plates containing different placing modes and carrying out inner corner point detection on the calibration plates;
and restoring the coordinates P _ i of the inner corner points in the image according to the positions of the detection frames of the inner corner points.
Further, before the step of determining a camera parameter transp _ i in a preset placing mode according to the P _ i and the Q _ i, the method further includes:
acquiring linear slopes K1 and K2 of the orthogonal sides of the berths in the image of the calibration board containing different putting modes, and acquiring slopes K1_ i and K2_ i of two actual orthogonal sides of the calibration board in the picture according to P _ i, wherein K1_ i < K2_ i;
comparing the characteristic slopes K1_ i and K2_ i of the calibration plates in different placing modes with K1 and K2 to judge whether the placing mode of the current calibration plate is configured correctly;
the step of determining the camera parameters transp _ i in different putting modes according to the P _ i and the Q _ i comprises the following steps:
and if the configuration of the placing mode of the current calibration plate is correct, determining a camera parameter transp _ i under different placing modes according to the P _ i and the Q _ i.
Further, the step of comparing the characteristic slopes K1_ i, K2_ i of the calibration boards in different placing modes with K1, K2 to determine whether the preset placing mode of the current calibration board is configured correctly includes:
if abs (K1_ i-K1) < th3 and abs (K2_ i-K2) < th3, confirming that the preset placing mode of the current calibration board is placing mode 1, wherein the slope error threshold is th 3;
if abs (K1_ i-K1) < th3 and abs (K2_ i-K2) > th3, confirming that the preset placing mode of the current calibration board is placing mode 2;
if abs (K2_ i-K1) > th3 and abs (K1_ i-K2) < th3, it is confirmed that the preset putting pattern of the current calibration board is the putting pattern 3.
Further, the step of obtaining the depth estimation function f _ depth _ i from the two-dimensional projection coordinates of the berth area to the corresponding camera coordinate system according to the transp _ i and the Qs comprises:
back-projecting the Qs to a camera coordinate system according to the transp _ i to obtain a three-dimensional coordinate Ps3d _ i and a two-dimensional projection coordinate Ps2d _ i of a corresponding picture;
and performing slicing interpolation according to the Ps2d _ i and the Ps3d _ i (z) to obtain the f _ depth _ i.
Further, the step of determining the size of the berth according to the transp _ i, the f _ depth _ i and the coordinates of the four vertexes of the berth in the picture comprises:
obtaining depth values Pb _ i _ z of four vertex coordinates of the parking space in a camera coordinate system through the depth estimation function f _ depth _ i;
and acquiring coordinates Qb _ i in a world coordinate system corresponding to the coordinates of four vertexes of the berth and two side lengths l1_ i and l2_ i corresponding to the berth according to the transp _ i.
In another aspect, the present invention provides a calibration plate based berth size determining system, comprising: the system comprises an acquisition module, a storage module and a display module, wherein the acquisition module is used for acquiring images of calibration plates containing different placing modes and coordinates Q _ i of inner corners of the calibration plates under the different placing modes in a world coordinate system, and the calibration plates are placed in a parking area through the different placing modes;
the determining module is used for acquiring a coordinate P _ i of an inner corner point of the calibration plate in the image according to the image of the calibration plate containing different putting modes, and determining a camera parameter transp _ i and a potential berth area coordinate relative to the coordinate of the calibration plate in different putting modes according to the P _ i and the Q _ i;
the dividing module is used for carrying out equidistant x-y plane grid division on the potential berth area coordinates to obtain a grid point coordinate set Qs;
the acquisition module is further used for acquiring a depth estimation function f _ depth _ i from the two-dimensional projection coordinates of the berth area to a corresponding camera coordinate system according to the transp _ i and the Qs;
the determining module is further configured to determine the size of the berth according to the transp _ i, the f _ depth _ i, and position information of four vertex coordinates of the berth in a world coordinate system in a picture acquired by a camera corresponding to the transp _ i.
Further, the obtaining module is specifically configured to configure the coordinates of the inner corner point of the lower left corner of the calibration board as the coordinates of the origin of the world coordinate system in the current mode, and calculate the coordinates of any inner corner point in the calibration board according to the actual physical size and the placement mode of the calibration board; and carrying out preset mode sequencing on any inner corner point coordinate in the calibration plate to obtain the Q _ i.
Further, the obtaining module is specifically configured to perform data contrast enhancement on a local picture in the image of the calibration board including different placement modes and perform inner corner detection on the calibration board; and restoring the coordinates P _ i of the inner corner points in the image according to the positions of the detection frames of the inner corner points.
Further, the obtaining module is further configured to obtain linear slopes K1 and K2 of the orthogonal sides of the berths in the image of the calibration board containing different putting modes, and obtain slopes K1_ i and K2_ i of two actual orthogonal sides of the calibration board in the picture according to P _ i, where K1_ i < K2_ i; comparing the characteristic slopes K1_ i and K2_ i of the calibration plates in different placing modes with K1 and K2 to judge whether the placing mode of the current calibration plate is configured correctly;
the determining module is specifically configured to determine, if the placement mode configuration of the current calibration board is correct, a camera parameter transp _ i in different placement modes according to the P _ i and the Q _ i.
Further, the obtaining module is specifically configured to, if abs (K1_ i-K1) < th3 and abs (K2_ i-K2) < th3, confirm that the preset placing mode of the current calibration board is placing mode 1, where a slope error threshold is th 3; if abs (K1_ i-K1) < th3 and abs (K2_ i-K2) > th3, confirming that the preset placing mode of the current calibration board is placing mode 2; if abs (K2_ i-K1) > th3 and abs (K1_ i-K2) < th3, it is confirmed that the preset putting pattern of the current calibration board is the putting pattern 3.
Further, the obtaining module is specifically configured to back-project the Qs into a camera coordinate system according to the transp _ i to obtain three-dimensional coordinates Ps3d _ i and two-dimensional projection coordinates Ps2d _ i of a corresponding picture; and performing slicing interpolation according to the Ps2d _ i and the Ps3d _ i (z) to obtain the f _ depth _ i.
Further, the obtaining module is specifically configured to obtain, through the depth estimation function f _ depth _ i, a depth value Pb _ i _ z of coordinates of four vertices of the parking space in a camera coordinate system; and acquiring coordinates Qb _ i in a world coordinate system corresponding to the coordinates of four vertexes of the berth and two side lengths l1_ i and l2_ i corresponding to the berth according to the transp _ i.
The invention provides a method and a system for determining a berth size based on a calibration plate, which are characterized in that the existing calibration plate detection model is utilized, calibration plate picture data containing different placing modes are screened out according to the difference of detection model results, then the calibration plate picture data of three placing modes are utilized, camera parameter estimation is carried out through inner corner coordinate detection and corresponding preset world coordinate system coordinates, and a depth estimation function of a surface water plane under the same coordinate system is obtained through data fitting; and finally, projecting the four vertex coordinates of the acquired parking position wire frame into a world coordinate system of a corresponding mode, and finally acquiring parking position size information in a mode of mode fusion. The invention can realize automatic, efficient, accurate and rapid berth size estimation without increasing excessive manual operation steps, and provides important data for subsequent three-dimensional scene construction.
Drawings
FIG. 1 is a flow chart of a method for determining a parking space size based on a calibration plate according to the present invention;
FIG. 2 is a schematic structural diagram of a calibration plate-based berth size determining system provided by the invention;
FIG. 3 shows a calibration board layout pattern 1 according to the present invention;
FIG. 4 shows a calibration board layout pattern 2 according to the present invention;
fig. 5 shows a calibration board layout pattern 3 according to the present invention.
Detailed Description
The technical solution of the present invention is further described in detail by the accompanying drawings and embodiments.
As shown in fig. 1, a method for determining a berth size based on a calibration plate according to an embodiment of the present invention includes the following steps:
101. the method comprises the steps of obtaining images of calibration plates with different placing modes and coordinates Q _ i of inner corners of the calibration plates in the different placing modes under a world coordinate system, and placing the calibration plates in a parking area through the different placing modes.
Before step 101, a yard configuration preparation operation needs to be performed, which specifically includes: firstly, selecting the central position which is closer to the camera and can completely shoot the checkerboard image, and then placing the calibration board according to three preset calibration board placing modes: assuming that two sides of the berth are defined as x and y axes and the vertical x-y plane is defined as z axis, the three-time placing planes of the calibration plate are required to be respectively parallel to the x-y plane, the x-z plane and the y-z plane and are respectively set as a mode 1, a mode 2 and a mode 3, and specific examples can refer to fig. 3, 4 and 5.
For the embodiment of the present invention, the step of acquiring the image of the calibration board including different placing modes may specifically include: firstly, setting a detection score threshold th1 and a feature similarity threshold th2, carrying out frame-by-frame calibration board detection on an acquired picture time sequence, recording picture codes, detection frame coordinates p1 and local feature vectors feat1 in a detection frame, storing a result of which the first detection score is greater than the threshold th1, and setting the picture as a mode 1 b; then, the same calibration board detection procedure is performed for the subsequent frame picture, and corresponding position information p _ j and feature information feat _ j are obtained, and compared with the detection result of the existing mode, parameters Dp _ ji ═ norm (p _ j-p _ i)/norm (p _ i), (j < i, j ═ 2,3), Dfeat _ ji ═ cos (feat _ j, feat _ i), (j < i, j ═ 2,3) are obtained, if Dp _ ji is greater than the threshold th1 and Dfeat _ ji is greater than the threshold th2, the detection result is saved, and the picture is set to be the mode jb.
For the embodiment of the present invention, the step of obtaining the coordinate Q _ i of the corner point in the fixed board in the world coordinate system under different placing modes includes: configuring the coordinates of the inner corner point of the left lower corner of the calibration board as the coordinates of the origin of a world coordinate system in the current mode, and calculating the coordinates of any inner corner point in the calibration board according to the actual physical size and the placing mode of the calibration board; and carrying out preset mode sequencing on any inner corner point coordinate in the calibration plate to obtain the Q _ i. Specifically, assuming that the coordinates of the inner corner point at the lower left corner of the calibration board are the coordinates of the origin of the world coordinate system in the current mode in any mode, the coordinates of the inner corner point at any position in the calibration board can be calculated according to the actual physical dimensions of the calibration board, such as the width and arrangement mode of the unit blocks, and after the calculation is completed, the point groups are subjected to specific mode sorting Q _ i, for example, sorting line by line or column by column from the upper left to the lower right, and the embodiment of the present invention is not limited.
102. And acquiring coordinates P _ i of the inner corner points of the calibration plate in the image according to the image of the calibration plate containing different putting modes, and determining a camera parameter transp _ i and potential berth area coordinates relative to the coordinates of the calibration plate in different putting modes according to the P _ i and the Q _ i.
For the embodiment of the present invention, the step of obtaining the coordinate P _ i of the inner corner point of the calibration board in the image according to the image of the calibration board including different placing modes includes: carrying out data contrast enhancement on local pictures in the images of the calibration plates containing different placing modes and carrying out inner corner point detection on the calibration plates; and restoring the coordinates P _ i of the inner corner points in the image according to the positions of the detection frames of the inner corner points.
For the embodiment of the present invention, in order to further improve the accuracy of estimating the berthage size, after step 102 and before step 103, the calibration board placement mode may be further modified, which specifically includes: acquiring linear slopes K1 and K2 of the orthogonal sides of the berths in the image of the calibration board containing different putting modes, and acquiring slopes K1_ i and K2_ i of two actual orthogonal sides of the calibration board in the picture according to P _ i, wherein K1_ i < K2_ i; if abs (K1_ i-K1) < th3 and abs (K2_ i-K2) < th3, confirming that the preset placing mode of the current calibration board is placing mode 1, wherein the slope error threshold is th 3; if abs (K1_ i-K1) < th3 and abs (K2_ i-K2) > th3, confirming that the preset placing mode of the current calibration board is placing mode 2; if abs (K2_ i-K1) > th3 and abs (K1_ i-K2) < th3, it is confirmed that the preset putting pattern of the current calibration board is the putting pattern 3.
Specifically, the step of acquiring the slope K1 and K2 of the straight line of the orthogonal side of the berth in the image of the calibration board containing different laying modes may include: if the image definition is high, the parking space line is clear and no vehicle is shielded, two orthogonal parking space side characteristic lines can be obtained by using an automatic straight line detection method, such as a Huffman transformation method, parking space line segmentation and other methods; if the definition of the berthage line in the picture is insufficient and an automatic method cannot be used, two orthogonal berthage edge characteristic lines are obtained by manually marking coordinates of four vertexes of the berthage, and the corresponding slope of the characteristic lines is further calculated.
103. And carrying out equidistant x-y plane meshing on the potential berth area coordinates to obtain a grid point coordinate set Qs.
104. And acquiring a depth estimation function f _ depth _ i from the two-dimensional projection coordinates of the berth region to the corresponding camera coordinate system according to the transp _ i and the Qs.
For the embodiment of the present invention, step 104 may specifically include: back-projecting the Qs to a camera coordinate system according to the transp _ i to obtain a three-dimensional coordinate Ps3d _ i and a two-dimensional projection coordinate Ps2d _ i of a corresponding picture; and performing slicing interpolation according to the Ps2d _ i and the Ps3d _ i (z) to obtain the f _ depth _ i.
105. And determining the size of the berth according to the transp _ i, the f _ depth _ i and the position information of the four vertex coordinates of the berth in the pictures acquired by the camera corresponding to the transp _ i under the world coordinate system.
For the embodiment of the present invention, step 105 may specifically include: obtaining depth values Pb _ i _ z of four vertex coordinates of the parking space in a camera coordinate system through the depth estimation function f _ depth _ i; and acquiring coordinates Qb _ i in a world coordinate system corresponding to the coordinates of four vertexes of the berth and two side lengths l1_ i and l2_ i corresponding to the berth according to the transp _ i.
The method for determining the berth size based on the calibration plate comprises the steps of screening calibration plate picture data containing different placing modes by utilizing an existing calibration plate detection model according to the difference of detection model results, then utilizing three placing modes to calibrate the plate picture data, carrying out camera parameter estimation through inner corner coordinate detection and corresponding preset world coordinate system coordinates, and obtaining a depth estimation function of a surface water plane under the same coordinate system by utilizing data fitting; and finally, projecting the four vertex coordinates of the acquired parking position wire frame into a world coordinate system of a corresponding mode, and finally acquiring parking position size information in a mode of mode fusion. The invention can realize automatic, efficient, accurate and rapid berth size estimation without increasing excessive manual operation steps, and provides important data for subsequent three-dimensional scene construction.
In order to implement the method provided by the embodiment of the present invention, an embodiment of the present invention provides a system for determining a parking space size based on a calibration plate, and as shown in fig. 2, the system includes: the device comprises an acquisition module 21, a determination module 22 and a division module 23.
The obtaining module 21 is configured to obtain an image of a calibration board including different placement modes and coordinates Q _ i of an inner corner of the calibration board in the different placement modes in a world coordinate system, where the calibration board is placed in a parking area in the different placement modes.
The determining module 22 is configured to obtain, according to the image of the calibration board including the different placement modes, a coordinate P _ i of an inner corner point of the calibration board in the image, and determine, according to the P _ i and the Q _ i, a camera parameter transp _ i and a potential parking area coordinate for the coordinate of the calibration board in the different placement modes.
And the dividing module 23 is configured to perform equidistant x-y planar grid division on the potential parking area coordinates to obtain a grid point coordinate set Qs.
The obtaining module 21 is further configured to obtain a depth estimation function f _ depth _ i from the two-dimensional projection coordinates of the parking space region to the corresponding camera coordinate system according to the transp _ i and the Qs.
The determining module 22 is further configured to determine the size of the berth according to the transp _ i, the f _ depth _ i, and position information of four vertex coordinates of the berth in a world coordinate system in a picture acquired by a camera corresponding to the transp _ i.
Further, the obtaining module 21 is specifically configured to configure the coordinates of the inner corner point of the lower left corner of the calibration board as the coordinates of the origin of the world coordinate system in the current mode, and calculate the coordinates of any inner corner point in the calibration board according to the actual physical size and the placement mode of the calibration board; and carrying out preset mode sequencing on any inner corner point coordinate in the calibration plate to obtain the Q _ i.
Further, the obtaining module 21 is specifically configured to perform data contrast enhancement on a local picture in the image of the calibration board including different placing modes and perform inner corner detection on the calibration board; and restoring the coordinates P _ i of the inner corner points in the image according to the positions of the detection frames of the inner corner points.
Further, the obtaining module 21 is further configured to obtain linear slopes K1 and K2 of the orthogonal sides of the berths in the image of the calibration board containing different putting modes, and obtain slopes K1_ i and K2_ i of two actual orthogonal sides of the calibration board in the picture according to P _ i, where K1_ i < K2_ i; comparing the characteristic slopes K1_ i and K2_ i of the calibration plates in different placing modes with K1 and K2 to judge whether the placing mode of the current calibration plate is configured correctly;
at this time, the determining module 22 is specifically configured to determine, according to the P _ i and the Q _ i, a camera parameter transp _ i in different placement modes if the placement mode configuration of the current calibration board is correct.
Further, the obtaining module 21 is further specifically configured to, if abs (K1_ i-K1) < th3 and abs (K2_ i-K2) < th3, confirm that the preset placing mode of the current calibration board is placing mode 1, where a slope error threshold is th 3; if abs (K1_ i-K1) < th3 and abs (K2_ i-K2) > th3, confirming that the preset placing mode of the current calibration board is placing mode 2; if abs (K2_ i-K1) > th3 and abs (K1_ i-K2) < th3, it is confirmed that the preset putting pattern of the current calibration board is the putting pattern 3.
Further, the obtaining module 21 is specifically configured to back-project the Qs into a camera coordinate system according to the transp _ i to obtain three-dimensional coordinates Ps3d _ i and two-dimensional projection coordinates Ps2d _ i of a corresponding picture; and performing slicing interpolation according to the Ps2d _ i and the Ps3d _ i (z) to obtain the f _ depth _ i.
Further, the obtaining module 21 is specifically configured to obtain, through the depth estimation function f _ depth _ i, a depth value Pb _ i _ z of coordinates of four vertices of the parking space in the camera coordinate system; and acquiring coordinates Qb _ i in a world coordinate system corresponding to the coordinates of four vertexes of the berth and two side lengths l1_ i and l2_ i corresponding to the berth according to the transp _ i.
According to the berth size determining system based on the calibration plate, provided by the embodiment of the invention, the existing calibration plate detection model is utilized, the calibration plate picture data containing different placing modes are screened out according to the difference of detection model results, then the calibration plate picture data of three placing modes are utilized, the camera parameter estimation is carried out through the inner corner coordinate detection and the corresponding preset world coordinate system coordinate, and the depth estimation function of the surface water plane under the same coordinate system is obtained through data fitting; and finally, projecting the four vertex coordinates of the acquired parking position wire frame into a world coordinate system of a corresponding mode, and finally acquiring parking position size information in a mode of mode fusion. The invention can realize automatic, efficient, accurate and rapid berth size estimation without increasing excessive manual operation steps, and provides important data for subsequent three-dimensional scene construction.
It should be understood that the specific order or hierarchy of steps in the processes disclosed is an example of exemplary approaches. Based upon design preferences, it is understood that the specific order or hierarchy of steps in the processes may be rearranged without departing from the scope of the present disclosure. The accompanying method claims present elements of the various steps in a sample order, and are not intended to be limited to the specific order or hierarchy presented.
In the foregoing detailed description, various features are grouped together in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments of the subject matter require more features than are expressly recited in each claim. Rather, as the following claims reflect, invention lies in less than all features of a single disclosed embodiment. Thus, the following claims are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate preferred embodiment of the invention.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. To those skilled in the art; various modifications to these embodiments will be readily apparent, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the disclosure. Thus, the present disclosure is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
What has been described above includes examples of one or more embodiments. It is, of course, not possible to describe every conceivable combination of components or methodologies for purposes of describing the aforementioned embodiments, but one of ordinary skill in the art may recognize that many further combinations and permutations of various embodiments are possible. Accordingly, the embodiments described herein are intended to embrace all such alterations, modifications and variations that fall within the scope of the appended claims. Furthermore, to the extent that the term "includes" is used in either the detailed description or the claims, such term is intended to be inclusive in a manner similar to the term "comprising" as "comprising" is interpreted when employed as a transitional word in a claim. Furthermore, any use of the term "or" in the specification of the claims is intended to mean a "non-exclusive or".
Those of skill in the art will further appreciate that the various illustrative logical blocks, units, and steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate the interchangeability of hardware and software, various illustrative components, elements, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design requirements of the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present embodiments.
The various illustrative logical blocks, or elements, described in connection with the embodiments disclosed herein may be implemented or performed with a general purpose processor, a digital signal processor, an Application Specific Integrated Circuit (ASIC), a field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a digital signal processor core, or any other similar configuration.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may be stored in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. For example, a storage medium may be coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC, which may be located in a user terminal. In the alternative, the processor and the storage medium may reside in different components in a user terminal.
In one or more exemplary designs, the functions described above in connection with the embodiments of the invention may be implemented in hardware, software, firmware, or any combination of the three. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media that facilitate transfer of a computer program from one place to another. Storage media may be any available media that can be accessed by a general purpose or special purpose computer. For example, such computer-readable media can include, but is not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to carry or store program code in the form of instructions or data structures and which can be read by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Additionally, any connection is properly termed a computer-readable medium, and, thus, is included if the software is transmitted from a website, server, or other remote source via a coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), or wirelessly, e.g., infrared, radio, and microwave. Such discs (disk) and disks (disc) include compact disks, laser disks, optical disks, DVDs, floppy disks and blu-ray disks where disks usually reproduce data magnetically, while disks usually reproduce data optically with lasers. Combinations of the above may also be included in the computer-readable medium.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are merely exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (14)
1. A method for determining a berth size based on a calibration plate is characterized by comprising the following steps:
acquiring images of calibration plates with different placing modes and coordinates Q _ i of inner corners of the calibration plates in the different placing modes in a world coordinate system, wherein the calibration plates are placed in a parking area through the different placing modes;
acquiring coordinates P _ i of inner corner points of the calibration plate in the image according to the image of the calibration plate containing different putting modes, and determining camera parameters transp _ i and potential berth area coordinates relative to the coordinates of the calibration plate in different putting modes according to the P _ i and the Q _ i;
carrying out equidistant x-y plane grid division on the potential berth area coordinates to obtain a grid point coordinate set Qs;
acquiring a depth estimation function f _ depth _ i from the two-dimensional projection coordinates of the berth area to the corresponding camera coordinate system according to the transp _ i and the Qs;
and determining the size of the berth according to the transp _ i, the f _ depth _ i and the position information of the four vertex coordinates of the berth in the pictures acquired by the camera corresponding to the transp _ i under the world coordinate system.
2. The method of claim 1, wherein the step of obtaining coordinates Q _ i of the corner point in the calibration board in the world coordinate system under different placing modes comprises:
configuring the coordinates of the inner corner point of the left lower corner of the calibration board as the coordinates of the origin of a world coordinate system in the current mode, and calculating the coordinates of any inner corner point in the calibration board according to the actual physical size and the placing mode of the calibration board;
and carrying out preset mode sequencing on any inner corner point coordinate in the calibration plate to obtain the Q _ i.
3. The method according to claim 1, wherein the step of obtaining coordinates P _ i of the corner points in the calibration board in the image according to the image of the calibration board containing different putting modes comprises:
carrying out data contrast enhancement on local pictures in the images of the calibration plates containing different placing modes and carrying out inner corner point detection on the calibration plates;
and restoring the coordinates P _ i of the inner corner points in the image according to the positions of the detection frames of the inner corner points.
4. The method of claim 1, wherein before the step of determining the camera parameter transp _ i in the preset parking mode according to the P _ i and the Q _ i, the method further comprises:
acquiring linear slopes K1 and K2 of the orthogonal sides of the berths in the image of the calibration board containing different putting modes, and acquiring slopes K1_ i and K2_ i of two actual orthogonal sides of the calibration board in the picture according to P _ i, wherein K1_ i < K2_ i;
comparing the characteristic slopes K1_ i and K2_ i of the calibration plates in different placing modes with K1 and K2 to judge whether the placing mode of the current calibration plate is configured correctly;
the step of determining the camera parameters transp _ i in different putting modes according to the P _ i and the Q _ i comprises the following steps:
and if the configuration of the placing mode of the current calibration plate is correct, determining a camera parameter transp _ i under different placing modes according to the P _ i and the Q _ i.
5. The method of claim 4, wherein the step of comparing the slope K1_ i, K2_ i of the calibration board feature with K1, K2 in different placement modes to determine whether the preset placement mode of the current calibration board is configured correctly comprises:
if abs (K1_ i-K1) < th3 and abs (K2_ i-K2) < th3, confirming that the preset placing mode of the current calibration board is placing mode 1, wherein the slope error threshold is th 3;
if abs (K1_ i-K1) < th3 and abs (K2_ i-K2) > th3, confirming that the preset placing mode of the current calibration board is placing mode 2;
if abs (K2_ i-K1) > th3 and abs (K1_ i-K2) < th3, it is confirmed that the preset putting pattern of the current calibration board is the putting pattern 3.
6. The calibration plate-based berthage size determining method according to claim 1, wherein the step of acquiring the depth estimation function f _ depth _ i of the two-dimensional projection coordinates of the berthage area to the corresponding camera coordinate system thereof according to the transp _ i and the Qs comprises:
back-projecting the Qs to a camera coordinate system according to the transp _ i to obtain a three-dimensional coordinate Ps3d _ i and a two-dimensional projection coordinate Ps2d _ i of a corresponding picture;
and performing slicing interpolation according to the Ps2d _ i and the Ps3d _ i (z) to obtain the f _ depth _ i.
7. The calibration plate-based berth size determining method according to claim 1, wherein the step of determining the berth size according to the transp _ i, the f _ depth _ i and the four vertex coordinates of the berth in a picture comprises:
obtaining depth values Pb _ i _ z of four vertex coordinates of the parking space in a camera coordinate system through the depth estimation function f _ depth _ i;
and acquiring coordinates Qb _ i in a world coordinate system corresponding to the coordinates of four vertexes of the berth and two side lengths l1_ i and l2_ i corresponding to the berth according to the transp _ i.
8. A calibration plate based berth size determination system, comprising:
the system comprises an acquisition module, a storage module and a display module, wherein the acquisition module is used for acquiring images of calibration plates containing different placing modes and coordinates Q _ i of inner corners of the calibration plates under the different placing modes in a world coordinate system, and the calibration plates are placed in a parking area through the different placing modes;
the determining module is used for acquiring a coordinate P _ i of an inner corner point of the calibration plate in the image according to the image of the calibration plate containing different putting modes, and determining a camera parameter transp _ i and a potential berth area coordinate relative to the coordinate of the calibration plate in different putting modes according to the P _ i and the Q _ i;
the dividing module is used for carrying out equidistant x-y plane grid division on the potential berth area coordinates to obtain a grid point coordinate set Qs;
the acquisition module is further used for acquiring a depth estimation function f _ depth _ i from the two-dimensional projection coordinates of the berth area to a corresponding camera coordinate system according to the transp _ i and the Qs;
the determining module is further configured to determine the size of the berth according to the transp _ i, the f _ depth _ i, and position information of four vertex coordinates of the berth in a world coordinate system in a picture acquired by a camera corresponding to the transp _ i.
9. A calibration plate based berth size determination system of claim 8,
the acquisition module is specifically used for configuring coordinates of an inner corner point of the left lower corner of the calibration board as coordinates of an origin of a world coordinate system in a current mode, and calculating coordinates of any inner corner point in the calibration board according to the actual physical size and the placing mode of the calibration board; and carrying out preset mode sequencing on any inner corner point coordinate in the calibration plate to obtain the Q _ i.
10. A calibration plate based berth size determination system of claim 8,
the acquisition module is specifically further configured to perform data contrast enhancement on a local picture in the image of the calibration board including different placement modes and perform inner corner detection on the calibration board; and restoring the coordinates P _ i of the inner corner points in the image according to the positions of the detection frames of the inner corner points.
11. A calibration plate based berth size determination system of claim 8,
the obtaining module is further configured to obtain linear slopes K1 and K2 of the orthogonal sides of the berths in the image of the calibration board including different placing modes, and obtain slopes K1_ i and K2_ i of two actual orthogonal sides of the calibration board in a picture according to P _ i, where K1_ i < K2_ i; comparing the characteristic slopes K1_ i and K2_ i of the calibration plates in different placing modes with K1 and K2 to judge whether the placing mode of the current calibration plate is configured correctly;
the determining module is specifically configured to determine, if the placement mode configuration of the current calibration board is correct, a camera parameter transp _ i in different placement modes according to the P _ i and the Q _ i.
12. The system of claim 11, wherein the obtaining module is further configured to determine that the preset placement mode of the current calibration board is placement mode 1 if abs (K1_ i-K1) < th3 and abs (K2_ i-K2) < th3, where the slope error threshold is th 3; if abs (K1_ i-K1) < th3 and abs (K2_ i-K2) > th3, confirming that the preset placing mode of the current calibration board is placing mode 2; if abs (K2_ i-K1) > th3 and abs (K1_ i-K2) < th3, it is confirmed that the preset putting pattern of the current calibration board is the putting pattern 3.
13. A calibration plate based berth size determination system of claim 8,
the obtaining module is specifically configured to back-project the Qs to a camera coordinate system according to the transp _ i to obtain a three-dimensional coordinate Ps3d _ i and a two-dimensional projection coordinate Ps2d _ i of a corresponding picture; and performing slicing interpolation according to the Ps2d _ i and the Ps3d _ i (z) to obtain the f _ depth _ i.
14. A calibration plate based berth size determination system of claim 8,
the obtaining module is specifically configured to obtain, through the depth estimation function f _ depth _ i, a depth value Pb _ i _ z of coordinates of four vertices of the parking space in a camera coordinate system; and acquiring coordinates Qb _ i in a world coordinate system corresponding to the coordinates of four vertexes of the berth and two side lengths l1_ i and l2_ i corresponding to the berth according to the transp _ i.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110783139.8A CN113436275B (en) | 2021-07-12 | 2021-07-12 | Berth size determining method and system based on calibration plate |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110783139.8A CN113436275B (en) | 2021-07-12 | 2021-07-12 | Berth size determining method and system based on calibration plate |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113436275A true CN113436275A (en) | 2021-09-24 |
| CN113436275B CN113436275B (en) | 2024-06-25 |
Family
ID=77759963
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110783139.8A Active CN113436275B (en) | 2021-07-12 | 2021-07-12 | Berth size determining method and system based on calibration plate |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113436275B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2010190024A (en) * | 2009-02-13 | 2010-09-02 | Technopark:Kk | Multistory parking system |
| US20180010917A1 (en) * | 2015-01-15 | 2018-01-11 | Nanjing University 5D Technology Co., Ltd. | Auxiliary berthing method and system for vessel |
| JP2018178438A (en) * | 2017-04-07 | 2018-11-15 | Ｉｈｉ運搬機械株式会社 | Self-travelling type parking device for automatic drive vehicle |
| CN110725586A (en) * | 2019-10-25 | 2020-01-24 | 浙江水利水电学院 | Ferris wheel type storage battery car parking garage |
| WO2020056874A1 (en) * | 2018-09-17 | 2020-03-26 | 魔门塔(苏州)科技有限公司 | Automatic parking system and method based on visual recognition |
| CN111243034A (en) * | 2020-01-17 | 2020-06-05 | 广州市晶华精密光学股份有限公司 | Panoramic auxiliary parking calibration method, device, equipment and storage medium |
-
2021
- 2021-07-12 CN CN202110783139.8A patent/CN113436275B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2010190024A (en) * | 2009-02-13 | 2010-09-02 | Technopark:Kk | Multistory parking system |
| US20180010917A1 (en) * | 2015-01-15 | 2018-01-11 | Nanjing University 5D Technology Co., Ltd. | Auxiliary berthing method and system for vessel |
| JP2018178438A (en) * | 2017-04-07 | 2018-11-15 | Ｉｈｉ運搬機械株式会社 | Self-travelling type parking device for automatic drive vehicle |
| WO2020056874A1 (en) * | 2018-09-17 | 2020-03-26 | 魔门塔(苏州)科技有限公司 | Automatic parking system and method based on visual recognition |
| CN110725586A (en) * | 2019-10-25 | 2020-01-24 | 浙江水利水电学院 | Ferris wheel type storage battery car parking garage |
| CN111243034A (en) * | 2020-01-17 | 2020-06-05 | 广州市晶华精密光学股份有限公司 | Panoramic auxiliary parking calibration method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113436275B (en) | 2024-06-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11043000B2 (en) | Measuring method and apparatus for damaged part of vehicle | |
| US8452081B2 (en) | Forming 3D models using multiple images | |
| JP4035978B2 (en) | Three-dimensional shape model evaluation method, generation method, and apparatus | |
| CN113205691B (en) | Method and device for identifying vehicle position | |
| CN103426184A (en) | Optical flow tracking method and device | |
| CN111080662A (en) | Lane line extraction method and device and computer equipment | |
| CN113191329A (en) | Vehicle berth matching method and system based on monocular vision picture | |
| CN111862097A (en) | Data enhancement method and device for micro defect detection rate | |
| CN111640180A (en) | Three-dimensional reconstruction method and device and terminal equipment | |
| CN110298891A (en) | The method and device that Camera extrinsic precision is assessed automatically | |
| CN113658279B (en) | Camera internal reference and external reference estimation method, device, computer equipment and storage medium | |
| CN111161348B (en) | Object pose estimation method, device and equipment based on monocular camera | |
| CN117315046A (en) | Method and device for calibrating looking-around camera, electronic equipment and storage medium | |
| Terpstra et al. | An Evaluation of Two Methodologies for Lens Distortion Removal when EXIF Data is Unavailable | |
| CN115994952B (en) | Calibration method and device for panoramic image system, computer equipment and storage medium | |
| CN113436275B (en) | Berth size determining method and system based on calibration plate | |
| CN115578441B (en) | Vehicle side image stitching and vehicle size measuring method based on deep learning | |
| CN114792343B (en) | Calibration method of image acquisition equipment, method and device for acquiring image data | |
| CN113744361B (en) | Three-dimensional high-precision map construction method and device based on three-dimensional vision | |
| CN116611994A (en) | Image stitching method and device, and glass through hole image positioning method and device | |
| CN115601336A (en) | Method and device for determining target projection and electronic equipment | |
| CN114463976A (en) | Vehicle behavior state determination method and system based on 3D vehicle track | |
| CN114842443A (en) | Target object identification and distance measurement method, device and equipment based on machine vision and storage medium | |
| CN114783172B (en) | Parking lot empty space recognition method and system and computer readable storage medium | |
| CN118015093A (en) | LIDAR and CAMERA combined calibration method and device under weak constraint environment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |