TWI831445B - Method and system for obtaining parameters ofan image capture device - Google Patents

Abstract

A method and system for obtaining parameters of an image capture device, wherein the method includes the following steps: obtaining a plurality of continuous street view images captured by an image capture device at an image capture location, wherein each continuous street view image includes at least one traffic sign image; obtaining a vertex image position of the plurality of the at least one traffic sign image; obtaining a vertex map position of at least one traffic sign of the at least one traffic sign image at the image capture location from a high-precision map system; comparing the vertex image position and the vertex map position to obtain a complex internal parameters and a complex radial distortion coefficients of the image capturing device; and calculating the parameters of the image capturing device by using the complex parameters and the complex radial distortion coefficients.

Description

Image capture device parameter acquisition method and system

The present invention relates to a method and system for obtaining parameters of an image capture device, and in particular, to a method and system for obtaining parameters of an image capture device from a plurality of images to capture the plurality of images.

Traditionally, when the obtained street view images are used for positioning or mapping calculations, internal camera parameters such as focal length, principal point, radial distortion, and tangential distortion need to be obtained first. (tangential distortion) can be used. The way to obtain camera parameters is to first use the reference image obtained by the camera (this image is not an image used for positioning or mapping calculations) as a benchmark, use a chess board for camera calibration, and use the straight lines on the chessboard The geometric relationship at right angles to the intersection of the straight line, and the actual size of the grid in the checkerboard are measured in advance, and the internal parameters of the camera can be calculated. In the future, when the calibrated camera captures street view images, these street images can be The internal parameters of the camera corresponding to the image are used in the calculation of positioning or mapping.

Of course, because the street view images used for positioning or mapping calculations must be calibrated using a checkerboard before the official mapping or positioning and imaging, if the internal parameters of the camera that captures the image are not corrected during the image shooting process, If the image data is changed, the internal parameters of the camera obtained by using the checkerboard positioning of the reference image cannot be used to change the internal parameters of the camera, resulting in the image data (such as videos or pictures) being unable to be used for processing because there are no internal parameters of the camera. Positioning or mapping related calculations. For example, if you want to use a driving recorder in the Street View fleet that does not have internal camera parameters to achieve crowdsourcing, but there is no way to call all the cars in the Street View fleet back to the factory for camera calibration, then there is no internal camera reference in the Street View fleet. The image information obtained by the driving recorder cannot be used for positioning or mapping related calculations, so there is a need for improvement.

The main purpose of the present invention is to provide a method for obtaining parameters of an image capturing device for capturing the plural images from the plural images.

Another main object of the present invention is to provide a system for obtaining parameters of an image capturing device for capturing the plural images from the plural images.

In order to achieve the above object, the method for obtaining parameters of an image capturing device of the present invention includes the following steps: obtaining a plurality of continuous street view images captured by an image capturing device at an image capturing location, where each continuous street view image includes at least one Traffic sign images; obtain a vertex image position of a plurality of at least one traffic sign image; obtain a vertex map position of at least one traffic sign corresponding to the image capture location from a high-precision map system; compare the vertex images The position and the vertex map position are used to obtain the complex internal parameter K (intrinsic parameter) and the complex radial distortion coefficient D (radial distortion) of the image capture device. coefficient); and, calculate the parameters of the image capture device through the complex internal parameter K and the complex radial distortion coefficient D.

The present invention also provides an image capture device parameter acquisition system, which includes a memory, an image information acquisition module, a high-precision map information acquisition module, a comparison module and a calculation module. The memory stores a plurality of continuous street view images captured by an image capture device at an image capture location, where each continuous street view image includes at least one traffic sign image. The image information acquisition module signal is connected to the memory, and the image information acquisition module acquires the vertex image position of a plurality of at least one traffic sign image. The high-precision map information acquisition module signal is connected to a high-precision map system. The high-precision map information acquisition module obtains one of at least one traffic sign corresponding to the image capture location corresponding to the at least one traffic sign image from a high-precision map system. Vertex map location. The comparison module signal is connected to the image information acquisition module and the high-precision map information acquisition module. The comparison module compares the vertex image position and the vertex map position to obtain the plural internal parameters K (intrinsic parameters) of the image capture device and Complex radial distortion coefficient D (radial distortion coefficient). The calculation module signal is connected to the comparison module. The calculation module calculates the parameters of the image capture device through the complex internal parameter K (intrinsic parameter) and the complex radial distortion coefficient D (radial distortion coefficient).

This method uses the street signs in the high-precision map and performs a projection comparison operation on the corresponding street signs in the image to find the vertex of the sign in the image, and then matches it. Through the scale information provided by the high-precision map , the internal parameters of the camera can be calculated. Therefore, this method can solve the problem of being unable to use image data captured by an imaging device without obtaining the internal parameters of the machine in advance.

This project takes advantage of the fact that the four sides of a rectangular road sign are straight lines, and the horizontal or vertical distance between two vertices of the same side of the same road sign is the same, using an image capture device By obtaining multiple continuous street view images, the corresponding relationship between the street sign image and the position of the street sign in the high-precision map (real world) can be found, that is, the longitude and latitude position of the same street sign at the same location in the high-precision map system. In this way, the complex internal parameter K (intrinsic parameter) and the complex radial distortion coefficient D (radial distortion coefficient) of the image capturing device 800 that captures the continuous street view images are found. In this way, after acquiring the street view image, the internal parameters of the camera that acquired the street view image can be calculated by comparing the image position of the rectangular road sign in the image with the corresponding position of the rectangular road sign in the high-precision map system. The source of the street view image required for the picture allows street view images obtained by cameras with unknown internal parameters (such as driving recorders) to be used, which solves the problem in the previous technology that only image capture with known internal parameters of the camera can be used. The limitations of the street view images obtained by the installation.

1: Image capture device parameter acquisition system

10:Memory

20:Image information acquisition module

30: High-precision map information acquisition module

40:Comparison module

50:Computing module

60:Optimize module

700: High-precision map system

710, 710a, 710b, 710c: Vertex map location

911: traffic sign images

21, 21a, 21b, 21c: vertex image position

930:Traffic signs

800:Image capture device

900: Image capture location

910, 910a: continuous street view images

FIG. 1 is a schematic diagram of the hardware architecture of an embodiment of the image capture device parameter acquisition system of the present invention.

Figure 2A is one of a plurality of continuous street view images captured by the image capture device at the image capture location.

Figure 2B is one of two consecutive street view images captured by the image capture device at the image capture location.

Figure 3 is the map information of the image capture location in the high-precision map system.

Figure 4 is a step flow chart of one embodiment of a method for obtaining parameters of an image capture device according to the present invention.

In order to better understand the technical content of the present invention, preferred specific embodiments are described below. Please refer to Figure 1, Figure 2A, Figure 2B, Figure 3 and Figure 4 below for a schematic diagram of the hardware architecture of one embodiment of the image capture device parameter acquisition system of the present invention, and the image capture device captures at the image capture location. Take multiple continuous street view images and the map information of the map of the location captured by the image in the high-precision map system.

As shown in Figure 1, Figure 2A and Figure 2B, in this embodiment, the image capture device parameter acquisition system 1 of the present invention includes a memory 10, an image information acquisition module 20, and a high-precision map information acquisition module 30 , comparison module 40, calculation module 50 and optimization module 60, wherein the memory 10 stores a plurality of continuous street view images 910, 910 ' captured by an image capture device 800 at an image capture location 900, wherein each The continuous street view images 910 and 910 ' each include at least one traffic sign image 911, 912, 913, 911 ' , 912 ' , and 913 ' . In this embodiment, the image capture device 800 can be a camera, a driving recorder, or other optical or electronic devices that can be used to capture images. The traffic sign images 911, 912, 913, 911 ' , 912 ' , and 913 ' are rectangular. The continuous image of the road sign refers to the continuous movement of the image capture device 800 in a certain direction, which can also be understood as the continuous movement of the viewfinder car on the lane, and the image capture device 800 on the car during the movement of the viewfinder car. Continuously capture street view images (Figure 2A and Figure 2B). The image information acquisition module 20 is connected to the memory 10 with a signal. The image information acquisition module 20 acquires the vertex image positions of a plurality of traffic sign images 911, 912, 913, 911 ' , 912 ' , and 913 ' . In this embodiment, the traffic sign Images 911, 912, 913, 911 ' , 912 ' , and 913 ' are in a rectangular shape. The traffic sign image 911 has four vertex image positions 911a, 911b, 911c, and 911d; the traffic sign image 912 has four vertex image positions 912a, 912b, 912c, 912d; the traffic sign image 913 has four vertex image positions 913a, 913b, 913c, 913d; the traffic sign image 911 ' has four vertex image positions 911a ' , 911b ' , 911c ' , 911d ' ; the traffic sign image 912 ' It has four vertex image positions 912a ' , 912b ' , 912c ' , 912d ' ; the traffic sign image 913 ' has four vertex image positions 913a ' , 913b ' , 913c ' , 913d ' . In this embodiment, the vertex image positions of the traffic sign images 911 , 912, 913, 911 ' , 912', 913 ' (such as the vertex image positions 911a, 912a, 913a, 911a ' , 912a ' , 913a ' ...) For these vertices in the corresponding street view images 910, 910 ' , the image positions of the vertices in the respective images to which they belong are marked using chess board or pixel positions.

As shown in Figures 1 and 3, the high-precision map information acquisition module 30 is connected to a high-precision map system 700 through signals, so that the high-precision map information acquisition module 30 obtains the image capture location 900 from the high-precision map system 700. One of the vertex map positions 710 of the traffic signs 711, 712, and 713 corresponding to the traffic sign images 911, 912, and 913, where the vertex map position 710 includes the traffic sign 711 and has four vertex map positions 711a, 711b, 711c, 711d, and the traffic sign 712 There are four vertex map locations 712a, 712b, 712c, 712d, and traffic signs 713. There are four vertex map locations 713a, 713b, 713c, and 713d. The vertex map location 710 provided by the high-precision map system 700 includes each traffic sign 711. , 712, and 713 correspond to the longitude and latitude positions of the vertex map positions 711a, 711b, 711c, 711d, 712a, 712b, 712c, 712d, 713a, 713b, 713c, and 713d. In other words, these vertices are in the real world. location in.

，其中

、

為各該交通標誌影像911、912、913、911'、912'、913之四個頂點位置於各該街景影像910、910a之頂點影像位置，如：頂點影像位置911a、911b、911c、911d、...於所屬街景影像910、910'中使用盤格(chess board)所標出的位置；x、y為高精地圖系統700中於影像擷取地點900對應交通標誌影像911、912、913、911'、912'、913之交通標誌711、712、713對應之各該頂點地圖位置711a、711b、711c、711d、712a、712b、712c、712d、713a、713b、713c、713d之經緯度位置，[h1 h2 h3]為交通標誌影像911之頂點位置與高精地圖系統700之頂點地圖尺度資訊(如：經緯度位置)的轉換關係。 The signal of the comparison module 40 is connected to the image information acquisition module 20 and the high-precision map information acquisition module 30. The comparison module 40 compares the vertex image positions 911a, 911b, 911c, 911d, 12a, 912b, 912c, 912d, 913a, 913b, 913c, 913d, 911a ' , 911b ' , 911c ' , 911d ' , 912a ' , 912b ' , 912c ' , 912d ' , 913a ' , 913b ' , 913c ' , 913d ' and vertex map positions 711a, 711b , 711c, 711d, 712a, 712b, 712c, 712d, 713a, 713b, 713c, 713d to obtain the complex internal parameter K (intrinsic parameter) and the complex radial distortion coefficient D (radial distortion coefficient) of the image capturing device 800. The vertex map positions (such as: 711a, 711b, 711c,...) of the high-precision map system 700 and the vertex image positions 911a, 911b, of the traffic sign images (such as: 911, 912, 913, 911 ' , 912 ' , 913 The corresponding relationship between 911c, 911d, 12a, 912b, 912c,...) is:

,in

,

The four vertex positions of each of the traffic sign images 911, 912 , 913, 911 ' , 912', and 913 are located at the vertex image positions of each of the street view images 910 and 910a, such as: vertex image positions 911a, 911b, 911c, 911d, ...the positions marked by chess boards in the corresponding street view images 910 and 910 ' ; x and y are the corresponding traffic sign images 911, 912 and 913 at the image capture location 900 in the high-precision map system 700 The latitude and longitude positions of the vertex map positions 711a , 711b, 711c, 711d, 712a, 712b, 712c, 712d, 713a, 713b, 713c, 713d corresponding to the traffic signs 711, 712, and 713 of 911 ' , 912', and 913, [ h 1 h 2 h 3] is the conversion relationship between the vertex position of the traffic sign image 911 and the vertex map scale information (such as longitude and latitude position) of the high-precision map system 700.

，複數徑向畸變係數D=[k1 k2 k3 p1 p2]。本實施例之複數內部參數K(intrinsic parameter)及該複數徑向畸變係數D(radial distortion coefficient)由單應性矩陣(homography matrix)計算比對得出，其中內部參數K與h1、h2、h3的關係為：

、

，藉由奇異值分解(SVD decomposition)與前述對應關係，即 可算出內部參數K。畸變係數D之k1、k2、k3係由下列算式取得：

=x+x[k ₁×(x ²+y ²)+k ₂×(x ²+y ²)²+k ₃×(x ²+y ²)³]、

=y+y[k ₁×(x ²+y ²)+k ₂×(x ²+y ²)²+k ₃×(x ²+y ²)³]。

、

為各該交通標誌影像911、912、913、911'、912'、913之四個頂點位置於各街景影像910、910a之頂點影像位置，x、y為高精地圖系統700中於影像擷取地點900對應交通標誌影像911、912、913、911'、912'、913之交通標誌711、712、713對應之各該頂點地圖位置711a、711b、711c、711d、712a、712b、712c、712d、713a、713b、713c、713d之經緯度位置。 The calculation module 50 is connected to the comparison module 40 via a signal. The calculation module 50 calculates the parameters of the image capture device 800 through a complex internal parameter K (intrinsic parameter) and a complex radial distortion coefficient D (radial distortion coefficient), where the complex internal parameter parameters

, complex radial distortion coefficient D=[ k 1 k 2 k 3 p 1 p 2]. In this embodiment, the complex internal parameter K (intrinsic parameter) and the complex radial distortion coefficient D (radial distortion coefficient) are calculated and compared with the homography matrix (homography matrix), where the internal parameter K and h1, h2, h3 The relationship is:

,

, through the singular value decomposition (SVD decomposition) and the aforementioned correspondence, the internal parameter K can be calculated. The k1, k2, and k3 of the distortion coefficient D are obtained by the following formula:

= x + x [ k ₁ ×( x ² + y ² )+ k ₂ ×( x ² + y ² ) ² + k ₃ ×( x ² + y ² ) ³ ],

= y + y [ k ₁ ×( x ² + y ² )+ k ₂ ×( x ² + y ² ) ² + k ₃ ×( x ² + y ² ) ³ ].

,

are the four vertex positions of the traffic sign images 911, 912 , 913, 911', 912 ' , and 913 in the vertex image positions of the street view images 910 and 910a, x and y are the image capture in the high-precision map system 700 The location 900 corresponds to the traffic sign images 911, 912 , 913, 911 ' , 912', and 913. The traffic signs 711, 712, and 713 correspond to the vertex map positions 711a, 711b, 711c, 711d, 712a, 712b, 712c, 712d, The latitude and longitude positions of 713a, 713b, 713c, and 713d.

以算出畸變係數D之p1、p2，其中F為高精地圖系統700中於影像擷取地點900對應交通標誌影像911、912、913、911'、912'、913之交通標誌711、712、713對應之各該頂點地圖位置711a、711b、711c、711d、712a、712b、712c、712d、713a、713b、713c、713d於街景影像910或街景影像910'中的投影，其中i為第i張影像，如：i=1代表街景影像910，i=2代表街景影像910'。b _ij 為頂點影像位置911a、911b、911c、911d、12a、912b、912c、912d、913a、913b、913c、913d、911a'、911b'、911c'、911d'、912a'、912b'、912c'、912d'、913a'、913b'、913c'、913d'的於各自所屬影像(街景影像910或街景影像910')之畫素點(pixel point)，也就是各頂點於所屬影像中的位置，R _i ,t _i 為從單應性矩陣擷取之第i張影像街景影像之相機姿態(camera pose)。通常來說，會先給予代價函數

F(K,D,R _i ,t _i ,M _j )∥2至3個起始值讓代價函數進行運算以找出畸變係數D之p1、p2。至此，複數內部參數K(intrinsic parameter)及複數徑向畸變係數D(radial distortion coefficient)可全部計算得出，供建圖或定位使用。由先前段所列算式可知，若影像擷取裝置800連續取得包括相同交通標誌的影像愈多，則所取得之畸變係數D越準確。在本實施例中，影像擷取裝置800之參數亦可包括焦距(focal point)、主點(principle point)、徑向畸變(radial distortion)及切向畸變(tangential distortion)，而當複數連續街景影像910、910'做為利用視覺同時定位與地圖建構技術重建或更新高精地圖時使用之影像時，前述的內部參數K(intrinsic parameter)及複數徑向畸變係數D(radial distortion coefficient)係用於校正複數連續街景影像910、910'中各項物件的位置。 The optimization module 60 is signal-connected to the calculation module 50. The optimization module 60 is used to optimize the projection error and minimize the cost function through the Levenberg-Marquardt algorithm.

To calculate p1 and p2 of the distortion coefficient D, where F is the traffic signs 711, 712, and 713 corresponding to the traffic sign images 911, 912, 913, 911 ' , 912 ' , and 913 at the image capture location 900 in the high-precision map system 700. The projection of the corresponding vertex map positions 711a, 711b, 711c, 711d, 712a, 712b, 712c, 712d, 713a, 713b, 713c, 713d in the street view image 910 or the street view image 910 ' , where i is the i-th image , for example: i=1 represents street view image 910, i=2 represents street view image 910 ' . b _ij is the vertex image position 911a, 911b, 911c, 911d, 12a, 912b, 912c, 912d, 913a, 913b, 913c, 913d, 911a ' , 911b ' , 911c ' , 911d ' , 912a ' , 912b ' , 912c ' , 912d ' , 913a ' , 913b ' , 913c ' , 913d ' are the pixel points of their respective images (street view image 910 or street view image 910 ' ), that is, the position of each vertex in the image to which they belong, R _i , t _i are the camera poses of the i-th street view image extracted from the homography matrix. Generally speaking, the cost function is first given

F ( K,D,R _i ,t _i ,M _j )∥2 to 3 starting values allow the cost function to operate to find p1 and p2 of the distortion coefficient D. At this point, the complex internal parameter K (intrinsic parameter) and the complex radial distortion coefficient D (radial distortion coefficient) can all be calculated for mapping or positioning. It can be seen from the calculation formula listed in the previous paragraph that if the image capture device 800 continuously acquires more images including the same traffic sign, the obtained distortion coefficient D will be more accurate. In this embodiment, the parameters of the image capture device 800 may also include focal point, principal point, radial distortion and tangential distortion. When multiple continuous street scenes When images 910 and 910 are used to reconstruct or update high-precision maps using simultaneous visual positioning and map construction technology, the aforementioned internal parameter K (intrinsic parameter) and complex radial distortion coefficient D (radial distortion coefficient) are used Correcting the positions of various objects in multiple continuous street view images 910, 910 ' .

As shown in Figure 1, the image capture device parameter acquisition system 1 of the present invention is, for example, one or several computer servers. It should be noted that, in addition to being configured as hardware devices, software programs, firmware, or a combination thereof, each of the above modules can also be configured by circuit loops or other appropriate types; and, in addition to being configured as separate modules, each module can also be configured in a separate form. , and can also be configured in combination. A preferred embodiment is that each module is a software program stored in the memory, and a processor (not shown) in the system 1 is used to obtain the parameters of the image capture device to execute each module to achieve the functions of the present invention. In addition, this embodiment only illustrates the preferred embodiments of the present invention. To avoid redundancy, all possible combinations of changes are not described in detail. However, those of ordinary skill in the art should understand that not all of the above-mentioned modules or components are necessary. In order to implement the present invention, other more detailed conventional modules or components may also be included. Each module or component may be omitted or modified as needed, and there may not be other modules or components between any two modules.

As shown in FIG. 4 , the image capture device parameter acquisition method of the present invention is used for reconstruction or precise high-altitude map system. The method includes steps S1 to S5. Each step will be described in detail below.

Step S1: Obtain a plurality of continuous street view images captured by an image capturing device at an image capturing location, where each continuous street view image includes at least one traffic sign image.

In this embodiment, the image capture device 800 can be a camera, a driving recorder, or other optical devices or electronic devices that can be used to capture images, for a plurality of continuous street view images 910, 910 captured at an image capture location 900. ' , each of the continuous street view images 910 and 910 ' includes at least one traffic sign image 911, 912, 913, 911 ' , 912 ' , 913 ' , and the traffic sign images 911, 912, 913, 911 ' , 912 ' , 913 ' is a rectangular road sign. Continuous images refer to the continuous movement of the image capture device 800 in a certain direction, which can also be understood as the continuous movement of the viewfinder car on the lane, and the image capture device 800 on the car continues to capture street view images while the viewfinder car is moving. (Figure 2A and Figure 2B).

Step S2: Obtain a plurality of vertex position data of the at least one traffic sign image.

Obtain the vertex image positions of a plurality of traffic sign images 911, 912, 913, 911 ' , 912 ' , and 913' in each continuous street view image 910, 910'. In this embodiment, the traffic sign images 911, 912, 913, 911 ' , 912 ' , 913 ' are rectangular, so the traffic sign image 911 has four vertex image positions 911a, 911b, 911c, 911d; the traffic sign image 912 has four vertex image positions 912a, 912b, 912c, 912d; the traffic sign image 913 There are four vertex image positions 913a, 913b, 913c, 913d; the traffic sign image 911 ' has four vertex image positions 911a ' , 911b ' , 911c ' , 911d ' ; the traffic sign image 912 ' has four vertex image positions 912a ' , 912b ' , 912c ' , 912d ' ; the traffic sign image 913 ' has four vertex image positions 913a ' , 913b ' , 913c ' , 913d ' . In this embodiment, the vertex image positions of the traffic sign images 911 , 912, 913, 911 ' , 912', 913 ' (such as the vertex image positions 911a, 912a, 913a, 911a ' , 912a ' , 913a ' ...) For these vertices in the corresponding street view images 910, 910 ' , the image positions of the vertices in the respective images to which they belong are marked using chess board or pixel positions.

Step S3: Obtain the vertex map information of the at least one traffic sign at the image capture location from a high-precision map system.

Obtain a vertex map position 710 of the traffic signs 711 , 712 , and 713 corresponding to the traffic sign images 911 , 912 , and 913 at the image capture location 900 from the high-precision map system 700 , where the vertex map position 710 includes the vertex map position of the traffic sign 711 711a, 711b, 711c, 711d, the vertex map positions 712a, 712b, 712c, 712d of the traffic sign 712, and the vertex map positions 713a, 713b, 713c, 713d of the traffic sign 713, among which the vertex map provided by the high-precision map system 700 The position 710 includes the longitude and latitude positions of the vertex map positions 711a, 711b, 711c, 711d, 712a, 712b, 712c, 712d, 713a, 713b, 713c, 713d corresponding to the traffic signs 711, 712, 713. In other words, It is the location of these vertices in the real world.

Step S4: Compare the vertex position data and the vertex map information to obtain the complex internal parameters K and the complex radial distortion coefficient D of the image capture device.

，其中

、

為各該交通標誌影像911、912、913、911'、912'、913之四個頂點位置於各該街景影像910、910a之頂點影像位置，如：頂點影像位置911a、911b、911c、911d、12a、912b、912c、912d、913a...於所屬街景影像910、910'中使用盤格(chess board)所標出的位置；x、y為高精地圖系統700中於影像擷取地點900對應交通標誌影像911、912、913、911'、912'、913之交通標誌711、712、713對應之各該頂點地圖位置711a、711b、711c、711d、712a、712b、712c、712d、713a、713b、713c、713d之經緯度位置，[h1 h2 h3]為交通標誌影像911之頂點位置與高精地圖系統700之頂點地圖尺度資訊(如：經緯度位置)的轉換關係。 Compare vertex image positions 911a, 911b, 911c, 911d, 12a, 912b, 912c, 912d, 913a, 913b, 913c, 913d, 911a ' , 911b ' , 911c ' , 911d ' , 912a ' , 912b ' , 912c ' , 912d ' , 913a ' , 913b ' , 913c ' , 913d ' and vertex map positions 711a, 711b, 711c, 711d, 712a, 712b, 712c, 712d, 713a, 713b, 713c, 713d to obtain the plurality of image capture devices 800 Internal parameter K (intrinsic parameter) and complex radial distortion coefficient D (radial distortion coefficient). The vertex map positions of the high-precision map system 700 (such as: 711a, 711b, 711c, 711d,...) and the vertex image positions of the traffic sign images 911, 912, 913, 911 ' , 912 ' , 913 (such as: 911a, The corresponding relationship between 911b, 911c, 911d, 12a, 912b, 912c, 912d, 913a, 913b, 913c, 913d, 911a ' ...) is:

,in

,

The four vertex positions of each of the traffic sign images 911, 912 , 913, 911 ' , 912', and 913 are located at the vertex image positions of each of the street view images 910 and 910a, such as: vertex image positions 911a, 911b, 911c, 911d, 12a, 912b, 912c, 912d, 913a... are the positions marked using chess boards in the corresponding street view images 910 and 910 ' ; x and y are the image capture locations 900 in the high-precision map system 700 The vertex map positions 711a, 711b, 711c, 711d, 712a, 712b, 712c, 712d, 713a, corresponding to the traffic signs 711, 712 , and 713 corresponding to the traffic sign images 911, 912, 913, 911' , 912', and 913 The longitude and latitude positions of 713b, 713c, and 713d, [ h 1 h 2 h 3 ], are the conversion relationship between the vertex position of the traffic sign image 911 and the vertex map scale information (such as the longitude and latitude position) of the high-precision map system 700 .

，複數徑向畸變係數D=[k1 k2 k3 p1 p2]。本實施例之複數內部參數K(intrinsic parameter)及該複數徑向畸變係數D(radial distortion coefficient)由單應性矩陣(homography matrix)計算比對得出，其中內部參數K與h1、h2、h3的關係為：

、

，藉由奇異值分解(SVD decomposition)與前述對應關係，即可算出內部參數K。畸變係數D之k1、k2、k3係由下列算式取得：

=x+x[k ₁×(x ²+y ²)+k ₂×(x ²+y ²)²+k ₃× (x ²+y ²)³]、

=y+y[k ₁×(x ²+y ²)+k ₂×(x ²+y ²)²+k ₃×(x ²+y ²)³]。

、

為各該交通標誌影像911、912、913、911'、912'、913之四個頂點位置於各街景影像910、910a之頂點影像位置，x、y為高精地圖系統700中於影像擷取地點900對應交通標誌影像911、912、913、911'、912'、913之交通標誌711、712、713對應之各該頂點地圖位置711a、711b、711c、711d、712a、712b、712c、712d、713a、713b、713c、713d之經緯度位置。 The parameters of the image capture device 800 are calculated by the complex internal parameter K (intrinsic parameter) and the complex radial distortion coefficient D (radial distortion coefficient), where the complex internal parameter

, complex radial distortion coefficient D=[ k 1 k 2 k 3 p 1 p 2]. In this embodiment, the complex internal parameter K (intrinsic parameter) and the complex radial distortion coefficient D (radial distortion coefficient) are calculated and compared with the homography matrix (homography matrix), where the internal parameter K and h1, h2, h3 The relationship is:

,

, through the singular value decomposition (SVD decomposition) and the aforementioned correspondence, the internal parameter K can be calculated. The k1, k2, and k3 of the distortion coefficient D are obtained by the following formula:

= x + x [ k ₁ × ( x ² + y ² ) + k ₂ × ( x ² + y ² ) ² + k ₃ × ( x ² + y ² ) ³ ],

= y + y [ k ₁ ×( x ² + y ² )+ k ₂ ×( x ² + y ² ) ² + k ₃ ×( x ² + y ² ) ³ ].

,

are the four vertex positions of the traffic sign images 911, 912 , 913, 911', 912 ' , and 913 in the vertex image positions of the street view images 910 and 910a, x and y are the image capture in the high-precision map system 700 The location 900 corresponds to the traffic sign images 911, 912 , 913, 911 ' , 912', and 913. The traffic signs 711, 712, and 713 correspond to the vertex map positions 711a, 711b, 711c, 711d, 712a, 712b, 712c, 712d, The latitude and longitude positions of 713a, 713b, 713c, and 713d.

Step S5: Calculate parameters of the image capturing device based on the complex internal parameter K and the complex radial distortion coefficient D.

以算出畸變係數D之p1、p2，其中F為高精地圖系統700中於影像擷取地點900對應交通標誌影像911、912、913、911'、912'、913之交通標誌711、712、713對應之各該頂點地圖位置711a、711b、711c、711d、712a、712b、712c、712d、713a、713b、713c、713d於街景影像910或街景影像910'中的投影，其中i為第i張影像，如：i=1代表街景影像910，i=2代表街景影像910'。b _ij 為頂點影像位置911a、911b、911c、911d、12a、912b、912c、912d、913a、913b、913c、913d、911a'、911b'、911c'、911d'、912a'、912b'、912c'、912d'、913a'、913b'、913c'、913d'的於各自所屬影像(街景影像910或街景影像910')之畫素點(pixel point)，也就是各頂點於所屬影像中的位置，R _i ,t _i 為從單應性矩陣擷取之第i張影像街景影像之相機姿態(camera pose)。通常來說，會先給予代價函數

2至3個起始值讓代價函數進行運算以找出 畸變係數D之p1、p2。至此，複數內部參數K(intrinsic parameter)及複數徑向畸變係數D(radial distortion coefficient)可全部計算得出，供建圖或定位使用。由先前段所列算式可知，若影像擷取裝置800連續取得包括相同交通標誌的影像愈多，則所取得之畸變係數D越準確，此外，根據本發明之另一實施例，若影像擷取裝置800連續取得矩形(如不同交通標誌)的影像(不需要是相同一個交通標誌)越多，則所取得之畸變係數D也會越準確。在本實施例中，影像擷取裝置800之參數亦可包括焦距(focal point)、主點(principle point)、徑向畸變(radial distortion)及切向畸變(tangential distortion)，而當複數連續街景影像910、910'做為利用視覺同時定位與地圖建構技術重建或更新高精地圖時使用之影像時，前述的內部參數K(intrinsic parameter)及複數徑向畸變係數D(radial distortion coefficient)係用於校正複數連續街景影像910、910'中各項物件的位置。 This step is to optimize the projection error and minimize the cost function using the Levenberg-Marquardt algorithm.

To calculate p1 and p2 of the distortion coefficient D, where F is the traffic signs 711, 712, and 713 corresponding to the traffic sign images 911, 912, 913, 911 ' , 912 ' , and 913 at the image capture location 900 in the high-precision map system 700. The projection of the corresponding vertex map positions 711a, 711b, 711c, 711d, 712a, 712b, 712c, 712d, 713a, 713b, 713c, 713d in the street view image 910 or the street view image 910 ' , where i is the i-th image , for example: i=1 represents street view image 910, i=2 represents street view image 910 ' . b _ij is the vertex image position 911a, 911b, 911c, 911d, 12a, 912b, 912c, 912d, 913a, 913b, 913c, 913d, 911a ' , 911b ' , 911c ' , 911d ' , 912a ' , 912b ' , 912c ' , 912d ' , 913a ' , 913b ' , 913c ' , 913d ' are the pixel points of their respective images (street view image 910 or street view image 910 ' ), that is, the position of each vertex in the image to which they belong, R _i , t _i are the camera poses of the i-th street view image extracted from the homography matrix. Generally speaking, the cost function is first given

2 to 3 starting values allow the cost function to operate to find p1 and p2 of the distortion coefficient D. At this point, the complex internal parameter K (intrinsic parameter) and the complex radial distortion coefficient D (radial distortion coefficient) can all be calculated for mapping or positioning. It can be seen from the calculation formula listed in the previous paragraph that if the image capture device 800 continuously acquires more images including the same traffic sign, the obtained distortion coefficient D will be more accurate. In addition, according to another embodiment of the present invention, if the image capture device 800 The more images of rectangles (such as different traffic signs) that the device 800 continuously acquires (not necessarily the same traffic sign), the more accurate the obtained distortion coefficient D will be. In this embodiment, the parameters of the image capture device 800 may also include focal point, principal point, radial distortion and tangential distortion. When multiple continuous street scenes When images 910 and 910 are used to reconstruct or update high-precision maps using simultaneous visual positioning and map construction technology, the aforementioned internal parameter K (intrinsic parameter) and complex radial distortion coefficient D (radial distortion coefficient) are used Correcting the positions of various objects in multiple continuous street view images 910, 910 ' .

This case takes advantage of the fact that the four sides of a rectangular road sign are straight lines, and the horizontal or vertical distance between two vertices of the same side of the same road sign is the same. The image capture device 800 is used to obtain multiple plural continuous street scene images. It can be found The corresponding relationship between the street sign and the real-world location of the street sign, that is, the longitude and latitude position of the same street sign at the same location in the high-longitude map system, thereby finding the plurality of image capture devices 800 that captured the continuous street view images. Internal parameter K (intrinsic parameter) and complex radial distortion coefficient D (radial distortion coefficient). In this way, after acquiring the street view image, the internal parameters of the camera that acquired the street view image can be calculated by comparing the image position of the rectangular road sign in the image with the corresponding position of the rectangular road sign in the high-precision map system. The source of the street view images required for the image is unknown. Street view images obtained by cameras with known internal parameters (such as driving recorders) can also be used, which solves the limitation of the previous technology that only street view images obtained by known internal parameters of the imaging camera can be used.

It should be noted that the above-mentioned embodiments are only examples for convenience of explanation, and the scope of rights claimed by the present invention should be subject to the scope of the patent application and is not limited to the above-mentioned embodiments.

Step S1 to Step S5

Claims (10)

A method for obtaining parameters of an image capture device, including the following steps: obtain a plurality of continuous street view images captured by an image capture device at an image capture location, wherein each of the continuous street view images includes at least one traffic sign image; obtain a plurality of the The vertex image position of at least one traffic sign image; obtaining the vertex map position of the at least one traffic sign corresponding to the image capture location from a high-precision map system; comparing the vertex image position with the The vertex map position is used to obtain the complex internal parameter K (intrinsic parameter) and the complex radial distortion coefficient D (radial distortion coefficient) of the image capture device; and the complex internal parameter K and the complex radial distortion coefficient D are used to calculate the Parameters of the image capture device. The image capture device parameter acquisition method as described in claim 1, wherein the at least one traffic sign image is in the shape of a rectangle, and the four vertex positions of the rectangle in the plurality of continuous street view images are obtained in the vertex images of each of the street view images. Location. The method for obtaining parameters of an image capture device as described in claim 2, wherein the vertex map position is four vertex map positions, and the four vertex map positions respectively correspond to the image capture location in the high-precision map system and the at least one A traffic sign image corresponds to the vertex map positions of four vertices of the at least one traffic sign. The method for obtaining image capture device parameters as described in claim 3, wherein the complex internal parameter K (intrinsic parameter) and the complex radial distortion coefficient D (radial distortion coefficient) are calculated from a homography matrix (homography matrix) . The method for obtaining parameters of an image capture device as described in any one of Claims 1 to 4, further comprising optimizing the parameters of the image capture device through Levenberg-Marquardt algorithm. . An image capture device parameter acquisition system includes: a memory that stores a plurality of continuous street view images captured by an image capture device at an image capture location, wherein each of the continuous street view images includes at least one traffic sign image; An image information acquisition module, the signal is connected to the memory, the image information acquisition module acquires a plurality of vertex image positions of the at least one traffic sign image; a high-precision map information acquisition module, the signal is connected to a high-precision map system, The high-precision map information acquisition module obtains a vertex map position of the at least one traffic sign corresponding to the image capture location from a high-precision map system; a comparison module is connected to the signal The image information acquisition module and the high-precision map information acquisition module, the comparison module compares the vertex image position and the vertex map position to obtain the plural internal parameters K (intrinsic parameters) and plural numbers of the image capture device Radial distortion coefficient D (radial distortion coefficient); and a calculation module, the signal is connected to the comparison module, the calculation module uses the complex internal parameter K (intrinsic parameter) and the complex radial distortion coefficient D (radial distortion coefficient) distortion coefficient) to calculate the parameters of the image capture device. The image capture device parameter acquisition system as described in claim 6, wherein the at least one traffic sign image is in the shape of a rectangle, and the image information acquisition module acquires the four vertex positions of the rectangle in each of the plurality of continuous street view images. The vertex image position of the street view image. The image capture device parameter acquisition system as described in claim 7, wherein the vertex map location is four vertex map locations, and the four vertex map locations respectively correspond to the image capture location in the high-precision map system and the at least A traffic sign image corresponds to the vertex map positions of four vertices of the at least one traffic sign. The image capture device parameter acquisition system as described in claim 8, wherein the complex internal parameter K (intrinsic parameter) and the complex radial distortion coefficient D (radial distortion coefficient) are determined by the comparison module using a homography matrix. ) calculated and compared. The image capture device parameter acquisition system as described in any one of claims 6 to 9 further includes an optimization module, the signal is connected to the calculation module, and the optimization module uses Levenberg-Marquardt An algorithm (Levenberg-Marquardt algorithm) optimizes the parameters of the image capture device.

Images