CN1975323A

CN1975323A - Method for making three-dimensional measurement of objects utilizing single digital camera to freely shoot

Info

Publication number: CN1975323A
Application number: CN 200610161274
Authority: CN
Inventors: 张丽艳; 郑建冬; 张辉; 卫炜
Original assignee: Nanjing University of Aeronautics and Astronautics
Current assignee: Nanjing University of Aeronautics and Astronautics
Priority date: 2006-12-19
Filing date: 2006-12-19
Publication date: 2007-06-06
Anticipated expiration: 2026-12-19
Also published as: CN100430690C

Abstract

The invention relates to a method for three-dimensional measurement of an object by free shooting with a single digital camera, which belongs to the testing technology. The method includes seven steps: measurement preparation, image capture, identification and positioning of coding points, camera pose determination, target curve extraction, automatic matching optimization of the same-named curve, and 3D reconstruction of the target curve. It is characterized in that: three-dimensional measurement uses a digital camera, a computer, a group of coding points and a ruler, and first marks the target curve to be measured on the measured object to facilitate image recognition; around the measured object Place a ruler and a set of code points; then hold a digital camera to obtain a set of images of the measured object in a free shooting manner; according to this set of images, automatically and accurately calculate the camera position and attitude during each shooting; provide user convenience It realizes semi-automatic extraction of marked curves and optimal matching of curves with the same name in different images, so as to automatically calculate the three-dimensional point sequence information on the marked curve structure.

Description

Method for Three-Dimensional Measurement of Objects by Using Single Digital Camera to Shoot Freely

一、技术领域1. Technical field

三维物体测量属于测量、测试技术领域。在国际分类表中的相应代码为G01B。The three-dimensional object measurement belongs to the technical field of measurement and testing. The corresponding code in the International Classification is G01B.

二、背景技术2. Background technology

三维测量技术在逆向工程、工业检测、质量控制等领域日益广泛的应用需求极大地推动了三维测量技术的迅速发展，出现了基于光学、声学、电磁学以及机械接触原理的各种测量方法，如三坐标测量机、激光扫描仪、结构光测量仪等。其中，三坐标测量机采用机械接触式传感，测量精度高，但一般需要专门的测量室和专用测量台，对测量环境要求很高，测量范围有限，测量效率低，不适合软性物体的测量；激光线扫描测量和结构光照射测量是目前三维几何外形测量的主流方法，通过在物体表面进行激光或结构光照射，可以快速获取模型表面密集的点云数据，但都受扫描范围、物体表面高光反射等的限制，且激光扫描仪和结构光测量仪均价格昂贵。尤其是对于机械产品而言，由于通常含有明显的结构特征，常常需要重点获取对重建被测物体的数字化模型起关键作用的角点、棱边以及模型表面的某些控制线等数据，结构光照射测量法、激光线扫描测量法等获得的都是表面的整体点云或网格数据，一方面输出的数据量十分庞大，另一方面却不能直接显式地获取所需的棱边特征及某些关键截面控制线数据。而且，这些方法输出的测量数据通常在模型的光滑平坦区域的效果比较好，而恰恰在关键的角点和棱边处的测量效果较差。The increasingly wide application requirements of 3D measurement technology in reverse engineering, industrial inspection, quality control and other fields have greatly promoted the rapid development of 3D measurement technology, and various measurement methods based on optics, acoustics, electromagnetics and mechanical contact principles have emerged, such as Coordinate measuring machine, laser scanner, structured light measuring instrument, etc. Among them, the three-coordinate measuring machine adopts mechanical contact sensing, which has high measurement accuracy, but generally requires a special measurement room and a special measurement table, which has high requirements for the measurement environment, limited measurement range, low measurement efficiency, and is not suitable for soft objects. Measurement; laser line scanning measurement and structured light irradiation measurement are the mainstream methods for three-dimensional geometric shape measurement. By irradiating the surface of the object with laser or structured light, dense point cloud data on the surface of the model can be quickly obtained, but they are all limited by the scanning range, object Surface specular reflection, etc., and laser scanners and structured light measuring instruments are expensive. Especially for mechanical products, because they usually contain obvious structural features, it is often necessary to focus on obtaining data such as corner points, edges and some control lines on the surface of the model that play a key role in reconstructing the digital model of the measured object. Irradiation measurement, laser line scanning measurement, etc. obtain the overall point cloud or grid data of the surface. On the one hand, the amount of output data is very large, on the other hand, it cannot directly and explicitly obtain the required edge features and grid data. Certain key section control line data. Moreover, the measurement data output by these methods usually have a better effect on the smooth and flat areas of the model, but the measurement effect on the key corners and edges is poor.

为了以更加简单的硬件条件和更加灵活方便的方式实现三维几何信息的测量，根据一个数码相机拍摄的多幅图像精确重建物体的位置和形状成为近年来的一个研究热点。其中，德国Gom公司的TriTop^系统，已经能够用单数码相机自由拍摄方式进行较高精度的三维坐标定位。该系统通过在场景中放置一组编码点和长度标尺，并在感兴趣的部位粘贴易于识别的标记点，然后使用者手持一个数码相机自由拍摄多幅图像，要求各幅图像相互之间有一定的重叠，所有图像输入相应的软件系统后，系统一次性自动计算各次拍摄时的相机位置、姿态以及所有标记点的空间坐标。该系统已经商品化并在我国进行销售，但是，该系统目前只能进行特定标记点目标(由一个黑色圆环围绕一个白色圆点构成)的空间坐标定位，一般用于配合其它测量方法进行多视角测量数据的拼合，而不能进行曲线目标的三维测量，不能用于具有复杂几何外形产品的三维数字化模型重建。In order to realize the measurement of 3D geometric information with simpler hardware conditions and a more flexible and convenient way, it has become a research hotspot in recent years to accurately reconstruct the position and shape of objects based on multiple images taken by a digital camera. Among them, the TriTop ^(R) system of the German Gom company has been able to perform relatively high-precision three-dimensional coordinate positioning with a single digital camera free shooting mode. The system places a set of coding points and length scales in the scene, and pastes easily identifiable marking points on the interested parts, and then the user holds a digital camera to shoot multiple images freely, requiring a certain distance between each image. After all the images are input into the corresponding software system, the system automatically calculates the camera position, attitude and the spatial coordinates of all marked points for each shooting at one time. The system has been commercialized and sold in our country, but at present, the system can only locate the spatial coordinates of a specific marker target (constituted by a black circle around a white dot), and is generally used to coordinate with other measurement methods for multiple The combination of viewing angle measurement data cannot be used for 3D measurement of curved objects, and cannot be used for 3D digital model reconstruction of products with complex geometric shapes.

三、发明内容3. Contents of the invention

本发明旨在用简单的硬件条件，实现一种面向工业产品测量建模的便于实施、精度较高、成本较低的实用测量方法。为此，本发明通过对被测物体的特征线和进行数字化模型重建所需要的物体表面的某些关键截面控制线进行标记，使其在颜色亮度上明显区别于被测物体本身的颜色，以利于图像识别；在被测物体周围放置一把标尺和一组经特殊设计的编码点；然后手持一个数码相机以自由拍摄方式获得被测物体的一组图像；根据这组图像，自动精确计算各次拍摄时的相机位置与姿态，同时，提供使用者方便的交互手段，实现对标识曲线的半自动提取和不同图像中同名曲线的优化匹配，进而自动计算出所标识曲线结构的三维点列信息。The invention aims at realizing a practical measurement method oriented to measurement and modeling of industrial products, which is easy to implement, high in precision and low in cost with simple hardware conditions. For this reason, the present invention marks the characteristic lines of the measured object and some key section control lines of the object surface required for digital model reconstruction, so that its color brightness is obviously different from the color of the measured object itself, so as to Facilitate image recognition; place a ruler and a set of specially designed coding points around the measured object; then hold a digital camera to obtain a set of images of the measured object in a free shooting manner; according to this set of images, automatically and accurately Calculate the position and attitude of the camera for each shooting, and at the same time, provide users with convenient interactive means to realize the semi-automatic extraction of the marked curve and the optimized matching of the same-named curve in different images, and then automatically calculate the three-dimensional point sequence information of the marked curve structure .

根据上述方案开发出了使用灵活方便、适用于不同大小物体上三维曲线结构测量的实用系统，可以很好地用于基于实物的机械产品三维数字化模型建立。本发明提出的物体三维测量方法，其特征在于测量只用一个数码相机、一台普通个人电脑，辅以一组编码点和一把标尺，无需复杂的测量硬件系统，也不需要对测量系统进行繁琐的标定；直接显式地产生对物体进行数字化模型重建所需的曲线数据，避免数据冗余，且便于进行高效率模型重建；全部测量数据自动位于一个世界坐标系下，避免了其他测量方法中多次测量的数据需要拼合的难点问题，也避免了多个数据集拼合带来的累计误差。本发明包括测量准备、图像摄取、相机位姿确定、目标曲线提取、同名曲线的自动匹配优化、目标曲线的三维重建等主要步骤。According to the above scheme, a flexible and convenient practical system suitable for measuring three-dimensional curve structures on objects of different sizes has been developed, which can be well used for the establishment of three-dimensional digital models of mechanical products based on physical objects. The object three-dimensional measurement method proposed by the present invention is characterized in that only one digital camera, one common personal computer is used for measurement, supplemented by a set of code points and a scale, and no complex measurement hardware system is needed, nor is it necessary to carry out the measurement of the measurement system. Complicated calibration; directly and explicitly generate the curve data required for digital model reconstruction of the object, avoid data redundancy, and facilitate high-efficiency model reconstruction; all measurement data are automatically located in a world coordinate system, avoiding other measurement methods The difficult problem of merging the data measured multiple times in the middle is also avoided, and the cumulative error caused by the merging of multiple data sets is also avoided. The invention includes the main steps of measurement preparation, image capture, camera pose determination, target curve extraction, automatic matching optimization of the same-named curve, three-dimensional reconstruction of the target curve, and the like.

测量准备与图像摄取Measurement preparation and image capture

测量准备主要包括三个方面的工作：1)根据数字化模型重建的需要对目标曲线(一般是自然曲面片的边界线、关键的截面控制线等)进行标记，使其在颜色亮度上明显区别于被测物体，以利于图像识别；2)在测量区域内布置若干编码点。每个编码点具有唯一身份编码，易于在不同图像中进行快速可靠的识别。标记点和编码点均可简单地在计算机上产生相应的图案，然后打印生成。把编码点贴到硬纸板、木片等的上面，可以重复使用；3)在测量场景内放置一个有两个编码点的标尺，其上的两个编码点中心的间距已知。放置标尺的目的是获得被测物体的实际尺寸，否则只能得到相差一个比例系数的三维结构。上述准备工作完成后，就可手持数码相机对被测物体进行多角度拍摄。要求各幅图像间具有一定的重叠，即一幅图像至少要与另一幅图像之间有共同可见的5个以上编码点和某些目标曲线。The measurement preparation mainly includes three aspects of work: 1) According to the needs of digital model reconstruction, mark the target curve (generally the boundary line of the natural surface patch, the key section control line, etc.), so that its color brightness is obviously different from that of The measured object is beneficial to image recognition; 2) Arranging several coding points in the measurement area. Each code point has a unique identity code, which is easy to carry out fast and reliable identification in different images. Both marking points and coding points can be simply generated on the computer corresponding pattern, and then printed. Paste the code points on cardboard, wood chips, etc., which can be reused; 3) Place a ruler with two code points in the measurement scene, and the distance between the centers of the two code points on it is known. The purpose of placing the ruler is to obtain the actual size of the measured object, otherwise only a three-dimensional structure with a difference of one scale factor can be obtained. After the above preparatory work is completed, the digital camera can be held in hand to take multi-angle shots of the measured object. It is required to have a certain overlap between the images, that is, at least one image must have more than 5 code points and some target curves that are common to another image.

编码点的识别与定位Identification and location of code points

编码点的图案设计成由中心白色圆点、中间黑色圆环、外圈圆环组成，其中外圈圆环被分成15个等分，每个等分为黑色或白色，黑色表示二进制码“0”，白色表示二进制码“1”，称为“编码带”。一组编码点中每个点的编码均不相同。根据这种编码，可以在不同图像中可靠地识别其身份，自动建立同名编码点在各图像间的对应关系。正是根据这些同名编码点在图像中的位置和在多幅图像间的对应，才能实现自由拍摄时相机的各个位置和姿态的自动计算。The pattern of the coding point is designed to be composed of a white dot in the center, a black ring in the middle, and an outer ring. The outer ring is divided into 15 equal parts, each of which is divided into black or white, and black means the binary code "0". ", white represents the binary code "1", which is called "coding band". Each point in a set of coded points is coded differently. According to this encoding, its identity can be reliably identified in different images, and the corresponding relationship between the same-named encoding points in each image can be automatically established. It is precisely based on the positions of these code points with the same name in the image and the correspondence between multiple images that the automatic calculation of each position and attitude of the camera during free shooting can be realized.

由于编码点的中心圆形经CCD成像后呈椭圆状，因此，为了识别和定位图像中的编码点，本发明首先采用Canny算子进行图像分割，在图像中提取代表不同区域的轮廓信息，然后根据各轮廓的尺寸、形状、椭圆拟合残差、区域灰度的均值、区域灰度的方差等5个约束条件，对候选编码点目标进行逐步过滤，从而实现编码点目标的提取。Since the central circle of the code point is elliptical after being imaged by the CCD, in order to identify and locate the code point in the image, the present invention first uses the Canny operator to segment the image, and extracts the contour information representing different regions in the image, and then According to the five constraints of size, shape, residual error of ellipse fitting, mean value of regional gray level, and variance of regional gray level, the candidate coding point targets are filtered step by step, so as to realize the extraction of coding point targets.

锁定一个编码点目标后，要对其进行解码，即确定其具体是哪个编码点。解码的依据是编码带上各扇区的灰度。本发明采用拟合编码带中间椭圆并对其上的每一象素取线性窗口进行中值滤波的方法，综合考虑了编码带内大多数象素的灰度值，能够消除孤立噪声的影响。对大量的现场拍摄图像的处理结果表明，这一方法对提高编码点身份识别的鲁棒性十分有效。After locking a code point target, it needs to be decoded, that is, to determine which code point it is. The basis of decoding is the gray level of each sector on the coded tape. The present invention adopts the method of fitting the ellipse in the middle of the encoding band and taking a linear window for each pixel on it to carry out median filtering, comprehensively considers the gray values of most pixels in the encoding band, and can eliminate the influence of isolated noise. The processing results of a large number of live images show that this method is very effective in improving the robustness of code point identification.

最后，根据识别出的编码点中心圆形区域内各象素点的灰度值，确定具有亚象素定位精度的编码点中心坐标。Finally, according to the gray value of each pixel in the identified central circular area of the coding point, the central coordinates of the coding point with sub-pixel positioning accuracy are determined.

相机位置和姿态确定Camera position and pose determination

根据至少5个同名编码点中心在两幅图像中的象素坐标，首先计算两幅图像之间的基本矩阵。由相机内参数和基本矩阵，可进一步恢复两幅图像对应的相机姿态及这两幅图中共同可见的编码点中心的三维坐标。然后，根据所获得的三维空间点与第三幅图像上编码点之间的同名对应关系，求解第三幅图像对应的相机姿态，进而获得更多的编码点中心三维坐标，再求解下一幅图像对应的相机姿态，如此递增，直到获得所有相机姿态和编码点中心三维坐标。最后采用光束平差法同时对所有相机参数及编码点中心三维坐标进行整体优化，以进一步提高精度。这种递增式方法和全局优化方法相结合的策略，使算法既有较高的效率，又能使相机定位达到较高的精度。Based on the pixel coordinates of the centers of at least 5 coded points with the same name in the two images, the fundamental matrix between the two images is first calculated. From the camera intrinsic parameters and the fundamental matrix, the camera poses corresponding to the two images and the three-dimensional coordinates of the centers of the coded points that are commonly visible in the two images can be further recovered. Then, according to the corresponding relationship between the obtained three-dimensional space point and the coding point on the third image, solve the camera pose corresponding to the third image, and then obtain more three-dimensional coordinates of the center of the coding point, and then solve the next one The camera pose corresponding to the image is incremented in this way until all camera poses and the three-dimensional coordinates of the center of the encoding point are obtained. Finally, the beam adjustment method is used to optimize all camera parameters and the three-dimensional coordinates of the center of the coding point at the same time, so as to further improve the accuracy. The strategy of combining the incremental method and the global optimization method makes the algorithm not only have higher efficiency, but also enable the camera positioning to achieve higher accuracy.

一次读入拍摄到的一组图像，测量系统即自动计算并记录各次拍摄时的相机位置与姿态。确定了各次拍摄时的相机位置和姿态，意味着确定了各幅图像在一个统一的世界坐标系下的位置和姿态，这使得后续算法根据不同图像对重建出的各目标曲线直接位于同一个坐标系中，无需数据拼合。Once a group of captured images is read in, the measurement system automatically calculates and records the camera position and attitude during each shooting. Determining the position and attitude of the camera at each shooting means determining the position and attitude of each image in a unified world coordinate system, which makes the target curves reconstructed by subsequent algorithms based on different image pairs directly located in the same In the coordinate system, no data flattening is required.

目标曲线的半自动提取Semi-automatic extraction of target curves

对于当前选中显示在两个图像窗口中的一个图像对，分别在两幅图像上用鼠标在共同可见的同一条标记曲线附近取点，使这些点的连线大致反映相应的图像曲线轮廓，然后测量系统根据能量优化的原理自动将这样的大致图像曲线轮廓最佳地贴合到图像曲线上。这一半自动提取过程中，使用者只需要在图像曲线附近依次取点即可，简便易行，且由于有较好的初始搜索位置而大大增加了目标曲线提取及后续自动匹配优化的稳定性，同时最佳贴合方法保证目标曲线的提取精度。For an image pair currently selected and displayed in two image windows, use the mouse to select points near the same marked curve that are both visible on the two images, so that the line connecting these points roughly reflects the corresponding image curve outline, and then Based on the principle of energy optimization, the measuring system automatically fits such an approximate image curve profile optimally to the image curve. In this half of the automatic extraction process, the user only needs to take points sequentially near the image curve, which is simple and easy, and because of the better initial search position, the stability of the target curve extraction and subsequent automatic matching optimization is greatly increased. At the same time, the best fitting method ensures the extraction accuracy of the target curve.

同名曲线的自动匹配优化Automatic matching optimization of curves with the same name

在一个图像对中提取出一对同名目标曲线后，一个关键的问题就是建立两幅图像间同名目标曲线上各象素点的对应关系。基于图像点的对应，就可以通过立体三角法重建出这些点的空间坐标。After extracting a pair of target curves with the same name from an image pair, a key problem is to establish the corresponding relationship between the pixels on the target curves with the same name between the two images. Based on the correspondence of image points, the spatial coordinates of these points can be reconstructed by stereo triangulation.

根据立体视觉的基本理论，相机在不同位置和角度获得的一个图像对间的同名点应满足极线约束。对于一个图像对中两个候选同名匹配点v₁和v₂，本发明用在第二幅图像上v₂到v₁的极线的距离，以及在第一幅图像上v₁到v₂的极线的距离之和来度量v₁和v₂的匹配程度。同时，两条同名目标曲线上的点列匹配应满足空间相关性约束，即一条曲线上按顺序排列的点列，一定对应另一条曲线上一个顺序排列的点列。基于这样的分析，本发明首先采用动态规划来获得同名曲线上离散象素点的初始匹配。得到同名图像曲线上点对的初始匹配后，再进一步进行曲线的匹配优化。设两条同名图像曲线分别用参数方程c₁(l)、c₂(l)表示，本发明优化下面的目标函数来达到c₁、c₂曲线上点的精确匹配According to the basic theory of stereo vision, the points of the same name between an image pair obtained by the camera at different positions and angles should satisfy the epipolar constraint. For two candidate matching points v ₁ and v ₂ in an image pair, the present invention uses the epipolar distance from v ₂ to v ₁ on the second image, and the distance from v ₁ to v ₂ on the first image The sum of the distances of the epipolar lines measures how well _v1 and _v2 match. At the same time, the point sequence matching on two target curves with the same name should satisfy the spatial correlation constraint, that is, the point sequence arranged in sequence on one curve must correspond to the point sequence arranged in sequence on the other curve. Based on such analysis, the present invention first uses dynamic programming to obtain the initial matching of discrete pixel points on the curve with the same name. After the initial matching of point pairs on the image curve with the same name is obtained, the matching optimization of the curve is further carried out. Assuming that two image curves with the same name are represented by parameter equations c ₁ (l) and c ₂ (l) respectively, the present invention optimizes the following objective function to achieve accurate matching of points on the c ₁ and c ₂ curves

$min min {&Integral; &Integral;}_{00}^{{L L}_{11}} \frac{| | {c c}_{22} {((σ σ ((l l))))}^{T T} F f {c c}_{11} ((l l))}{| | | | eF f {c c}_{11} ((l l)) | | | |} + + \frac{| | {c c}_{22} {((σ σ ((l l))))}^{T T} F f {c c}_{11} ((l l)) | |}{| | | | {c c}_{22} {((σ σ ((l l))))}^{T T} F f {e e}^{T T}} dl dl - - - - - - ((2828))$

其中， $e = [\begin{matrix} 0 & - 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 0 \end{matrix}];$ F是图像对间的基本矩阵，在各次拍摄时的相机位置和姿态已经确定的情况下时已知的；σ(l)为待求的映射函数，表示c₁曲线上参数为l的点在曲线c₂上的参数值。in, $e = [\begin{matrix} 0 & - 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 0 \end{matrix}];$ F is the basic matrix between image pairs, which is known when the camera position and attitude have been determined for each shooting; σ(l) is the mapping function to be obtained, indicating the point on the c ₁ curve with parameter l Parameter values on curve _c2 .

目标曲线的三维重建3D reconstruction of target curve

完成同名曲线上所有象素点的匹配后，由于图像对在各自拍摄时的相机位置和姿态已经自动计算得出，因此可以用双目立体视觉中成熟的三角测量法重建出这些点的空间坐标，从而完成整条曲线的重建。After completing the matching of all pixel points on the curve with the same name, since the camera position and attitude of the image pair at the time of shooting have been automatically calculated, the spatial coordinates of these points can be reconstructed using the mature triangulation method in binocular stereo vision , so as to complete the reconstruction of the entire curve.

本发明具有测量硬件简单(一个数码相机、一台普通个人电脑、一把标尺、一组打印生成的编码点)、测量方式十分灵活(自由拍摄)、测量范围不受限制、各角度测量数据自动拼合、测量输出数据无冗余、使用方便、成本低等突出优点，不但能够用于空间点的定位，而且能够根据机械产品测量建模的需要，测量出被测物体上的棱边、特征线、关键截面控制线等三维信息，在逆向工程、产品质量检测等领域有广泛的应用前景。The present invention has the advantages of simple measurement hardware (a digital camera, an ordinary personal computer, a ruler, and a set of code points generated by printing), very flexible measurement methods (free shooting), unlimited measurement range, and automatic measurement data at each angle. Stitching, measurement output data without redundancy, easy to use, low cost and other outstanding advantages, not only can be used for the positioning of spatial points, but also can measure the edges and characteristic lines on the measured object according to the needs of measurement and modeling of mechanical products 3D information, such as control lines of key sections, has broad application prospects in reverse engineering, product quality inspection and other fields.

附图说明Description of drawings

图1：本发明提出的测量方法的基本流程图。Figure 1: Basic flowchart of the measurement method proposed by the present invention.

图2：编码点示意图。图2(a)描述编码点结构，即由中心白色圆点、中间黑色圆环、外圈圆环组成，其中外圈圆环被分成15个等分，根据每个扇区分的颜色确定其身份，黑色表示二进制码“0”，白色表示二进制码“1”；图2(b)是三个编码点示例。Figure 2: Schematic representation of codepoints. Figure 2(a) describes the code point structure, which is composed of a central white dot, a middle black ring, and an outer ring. The outer ring is divided into 15 equal parts, and its identity is determined according to the color of each sector , black represents the binary code "0", and white represents the binary code "1"; Figure 2(b) is an example of three code points.

图3：实施例测量系统软件图形界面分区示意图。1.菜单区；2.图标工具栏；3.显示图像文件的列表；4.显示当前活动图像对中的一幅图像；5.显示当前活动图像对中的另一幅图像；6.重建出的目标曲线的三维图形显示区。通过点击区域3中的图像文件列表来指定4、5两个窗口内当前显示的图像，用户在4、5两个图像窗口中依照图像上的标识曲线交互勾勒出一对同名曲线的大致形状后，系统即自动计算出该曲线上的三维点列并显示于三维图形区。Fig. 3: Schematic diagram of partitioning of the graphical interface of the measurement system software of the embodiment. 1. Menu area; 2. Icon toolbar; 3. Display the list of image files; 4. Display one image in the current active image pair; 5. Display another image in the current active image pair; 6. Reconstruct The three-dimensional graphic display area of the target curve. Click the image file list in area 3 to specify the currently displayed image in the two windows 4 and 5. After the user interactively outlines the approximate shape of a pair of curves with the same name in the two image windows 4 and 5 according to the identification curve on the image , the system automatically calculates the 3D point series on the curve and displays it in the 3D graphics area.

具体实施方式Detailed ways

本发明提出的物体三维曲线结构测量方法的实施例说明如下：数码相机采用具有内置闪光灯、分辨率为4256×2848的Nikon手动调焦数码相机，计算机采用主频2.8GHz、内存512MB的Pentium IV微机，测量软件系统在Visual C++ 6.0平台上实现。The embodiment of object three-dimensional curve structure measurement method that the present invention proposes is described as follows: digital camera adopts the Nikon manual focusing digital camera that has built-in flashlight, and resolution is 4256 * 2848, and computer adopts the Pentium IV microcomputer of main frequency 2.8GHz, internal memory 512MB , the measurement software system is implemented on the Visual C++ 6.0 platform.

根据图1叙述本发明的具体实施方式及原理：测量之前要进行一定的测量准备，包括对被测物体上需要测量的目标曲线(一般是物体的特征线和进行数字化模型重建所需要的物体表面的某些关键截面控制线)进行标记，使其在颜色亮度上明显区别于被测物体本身的颜色，以利于图像识别；在被测物体周围放置一把标尺和一组经特殊设计的编码点，这组编码点中每个点的编码均不相同，即每个编码点具有身份唯一性。完成上述测量准备工作以后，手持一个数码相机以自由拍摄方式获得被测物体的一组图像，要求各幅图像间具有一定的重叠，即一幅图像至少要与另一幅图像之间有共同可见的5个以上编码点和某些目标曲线。根据这组图像，测量系统自动识别并精确定位各幅图像中的编码点，然后自动精确计算各次拍摄时的相机位置与姿态。使用者经简单的交互完成当前活动图像对中一条标识曲线的半自动提取，测量系统随后自动进行同名图像曲线的优化匹配，进而自动计算这条出所标识曲线结构的三维点列信息。如果尚有未处理的目标曲线，则对下一条目标曲线(可能出现在不同的图像对中)重复上述半自动提取、自动匹配、三维曲线重建的过程，直至完成所有目标曲线的三维重建。以下就图1中的几个主要步骤的具体实施方式做详细说明。Describe the specific embodiment and principle of the present invention according to Fig. 1: before the measurement, certain measurement preparations will be carried out, including the target curve to be measured on the measured object (generally the characteristic line of the object and the object surface required for digital model reconstruction) Some key section control lines) to mark, so that the color brightness is obviously different from the color of the measured object itself, so as to facilitate image recognition; a ruler and a set of specially designed coding points are placed around the measured object , the codes of each point in this group of code points are different, that is, each code point has a unique identity. After completing the above measurement preparations, hold a digital camera in hand to obtain a group of images of the measured object in a free shooting manner. It is required that there be a certain overlap between each image, that is, one image must have at least the same image as another image. Visible 5+ code points and certain target curves. According to this group of images, the measurement system automatically identifies and accurately locates the code points in each image, and then automatically and accurately calculates the camera position and attitude during each shooting. The user completes the semi-automatic extraction of a marked curve in the current active image pair through simple interaction, and then the measurement system automatically performs optimal matching of the image curve of the same name, and then automatically calculates the three-dimensional point sequence information of the marked curve structure. If there are unprocessed target curves, repeat the above process of semi-automatic extraction, automatic matching, and 3D curve reconstruction for the next target curve (which may appear in different image pairs) until the 3D reconstruction of all target curves is completed. The specific implementation of several main steps in FIG. 1 will be described in detail below.

编码点身份识别code point identification

本发明的三维测量方法基于对拍摄得到的一组图像进行分析处理，首先就是识别图2中所示的编码点。每个编码点的中心为圆形“目标点”，周围为环状“编码带”，“编码带”按照角度平均分为15份，每24度一份，相当于一个二进制位，取白色为前景色，相应的二进制码为“1”，黑色为背景色，相应的二进制码为“0”。对于每一个编码点，存在15种可能的二进制编码，取这15个二进制数里最小的数对应的十进制数作为编码点的ID。The three-dimensional measuring method of the present invention is based on analyzing and processing a group of photographed images, firstly identifying the coded points shown in FIG. 2 . The center of each coding point is a circular "target point", surrounded by a ring-shaped "coding belt". Foreground color, the corresponding binary code is "1", black is the background color, the corresponding binary code is "0". For each code point, there are 15 possible binary codes, and the decimal number corresponding to the smallest number among the 15 binary numbers is taken as the ID of the code point.

本发明的编码点自动检测算法主要包括以下三个主要过程：①编码点目标的提取，即在图像中寻找“目标点”；②根据编码点“编码带”上的信息确定编码点的唯一身份，即编码点解码；③编码点中心的亚象素定位。The code point automatic detection algorithm of the present invention mainly includes the following three main processes: 1. the extraction of the code point target, that is, searching for the "target point" in the image; 2. determining the unique identity of the code point according to the information on the code point "code belt" , that is, code point decoding; ③The sub-pixel positioning of the code point center.

(1)编码点目标提取(1) code point target extraction

编码点的中心圆形经CCD成像后呈椭圆状。因此，首先采用Canny算子进行图像分割，在图像中提取代表不同区域的轮廓信息，然后采用逐步过滤的方法进行编码点目标提取。首先根据标记点目标的尺寸、形状对可能的目标点进行初步过滤，满足如下条件的封闭轮廓进入进一步的识别过程：The central circle of the coding point is elliptical after being imaged by the CCD. Therefore, firstly, the Canny operator is used to segment the image, and the contour information representing different regions is extracted in the image, and then the coded point target is extracted by using the method of step-by-step filtering. Firstly, the possible target points are preliminarily filtered according to the size and shape of the marked point target, and the closed contours that meet the following conditions enter the further recognition process:

P_min≤P≤P_max (1)P _min ≤ P ≤ P _max (1)

1≤P²/4πA≤1.5 (2)1≤P ² /4πA≤1.5 (2)

式中，P和A分别为封闭轮廓的周长和面积，P_min、P_max分别是轮廓周长的最小和最大阈值。式(1)是对封闭轮廓大小的限定，式(2)则度量了其与圆的接近程度。In the formula, P and A are the perimeter and area of the closed contour, respectively, and P _min and P _max are the minimum and maximum thresholds of the contour perimeter, respectively. Formula (1) limits the size of the closed contour, and formula (2) measures its closeness to a circle.

对于满足(1)(2)式的封闭轮廓，采用最小二乘方法进行椭圆拟合，剩余误差ε_eli满足给定允差ε_τ方为候选编码点的中心圆，即For the closed contour that satisfies (1) and (2), the least squares method is used for ellipse fitting, and the residual error ε _eli satisfies the given tolerance ε _τ square is the central circle of the candidate coding point, that is

ε_eli≤ε_τ (3)ε _eli ≤ ε _τ (3)

经过最小二乘模板匹配后，图像中所有椭圆轮廓都已找到。但是，在真实场景中往往存在一些不是标记点目标而具有椭圆形状或者与椭圆形状接近的轮廓也被误认为是标记点目标，但由于本方法采用的标记点目标前景灰度为白色，背景灰度为黑色，两者对比度强烈。这是标记点目标区别于其它非标记点目标的显著特征。因此，要根据标记点的这种灰度特征来进一步排除非编码点目标。由于已经通过式(3)所示的椭圆拟合准则，因此可以确定内部中心椭圆和黑色圆环的区域。记中心白点内部区域的灰度均值为M_I，黑色圆环区域的灰度均值为M_O，则M_I和M_O应满足：After least squares template matching, all ellipse contours in the image have been found. However, in real scenes, there are often some outlines that are not marker targets but have an ellipse shape or are close to the ellipse shape and are also mistaken for marker targets. The degree is black, and the contrast between the two is strong. This is a distinctive feature that distinguishes marked point targets from other non-marked point targets. Therefore, it is necessary to further exclude non-coded point targets based on this gray feature of marked points. Since the ellipse fitting criterion shown in formula (3) has been passed, the area of the inner central ellipse and the black circle can be determined. Note that the average gray value of the inner area of the central white point is M _I , and the average gray value of the black ring area is M _O , then M _I and M _O should satisfy:

$\{\begin{matrix} {M m}_{I I} {&GreaterEqual; &Greater Equal; M m}_{t t} \\ {M m}_{O o} {\leq \leq M m}_{t t} \\ {M m}_{I I} - - {M m}_{O o} &GreaterEqual; &Greater Equal; Δ Δ {M m}_{t t} \end{matrix} - - - - - - ((44))$

其中，M_t为区分前景灰度与背景灰度的阈值；ΔM_t为前景灰度与背景灰度之差应满足的最小值。Among them, M _t is the threshold for distinguishing the foreground gray level from the background gray level; ΔM _t is the minimum value that the difference between the foreground gray level and the background gray level should satisfy.

另外，约束中心白点内部区域的灰度方差V_I和黑色圆环区域的灰度方差满足：In addition, the gray variance _V of the inner area of the constrained central white point and the gray variance of the black ring area satisfy:

$\{\begin{matrix} {V V}_{I I} {\leq \leq δ δ}_{I I} \\ {V V}_{O o} {\leq \leq δ δ}_{O o} \end{matrix} - - - - - - ((55))$

其中，δ_I，δ_O是允许的最大灰度方差。条件(5)约束编码点的中心必须满足一定的灰度均匀性。满足上述(1)～(5)式，则进入编码点解码过程。Among them, δ _I , δ _O are the maximum gray variance allowed. Condition (5) constrains that the center of the coding point must satisfy a certain gray uniformity. If the above formulas (1)-(5) are satisfied, the code point decoding process is entered.

(2)编码点解码(2) Code point decoding

本发明提出的编码点解码算法的具体实现步骤如下：The specific implementation steps of the code point decoding algorithm proposed by the present invention are as follows:

Step1：拟合编码点中心圆点的外轮廓椭圆(记为椭圆A)、中间黑色圆环的外轮廓椭圆(记为椭圆B)、各白色扇区所在的圆环的外轮廓椭圆(记为椭圆C)。再拟合一个位于椭圆B、C中间的椭圆，其中心及旋转角与椭圆B、C的中心及旋转角相同，长、短轴分别取B、C长、短轴的均值。再采用椭圆绘制的算法，获得椭圆D上各象素点的位置坐标。Step1: Fitting the outer contour ellipse of the center point of the coding point (denoted as ellipse A), the outer contour ellipse of the middle black circle (denoted as ellipse B), and the outer contour ellipse of the ring where each white sector is located (denoted as Ellipse C). Then fit an ellipse located in the middle of ellipses B and C, whose center and rotation angle are the same as the centers and rotation angles of ellipses B and C, and the long and short axes are the mean values of the long and short axes of B and C respectively. Then use the ellipse drawing algorithm to obtain the position coordinates of each pixel point on the ellipse D.

Step2：计算A所包围的区域内所有象素灰度的中值作为前景灰度，A和B之间区域内所有象素灰度的中值作为背景灰度。前景灰度与背景灰度的均值作为阈值，用于后续确定编码点各二进制位的码值。Step2: Calculate the median gray value of all pixels in the area surrounded by A as the foreground gray value, and the median gray value of all pixels in the area between A and B as the background gray value. The mean value of the foreground gray level and the background gray level is used as a threshold value for subsequent determination of the code value of each binary bit of the code point.

Step3：对于椭圆D上的任一个象素点TD，过椭圆中心做一条射线，记该射线与椭圆B、C的交点为TB和TC。将线段TBTC上所有象素的灰度值进行排序，取最中间的一个象素灰度值作为TD的新灰度值。Step3: For any pixel point TD on the ellipse D, make a ray through the center of the ellipse, and record the intersection points of this ray with ellipses B and C as TB and TC. Sort the gray values of all pixels on the line segment TBTC, and take the gray value of the middle pixel as the new gray value of TD.

Step4：对椭圆D上各点按照公式(6)做逆仿射变换，使椭圆D对应一个单位圆，单位圆上各点的灰度对应于椭圆B的新灰度值。Step4: Perform an inverse affine transformation on each point on the ellipse D according to formula (6), so that the ellipse D corresponds to a unit circle, and the grayscale of each point on the unit circle corresponds to the new grayscale value of the ellipse B.

${X x}^{' '} = = [\begin{matrix} {a a}^{- - 11} \\ {b b}^{- - 11} \end{matrix}] [\begin{matrix} cos cos α α & sin sin α α \\ - - sin sin α α & cos cos α α \end{matrix}] ((X x - - {X x}_{o o})) - - - - - - ((66))$

式中，X′是与TD对应的单位圆上点的坐标，X是点TD的坐标，X_o是椭圆D的中心O的坐标，a，b分别是椭圆D的长轴和短轴的长度，α是椭圆D的旋转角。In the formula, X' is the coordinate of the point on the unit circle corresponding to TD, X is the coordinate of the point TD, X _o is the coordinate of the center O of the ellipse D, a, b are the lengths of the major axis and the minor axis of the ellipse D respectively , α is the rotation angle of the ellipse D.

Step5：对单位圆上像素作二值化，取其中的一个边缘点作为起始点。Step5: Binarize the pixels on the unit circle, and take one of the edge points as the starting point.

Step6：从起始点开始，在该单位圆上每隔24°为一个二进制位，计算各位内所有象素点的平均灰度值。若某位的灰度平均值大于阈值，则该位取二进制码为“1”；否则取“0”。从而可以得到编码点的一个二进制编码。找到与该二进制数对应的最小十进制数，该十进制数即为编码点的ID。Step6: Starting from the starting point, every 24° on the unit circle is a binary bit, and calculate the average gray value of all pixels in each bit. If the gray average value of a certain bit is greater than the threshold, the bit takes the binary code as "1"; otherwise it takes "0". A binary encoding of the code point can thus be obtained. Find the smallest decimal number corresponding to the binary number, which is the ID of the code point.

上述Step3相当于对点TD在一个线形窗口内进行一次中值滤波，滤波窗口内包含的象素即为线段TBTC上的象素。采用中值滤波后椭圆D上新的灰度值来确定编码点每一位的二进制码，考虑了编码带内所有象素的灰度值，能够消除孤立噪声的影响。对大量的现场拍摄图像的处理结果表明，这一方法对提高编码点身份识别的鲁棒性十分有效。The above Step 3 is equivalent to performing a median filter on the point TD in a linear window, and the pixels contained in the filter window are the pixels on the line segment TBTC. The new gray value on the ellipse D after median filtering is used to determine the binary code of each bit of the coding point, and the gray value of all pixels in the coding band is considered, which can eliminate the influence of isolated noise. The processing results of a large number of live images show that this method is very effective in improving the robustness of code point identification.

(3)标记点中心计算(3) Mark point center calculation

采用式(7)来进行编码点的中心亚象素定位Use equation (7) to locate the center sub-pixel of the code point

$\{\begin{matrix} {x x}_{c c} = = \underset{j j}{Σ Σ} \underset{i i}{Σ Σ} i i \cdot \cdot {I I}_{i i,, j j} / / \underset{j j}{Σ Σ} \underset{i i}{Σ Σ} {I I}_{i i,, j j} \\ {y the y}_{c c} = = \underset{j j}{Σ Σ} \underset{i i}{Σ Σ} j j \cdot &Center Dot; {I I}_{i i,, j j} / / \underset{j j}{Σ Σ} \underset{i i}{Σ Σ} {I I}_{i i,, j j} \end{matrix} - - - - - - ((77))$

式中，(x_c，y_c)为编码点中心坐标，I_i，j为中心圆形区域内象素点(i，j)的灰度值。In the formula, (x _c , y _c ) is the coordinates of the center of the coding point, and I _{i, j} is the gray value of the pixel point (i, j) in the central circular area.

相机位姿自动确定Automatic camera pose determination

在齐次坐标表示下，三维空间点X在摄像机成像平面上的投影x可表示为：Under the representation of homogeneous coordinates, the projection x of a point X in three-dimensional space on the imaging plane of the camera can be expressed as:

x＝K[R|t]X＝PX (8)x＝K[R|t]X＝PX (8)

其中，K是摄像机内参数矩阵；R和t分别为从世界坐标系到摄像机坐标系的旋转变换矩阵和平移变换向量；P为3×4的投影变换矩阵。Among them, K is the internal parameter matrix of the camera; R and t are the rotation transformation matrix and translation transformation vector from the world coordinate system to the camera coordinate system respectively; P is the 3×4 projection transformation matrix.

设相机在两个不同的位置和朝向对同一场景拍摄了两幅图像，两个相机之间存在旋转矩阵R₁₂和非零平移向量t₁₂，则由极线几何可知两幅图像之间存在以下约束Assuming that the camera has taken two images of the same scene at two different positions and orientations, and there is a rotation matrix R ₁₂ and a non-zero translation vector t ₁₂ between the two cameras, it can be known from the epipolar geometry that the following exists between the two images constraint

${x x}_{22}^{T T} F f {x x}_{11} = = 00 - - - - - - ((99))$

式中x₁和x₂分别是三维空间点X在第一、第二两幅图像上的投影点；F是3×3的基本矩阵，它映射右图像上的一点x₂到左图像相应的对极线Fx₁上。In the formula, x ₁ and x ₂ are the projection points of the three-dimensional space point X on the first and second images respectively; F is a 3×3 basic matrix, which maps a point x ₂ on the right image to the corresponding point on the left image on the epipolar line Fx ₁ .

根据两幅图像之间编码点的对应关系 l＝1，L，N，N≥5，我们首先采用MLESAC(Maximum Likelihood Estimation SAmple Consensus)方法计算两幅图像之间的基本矩阵F。According to the correspondence between the coding points between the two images l=1, L, N, N≥5, we first use the MLESAC (Maximum Likelihood Estimation SAmple Consensus) method to calculate the fundamental matrix F between two images.

根据相机标示的焦距等内参数，可以构建内参数矩阵K的初值(产品标示参数只作为初值，测量系统在后续过程中会对其做进一步优化)。这样，根据基本矩阵F，可进一步计算两幅图像之间的本质矩阵EAccording to the internal parameters such as the focal length marked by the camera, the initial value of the internal parameter matrix K can be constructed (the product marked parameters are only used as initial values, and the measurement system will further optimize them in the subsequent process). In this way, according to the fundamental matrix F, the essential matrix E between the two images can be further calculated

E＝K^TFK (10)E＝K ^T F K (10)

又根据本质矩阵的定义E＝[t]_×R(式中[g]_×表示向量的反对称矩阵)，利用旋转矩阵的正交性可以很容易推导出And according to the definition of the essential matrix E=[t] _× R (where [g] _× represents the anti-symmetric matrix of the vector), it can be easily deduced by using the orthogonality of the rotation matrix

${\overset{^^}{E E.}}^{T T} \overset{^^}{E E.} = = [\begin{matrix} 11 - - {\overset{^^}{t t}}_{x x}^{22} & - - {\overset{^^}{t t}}_{x x} {\overset{^^}{t t}}_{y the y} & - - {\overset{^^}{t t}}_{x x} {\overset{^^}{t t}}_{z z} \\ - - {\overset{^^}{t t}}_{y the y} {\overset{^^}{t t}}_{x x} & 11 - - {\overset{^^}{t t}}_{y the y}^{22} & - - {\overset{^^}{t t}}_{y the y} {\overset{^^}{t t}}_{z z} \\ - - {\overset{^^}{t t}}_{z z} {\overset{^^}{t t}}_{x x} & - - {\overset{^^}{t t}}_{z z} {\overset{^^}{t t}}_{y the y} & 11 - - {\overset{^^}{t t}}_{z z}^{22} \end{matrix}] - - - - - - ((1111))$

式中 $\hat{E} = E / \sqrt{Tr (E^{T} E) / 2},$ Tr(g)表示矩阵的迹， ${\hat{t}}_{12} = t_{12} / | | t_{12} | |$ 是归一化后的平移向量。这样，根据(10)式求出的E矩阵和式(11)可以很容易求得归一化后的平移向量 ${\hat{t}}_{12} = {({\hat{t}}_{x}, {\hat{t}}_{y}, {\hat{t}}_{z})}^{T} .$ 由于 ${(- \hat{E})}^{T} (- \hat{E}) = {\hat{E}}^{T} \hat{E},$ 因此，归一化得到的矩阵

可能与实际的相差一个符号。另外，由于矩阵

的每一个元素都是关于向量

的二次项，因此根据式(11)计算出的向量也具有二义性，即都满足(11)式。后文将给出解决和

的二义性的方法。In the formula

\hat{E.} = E. / \sqrt{Tr ({E.}^{T} E.) / 2},

Tr(g) represents the trace of the matrix,

{\hat{t}}_{12} = t_{12} / | | t_{12} | |

is the normalized translation vector. In this way, the normalized translation vector can be easily obtained according to the E matrix obtained by formula (10) and formula (11)

{\hat{t}}_{12} = {({\hat{t}}_{x}, {\hat{t}}_{the y}, {\hat{t}}_{z})}^{T} .

because

{(- \hat{E.})}^{T} (- \hat{E.}) = {\hat{E.}}^{T} \hat{E.},

Therefore, the matrix obtained by normalization

possible and actual differ by one sign. Also, since the matrix

Each element of is about the vector

The quadratic term of , so the vector calculated according to formula (11) is also ambiguous, namely All satisfy (11). The solution will be given later and

ambiguous method.

为了计算第一幅图与第二幅图之间的旋转矩阵R₁₂，定义To calculate the rotation matrix R ₁₂ between the first image and the second image, define

${w w}_{i i} = = {\overset{^^}{E E.}}_{i i} \times \times {\overset{^^}{t t}}_{1212} ((i i = = 1,2,3 1,2,3)) - - - - - - ((1212))$

式中代表矩阵的各行向量。设r_i是旋转矩阵R₁₂的各行向量，则In the formula representative matrix The row vectors of . Let r _i be the row vectors of the rotation matrix R ₁₂ , then

r_i＝w_i+w_j×w_k (13)r _i =w _i +w _j ×w _k (13)

式中(i，j，k)是(1，2，3)的循环组合。这样就确定了前两幅视图的相机姿态。Where (i, j, k) is a cyclic combination of (1, 2, 3). This determines the camera pose for the first two views.

将世界坐标系建立在第一个相机上，根据成像几何关系不难导出空间点X在第一个相机坐标系下的Z方向坐标为Establish the world coordinate system on the first camera. According to the imaging geometric relationship, it is not difficult to derive the Z coordinate of the space point X in the first camera coordinate system as

${Z Z}_{11} = = f f \frac{{(({fr fr}_{11} - - {x x}_{22} {r r}_{33}))}^{T T} {\overset{^^}{t t}}_{1212}}{{(({fr fr}_{11} - - {x x}_{22} {r r}_{33}))}^{T T} {x x}_{11}} - - - - - - ((1414))$

进一步可求得另外两个坐标分量Further, the other two coordinate components can be obtained

X₁＝x₁Z₁/f，Y₁＝y₁Z₁/f (15)X ₁ =x ₁ Z ₁ /f, Y ₁ =y ₁ Z ₁ /f (15)

X在第二个相机坐标系下的坐标为The coordinates of X in the second camera coordinate system are

X₂＝R₁₂(X₁-t₁₂) (16)X ₂ =R ₁₂ (X ₁ -t ₁₂ ) (16)

由于

和的二义性，可能产生四对不同的根据拍摄时的实际情况，只有当采用某个对重建的所有点均同时在两个相机前面，即所有点(这里是指两个视图共同可见的各个编码点中心)的Z₁和Z₂都为正时，才表明重建结果正确，相应的一组即为正确解。because

and ambiguity, four different pairs of According to the actual situation at the time of shooting, only when a certain All reconstructed points are in front of the two cameras at the same time, that is, when Z ₁ and Z ₂ of all points (here refers to the center of each coding point visible to both views) are both positive, it indicates that the reconstruction result is correct, and the corresponding A group is the correct solution.

由于上述重建算法中，两个相机之间基线长度未知，因此只能得到归一化的平移向量从式(14)容易看出，重建出来的场景与实际场景相差一个固定的比例因子。为此，在场景中放置标尺，标尺上两个标记点之间的距离已知，从而可以确定一个比例因子，得到被测物体的实际尺寸。Since the baseline length between the two cameras is unknown in the above reconstruction algorithm, only the normalized translation vector can be obtained It is easy to see from formula (14) that the reconstructed scene differs from the actual scene by a fixed scale factor. To this end, a scale is placed in the scene, and the distance between two marked points on the scale is known, so that a scale factor can be determined to obtain the actual size of the measured object.

在两视图相机姿态确定和编码点中心三维坐标重建的基础上，需要进一步依次确定其它各次拍摄时的相机姿态。对第j幅图像进行处理时，要求该图像中存在至少6个已在前面的步骤中重建出了三维坐标的编码点，即已知图像点和空间点的对应X_ix_i，i＝1，L L，L≥6。将这些约束带入投影方程(8)中可得On the basis of determining the pose of the two-view camera and reconstructing the three-dimensional coordinates of the center of the encoding point, it is necessary to further determine the pose of the camera in other shots in sequence. When processing the jth image, it is required that there are at least 6 coding points in the image whose three-dimensional coordinates have been reconstructed in the previous steps, that is, the corresponding X _i  x _i of known image points and spatial points, i= 1, L L, L≥6. Bringing these constraints into the projection equation (8) gives

x_i＝P_jX_i，i＝1，L L，L≥6 (17)x _i =P _j X _i , i=1, L L, L≥6 (17)

由于每组X_ix_i的对应产生两个线性方程，因此，根据(17)式采用最小二乘法即可求解出第j幅图像的投影矩阵P_j中的11个未知元素。Since the correspondence of each group of X _i  x _i produces two linear equations, the 11 unknown elements in the projection matrix P _j of the jth image can be solved by using the least square method according to formula (17).

将3×4的投影矩阵P_j表示成Express the 3×4 projection matrix P _j as

其中，M是矩阵P_j前3×3的子矩阵，p₄表示矩阵P_j的第四列。从式(18)中很容易确定出平移向量t_j＝K^-1p₄。Among them, M is the first 3×3 sub-matrix of matrix P _j , and p ₄ represents the fourth column of matrix P _j . It is easy to determine the translation vector t _j =K ⁻¹ p ₄ from formula (18).

由于内参数矩阵是上三角的且旋转矩阵是正交的，因此，对矩阵进行QR分解就可得旋转矩阵R_j。在估计出拍摄第j幅图像时相机相对于世界坐标系的外部姿态参数R_j和t_j的基础上，我们进一步通过光学三角形法，重建出第j幅图像中新出现的且可在前j-1幅图像中找到同名匹配的编码点的三维空间点坐标。至此，就完成了对应第j幅图像的相机姿态确定和编码点三维坐标计算。然后继续下一幅图像的处理，直至处理完所有图像。Since the internal parameter matrix is upper triangular and the rotation matrix is orthogonal, the rotation matrix R _j can be obtained by performing QR decomposition on the matrix. On the basis of estimating the external attitude parameters R _j and t _j of the camera relative to the world coordinate system when the j-th image is taken, we further use the optical triangle method to reconstruct -The 3D space point coordinates of the coded point with the same name matching found in 1 image. So far, the determination of the camera pose corresponding to the jth image and the calculation of the three-dimensional coordinates of the encoding point are completed. Then continue to process the next image until all images are processed.

(3)相机姿态优化(3) Camera pose optimization

由于图像噪声等因素的存在，三维空间点X_i经投影矩阵P_j变换后的像点，与实际识别出的X_i在第j幅图像中的像点坐标x_ij并不重合。为了进一步提高系统精度，本发明基于光束平差算法，以再投影误差最小建立目标函数Due to the existence of factors such as image noise, the image point of the three-dimensional space point X _i transformed by the projection matrix P _j does not coincide with the actually recognized image point coordinate x _ij of X _i in the jth image. In order to further improve the system accuracy, the present invention is based on the beam adjustment algorithm, and establishes the objective function with the minimum reprojection error

$\underset{ij ij}{Σ Σ} d d {(({P P}_{j j} {X x}_{i i},, {x x}_{ij ij}))}^{22} &RightArrow; &Right Arrow; min min - - - - - - ((1919))$

对前面求出的相机参数和三维空间点坐标进行全局优化。具体求解采用LM(Levenberg-Marquardt)算法。由于是以前面求出的已经比较接近真实值的X_i和P_j作为初值，因此，全局优化可以较快收敛。Globally optimize the camera parameters and 3D space point coordinates obtained earlier. The specific solution adopts LM (Levenberg-Marquardt) algorithm. As the initial values of Xi and _Pj _, which are already close to the real values obtained earlier, are used as the initial values, so the global optimization can converge quickly.

目标曲线的半自动提取Semi-automatic extraction of target curves

本发明开发的物体曲线结构三维测量软件系统图形界面示意图如附图3所示，该图的左侧显示所有拍摄得到的图像文件的列表，该图的右侧下部为当前活动图像对的两个图像显示窗口，通过点击左侧的图像文件列表来指定两个窗口内当前显示的图像，该图的右侧上部为重建出的目标曲线的三维图形显示区。The schematic diagram of the graphical interface of the object curve structure three-dimensional measurement software system developed by the present invention is as shown in accompanying drawing 3, the left side of this figure shows the list of all image files that are photographed, and the lower right side of this figure is the two pairs of current active images. In the image display window, click the image file list on the left to specify the currently displayed images in the two windows. The upper right part of the figure is the three-dimensional graphic display area of the reconstructed target curve.

本发明采用能量优化的基本思想将在两个活动图像上交互勾勒出的目标曲线的大致轮廓通过后台算法自动“贴合”到相应的目标曲线上。The present invention adopts the basic idea of energy optimization to automatically "fit" the rough outline of the target curve drawn interactively on the two moving images to the corresponding target curve through the background algorithm.

具体实现时，首先连接用户输入的目标曲线上的点构成一条折线(封闭情况下为多边形)，然后本发明采用计算机图形学中直线段光栅扫描转换的DDA算法快速得到折线经过的所有象素点，从这些象素点中按固定间隔(实施例中每隔两个象素取一个)进行During concrete realization, at first connect the point on the target curve of user input to form a broken line (be polygon under closed situation), then the present invention adopts the DDA algorithm of raster scan conversion of straight line segment in computer graphics to obtain all pixel points that broken line passes through fast , from these pixels at fixed intervals (take one every two pixels in the embodiment)

采样，记为v_i，i＝(0，1，L，n)。这里需要进一步利用Canny边缘检测算子自动检测出的图像边缘信息，因此记检测出的边缘点集为P。另记v_ij，j＝(1，L，8)为点v_i的八邻域象素点，同时为了叙述方便，记v_i0＝v_i。Sampling, denoted as v _i , i=(0, 1, L, n). Here it is necessary to further utilize the image edge information automatically detected by the Canny edge detection operator, so record the detected edge point set as P. In addition, record v _ij , j=(1, L, 8) as the eight-neighborhood pixel points of point v _i , and record v _i0 =v _i for the convenience of description.

本发明建立每个点v_i及其八邻域点的如下能量函数：The present invention establishes the following energy functions of each point v _i and its eight neighborhood points:

E(v_ij)＝[αE_tension(v_ij)+βE_bend(v_ij)+γE_img(v_ij)+δE_attr(v_ij)] (20)E(v _ij )＝[αE _tension (v _ij )+βE _bend (v _ij )+γE _img (v _ij )+δE _attr (v _ij )] (20)

其中，E_tension(v_ij)，E_bend(v_ij)，E_img(v_ij)，E_attr(v_ij)分别为v_ij点处的拉伸能量、弯曲能量、图像能量和边缘点引力产生的能量，α，β，γ，δ分别为各能量项的权值，用以调节各能量项的比重。为了平衡各项的影响，各能量项全都归一化到区间[0，1]：Among them, E _tension (v _ij ), E _bend (v _ij ), E _img (v _ij ), E _attr (v _ij ) are the stretching energy, bending energy, image energy and edge point gravitational force at point v _ij respectively The energy of , α, β, γ, δ are the weights of each energy item, which are used to adjust the proportion of each energy item. In order to balance the influence of each item, each energy item is normalized to the interval [0, 1]:

${E E.}_{tension tension} (({v v}_{ij ij})) = = \frac{| | \overset{&OverBar; &OverBar;}{d d} - - | | {v v}_{ij ij} - - {v v}_{i i - - 11} | | | |}{\underset{00 \leq \leq j j \leq \leq 88}{max max} {{| | \overset{&OverBar; &OverBar;}{d d} - - | | {v v}_{ij ij} - - {v v}_{i i - - 11} | | | |}}},,$ $\overset{&OverBar; &OverBar;}{d d} = = \frac{11}{n no} {Σ Σ}_{i i = = 11}^{n no} | | {v v}_{i i} - - {v v}_{i i - - 11} | |$

${E E.}_{bend bend} (({v v}_{ij ij})) = = \frac{{| | {v v}_{i i - - 11} - - 22 {v v}_{ij ij} + + {v v}_{i i + + 11} | |}^{22}}{\underset{00 \leq \leq j j \leq \leq 88}{max max} {{{| | {v v}_{i i - - 11} - - {22 v v}_{ij ij} + + {v v}_{i i + + 11} | |}^{22}}}}$ ${E E.}_{img img} (({v v}_{ij ij})) = = \frac{\underset{00 \leq \leq j j \leq \leq 88}{min min} (({E E.}_{img img} (({v v}_{ij ij})))) - - {E E.}_{img img} (({v v}_{ij ij}))}{\underset{00 \leq \leq j j \leq \leq 88}{max max} (({E E.}_{img img} (({v v}_{ij ij})))) - - \underset{00 \leq \leq j j \leq \leq 88}{min min} (({E E.}_{img img} (({v v}_{ij ij}))))}$

${E E.}_{attr attr} (({v v}_{ij ij})) = = \frac{| | {v v}_{ij ij} - - {p p}_{ij ij} | |}{\underset{00 \leq \leq j j \leq \leq 88}{max max} {{| | {v v}_{ij ij} - - {p p}_{ij ij} | |}}}$

其中，E_img＝-|I(x，y)|²，E_attr(v_ij)中的p_ij∈P是与v_i距离最近的边缘点。本发明在拉伸能、弯曲、图像能之外增加边缘点引力能E_attr(v_i)的目的是进一步促使点列向目标曲线收敛。p_ij的搜索限定在以v_i为中心的一个窗口内进行。如果该窗口内没有任何边缘点，E_attr(v_ij)＝0，j＝0，L，8。Wherein, E _img =-|I(x, y)| ² , p _ij ∈P in E _attr (v _ij ) is the edge point closest to v _i . The purpose of the present invention to increase the gravitational energy E _attr (v _i ) of the edge points in addition to the stretching energy, bending, and image energy is to further promote the convergence of the point sequence to the target curve. The search of p _ij is limited to a window centered on v _i . If there is no edge point in the window, E _attr (v _ij )=0, j=0,L,8.

通过迭代的方法使点列v_i，i＝(0，1，L，n)向使式(20)能量极小的位置移动，最终锁定在图像特征附近，形成光滑的目标曲线点列。再用均匀B样条曲线来拟合这些点，以备后续曲线匹配使用。对于开曲线，约束其两端点始终保持位置不变，从而防止开曲线的点列收缩为一点产生退化。The point sequence v _i , i=(0, 1, L, n) is moved to the position where the energy of formula (20) is minimal by iterative method, and finally locked near the image feature to form a smooth target curve point sequence. These points are then fitted with a uniform B-spline curve for subsequent curve matching. For the open curve, the two ends of the constraint are always kept in the same position, so as to prevent the point sequence of the open curve from shrinking into one point and degenerate.

(1)匹配程度度量(1) Matching degree measurement

相机在不同位置和角度获得的一个图像对间同名点应满足的一个基本约束是(9)式描述的极线约束。在相机内参数和相对位置及姿态已知的情况，图像对间的基本矩阵F是已知的。对于一个图像对中两个候选匹配点v₁和v₂，可以用在第二幅图像上v₂到v₁的极线的距离A basic constraint that points with the same name between an image pair obtained by the camera at different positions and angles should satisfy is the epipolar constraint described in (9). In the case where the camera intrinsic parameters and relative position and pose are known, the fundamental matrix F between image pairs is known. For two candidate matching points v ₁ and v ₂ in an image pair, the distance from v ₂ to the epipolar line of v ₁ on the second image can be used

${D D.}_{22} (({v v}_{11},, {v v}_{22})) = = \frac{| | {v v}_{22}^{T T} F f {v v}_{11} | |}{| | | | eF f {v v}_{11} | | | |} - - - - - - ((21 twenty one))$

以及在第一幅图像上v₁到v₂的极线的距离and the distance of epipolar lines from _v1 to _v2 on the first image

${D D.}_{11} (({v v}_{11},, {v v}_{22})) = = \frac{| | {v v}_{22}^{T T} F f {v v}_{11} | |}{| | | | {v v}_{22}^{T T} F f {e e}^{T T} | | | |} - - - - - - ((22 twenty two))$

度量v₁和v₂的匹配程度，其中， $e = [\begin{matrix} 0 & - 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 0 \end{matrix}] .$ 本发明正是根据式(21)和式(22)的极线约束条件，建立同名曲线间点对匹配的优化目标函数。Measures how well _v1 and _v2 match, where, $e = [\begin{matrix} 0 & - 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 0 \end{matrix}] .$ The present invention is just based on the epipolar constraint conditions of formula (21) and formula (22), to establish the optimization objective function of point pair matching between curves with the same name.

(2)图像曲线重采样(2) Image curve resampling

为了得到拟合出来的均匀B样条曲线p(u)，u∈[0，1]上的各象素点，取离散参数间距Δu＝1/L，L为各型值点的累加弦长，得到均匀B样条曲线上的离散象素点v_i，i＝0，1，...，N，构造分段线性插值曲线满足：In order to obtain the fitted uniform B-spline curve p(u), each pixel point on u∈[0,1], take the discrete parameter distance Δu=1/L, L is the accumulated chord length of each value point , to obtain discrete pixel points v _i on the uniform B-spline curve, i=0, 1, ..., N, and construct a piecewise linear interpolation curve satisfying:

$\{\begin{matrix} c c (({l l}_{i i})) = = {v v}_{i i},, & i i = = 00,, . . . . . .,, N N \\ c c ((l l)) = = \frac{{l l}_{i i + + 11} - - l l}{{l l}_{i i + + 11} - - {l l}_{i i}} {v v}_{i i} + + \frac{l l - - {l l}_{i i}}{{l l}_{i i + + 11} - - {l l}_{i i}} {v v}_{i i + + 11},, & {l l}_{i i} \leq \leq l l < < {l l}_{i i + + 11} \end{matrix} - - - - - - ((23 twenty three))$

其中l₀＝0.0， $l_{i} = Σ_{j = 1}^{i} | v_{j} - v_{j - 1} | .$ 记 $L = Σ_{j = 1}^{N} | v_{j} - v_{j - 1} | .$ 不失一般性，将两条同名图像曲线中象素点较多的一条记为c₁(l)，较少的那条记为c₂(l)。c₁(l)上的离散象素点记为v_k ⁽¹⁾，0≤k≤N₁；c₂(l)上的离散象素点记为v_j ⁽²⁾，0≤j≤N₂。寻求c₁(l)，l∈[0，L₁]上的各象素点在c₂(l)，l∈[0，L₂]上的匹配点，可以实现其在c₂(l)上的亚象素匹配，从而实现高精度三维曲线重建。where l ₀ =0.0, $l_{i} = Σ_{j = 1}^{i} | v_{j} - v_{j - 1} | .$ remember $L = Σ_{j = 1}^{N} | v_{j} - v_{j - 1} | .$ Without loss of generality, among the two image curves with the same name, the one with more pixels is denoted as c ₁ (l), and the one with fewer pixels is denoted as c ₂ (l). The discrete pixel points on c ₁ (l) are recorded as v _k ⁽¹⁾ , 0≤k≤N ₁ ; the discrete pixel points on c ₂ (l) are recorded as v _j ⁽²⁾ , 0≤j≤N ₂ . Seeking the matching points of each pixel point on c ₁ (l), l∈[0, L ₁ ] on c ₂ (l), l∈[0, L ₂ ], it can be achieved in c ₂ (l) On the sub-pixel matching, so as to achieve high-precision three-dimensional curve reconstruction.

(3)匹配优化(3) Matching optimization

首先采用动态规划来初始匹配对应曲线上的离散象素点。同名曲线一个对应点对的累积代价函数定义为First, dynamic programming is used to initially match the discrete pixel points on the corresponding curve. The cumulative cost function for a corresponding point pair of the curve of the same name is defined as

$C C (({v v}_{k k}^{((11))},, {v v}_{j j}^{((22))})) = = D D. (({v v}_{k k}^{((11))},, {v v}_{j j}^{((22))})) + + \underset{m m &Element; &Element; {G G}_{kj kj}}{min min} C C (({v v}_{k k - - 11}^{((11))},, {v v}_{m m}^{((22))})) - - - - - - ((24 twenty four))$

其中 $D (v_{k}^{(1)}, v_{j}^{(2)}) = D_{1} (v_{k}^{(1)}, v_{j}^{(2)}) + D_{2} (v_{k}^{(1)}, v_{j}^{(2)}),$ G_kj表示在v_k ⁽¹⁾与v_j ⁽²⁾已经匹配的情况下m的所有可能取值。由于两条同名图像曲线上的象素点个数不等，动态规划匹配的结果会出现多对一的情况，即较长的c₁曲线上的多个象素点对应较短的c₂曲线上的同一个象素点。in $D. (v_{k}^{(1)}, v_{j}^{(2)}) = {D.}_{1} (v_{k}^{(1)}, v_{j}^{(2)}) + {D.}_{2} (v_{k}^{(1)}, v_{j}^{(2)}),$ G _kj represents all possible values of m when v _k ⁽¹⁾ and v _j ⁽²⁾ have been matched. Since the number of pixels on the two image curves with the same name is not equal, the result of dynamic programming matching will appear many-to-one, that is, multiple pixel points on the longer _c1 curve correspond to the shorter _c2 curve on the same pixel.

得到图像曲线上点对的初始匹配后，再进行曲线的匹配优化。根据式(21)和式(22)，本发明优化下面的目标函数来达到c₁、c₂曲线上点的精确匹配After the initial matching of point pairs on the image curve is obtained, the matching optimization of the curve is performed. According to formula (21) and formula (22), the present invention optimizes the following objective function to achieve the exact matching of points on the c ₁ and c ₂ curves

$min min {&Integral; &Integral;}_{00}^{{L L}_{11}} \frac{| | {c c}_{22} {((σ σ ((l l))))}^{T T} F f {c c}_{11} ((l l)) | |}{| | | | eF f {c c}_{11} ((l l)) | | | |} + + \frac{| | {c c}_{22} {((σ σ ((l l))))}^{T T} F f {c c}_{11} ((l l)) | |}{| | | | {c c}_{22} {((σ σ ((l l))))}^{T T} F f {e e}^{T T}} dl dl - - - - - - ((2525))$

其中σ(l)为待求的映射函数，表示c₁曲线上参数为l的点在曲线c₂上的参数值。将式(25)的积分形式改写为求和的形式Among them, σ(l) is the mapping function to be obtained, which means the parameter value of the point on the curve c ₂ whose parameter is l on the curve c ₁ . Rewrite the integral form of formula (25) into the form of summation

(26)式中l₀，l₁，L，l_N1分别是v_k ⁽¹⁾，0≤k≤N₁对应c₁(l)上的参数值。以上述动态规划法产生的粗匹配为迭代初始条件，采用共轭梯度法即可求解式(26)的最小化问题。In formula (26), l ₀ , l ₁ , L, and l _N1 are v _k ⁽¹⁾ respectively, and 0≤k≤N ₁ corresponds to the parameter value on c ₁ (l). Taking the rough matching generated by the above dynamic programming method as the initial condition of iteration, the minimization problem of formula (26) can be solved by using the conjugate gradient method.

曲线三维重建Curved 3D reconstruction

完成同名曲线上所有象素点的匹配后，由于相机内参数已知，且图像对在各自拍摄时的相机位置和姿态已经自动计算得出，即相对于世界坐标系的旋转矩阵和平移向量也已知，因此可以用双目立体视觉中成熟的三角测量法重建出这些点的空间坐标，具体计算式可表达为After completing the matching of all pixels on the curve with the same name, since the internal parameters of the camera are known, and the camera position and attitude of the image pair when they are shot have been automatically calculated, that is, the rotation matrix and translation vector relative to the world coordinate system are also It is known, so the spatial coordinates of these points can be reconstructed by the mature triangulation method in binocular stereo vision, and the specific calculation formula can be expressed as

$\begin{matrix} {x x}^{((j j))} = = f f \frac{{R R}_{1111}^{((j j))} X x + + {R R}_{1212}^{((j j))} Y Y + + {R R}_{1313}^{((j j))} Z Z + + {T T}_{x x}^{((j j))}}{{R R}_{3131}^{((j j))} X x + + {R R}_{3232}^{((j j))} Y Y + + {R R}_{3333}^{((j j))} Z Z + + {T T}_{z z}^{((j j))}} \\ {y the y}^{((j j))} = = f f \frac{{R R}_{21 twenty one}^{((j j))} X x + + {R R}_{22 twenty two}^{((j j))} Y Y + + {R R}_{23 twenty three}^{((j j))} Z Z + + {T T}_{y the y}^{((j j))}}{{R R}_{3131}^{((j j))} X x + + {R R}_{3232}^{((j j))} Y Y + + {R R}_{3333}^{((j j))} Z Z + + {T T}_{z z}^{((j j))}} \end{matrix}\},, j j = = 1,2 1,2 - - - - - - ((2727))$

其中，f是相机的焦距，x^(j)和y^(j)分别是同名匹配点在第j幅图像上的象素坐标的两个分量，R.^(j)是第j次拍摄时相机相对于世界坐标系的旋转矩阵的各个分量，T.^(j)是第j次拍摄时相机相对于世界坐标系的各个平移分量，根据(27)式中的4个方程用最小二乘法即可解出该点的空间坐标的3个未知分量(X，Y，Z)。对同名曲线上每一个匹配点对执行上述求解过程，就完成了整条目标曲线点列的三维测量。完成所有目标曲线的三维测量后，可以进一步根据这些点列构造出曲线的参数方程及模型的曲面方程。Among them, f is the focal length of the camera, x ^(j) and y ^(j) are the two components of the pixel coordinates of the matching point with the same name on the jth image, and R. ^(j) is the relative Each component of the rotation matrix of the world coordinate system, T. ^(j) is each translation component of the camera relative to the world coordinate system at the jth shooting, and can be solved by the least square method according to the four equations in (27) 3 unknown components (X, Y, Z) of the spatial coordinates of the point. The above solution process is performed for each pair of matching points on the curve with the same name, and the three-dimensional measurement of the point sequence of the entire target curve is completed. After completing the three-dimensional measurement of all target curves, the parameter equation of the curve and the surface equation of the model can be constructed further according to these point columns.

Claims

1. A method for carrying out three-dimensional measurement on an object by utilizing free shooting of a single digital camera is characterized by comprising seven steps of measurement preparation, image shooting, identification and positioning of encoding points, camera pose determination, target curve extraction, automatic matching optimization of homonymous curves and three-dimensional reconstruction of target curves, and the specific method comprises the following steps: firstly, marking a characteristic line of a measured object and a key section control line on the surface of the object required for digital model reconstruction, so that the characteristic line is obviously different from the color of the measured object in color brightness, and the identification of images is facilitated; a ruler and a group of coding points are arranged around a measured object; then, holding a digital camera by hand to obtain a group of images of the measured object in a free shooting mode; according to the group of images, the position and the posture of the camera during each shooting are automatically and accurately calculated, meanwhile, a convenient interaction means is provided for a user, the semi-automatic extraction of the identification curve and the optimized matching of homonymous curves in different images are realized, and further, the three-dimensional point list information of the identified curve structure is automatically calculated.

2. The method of claim 1, wherein the identification of the code points is that the codes of each point in a set of designed code points are different, and the candidate code point targets are gradually filtered by adopting 5 constraint conditions of size, shape, ellipse fitting residual, area gray mean, and area gray variance; the gray values of most pixels in a coding band are comprehensively considered in the coding point decoding process, so that the effects of eliminating noise influence and improving the robustness of the identification of the coding points are achieved; and (4) positioning the sub-pixels in the center of the coding point by adopting a gray scale weighting method in the target area.

3. The method of claim 1, wherein the camera pose determination is based on identity uniqueness of the code points, and automatically establishes correspondence between the code points of the same name in each image; and recovering the camera postures corresponding to the two images and the three-dimensional coordinates of the centers of the coding points which are commonly visible in the two images according to the pixel coordinates of the centers of at least 5 coding points with the same name in the two images, and further adopting a strategy of combining an incremental method and a global optimization method to realize the automatic solution of the camera positions and postures corresponding to all the images.

4. The method as claimed in claim 1, wherein the target curve extraction is performed by a user by using a mouse to pick points on the two images near the same identification curve, and making the connecting line of the points approximately reflect the corresponding image curve contour, and then the measurement software system iteratively minimizes the energy of the image curve contour, thereby automatically and optimally fitting the interactively sketched approximate image curve contour to the image curve.

5. The method as claimed in claim 1, wherein the automatic matching optimization of the homonymous curve is performed by using a dynamic programming method to obtain an initial matching of pixel points on the corresponding curve, and then using a nonlinear optimization method to minimize the sum of distances from all matching points on the corresponding curve to respective polar lines, so as to achieve an optimal matching of the homonymous target curve point rows on the image, and further calculate the three-dimensional coordinates of the point rows.