CN110766669B - Pipeline measuring method based on multi-view vision - Google Patents

Pipeline measuring method based on multi-view vision Download PDF

Info

Publication number
CN110766669B
CN110766669B CN201910993518.2A CN201910993518A CN110766669B CN 110766669 B CN110766669 B CN 110766669B CN 201910993518 A CN201910993518 A CN 201910993518A CN 110766669 B CN110766669 B CN 110766669B
Authority
CN
China
Prior art keywords
pipeline
point
camera
centerline
outer diameter
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910993518.2A
Other languages
Chinese (zh)
Other versions
CN110766669A (en
Inventor
王健
李明洲
方定君
吕琦
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing University
Original Assignee
Nanjing University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing University filed Critical Nanjing University
Priority to CN201910993518.2A priority Critical patent/CN110766669B/en
Publication of CN110766669A publication Critical patent/CN110766669A/en
Application granted granted Critical
Publication of CN110766669B publication Critical patent/CN110766669B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0004Industrial image inspection
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01BMEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00Measuring arrangements characterised by the use of optical techniques
    • G01B11/08Measuring arrangements characterised by the use of optical techniques for measuring diameters
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01CMEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C11/00Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/13Edge detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/60Analysis of geometric attributes
    • G06T7/62Analysis of geometric attributes of area, perimeter, diameter or volume
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/80Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30108Industrial image inspection
    • G06T2207/30164Workpiece; Machine component

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Multimedia (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Geometry (AREA)
  • Image Processing (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

本发明公开了一种基于多目视觉的管线测量方法,该方法由图像采集和预处理,中心线及外径提取,相机标定以及相机矩阵转换,管道中心线匹配和外径扩展四部分组成。系统将相机放置于搭建好的测量框架上,进行图像采集和预处理,得到管道灰度图片。运用距离变换和最小路径方法获取管道中心线,提取管道中心线特征点并对特征点进行离散化操作;对相对应的双目相机(前侧相机,后侧相机)进行标定,并获取从后侧相机坐标系至前侧相机坐标系的转换矩阵;根据获取的中心线特征点和相机标定参数,采用一种动态匹配方法,恢复管路中心线的三维信息,并将之前不同视角拍摄图片处理获得的外径信息在三维管路中心线上扩展,最终得到管路中心线的空间位置以及对应的外径信息。

Figure 201910993518

The invention discloses a pipeline measurement method based on multi-eye vision. The method consists of four parts: image acquisition and preprocessing, centerline and outer diameter extraction, camera calibration and camera matrix conversion, pipeline centerline matching and outer diameter expansion. The system places the camera on the built measurement frame, performs image acquisition and preprocessing, and obtains a grayscale image of the pipeline. Use the distance transformation and the minimum path method to obtain the pipeline centerline, extract the pipeline centerline feature points and discretize the feature points; calibrate the corresponding binocular cameras (front camera, rear camera), and obtain the The transformation matrix from the side camera coordinate system to the front side camera coordinate system; according to the obtained centerline feature points and camera calibration parameters, a dynamic matching method is used to restore the three-dimensional information of the pipeline centerline, and the previous pictures taken from different perspectives are processed. The obtained outer diameter information is expanded on the three-dimensional pipeline centerline, and finally the spatial position of the pipeline centerline and the corresponding outer diameter information are obtained.

Figure 201910993518

Description

一种基于多目视觉的管线测量方法A Multi-Vision-Based Pipeline Measurement Method

技术领域technical field

本发明涉及视觉测量技术领域,主要涉及一种基于多目视觉的管线测量方法。The invention relates to the technical field of visual measurement, and mainly relates to a pipeline measurement method based on multi-eye vision.

背景技术Background technique

管路系统大量应用于核堆、航空、航天、船舶和汽车工业中,负责气体、液体等介质的传输和测量,是高端机电产品的重要组成部分,管路零件的精确加工对产品的性能和安全性有着重要的影响。因此高精度的管路加工是当前的热门问题。Piping systems are widely used in nuclear reactors, aviation, aerospace, shipbuilding and automobile industries. They are responsible for the transmission and measurement of gas, liquid and other media. They are an important part of high-end mechanical and electrical products. Security has important implications. Therefore, high-precision pipeline processing is currently a hot issue.

管路加工后的空间几何形态测量主要采用靠模法、三坐标测量测量仪和基于激光CCD器件的测量方法等。靠模法只能对管路的形状粗略的检验,而无法精确测量管路端点的位置以及端点之间的距离。三坐标测量仪测量精度虽然较高,但对测量环境要求比较严格,且测量范围有限。基于激光CCD器件的测量方法在测量端点时要求工人小心地移动测量光叉,操作较为困难。The measurement of the spatial geometry after the pipeline processing mainly adopts the model method, the three-coordinate measuring instrument and the measurement method based on the laser CCD device. The profiling method can only roughly check the shape of the pipeline, but cannot accurately measure the position of the endpoints of the pipeline and the distance between the endpoints. Although the measurement accuracy of the CMM is high, it has strict requirements on the measurement environment and limited measurement range. The measurement method based on the laser CCD device requires workers to move the measuring fork carefully when measuring the endpoint, which is difficult to operate.

学术界方面,张天的论文《基于多目视觉的管路数字化测量方法研究》提出了一种基于中心线匹配和机器学习的管线重建技术。采用机器视觉算法计算管路控制点坐标,并利用管路控制点坐标拟合管路中心线,进而重构出管路的三维模型。In academia, Zhang Tian's paper "Research on Digital Pipeline Measurement Method Based on Multi-ocular Vision" proposed a pipeline reconstruction technology based on centerline matching and machine learning. The machine vision algorithm is used to calculate the coordinates of the pipeline control points, and the pipeline control point coordinates are used to fit the pipeline centerline, and then the three-dimensional model of the pipeline is reconstructed.

综合以上分析,管道测量领域现有技术存着在以下缺点:传统的管路空间几何形态测量方法要么精准度不够,要么对测量环境有比较严格的要求,要么需要使用一些高成本的仪器,或是采用人工交互的方法,成本高,效率低。与张天提出的方法不同之处在于:一、张天的论文中在中心线提取环节使用的是图像学的方法,但这种方法并不能保证提取的是精准的中心线,本发明提出了更高效的距离变换及最小路径方法;二、在中心线匹配环节,张天的论文采取了NURBS曲线拟合方法,本发明提出了一种动态匹配的直接计算方法,更为简洁;此外,本发明提出了基于形心变换的边缘检测方法计算了管线外径。Based on the above analysis, the existing technology in the field of pipeline measurement has the following shortcomings: the traditional measurement method of pipeline space geometry is either not accurate enough, or has strict requirements on the measurement environment, or needs to use some high-cost instruments, or It is a method of manual interaction, which has high cost and low efficiency. The difference from the method proposed by Zhang Tian is as follows: 1. In Zhang Tian's paper, the method of image science is used in the extraction of the center line, but this method cannot guarantee that the exact center line is extracted. More efficient distance transformation and minimum path method; 2. In the centerline matching link, Zhang Tian's paper adopts the NURBS curve fitting method, and the present invention proposes a direct calculation method for dynamic matching, which is more concise; The invention proposes an edge detection method based on centroid transformation to calculate the outer diameter of the pipeline.

发明内容SUMMARY OF THE INVENTION

发明目的:为了克服现有技术中存在的不足,本发明提供一种基于多目视觉的管线测量方法与系统,用以解决当前管线测量技术中存在的流程复杂,成本高,精度低的问题。本方法使用基于中心线匹配的管路重建技术可以高精度的识别管线路径,外径大小,配合建模软件可以获得管路完整的3D模型。Purpose of the invention: In order to overcome the deficiencies in the prior art, the present invention provides a pipeline measurement method and system based on multi-eye vision to solve the problems of complex process, high cost and low precision in the current pipeline measurement technology. In this method, the pipeline reconstruction technology based on centerline matching can be used to identify the pipeline path and the outer diameter with high precision, and a complete 3D model of the pipeline can be obtained by cooperating with the modeling software.

技术方案:为实现上述目的,本发明采用的技术方案为:Technical scheme: In order to realize the above-mentioned purpose, the technical scheme adopted in the present invention is:

1、一种基于多目视觉的管线测量方法,包括以下步骤:1. A pipeline measurement method based on multi-eye vision, comprising the following steps:

(1)图像采集和预处理(1) Image acquisition and preprocessing

步骤1、搭建图像采集系统,通过分布在管道前后两侧的双目相机进行图片抓取,获得拍摄的管道灰度图;对所述灰度图进行二值化处理,使背景与所述管线互相区分;采用轮廓对比技术,对二值化图片进行目标区域ROI提取,获取管道轮廓信息;Step 1. Build an image acquisition system, and capture pictures through binocular cameras distributed on the front and rear sides of the pipeline to obtain a grayscale image of the pipeline; binarize the grayscale image to make the background and the pipeline Distinguish each other; use contour comparison technology to extract the target area ROI from the binarized image to obtain pipeline contour information;

(2)中心线及外径提取(2) Extraction of center line and outer diameter

步骤2.1、根据管道轮廓信息,采用欧式距离变换法计算管道内每点到管道轮廓的欧氏距离,获取管路的距离变换图,记录管线轮廓内部每个点到管线边界的欧式距离;Step 2.1. According to the pipeline outline information, use the Euclidean distance transformation method to calculate the Euclidean distance from each point in the pipeline to the pipeline outline, obtain the distance transformation map of the pipeline, and record the Euclidean distance from each point inside the pipeline outline to the pipeline boundary;

步骤2.2、通过图像处理搜索管道边界,获取管道两侧端点的像素坐标,采用Dijkstra最小路径算法,以中心线一侧端点为起点,另一侧端点为终点,得到两点之间的最小路径,即管道的中心线;Step 2.2. Search the pipeline boundary through image processing, obtain the pixel coordinates of the endpoints on both sides of the pipeline, and use the Dijkstra minimum path algorithm to obtain the minimum path between the two points with the endpoint on one side of the centerline as the starting point and the endpoint on the other side as the endpoint. the centerline of the pipe;

步骤2.3、采用局部特征点离散方法来减少特征值,将中心线视为函数,所求特征点即是函数的一部分驻点,对所有的驻点进行优化,得到中心线特征点点集;Step 2.3. Use the local feature point discrete method to reduce the eigenvalue, regard the center line as a function, and the required feature point is a part of the stagnation point of the function, optimize all the stagnation points, and obtain the center line feature point set;

步骤2.4、采用基于形体质心的边缘检测方法,获取中心线特征点点集中每个特征点对应的外径大小并存储;Step 2.4, adopt the edge detection method based on the centroid of the shape, obtain and store the size of the outer diameter corresponding to each feature point in the centerline feature point set;

(3)相机标定及多相机矩阵转换(3) Camera calibration and multi-camera matrix conversion

步骤3.1、通过相机标定得到由旋转矩阵和平移矩阵组成的外参数以及相机的内参数,根据内外参数获得世界坐标系下管道坐标向相机窗口像素坐标转换的矩阵;Step 3.1. Obtain the external parameters composed of the rotation matrix and the translation matrix and the internal parameters of the camera through the camera calibration, and obtain the matrix of the transformation from the pipeline coordinates in the world coordinate system to the pixel coordinates of the camera window according to the internal and external parameters;

步骤3.2、根据相机外参信息进行相机坐标系转换,对后侧相机与前侧相机采用同一个标定板同时标定,求出后侧相机坐标系向前侧相机坐标系转换所需的旋转矩阵以及平移矩阵;Step 3.2. Convert the camera coordinate system according to the external parameter information of the camera, use the same calibration plate to calibrate the rear camera and the front camera at the same time, and obtain the rotation matrix required for the transformation of the rear camera coordinate system to the front camera coordinate system and translation matrix;

(4)管道中心线匹配和外径扩展(4) Pipe centerline matching and outer diameter expansion

步骤4.1、对前侧、后侧相机所得的中心线特征点分别进行视口变换恢复,透视投影变换恢复,视图变换恢复,模型变换恢复,得到前、后侧相机中心指向前,后侧相机成像平面的两组射线族;Step 4.1. Perform viewport transformation recovery, perspective projection transformation recovery, view transformation recovery, and model transformation recovery on the centerline feature points obtained by the front and rear cameras, respectively, to obtain the front and rear camera centers pointing forward, and the rear camera imaging Two sets of ray families of the plane;

步骤4.2、根据步骤3.2获得的旋转矩阵和平移矩阵将后侧相机中心指向后侧相机成像平面的射线族转化为在前侧相机坐标系下的射线族;Step 4.2, according to the rotation matrix and translation matrix obtained in step 3.2, convert the ray family with the center of the rear camera pointing to the imaging plane of the rear camera into the ray family in the front camera coordinate system;

步骤4.3、采用动态匹配法,根据步骤2.2得出的中心线端点,从一侧端点开始进行匹配,根据此动态方法依次匹配直到另一侧端点,得到管道中心线的空间位置,由一组折线段表示;Step 4.3. Use the dynamic matching method. According to the centerline endpoints obtained in step 2.2, start matching from one side endpoint. According to this dynamic method, match until the other side endpoints in turn to obtain the spatial position of the pipeline centerline. line segment representation;

步骤4.4、将每个视图下中心线特征点处的外径进行扩展,获取待测管线全部方位的外径信息;采用基于最小二乘法的曲线拟合方法,将特征点处外径大小视为函数值,特征点距离端点距离视为自变量,对这一系列点进行多项式曲线拟合,按照偏差平方和最小的原则选择拟合曲线,最终得到的函数即为距离端点不同距离下的外径长度;Step 4.4. Expand the outer diameter of the centerline feature point under each view to obtain the outer diameter information of all directions of the pipeline to be tested; adopt the curve fitting method based on the least squares method, and regard the outer diameter at the feature point as The function value, the distance between the feature point and the end point is regarded as an independent variable, a polynomial curve fitting is performed on this series of points, and the fitting curve is selected according to the principle of the smallest sum of deviation squares, and the final function obtained is the outer diameter at different distances from the end point. length;

根据待测管线中心线在前侧相机坐标系下的空间坐标和外径信息即可重建待测管线。The pipeline to be tested can be reconstructed according to the spatial coordinates and outer diameter information of the centerline of the pipeline to be tested in the front camera coordinate system.

进一步地,所述步骤2.4中所述基于形体质心的边缘检测方法如下:Further, the edge detection method based on the shape centroid described in the step 2.4 is as follows:

不考虑噪声情况下,图像信息的一维数学模型简单表示为:Without considering noise, the one-dimensional mathematical model of image information is simply expressed as:

f(x)=u(x)*g(x)f(x)=u(x)*g(x)

其中:u(x)为原始理想信号,f(x)为一维图像信息;g(x)为点扩散函数,一般近似为高斯函数:Among them: u(x) is the original ideal signal, f(x) is the one-dimensional image information; g(x) is the point spread function, which is generally approximated as a Gaussian function:

Figure BDA0002239037690000031
Figure BDA0002239037690000031

将u(x)分别设置为理想阶跃边缘,理想脉冲边缘,理想屋脊边缘,经过点扩散函数作用后,尖锐的边缘被平滑成模糊边缘,对导数进行对称性分析,得出边界的计算公式为:Set u(x) as ideal step edge, ideal impulse edge, and ideal roof edge respectively. After the point spread function, the sharp edge is smoothed into a fuzzy edge, and the symmetry analysis of the derivative is carried out to obtain the calculation formula of the boundary. for:

Figure BDA0002239037690000032
Figure BDA0002239037690000032

对公式进行离散化,fi为f(x)在xi处的采样值,xi处的微分值利用此处的前向差分和后向差分的均值代替如下:Discretize the formula, f i is the sampling value of f(x) at x i , and the differential value at x i is replaced by the mean of the forward difference and the backward difference here as follows:

Figure BDA0002239037690000033
Figure BDA0002239037690000033

可将公式改为:The formula can be changed to:

Figure BDA0002239037690000034
Figure BDA0002239037690000034

计算差分矩阵,使用行差分模板和列差分模板分别与管线灰度图片做卷积,记录所得的矩阵D1,D2;根据管线矩阵的统计特性选取差分阈值T;算选取计算区间,对矩阵D1,D2中的元素小于T者置0,两个矩阵的非零连续区间就是边缘过渡区间;利用离散化的公式计算边源点值,存入灰度图像边缘;根据中心线特征点点集,外径大小就是在距离变换中得到的像素数加上前面计算出来的边缘点值。Calculate the difference matrix, use the row difference template and the column difference template to convolve the grayscale images of the pipeline respectively, and record the resulting matrices D 1 , D 2 ; select the difference threshold T according to the statistical characteristics of the pipeline matrix; If the elements in D 1 , D 2 are smaller than T, set to 0, and the non-zero continuous interval of the two matrices is the edge transition interval; use the discretization formula to calculate the value of the edge source point and store it into the edge of the gray image; according to the center line feature point point Set, the outer diameter is the number of pixels obtained in the distance transformation plus the edge point value calculated earlier.

进一步地,所述步骤4.3中的动态匹配策略如下:Further, the dynamic matching strategy in the step 4.3 is as follows:

根据图像处理环节得出的端点信息,从一侧端点开始进行匹配,记录此端点所对应的射线,以此为基准射线;According to the endpoint information obtained from the image processing link, start matching from the endpoint on one side, record the ray corresponding to this endpoint, and use this as the reference ray;

根据双目成像原理,选择在前侧相机坐标系下前侧相机中心指向前侧相机成像平面投影点的基准射线和下条射线,组成一个平面,选择该坐标系下后侧相机中心指向后侧相机成像平面投影点的基准射线和下条射线组成另一个平面,两个平面求交线;交线与四条射线形成四个交点,根据此前记录的基准射线,选择这两射线与交线的交点为基准点,远离此交点的第一个点就是这次匹配获得的待测管线中心线特征点,将此点作为下次匹配时的基准点,此点对应的射线族下次匹配中向后选择一个射线;经过匹配,得到待测管线中心线的空间特征点,连接即可得到待测管线中心线According to the principle of binocular imaging, select the reference ray and the next ray with the center of the front camera pointing to the projection point of the imaging plane of the front camera in the front camera coordinate system to form a plane, and select the center of the rear camera to point to the rear side in this coordinate system The reference ray of the projection point of the camera imaging plane and the next ray form another plane, and the intersection of the two planes is obtained; the intersection and the four rays form four intersections, and the intersection of these two rays and the intersection is selected according to the previously recorded reference rays. As the reference point, the first point far from this intersection is the feature point of the pipeline center line to be tested obtained in this match, and this point is used as the reference point in the next match, and the ray family corresponding to this point is backward in the next match Select a ray; after matching, the spatial feature points of the pipeline center line to be tested are obtained, and the center line of the pipeline to be tested can be obtained by connecting

有益效果:本方法针对广泛应用的管路零件精加工与测量提出了一种基于路径变换的管路中心线提取方法以及基于管路中心线匹配的管路中心重建测量方法。同时,针对管路外径的测量提出了一种基于形心变换的亚像素级检测方法。本方法可以有效地解决当前管路测量应用中出现的测量流程复杂、测量成本高、测量精度低、管路模型重建困难的问题,获得管路中心线折线族以及对应的外径信息,可以完整的表示管路的全部信息。Beneficial effects: This method proposes a pipeline centerline extraction method based on path transformation and a pipeline centerline reconstruction measurement method based on pipeline centerline matching, which are widely used in the finishing and measurement of pipeline parts. At the same time, a sub-pixel detection method based on centroid transformation is proposed for the measurement of the outer diameter of the pipeline. The method can effectively solve the problems of complex measurement process, high measurement cost, low measurement accuracy, and difficult pipeline model reconstruction in the current pipeline measurement application, and obtains the pipeline centerline polyline family and the corresponding outer diameter information, which can be completely to indicate all the information of the pipeline.

附图说明Description of drawings

图1是本发明系统各结构关系示意图Fig. 1 is a schematic diagram of each structural relationship of the system of the present invention

图2是本发明系统框架搭建示意图;2 is a schematic diagram of the construction of the system framework of the present invention;

图3是待测管线到拍摄所得图片经过的转化过程示意图;Fig. 3 is a schematic diagram of the conversion process from the pipeline to be tested to the captured picture;

图4是透视投影原理示意图;Fig. 4 is the schematic diagram of perspective projection principle;

图5是本发明双目成像原理示意图;Fig. 5 is the schematic diagram of the binocular imaging principle of the present invention;

图6是本发明动态匹配策略流程图。FIG. 6 is a flow chart of the dynamic matching strategy of the present invention.

具体实施方式Detailed ways

下面结合附图对本发明作更进一步的说明。The present invention will be further described below in conjunction with the accompanying drawings.

本发明提供的基于多目视觉的管线测量方法,主要包括(1)图像采集和预处理;(2)中心线及外径提取;(3)相机标定以及相机矩阵转换;(4)管道中心线匹配和外径扩展。The multi-vision-based pipeline measurement method provided by the present invention mainly includes (1) image acquisition and preprocessing; (2) centerline and outer diameter extraction; (3) camera calibration and camera matrix conversion; (4) pipeline centerline Matching and OD expansion.

下面给出一种具体实施例A specific embodiment is given below

(1)图像采集和预处理(1) Image acquisition and preprocessing

首先搭建图像采集系统,使用长度在1.5m左右的钢架台作为图像采集系框架,在框架底部铺设发光板,用以消除高处光源照明所带来的阴影,简化图像处理中对管道背景的操作。如果采用桌面作为载物台,桌面上的纹理信息会带来很大困扰,铺设发光板的强光可以使管道图片的背景接近白色。First build an image acquisition system, use a steel frame with a length of about 1.5m as the frame of the image acquisition system, and lay a light-emitting board at the bottom of the frame to eliminate the shadow caused by the lighting of the high-level light source and simplify the pipeline background in image processing. operate. If the desktop is used as the stage, the texture information on the desktop will cause great trouble, and the strong light of laying the light-emitting board can make the background of the pipe picture close to white.

根据对理想折线段的三维重建模拟进行的精度分析,图像采集模块需要使用至少500万像素的摄像头。将摄像头放置于系统框架中,如图1-2所示,前后各放置一个相机,形成一对相机组。对放置在发光板上的待测量管线进行拍摄。相机使用python和opencv来进行图片抓取。因后续图像处理不需要颜色信息,因此需要对摄像头参数进行调节,目标是尽量获取接近灰度图的原始图片。According to the accuracy analysis of the 3D reconstruction simulation of the ideal polyline segment, the image acquisition module needs to use a camera with at least 5 million pixels. Place the cameras in the system frame, as shown in Figure 1-2, and place one camera in the front and the rear to form a pair of camera groups. Take a picture of the pipeline to be measured placed on the luminous plate. The camera uses python and opencv for image capture. Because the subsequent image processing does not require color information, the camera parameters need to be adjusted, and the goal is to obtain the original image as close to the grayscale image as possible.

将拍摄的管道图像转化为灰度图,对灰度图进行二值化处理,选择合适的阈值,使得背景和管线及一些其他边界能够区分出来。对二至图片进行进行ROI区域提取,采用轮廓比对技术,由于摄像头在放置时是对准管道的,所拍摄的图片中最大的轮廓就是管线的轮廓。运用opencv提取图片所包含的所有图形轮廓,消除小的轮廓,保留最大的轮廓,即完成ROI区域提取。Convert the captured pipeline image into a grayscale image, perform binarization processing on the grayscale image, and select an appropriate threshold to distinguish the background from the pipeline and some other boundaries. The ROI area is extracted for the second picture, and the contour comparison technology is used. Since the camera is aligned with the pipeline when it is placed, the largest contour in the captured picture is the contour of the pipeline. Use opencv to extract all graphic contours contained in the image, eliminate small contours, and retain the largest contour, that is, to complete the ROI area extraction.

(2)中心线及外径提取(2) Extraction of center line and outer diameter

距离变换是指计算物体内部每个点到物体边界的最短距离。管道的中心线就是距离管道边界最远的点的集合。距离变换有欧氏距离变换与非欧氏距离变换,非欧氏距离变换具有低复杂度与高效的特点,可以得到相似的结果,而欧氏距离变换有着较高复杂度,可以得到精确结果,本发明使用欧氏距离变换。通过欧式距离变换,可以得到一个标量场,记录管线轮廓内部每个点到管线边界的欧式距离。Distance transformation refers to calculating the shortest distance from each point inside the object to the object boundary. The centerline of a pipe is the set of points furthest from the pipe boundary. Distance transformation includes Euclidean distance transformation and non-Euclidean distance transformation. Non-Euclidean distance transformation has the characteristics of low complexity and high efficiency, and can obtain similar results, while Euclidean distance transformation has higher complexity and can obtain accurate results. The present invention uses the Euclidean distance transform. Through the Euclidean distance transformation, a scalar field can be obtained to record the Euclidean distance from each point inside the pipeline outline to the pipeline boundary.

由于管线中心线是管线内部距离边界最远的点的集合,得到管线内部每个点到边界的欧氏距离之后,中心线提取问题就变成了从管线一侧端点到另一侧端点的最小路径选择问题。通过图形学的方法得到管线两侧端点,具体操作是不断细化管道,对管线最外侧的轮廓进行识别并去除,直到管线只剩单像素,这是此单像素曲线的端点就是管线两侧的端点。需要说明的是此单像素曲线和管线中心线有一定误差,因为细化轮廓时是环绕轮廓操作,因此与真实中心线会有一到两个像素的误差。Since the pipeline centerline is the set of points farthest from the boundary inside the pipeline, after obtaining the Euclidean distance from each point inside the pipeline to the boundary, the centerline extraction problem becomes the minimum distance from the endpoint on one side of the pipeline to the endpoint on the other side. Path selection problem. The endpoints on both sides of the pipeline are obtained by graphical methods. The specific operation is to continuously refine the pipeline, identify and remove the outermost outline of the pipeline until only a single pixel remains in the pipeline. This is the endpoint of the single-pixel curve on both sides of the pipeline. endpoint. It should be noted that there is a certain error between this single-pixel curve and the pipeline center line, because the contour is refined around the contour operation, so there will be an error of one to two pixels with the real center line.

获取管道两侧端点的坐标后。选择一侧端点作为路径起点,要注意的是,由于两个相机是相对放置的,所拍摄的图片是对称的。因此若前侧相机选择一侧端点作为路径起点,那后侧相机就要选择另一侧端点。区分端点可以用像素横纵坐标相加值的大小来区分。选好起点之后另一侧端点就是终点。After getting the coordinates of the endpoints on both sides of the pipe. Select one end point as the starting point of the path. It should be noted that since the two cameras are placed opposite to each other, the captured images are symmetrical. Therefore, if the front camera selects one end point as the starting point of the path, the back side camera must select the other end point. The endpoints can be distinguished by the size of the sum of the horizontal and vertical coordinates of the pixels. After selecting the starting point, the other end point is the end point.

管道中心线就是起点和终点之间的最小路径。本发明采用Dijkstra最小路径算法。具体流程为:寻找起点开始,相邻的最短路径;选另一条路径,比较同样前往此节点的,更新开销最小的,并更新路径;重复此步骤,直到最后一个节点;计算最终路径。The pipe centerline is the minimum path between the start and end points. The present invention adopts Dijkstra minimum path algorithm. The specific process is: find the starting point and the adjacent shortest path; choose another path, compare the ones that also go to this node, update the path with the least cost, and update the path; repeat this step until the last node; calculate the final path.

获取中心线之后,进行特征点的离散。将中心线视为函数,中心线的特征点就是函数的某些驻点,驻点筛选策略是:计算某特征点到曲线两端点的距离,与该点和两个端点练成的两条线距离之和进行比对,达到某种程度便舍弃,否则保留。得到离散化的中心线特征点点集。After the center line is obtained, the feature points are discretized. The center line is regarded as a function, and the feature points of the center line are some stagnation points of the function. The stagnation point screening strategy is to calculate the distance from a feature point to the two ends of the curve, and the two lines formed by this point and the two endpoints The sum of the distances is compared, and if it reaches a certain level, it will be discarded, otherwise it will be retained. Get the discretized centerline feature point set.

其中,针对管线外径,本发明采用一种基于形体质心的边缘检测方法。对阶跃型边缘、脉冲型边缘、屋脊型边缘3种基本类型边缘的特点进行详细分析后发现,这3种基本边缘类型的一阶导数是关于边缘点的对称函数,它们的绝对值则是一个关于边缘点的偶对称函数.而从图形的角度考虑,此对称点恰是图形的质心.因此亚像素级边缘检测问题就可以转化为求解图形质心问题。一般认为图像是原始理想信息在点扩散函数的卷积作用下形成的.在不考虑噪声影响的情况下,其一维数学模型简单表示为:Among them, for the outer diameter of the pipeline, the present invention adopts an edge detection method based on the centroid of the shape. After a detailed analysis of the characteristics of the three basic types of edges: the step edge, the impulse edge and the ridge edge, it is found that the first-order derivatives of these three basic edge types are symmetric functions about the edge points, and their absolute values are An even symmetric function about the edge point. From the point of view of the graph, this symmetric point is just the centroid of the graph. Therefore, the sub-pixel edge detection problem can be transformed into the problem of solving the graph centroid. It is generally believed that the image is formed by the original ideal information under the convolution of the point spread function. Without considering the influence of noise, its one-dimensional mathematical model is simply expressed as:

f(x)=u(x)*g(x)f(x)=u(x)*g(x)

其中:u(x)为原始理想信号,f(x)为一维图像信息;g(x)为点扩散函数,一般近似为高斯函数:Among them: u(x) is the original ideal signal, f(x) is the one-dimensional image information; g(x) is the point spread function, which is generally approximated as a Gaussian function:

Figure BDA0002239037690000061
Figure BDA0002239037690000061

将u(x)分别设置为理想阶跃边缘,理想脉冲边缘,理想屋脊边缘,经过点扩散函数作用后,尖锐的边缘呗平滑成模糊边缘,经过对导数的对称性分析,得出边界的计算公式为:Set u(x) as the ideal step edge, ideal pulse edge, and ideal roof edge respectively. After the point spread function, the sharp edge is smoothed into a fuzzy edge. After the symmetry analysis of the derivative, the calculation of the boundary is obtained. The formula is:

Figure BDA0002239037690000062
Figure BDA0002239037690000062

为了便于计算机实现此算法,需要对公式进行离散化,设fi为f(x)在xi处的采样值,xi处的微分值利用此处的前向差分和后向差分的均值代替,记为:In order to facilitate the computer to implement this algorithm, it is necessary to discretize the formula. Let f i be the sampling value of f(x) at x i , and the differential value at x i is replaced by the mean value of the forward difference and the backward difference here. , denoted as:

Figure BDA0002239037690000071
Figure BDA0002239037690000071

可将公式改为:The formula can be changed to:

Figure BDA0002239037690000072
Figure BDA0002239037690000072

实现这个算法的具体过程是:计算差分矩阵,使用行差分模板和列差分模板分别与管线灰度图片做卷积,记录所得的矩阵D1,D2;根据管线矩阵的统计特性选取差分阈值T;算选取计算区间,对矩阵D1,D2中的元素小于T者置0,两个矩阵的非零连续区间就是边缘过渡区间;利用离散化的公式计算边源点值,存入灰度图像边缘;根据中心线特征点点集,外径大小就是在距离变换中得到的像素数加上前面计算出来的边缘点值。The specific process of realizing this algorithm is: calculate the difference matrix, use the row difference template and the column difference template to convolve with the pipeline grayscale image, and record the resulting matrices D 1 , D 2 ; select the difference threshold T according to the statistical characteristics of the pipeline matrix ;Calculate and select the calculation interval, set 0 for the elements in the matrix D 1 , D 2 less than T, the non-zero continuous interval of the two matrices is the edge transition interval; use the discretization formula to calculate the value of the edge source point and store it in the grayscale Image edge; according to the centerline feature point set, the outer diameter is the number of pixels obtained in the distance transformation plus the edge point value calculated earlier.

(3)相机标定及多相机矩阵转换(3) Camera calibration and multi-camera matrix conversion

对两个相机进行标定。相机标定是计算机视觉的基本问题之一,是由二维图像获取三维信息的前提。标定是利用待标定相机所拍摄的二维图像中的已知信息,求取相机成像模型中的所有未知参数,包括表示相机内部结构的内部参数和相机空间位姿的外部参数。Calibrate both cameras. Camera calibration is one of the basic problems of computer vision, and it is the premise of obtaining three-dimensional information from two-dimensional images. Calibration is to use the known information in the two-dimensional image captured by the camera to be calibrated to obtain all the unknown parameters in the camera imaging model, including the internal parameters representing the internal structure of the camera and the external parameters of the camera's spatial pose.

相机标定方法分为以下四类:①利用三维标定物体的标定方法;②利用二维标定物体的标定方法;③利用一维标定物的标定方法;④自标定方法。利用二维标定物的标定方法精度高、操作简便,逐渐成为实际应用中普遍使用的标定方法。相机标定的过程即通过解投影方程优化求解相机内、外参数的过程。基于二维平面标定方法的思想,在多目相机分组后,双目立体相机同时进行标定。Camera calibration methods are divided into the following four categories: ① calibration methods using three-dimensional calibration objects; ② calibration methods using two-dimensional calibration objects; ③ calibration methods using one-dimensional calibration objects; ④ self-calibration methods. The calibration method using two-dimensional calibration objects has high precision and simple operation, and has gradually become a commonly used calibration method in practical applications. The process of camera calibration is to optimize the process of solving the internal and external parameters of the camera by solving the projection equation. Based on the idea of the two-dimensional plane calibration method, after the multi-cameras are grouped, the binocular stereo cameras are calibrated at the same time.

二维平面标定的方法需要使用标定板,标定板的形式和制作质量将影响相机标定精度,标定板主要有棋盘格标定板和圆形标定板,本发明采用棋盘格标定板。棋盘格标定板将网格直角点作为标定点,采用python进行图片抓取,两个相机各抓取15-20张图片,尽量使棋盘格覆盖尽可能大的面积,每张图片棋盘格都要有不同的角度。图像处理包的标定工具,比如Matlab内置的标定工具,进行相机标定,可以得到如下参数:相机的旋转矩阵,平移矩阵,内参矩阵。The two-dimensional plane calibration method requires the use of a calibration plate. The form and production quality of the calibration plate will affect the calibration accuracy of the camera. The calibration plate mainly includes a checkerboard calibration plate and a circular calibration plate. The present invention uses a checkerboard calibration plate. The checkerboard calibration board uses the grid right-angle points as the calibration points, and uses python to capture pictures. The two cameras each capture 15-20 pictures, and try to make the checkerboard cover as large an area as possible. There are different angles. The calibration tool of the image processing package, such as the built-in calibration tool of Matlab, to calibrate the camera, the following parameters can be obtained: the rotation matrix of the camera, the translation matrix, and the internal parameter matrix.

根据相机外参信息进行相机之间的矩阵转换,要对两个相机使用同一个标定板同时标定,这样两个相机标定时就有相同的世界坐标系。对标定盘上的某个点P(世界坐标系下)来说,它经过后侧相机的外参矩阵转化为后侧相机坐标系下的Pt,经过前侧相机的外参矩阵转化为前侧相机坐标系下的Pr。Pt,Pr可以通过一个旋转矩阵R,平移矩阵T来进行转换:The matrix conversion between cameras is performed according to the camera external parameter information, and the same calibration board is used to calibrate the two cameras at the same time, so that the two cameras have the same world coordinate system when they are calibrated. For a certain point P on the calibration disk (under the world coordinate system), it is transformed into P t in the rear camera coordinate system through the external parameter matrix of the rear camera, and is transformed into the front camera through the external parameter matrix of the front camera. Pr in the side camera coordinate system. P t , P r can be transformed by a rotation matrix R and a translation matrix T:

Pr=RPt+TP r =RP t +T

其中旋转矩阵表达式为:The rotation matrix expression is:

Figure BDA0002239037690000081
Figure BDA0002239037690000081

Rr为前侧相机单独标定时得到的旋转矩阵,

Figure BDA0002239037690000082
为后侧相机单独标定时得到的旋转矩阵求逆R r is the rotation matrix obtained when the front camera is independently calibrated,
Figure BDA0002239037690000082
Invert the rotation matrix obtained when the rear camera is individually calibrated

平移矩阵表达式为:The translation matrix expression is:

T=Tr-RTt T =Tr-RT t

Tr为前侧相机单独标定的平移矩阵,Tt为后侧相机单独标定时的平移矩阵。T r is the translation matrix when the front camera is individually calibrated, and T t is the translation matrix when the rear camera is independently calibrated.

(4)管道中心线匹配和外径扩展(4) Pipe centerline matching and outer diameter expansion

经过图像采集和预处理,中心线提取及离散化,得到了相机视图下的特征点像素位置信息。经过相机标定,获得了世界坐标系向相机坐标系转换的矩阵参数以及透视投影矩阵信息。根据相机参数,获得成像平面一点到图片像素点的转化关系。如图3所示,对于待测管线上的每个点,经历了图3所示的变换,成为相机成像图片中的一个像素点。本模块的目的就是利用立体成像的原理由像素点恢复世界坐标系中的点的位置信息。After image acquisition and preprocessing, centerline extraction and discretization, the pixel position information of feature points under the camera view is obtained. After camera calibration, the matrix parameters and perspective projection matrix information of the transformation from the world coordinate system to the camera coordinate system are obtained. According to the camera parameters, the conversion relationship from the imaging plane point to the picture pixel point is obtained. As shown in Figure 3, each point on the pipeline to be tested undergoes the transformation shown in Figure 3 to become a pixel in the image of the camera. The purpose of this module is to use the principle of stereo imaging to restore the position information of points in the world coordinate system from pixel points.

对后侧相机所得管线图片的特征点点集中的每个点,进行反向视图变换,根据图片大小和起始位置,将像素信息还原为相机坐标系下相机成像面上的二维点坐标。由于透视投影时舍弃了深度信息,如图4所示,在透过相机中心eye和成像面上点P的直线上任意一点(例如点M,点N)都会投影到成像平面上的P点,之后经过视图变换就会得到相同的一个像素点。因此还原透视投影无法恢复原来的点的信息,只能由相机中心eye指向成像面上点P的一条射线,记为Li。以后侧相机坐标系中两点(eye,P)表示。Perform reverse view transformation for each point in the feature point set of the pipeline picture obtained by the rear camera, and restore the pixel information to the two-dimensional point coordinates on the camera imaging surface under the camera coordinate system according to the picture size and starting position. Since the depth information is discarded in perspective projection, as shown in Figure 4, any point (such as point M, point N) on the line passing through the center eye of the camera and point P on the imaging plane will be projected to point P on the imaging plane, After the view transformation, the same pixel will be obtained. Therefore, restoring the perspective projection cannot restore the original point information, and can only point to a ray of point P on the imaging surface from the center eye of the camera, denoted as L i . It is represented by two points (eye, P) in the rear camera coordinate system.

对前侧相机的每个特征点同样可以得到由相机中心指向成像面上一点(反向视图变换得到的点)的射线,但是这条射线是在前侧相机坐标系下表示的,要运用相机标定模块得到的相机坐标系转化关系,将上步骤得到的后侧相机坐标系下的射线,转化为前侧相机坐标系下的一条射线。将此操作应用于两个相机拍摄图片所得中心线上的所有特征点,得到在前侧相机坐标系下的两组射线,分别是前侧相机中心指向前侧相机成像平面投影点的射线族和后侧相机中心指向后侧相机成像平面投影点的射线族。For each feature point of the front camera, a ray pointing from the center of the camera to a point on the imaging surface (the point obtained by the reverse view transformation) can also be obtained, but this ray is represented in the coordinate system of the front camera. The camera coordinate system conversion relationship obtained by the calibration module converts the ray in the rear camera coordinate system obtained in the previous step into a ray in the front camera coordinate system. This operation is applied to all the feature points on the center line of the images captured by the two cameras, and two sets of rays in the front camera coordinate system are obtained, which are the ray family of the front camera center pointing to the projection point of the front camera imaging plane and The ray family with the center of the rear camera pointing to the projection point of the imaging plane of the rear camera.

利用双目成像原理,如图5所示,两条射线可以确定一个平面,管线中心线就位于两个平面的交线中。具体来说是四条射线所与该交线产生的四个交点中某两点之间部分。确定是哪两点的方案与匹配策略有关。采用一种动态匹配的方法,根据图像处理环节得出的端点信息,从一侧端点开始进行匹配,记录此端点所对应的射线,以此为基准射线。匹配方法是根据双目成像原理,选择在前侧相机坐标系下前侧相机中心指向前侧相机成像平面投影点的基准射线和下条射线,组成一个平面,选择该坐标系下后侧相机中心指向后侧相机成像平面投影点的基准射线和下条射线组成另一个平面,两个平面求交线。交线与四条射线形成四个交点,根据此前记录的基准射线,选择这两射线与交线的交点为基准点,远离此交点的第一个点就是这次匹配获得的待测管线中心线特征点。将此点作为下次匹配时的基准点,此点对应的射线族下次匹配中向后选择一个射线,如图6所示。经过匹配,得到待测管线中心线的空间特征点,连接即可得到待测管线中心线。Using the principle of binocular imaging, as shown in Figure 5, two rays can determine a plane, and the centerline of the pipeline is located in the intersection of the two planes. Specifically, it is the part between two points among the four intersection points generated by the four rays and the intersection line. The solution to determining which two points are related to the matching strategy. A dynamic matching method is adopted. According to the endpoint information obtained from the image processing, the matching is performed from one endpoint, and the ray corresponding to this endpoint is recorded, which is used as the reference ray. The matching method is based on the principle of binocular imaging, select the reference ray and the next ray that point to the projection point of the front camera imaging plane from the front camera center in the front camera coordinate system to form a plane, and select the rear camera center in this coordinate system. The reference ray and the next ray pointing to the projection point of the imaging plane of the rear camera form another plane, and the two planes are intersected. The intersection line and the four rays form four intersection points. According to the previously recorded reference rays, the intersection point of these two rays and the intersection line is selected as the reference point. The first point far from this intersection point is the centerline feature of the pipeline to be tested obtained by this match. point. This point is used as the reference point for the next matching, and a ray is selected backward in the next matching of the ray family corresponding to this point, as shown in Figure 6. After matching, the spatial feature points of the center line of the pipeline to be tested are obtained, and the center line of the pipeline to be tested can be obtained by connecting.

将每个视图下中心线特征点处的外径进行扩展,得到待测管线全部方位的外径信息。采用基于最小二乘法的曲线拟合方法。将特征点处外径大小视为函数值,特征点距离端点距离视为自变量,对这一系列点进行多项式曲线拟合。按照偏差平方和最小的原则选择拟合曲线,最终得到的函数就代表了距离端点不同距离下的外径长度。Expand the outer diameter of the centerline feature point under each view to obtain the outer diameter information of the pipeline to be tested in all directions. A curve fitting method based on the least squares method was used. The outer diameter at the feature point is regarded as a function value, and the distance from the feature point to the end point is regarded as an independent variable, and a polynomial curve fitting is performed on this series of points. The fitting curve is selected according to the principle of the smallest sum of deviation squares, and the final function represents the length of the outer diameter at different distances from the end point.

根据待测管线中心线空间坐标(前侧相机坐标系下)和外径信息即可重建待测管线。The pipeline to be tested can be reconstructed according to the spatial coordinates of the centerline of the pipeline to be tested (under the coordinate system of the front camera) and the outer diameter information.

以上所述仅是本发明的优选实施方式,应当指出:对于本技术领域的普通技术人员来说,在不脱离本发明原理的前提下,还可以做出若干改进和润饰,这些改进和润饰也应视为本发明的保护范围。The above is only the preferred embodiment of the present invention, it should be pointed out that: for those skilled in the art, without departing from the principle of the present invention, several improvements and modifications can also be made, and these improvements and modifications are also It should be regarded as the protection scope of the present invention.

Claims (2)

1.一种基于多目视觉的管线测量方法,其特征在于:包括以下步骤:1. a pipeline measurement method based on multi-eye vision, is characterized in that: comprise the following steps: (1)图像采集和预处理(1) Image acquisition and preprocessing 步骤1、搭建图像采集系统,通过分布在管道前后两侧的双目相机进行图片抓取,获得拍摄的管道灰度图;对所述灰度图进行二值化处理,使背景与所述管道互相区分;采用轮廓对比技术,对二值化图片进行目标区域ROI提取,获取管道轮廓信息;Step 1. Build an image acquisition system, capture pictures through binocular cameras distributed on the front and rear sides of the pipeline, and obtain a grayscale image of the pipeline; binarize the grayscale image to make the background and the pipeline Distinguish each other; use contour comparison technology to extract the target area ROI from the binarized image to obtain pipeline contour information; (2)中心线及外径提取(2) Extraction of center line and outer diameter 步骤2.1、根据管道轮廓信息,采用欧式距离变换法计算管道内每点到管道轮廓的欧氏距离,获取管道的距离变换图,记录管道轮廓内部每个点到管道边界的欧式距离;Step 2.1. According to the pipeline outline information, use the Euclidean distance transformation method to calculate the Euclidean distance from each point in the pipeline to the pipeline outline, obtain the distance transformation map of the pipeline, and record the Euclidean distance from each point inside the pipeline outline to the pipeline boundary; 步骤2.2、通过图像处理搜索管道边界,获取管道两侧端点的像素坐标,采用Dijkstra最小路径算法,以中心线一侧端点为起点,另一侧端点为终点,得到两点之间的最小路径,即管道的中心线,记为管线;Step 2.2. Search the pipeline boundary through image processing, obtain the pixel coordinates of the endpoints on both sides of the pipeline, and use the Dijkstra minimum path algorithm to obtain the minimum path between the two points with the endpoint on one side of the centerline as the starting point and the endpoint on the other side as the endpoint. That is, the center line of the pipeline, recorded as the pipeline; 步骤2.3、采用局部特征点离散方法来减少特征值,将中心线视为函数,所求特征点即是函数的一部分驻点,对所有的驻点进行优化,得到中心线特征点点集;Step 2.3. Use the local feature point discrete method to reduce the eigenvalue, regard the center line as a function, and the required feature point is a part of the stagnation point of the function, optimize all the stagnation points, and obtain the center line feature point set; 步骤2.4、采用基于形体质心的边缘检测方法,获取中心线特征点点集中每个特征点对应的外径大小并存储;Step 2.4, adopt the edge detection method based on the centroid of the shape, obtain and store the size of the outer diameter corresponding to each feature point in the centerline feature point set; (3)相机标定及多相机矩阵转换(3) Camera calibration and multi-camera matrix conversion 步骤3.1、通过相机标定得到由旋转矩阵和平移矩阵组成的外参数以及相机的内参数,根据内外参数获得世界坐标系下管道坐标向相机窗口像素坐标转换的矩阵;Step 3.1. Obtain the external parameters composed of the rotation matrix and the translation matrix and the internal parameters of the camera through the camera calibration, and obtain the matrix of the transformation from the pipeline coordinates in the world coordinate system to the pixel coordinates of the camera window according to the internal and external parameters; 步骤3.2、根据相机外参信息进行相机坐标系转换,对后侧相机与前侧相机采用同一个标定板同时标定,求出后侧相机坐标系向前侧相机坐标系转换所需的旋转矩阵以及平移矩阵;Step 3.2. Convert the camera coordinate system according to the external parameter information of the camera, use the same calibration plate to calibrate the rear camera and the front camera at the same time, and obtain the rotation matrix required for the transformation of the rear camera coordinate system to the front camera coordinate system and translation matrix; (4)管道中心线匹配和外径扩展(4) Pipe centerline matching and outer diameter expansion 步骤4.1、对前侧、后侧相机所得的中心线特征点分别进行视口变换恢复,透视投影变换恢复,视图变换恢复,模型变换恢复,得到前、后侧相机中心指向前,后侧相机成像平面的两组射线族;Step 4.1. Perform viewport transformation recovery, perspective projection transformation recovery, view transformation recovery, and model transformation recovery on the centerline feature points obtained by the front and rear cameras, respectively, to obtain the front and rear camera centers pointing forward, and the rear camera imaging Two sets of ray families of the plane; 步骤4.2、根据步骤3.2获得的旋转矩阵和平移矩阵将后侧相机中心指向后侧相机成像平面的射线族转化为在前侧相机坐标系下的射线族;Step 4.2, according to the rotation matrix and translation matrix obtained in step 3.2, convert the ray family with the center of the rear camera pointing to the imaging plane of the rear camera into the ray family in the front camera coordinate system; 步骤4.3、采用动态匹配法,根据步骤2.2得出的中心线端点,从一侧端点开始进行匹配,根据此动态方法依次匹配直到另一侧端点,得到管道中心线的空间位置,由一组折线段表示;具体地,Step 4.3. Use the dynamic matching method. According to the center line endpoints obtained in step 2.2, start matching from one side endpoint. According to this dynamic method, match sequentially to the other side endpoint to obtain the spatial position of the pipeline center line. Line segment representation; specifically, 根据图像处理环节得出的端点信息,从一侧端点开始进行匹配,记录此端点所对应的射线,以此为基准射线;According to the endpoint information obtained from the image processing link, start matching from the endpoint on one side, record the ray corresponding to this endpoint, and use this as the reference ray; 根据双目成像原理,选择在前侧相机坐标系下前侧相机中心指向前侧相机成像平面投影点的基准射线和下条射线,组成一个平面,选择该坐标系下后侧相机中心指向后侧相机成像平面投影点的基准射线和下条射线组成另一个平面,两个平面求交线;交线与四条射线形成四个交点,根据此前记录的基准射线,选择这两射线与交线的交点为基准点,远离此交点的第一个点就是这次匹配获得的待测管道中心线特征点,将此点作为下次匹配时的基准点,此点对应的射线族下次匹配中向后选择一个射线;经过匹配,得到待测管道中心线的空间特征点,连接即可得到待测管道中心线;According to the principle of binocular imaging, select the reference ray and the next ray with the center of the front camera pointing to the projection point of the imaging plane of the front camera in the front camera coordinate system to form a plane, and select the center of the rear camera to point to the rear side in this coordinate system The reference ray of the projection point of the camera imaging plane and the next ray form another plane, and the intersection line of the two planes is obtained; the intersection line and the four rays form four intersection points, and the intersection point of these two rays and the intersection line is selected according to the previously recorded reference rays. As the reference point, the first point far from this intersection is the feature point of the pipeline centerline obtained by this match, and this point is used as the reference point for the next match, and the ray family corresponding to this point is backward in the next match Select a ray; after matching, the spatial feature points of the center line of the pipeline to be tested are obtained, and the center line of the pipeline to be tested can be obtained by connecting; 步骤4.4、将每个视图下中心线特征点处的外径进行扩展,获取待测管线全部方位的外径信息;采用基于最小二乘法的曲线拟合方法,将特征点处外径大小视为函数值,特征点距离端点距离视为自变量,对这一系列点进行多项式曲线拟合,按照偏差平方和最小的原则选择拟合曲线,最终得到的函数即为距离端点不同距离下的外径长度;Step 4.4. Expand the outer diameter of the centerline feature point under each view to obtain the outer diameter information of all directions of the pipeline to be tested; adopt the curve fitting method based on the least squares method, and regard the outer diameter at the feature point as The function value, the distance between the feature point and the end point is regarded as an independent variable, a polynomial curve fitting is performed on this series of points, and the fitting curve is selected according to the principle of the smallest sum of deviation squares, and the final function obtained is the outer diameter at different distances from the end point. length; 根据待测管道中心线在前侧相机坐标系下的空间坐标和外径信息即可重建待测管道。The pipeline to be tested can be reconstructed according to the spatial coordinates and outer diameter information of the centerline of the pipeline to be tested in the front camera coordinate system. 2.根据权利要求1所述的一种基于多目视觉的管线测量方法,其特征在于:所述步骤2.4中所述基于形体质心的边缘检测方法如下:2. a kind of pipeline measurement method based on multi-eye vision according to claim 1 is characterized in that: the edge detection method based on shape centroid described in described step 2.4 is as follows: 不考虑噪声情况下,图像信息的一维数学模型简单表示为:Without considering noise, the one-dimensional mathematical model of image information is simply expressed as: f(x)=u(x)*g(x)f(x)=u(x)*g(x) 其中:u(x)为原始理想信号,f(x)为一维图像信息;g(x)为点扩散函数,一般近似为高斯函数:Among them: u(x) is the original ideal signal, f(x) is the one-dimensional image information; g(x) is the point spread function, which is generally approximated as a Gaussian function:
Figure FDA0003474759230000021
Figure FDA0003474759230000021
将u(x)分别设置为理想阶跃边缘,理想脉冲边缘,理想屋脊边缘,经过点扩散函数作用后,尖锐的边缘被平滑成模糊边缘,对导数进行对称性分析,得出边界的计算公式为:Set u(x) as ideal step edge, ideal impulse edge, and ideal roof edge respectively. After the point spread function, the sharp edge is smoothed into a fuzzy edge, and the symmetry analysis of the derivative is carried out to obtain the calculation formula of the boundary. for:
Figure FDA0003474759230000031
Figure FDA0003474759230000031
对公式进行离散化,fi为f(x)在xi处的采样值,xi处的微分值利用此处的前向差分和后向差分的均值代替如下:Discretize the formula, f i is the sampling value of f(x) at x i , and the differential value at x i is replaced by the mean of the forward difference and the backward difference here as follows:
Figure FDA0003474759230000032
Figure FDA0003474759230000032
可将公式改为:The formula can be changed to:
Figure FDA0003474759230000033
Figure FDA0003474759230000033
计算差分矩阵,使用行差分模板和列差分模板分别与管线灰度图片做卷积,记录所得的矩阵D1,D2;根据管线矩阵的统计特性选取差分阈值T;算选取计算区间,对矩阵D1,D2中的元素小于T者置0,两个矩阵的非零连续区间就是边缘过渡区间;利用离散化的公式计算边源点值,存入灰度图像边缘;根据中心线特征点点集,外径大小就是在距离变换中得到的像素数加上前面计算出来的边缘点值。Calculate the difference matrix, use the row difference template and the column difference template to convolve the grayscale images of the pipeline respectively, and record the resulting matrices D 1 , D 2 ; select the difference threshold T according to the statistical characteristics of the pipeline matrix; If the elements in D 1 , D 2 are smaller than T, set to 0, and the non-zero continuous interval of the two matrices is the edge transition interval; use the discretization formula to calculate the value of the edge source point and store it into the edge of the gray image; according to the center line feature point point Set, the outer diameter is the number of pixels obtained in the distance transformation plus the edge point value calculated earlier.
CN201910993518.2A 2019-10-18 2019-10-18 Pipeline measuring method based on multi-view vision Active CN110766669B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910993518.2A CN110766669B (en) 2019-10-18 2019-10-18 Pipeline measuring method based on multi-view vision

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910993518.2A CN110766669B (en) 2019-10-18 2019-10-18 Pipeline measuring method based on multi-view vision

Publications (2)

Publication Number Publication Date
CN110766669A CN110766669A (en) 2020-02-07
CN110766669B true CN110766669B (en) 2022-06-21

Family

ID=69332505

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910993518.2A Active CN110766669B (en) 2019-10-18 2019-10-18 Pipeline measuring method based on multi-view vision

Country Status (1)

Country Link
CN (1) CN110766669B (en)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111666876B (en) * 2020-06-05 2023-06-09 阿波罗智联(北京)科技有限公司 Method and device for detecting obstacle, electronic equipment and road side equipment
CN111986257A (en) * 2020-07-16 2020-11-24 南京模拟技术研究所 A method and system for automatic calibration of bullet point recognition supporting variable distance
CN111882488B (en) * 2020-07-22 2022-07-19 浙江大学 Indoor pipeline position recording and projecting method
JP7539320B2 (en) 2021-01-15 2024-08-23 大阪瓦斯株式会社 Piping completion drawing creation system and piping completion drawing creation method
CN113436209B (en) * 2021-06-23 2023-11-17 南通大学 A new weld centerline extraction method based on layer-by-layer indentation strategy
CN114066859A (en) * 2021-11-18 2022-02-18 新拓三维技术(深圳)有限公司 Pipeline measuring method and device
CN114777658A (en) * 2022-03-31 2022-07-22 深圳禾思众成科技有限公司 Alignment detection method and alignment detection equipment for semiconductor device
CN115115602B (en) * 2022-05-31 2023-09-19 江苏濠汉信息技术有限公司 Algorithm for texture positioning in wire diameter measurement process
CN114897902B (en) * 2022-07-13 2022-11-11 深圳金正方科技股份有限公司 BWFRP pipeline online monitoring method and system based on multiple cameras
CN119043217B (en) * 2024-11-04 2025-01-24 中国特种设备检测研究院 Elevator guide rail straightness measuring device and method

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002071547A (en) * 2000-08-29 2002-03-08 Pal Giken:Kk On-line analyzer for image of particle in liquid
CN102135236A (en) * 2011-01-05 2011-07-27 北京航空航天大学 Automatic non-destructive testing method for internal wall of binocular vision pipeline
CN102410811A (en) * 2011-07-27 2012-04-11 北京理工大学 Method and system for measuring parameters of bent pipe
CN107563356A (en) * 2017-09-29 2018-01-09 西安因诺航空科技有限公司 A kind of unmanned plane inspection pipeline target analysis management method and system
CN109583377A (en) * 2018-11-30 2019-04-05 北京理工大学 A kind of control method, device and host computer that pipeline model is rebuild
CN109615654A (en) * 2019-01-09 2019-04-12 中国矿业大学(北京) Drainage pipeline inside corrosion depth and area measurement method based on binocular vision

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002071547A (en) * 2000-08-29 2002-03-08 Pal Giken:Kk On-line analyzer for image of particle in liquid
CN102135236A (en) * 2011-01-05 2011-07-27 北京航空航天大学 Automatic non-destructive testing method for internal wall of binocular vision pipeline
CN102410811A (en) * 2011-07-27 2012-04-11 北京理工大学 Method and system for measuring parameters of bent pipe
CN107563356A (en) * 2017-09-29 2018-01-09 西安因诺航空科技有限公司 A kind of unmanned plane inspection pipeline target analysis management method and system
CN109583377A (en) * 2018-11-30 2019-04-05 北京理工大学 A kind of control method, device and host computer that pipeline model is rebuild
CN109615654A (en) * 2019-01-09 2019-04-12 中国矿业大学(北京) Drainage pipeline inside corrosion depth and area measurement method based on binocular vision

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
通用亚像素边缘检测算法;盛遵冰;《上海交通大学学报》;20070630;第41卷(第6期);第911-915页 *

Also Published As

Publication number Publication date
CN110766669A (en) 2020-02-07

Similar Documents

Publication Publication Date Title
CN110766669B (en) Pipeline measuring method based on multi-view vision
CN103868460B (en) Binocular stereo vision method for automatic measurement based on parallax optimized algorithm
CN107945268B (en) A kind of high-precision three-dimensional method for reconstructing and system based on binary area-structure light
CN103971406B (en) Submarine target three-dimensional rebuilding method based on line-structured light
CN102938142B (en) Based on the indoor LiDAR missing data complementing method of Kinect
WO2020093436A1 (en) Three-dimensional reconstruction method for inner wall of pipe
CN103411553B (en) The quick calibrating method of multi-linear structured light vision sensors
CN104835158B (en) 3D Point Cloud Acquisition Method Based on Gray Code Structured Light and Epipolar Constraint
CN109544628B (en) Accurate reading identification system and method of a pointer meter
CN108335350A (en) The three-dimensional rebuilding method of binocular stereo vision
CN109470149B (en) A method and device for measuring the pose of a pipeline
CN106327532A (en) Three-dimensional registering method for single image
CN105180890A (en) Rock mass structural plane attitude measuring method integrating laser point cloud and digital image
CN104134188A (en) Three-dimensional visual information acquisition method based on two-dimensional and three-dimensional video camera fusion
CN105066909A (en) Hand-held multi-laser-stripe quick three-dimensional measuring method
CN110500954A (en) An Aircraft Pose Measurement Method Based on Circle Feature and P3P Algorithm
CN112037318A (en) Construction method and system of three-dimensional rock mass structure model and application of model
CN102510506A (en) Virtual and real occlusion handling method based on binocular image and range information
CN103822581B (en) A kind of irregularly shaped object volume measuring method based on compressed sensing
CN107218928A (en) A kind of complicated multi- piping branch system detection method
CN103196370A (en) Measuring method and measuring device of conduit connector space pose parameters
CN103308000B (en) Based on the curve object measuring method of binocular vision
CN116229017A (en) A Reconstruction Method of 3D Model of Pipe Fittings Based on Reverse Projection
CN105469389A (en) Grid ball target for visual sensor calibration and corresponding calibration method
CN105574812A (en) Multi-angle three-dimensional data registration method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant