WO2024032233A1 - 基于双目视觉的立体摄影测量方法 - Google Patents

基于双目视觉的立体摄影测量方法 Download PDF

Info

Publication number
WO2024032233A1
WO2024032233A1 PCT/CN2023/104336 CN2023104336W WO2024032233A1 WO 2024032233 A1 WO2024032233 A1 WO 2024032233A1 CN 2023104336 W CN2023104336 W CN 2023104336W WO 2024032233 A1 WO2024032233 A1 WO 2024032233A1
Authority
WO
WIPO (PCT)
Prior art keywords
cost
image
aggregation
pixel
matching
Prior art date
Application number
PCT/CN2023/104336
Other languages
English (en)
French (fr)
Inventor
索依娜
宁学斌
于复兴
王然
Original Assignee
华北理工大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华北理工大学 filed Critical 华北理工大学
Priority to US18/541,978 priority Critical patent/US11959744B1/en
Publication of WO2024032233A1 publication Critical patent/WO2024032233A1/zh

Links

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C3/00—Measuring distances in line of sight; Optical rangefinders
    • G01C3/10—Measuring distances in line of sight; Optical rangefinders using a parallactic triangle with variable angles and a base of fixed length in the observation station, e.g. in the instrument
    • G01C3/14—Measuring distances in line of sight; Optical rangefinders using a parallactic triangle with variable angles and a base of fixed length in the observation station, e.g. in the instrument with binocular observation at a single point, e.g. stereoscopic type
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
    • G01C11/04—Interpretation of pictures
    • G01C11/06—Interpretation of pictures by comparison of two or more pictures of the same area
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
    • G01C11/04—Interpretation of pictures
    • G01C11/30—Interpretation of pictures by triangulation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/149—Segmentation; Edge detection involving deformable models, e.g. active contour models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G06T7/593—Depth or shape recovery from multiple images from stereo images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G06T2207/10012—Stereo images
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Definitions

  • the invention relates to the field of stereo photogrammetry, specifically a stereo photogrammetry method based on binocular vision.
  • the method of obtaining the three-dimensional information of the measured object can be divided into passive visual measurement and active visual measurement.
  • the passive visual measurement method does not require special lighting projection devices. It only uses the camera to capture the image of the measured object, establish the relative position relationship between the measured object and the camera, and thereby obtain the three-dimensional information of the measured object.
  • the distance measurement method based on binocular vision can not only accurately measure the size of objects, but also can be applied in a variety of environments. Therefore, the research on size measurement of binocular vision has important research value and application value.
  • stereo matching algorithms are mainly used to perform stereo matching on images captured by binocular cameras to obtain depth maps, and then perform three-dimensional reconstruction to measure the size of objects.
  • some improved methods have emerged.
  • Hirschmüller combined the advantages and disadvantages of the global stereo matching algorithm and the local stereo matching algorithm and proposed the semi-global stereo matching algorithm SGM.
  • SGM the semi-global stereo matching algorithm
  • Humenberger proposed a method to calculate the cost using Census transform and Hamming distance to reduce the time complexity and memory consumption of the algorithm.
  • Wang Yunfeng and others combined the absolute difference cost (AD) with the Census cost to obtain more accurate matching accuracy.
  • the local stereo matching algorithm still has the problem of low matching accuracy, resulting in large errors in measuring object dimensions.
  • the present invention proposes a stereophotogrammetry method based on binocular vision for size measurement of the same object photographed by binocular photography equipment.
  • a stereophotogrammetry method based on binocular vision includes the following steps:
  • Step 1 Use a binocular camera to shoot the object, obtain the images taken by the left and right cameras, and use the internal and external parameters of the camera to correct the photos so that the pictures are corrected to be taken on the same plane;
  • Step 2 Continuously downsample the corrected photo images to obtain images of different sizes
  • Step 3 Stereo matching, perform cost matching and cost aggregation on images of different sizes in step 2.
  • Cost matching combines the image pixel cost, color cost and gradient cost to obtain the matching cost of the image; the absolute value of the RBG difference between the three color components of the image is averaged as the color cost, and the Sobel operator algorithm is used to obtain the image gradient information. , average the absolute values of image gradient differences as the gradient cost;
  • Cost aggregation uses the aggregation strategy of minimum spanning tree and scan line optimization to calculate and obtain the initial disparity map of each size image for the cost obtained by matching the costs of images of different sizes; and then obtain the optimal disparity map of the original size image based on the multi-size aggregation model. aggregation cost;
  • Step 4 Perform disparity calculation and disparity optimization on the obtained aggregation cost to obtain a disparity map
  • Step 5 Perform image segmentation on the corrected image to determine the edge pixels of the object to be measured
  • Step 6 Based on the triangulation method, calculate the depth of the edge pixels of the object to be measured, calculate the distance of each vertex from the camera, construct the three-dimensional coordinates of the vertex in the real world, and complete the object size measurement.
  • the fusion of image pixel cost, color cost and gradient cost is used as the matching cost, which enhances the matching accuracy of the contour pixels of the object to be measured.
  • the cost aggregation stage a combination of minimum spanning tree and scan line optimization aggregation is used, and multiple size images are used to obtain the aggregation cost for fusion.
  • the obtained disparity map is more accurate and the object size measurement accuracy is higher.
  • Figure 1 Overall flow chart of the measurement method of the present invention
  • Figure 2 Minimum spanning tree bottom-up and top-down cost aggregation
  • the stereophotogrammetry method based on binocular vision of the present invention uses a calibrated binocular camera to collect images and an image acquisition module to correct the image, and a stereoscopic module to match the pixel points of the corrected image.
  • the matching module the image segmentation module that performs image segmentation on the corrected image to obtain the object to be measured and uses the minimum quadrilateral frame to outline the object's outline, and the size measurement module that obtains the quadrilateral coordinates of the contour of the object to be measured based on the disparity map obtained by stereo matching and image segmentation to obtain the true size of the object.
  • the specific steps of this method are:
  • Image collection Use a binocular stereo vision camera that has been calibrated to capture the object and obtain the color images captured by the left and right cameras. According to the internal and external camera parameters obtained by camera calibration, the image is subjected to stereoscopic correction and epipolar alignment processing to obtain the corrected image.
  • the corrected photo image is down-sampled 4 times continuously.
  • the down-sampling operation reduces the image size and retains some effective information, and obtains 5 images of different sizes.
  • the binocular image is converted into a grayscale image, and a 9 ⁇ 7 matching window is established with the pixel to be matched on the grayscale image on the left as the center. Calculate the average value of the two spaced pixels in the four directions up, down, left, and right of the central pixel with the central pixel, and select the maximum and minimum values. Then each pixel in the window is compared with the center pixel, the maximum value and the minimum value, and finally the average cost of them is calculated as the image pixel cost; the binocular image obtains three-channel color information based on the different RGB three-color channels.
  • Obtain the image gradient information calculate the difference between the gradient value of the left image and the gradient of the right image, set the gradient threshold to 2, take the difference between the gradient difference greater than 2 as 2, and take the average absolute value of the difference as the gradient cost; combine the color cost and gradient
  • the costs are added with a weight of 0.11:0.89 to obtain the joint cost.
  • the image pixel cost and joint cost are fused using a normalized combination method to obtain the cost matching cost.
  • top-down polymerization is required.
  • the minimum spanning tree on the right side of the image is top-down aggregation.
  • node V3 we assume that V3 is the root node, and its parent node V4 is converted into its child node.
  • the aggregation cost of the V3 node is the same as the aggregation cost of V4.
  • the calculation method is the same, but the calculation of the aggregation cost of the V3 node requires adding the product of the aggregation value of the V4 node and its weight.
  • the aggregation value of the V4 node has already considered the influence of the V3 node, so the aggregation value of the V4 node needs to be subtracted first.
  • the aggregate value of the V3 node is multiplied by the node weight and added to the V3 node cost as the V3 cost.
  • the other nodes calculate the aggregation cost from top to bottom, and the result is the final minimum spanning tree cost.
  • the scanline optimization aggregation strategy is used for aggregation.
  • the scan line direction is shown in Figure 3.
  • the four scan line optimizations are independent of each other.
  • the costs are aggregated from the four directions of up, down, left and right. Finally, the optimization costs in the four directions are averaged as the aggregation cost. result.
  • Disparity calculation and optimization using the winner-take-all algorithm at the optimal aggregation cost, determine the disparity value corresponding to the minimum cost of each pixel, and determine the initial disparity map.
  • the disparity optimization of the disparity map uses consistency check, uniqueness detection and median filtering methods to obtain the optimal disparity map.
  • the distance of each vertex from the camera is calculated, that is, the three-dimensional coordinates of each vertex in the real world are obtained.
  • the real distance between the four vertices is calculated, thus achieving the measurement of the size of the object.
  • the actual size of the object can be conveniently measured, and it is widely used in the fields of industrial parts measurement and object measurement.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Multimedia (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Electromagnetism (AREA)
  • Image Processing (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Image Analysis (AREA)

Abstract

本发明公开一种基于双目视觉的立体摄影测量方法，包含以下步骤：图像采集、图像矫正、立体匹配，分别对矫正后不同尺寸的图像进行代价匹配和代价聚合，代价匹配，把图像像素代价、颜色代价和梯度代价相融合，得到图像的匹配代价；代价聚合，对不同尺寸图像的代价匹配所得代价采用最小生成树和扫描线优化的聚合策略，计算并获得每个尺寸图像的初始视差图；再根据多尺寸聚合模型，获取原始尺寸图像的最优聚合代价；对所得聚合代价进行视差计算与视差优化，获取视差图；对矫正图片进行图像分割，确定待测量物体边缘像素点；计算待测物体边缘像素距离，完成物体尺寸测量。本发明方法增强了待测物体轮廓像素的匹配精度，提高了测量精度。

Description

基于双目视觉的立体摄影测量方法 技术领域
本发明涉及立体摄影测量领域，具体是一种基于双目视觉的立体摄影测量方法。
 背景技术
在工业生产过程中，对产品参数的检测、记录和控制是十分重要的。人工检测其测量的周期长且浪费人力，对于一些不易测量的产品还可能存在人身安全的潜在危险。
根据获取被测物体三维信息的方法可以分为被动视觉测量和主动视觉测量。被动视觉测量方法不需要特殊的照明投射装置，仅利用相机拍拍摄被测物体的图像，建立被测物体与相机间的相对位置关系，从而获取被测物体的三维信息。基于双目视觉的测距方法不仅能够对物体的尺寸进行精度测量，而且可以应用于各种各样的环境中，因此对双目视觉的尺寸测量研究具有重要的研究价值和应用价值。
目前在立体摄影测量领域，主要采用立体匹配算法对双目相机拍摄图像进行立体匹配获取深度图，然后进行三维重建，测量物体的尺寸。针对测量步骤中重要的立体匹配算法，出现了一些改进方法，如Hirschmüller综合了全局立体匹配算法和局部立体匹配算法的优缺点，提出了半全局立体匹配算法SGM。Humenberger在SGM的基础上，提出了一种使用Census变换和汉明距离计算代价的方法，以降低算法的时间复杂度和内存消耗。王云峰等针对单一匹配代价无法获得高精度视差图问题，将绝对差代价(AD)与Census代价相结合，以获得更准确的匹配精度。然而，局部立体匹配算法仍然存在匹配精度低的问题，导致测量物体尺寸的误差较大。
 技术问题
为解决背景技术中涉及的问题，本发明提出一种基于双目视觉的立体摄影测量方法用于双目摄影设备拍摄的同一物体的尺寸测量。
 技术解决方案
一种基于双目视觉的立体摄影测量方法包括以下步骤：
1.基于双目视觉的立体摄影测量方法，其特征在于，包含以下步骤：
步骤一：采用双目相机对物体进行拍摄，获取左、右两个相机拍摄的图像，并利用相机内外参数对照片进行矫正，使图片矫正为在同一平面拍摄所得；
步骤二：将矫正后的照片图像进行连续下采样，获得不同尺寸的图像；
步骤三：立体匹配，分别对步骤二中不同尺寸的图像进行代价匹配和代价聚合，
代价匹配，把图像像素代价、颜色代价和梯度代价相融合，得到图像的匹配代价；利用图像的三个颜色分量RBG差的绝对值取平均作为颜色代价，同时利用Sobel算子算法获取图像梯度信息，将图像梯度差的绝对值取平均作为梯度代价；
代价聚合，对不同尺寸图像的代价匹配所得代价采用最小生成树和扫描线优化的聚合策略，计算并获得每个尺寸图像的初始视差图；再根据多尺寸聚合模型，获取原始尺寸图像的最优聚合代价；
步骤四：对所得聚合代价进行视差计算与视差优化，获取视差图；
步骤五：对矫正图片进行图像分割，确定待测量物体边缘像素点；
步骤六：根据三角测量法，计算出待测物体边缘像素点深度，计算出各顶点距离相机的距离，构建顶点现实世界的三维坐标，完成物体尺寸测量。
 有益效果
代价计算阶段，使用图像像素代价与颜色代价、梯度代价相融合作为匹配代价，增强了待测物体轮廓像素的匹配精度。
代价聚合阶段，使用最小生成树和扫描线优化聚合相结合，且使用多个尺寸图像获取聚合代价进行融合，获取的视差图更精准，令物体尺寸测量精度更高。
 附图说明
图1：本发明测量方法整体流程图；
图2：最小生成树自底向上和自顶向下代价聚合；
图3：扫描线优化聚合方向；
图4：三角测量原理图。
 本发明的最佳实施方式
如图1所示，本发明的基于双目视觉的立体摄影测量方法，采用经过标定后的双目相机对图像进行采集和对图像矫正的图像采集模块、对矫正后图像像素点进行匹配的立体匹配模块、对矫正图像进行图像分割获取待测物体并使用最小四边形框出物体轮廓的图像分割模块以及根据立体匹配所得视差图和图像分割获得待测物体轮廓四边形坐标获取物体真实尺寸的尺寸测量模块。该方法具体步骤为：
S1、图像采集；使用进行过相机标定后的双目立体视觉相机对物体进行拍摄，获取左、右相机拍摄的彩色图像。根据相机标定所得的相机内外参数，对图像进行立体矫正和极线对齐处理，得到矫正后图片。
S2、将矫正后的照片图像进行连续4次下采样，下采样操作缩小了图片尺寸，并保留了部分有效信息，获得5张不同尺寸的图像 。
S3、立体匹配
S3.1、代价计算；双目图像转换为灰度图像，以左图的灰度图像上的待匹配像素点为中心建立9×7的匹配窗口。将中心像素点上下左右4个方向的2间隔像素点分别与中心像素点求平均值，并选出最大值和最小值。然后将窗口内各像素点分别与中心像素点、最大值和最小值作比较，最后计算它们的代价均值作为图像像素代价；双目图像根据RGB三色通道不同，得到三通道颜色信息。将左图RGB值分别与右图RGB求差，设置颜色阈值为7将颜色之差大于7的差值取为7，取差的平均绝对值作为颜色代价；分别对左右图使用Sobel算子算法获取图像梯度信息，将左图梯度值分别与右图梯度求差，设置梯度阈值为2将梯度差大于2的差值取为2，取差的平均绝对值作为梯度代价；将颜色代价和梯度代价以0.11：0.89的权重进行相加，得到联合代价。将图像像素代价和联合代价，采用归一化结合的方法进行融合，得到代价匹配代价。
S3.2、代价聚合；根据最小生成树原理，把图像视为一个四联通区域，图像两点构成的边的权值为像素灰度值之差，其值代表相邻像素的相似度来构建最小生成树。针对图像生成的最小生成树，其聚合方法如图2所示，图像左侧最小生成树为自底向上聚合，以节点V4为例，计算V4节点的聚合代价值可以直接计算子节点（V3,V4）和（V5，V4）的聚合值与他们各自权值乘积的集合，从最小生成树底部一层层计算直到根节点。经过自底向上聚合后，需要进行自顶向下聚合。图像右侧最小生成树为自顶向下聚合，以节点V3为例，此时我们假设V3为根节点，它的父节点V4转换成了它的子节点，V3节点的聚合代价与V4聚合代价计算方式相同，但V3节点的聚合代价计算需要加上V4节点的聚合值与它的权值乘积，而V4节点的聚合值已经考虑了V3节点的影响，因此需要先将V4节点聚合值减去V3节点的聚合值，将差值与节点权值相乘并与V3节点代价相加作为V3代价。其他节点自顶向下依次计算聚合代价，结果为最终最小生成树代价。完成最小生成树聚合后，为了进一步缓解匹配歧义，采用扫描线优化聚合策略进行聚合。扫描线方向如图3所示，四个扫描线优化是相互独立的，分别从上、下、左和右四个方向对代价进行聚合，最后将四个方向的优化代价取平均值作为聚合代价结果。
S4、获取视差图；使用5张不同尺寸图像分别进行S3.1和S3.2步骤，获取每个尺寸图像的视差图。再根据多尺寸聚合模型，获取原始尺寸图像的最优聚合代价；
视差计算与优化；在最优聚合代价采用赢者通吃算法，确定每个像素最小代价所对应视差值，确定初始视差图。对视差图进行视差优化采用一致性检查、唯一性检测及中值滤波方法，得到最优视差图。
S5、图像分割，应用Segment Anything算法的API实现对图像的分割，由mask_generator.generate()函数获取图像的所有蒙版；使用鼠标对图像待测物测进行点选，根据点选坐标点对蒙版进行选取，得到待测物体蒙版。对待测物体蒙版区域进行最小四边形拟合，得到最小四边形顶点坐标。
S6、尺寸测量；根据三角测量法，计算出待测物体边缘像素点深度。原理如图4所示，点P为待测物体上的某一点，O l和O r分别为两个相机的光心，点P在两个相机感光器上的成像点分别为X l和X r，X l和X r分别为成像点X l和X r到各自平面左边缘的距离，f为相机焦距，B为两相机中心距，Z为欲求得的深度信息，经过矫正后两个相机像平面精准位于同一平面上，利用三角形关系，得到像素点距离相机的距离 。通过S5中所得待测物体的最小四边形顶点坐标，计算出各顶点距离相机的距离，即得出了各顶点现实世界的三维坐标。使用欧式方程，计算出四个顶点间的真实距离，至此实现了对物体尺寸的测量。
工业实用性
通过上述测量方法可以方便的测量物体的实际尺寸，在工业零件测量，物体测量领域有着广泛的应用。

Claims (5)

  1. 基于双目视觉的立体摄影测量方法，其特征在于，包含以下步骤：
    步骤一：采用双目相机对物体进行拍摄，获取左、右两个相机拍摄的图像，并利用相机内外参数对照片进行矫正，使图片矫正为在同一平面拍摄所得；
    步骤二：将矫正后的照片图像进行连续下采样，获得不同尺寸的图像；
    步骤三：立体匹配，分别对步骤二中不同尺寸的图像进行代价匹配和代价聚合；
    代价匹配，把图像像素代价、颜色代价和梯度代价相融合，得到图像的匹配代价；利用图像的三个颜色分量RBG差的绝对值取平均作为颜色代价，同时利用Sobel算子算法获取图像梯度信息，将图像梯度差的绝对值取平均作为梯度代价；代价计算；双目图像转换为灰度图像，以左图的灰度图像上的待匹配像素点为中心建立9×7的匹配窗口，将中心像素点上下左右4个方向的2间隔像素点分别与中心像素点求平均值，并选出最大值和最小值，然后将窗口内各像素点分别与中心像素点、最大值和最小值作比较，最后计算它们的代价均值作为图像像素代价；双目图像根据RGB三色通道不同，得到三通道颜色信息，将左图RGB值分别与右图RGB求差，设置颜色阈值为7将颜色之差大于7的差值取为7，取差的平均绝对值作为颜色代价；分别对左右图使用Sobel算子算法获取图像梯度信息，将左图梯度值分别与右图梯度求差，设置梯度阈值为2将梯度差大于2的差值取为2，取差的平均绝对值作为梯度代价；将颜色代价和梯度代价以0.11：0.89的权重进行相加，得到联合代价，将图像像素代价和联合代价，采用归一化结合的方法进行融合，得到代价匹配代价；图像像素代价和联合代价的融合权值分别为15和35，融合公式为C(p,d)=2-exp(-C AG(p,d)/35)-exp(-C Census(p,d)/15),式中C AG为颜色代价和梯度代价的联合代价 ,C Census为像素代价；
    代价聚合，对不同尺寸图像的代价匹配所得代价采用最小生成树和扫描线优化的聚合策略，计算并获得每个尺寸图像的初始视差图；再根据多尺寸聚合模型，获取原始尺寸图像的最优聚合代价；
    步骤四：对所得聚合代价进行视差计算与视差优化，获取视差图；
    步骤五：对矫正图片进行图像分割，确定待测量物体边缘像素点；
    步骤六：根据三角测量法，计算出待测物体边缘像素点深度，计算出各顶点距离相机的距离，构建顶点现实世界的三维坐标，完成物体尺寸测量，各顶点距离相机的距离计算步骤：点P为待测物体上的某一点，O l和O r分别为两个相机的光心，点P在两个相机感光器上的成像点分别为X l和X r，X l和X r分别为成像点X l和X r到各自平面左边缘的距离，f为相机焦距，B为两相机中心距，Z为欲求得的深度信息，经过矫正后两个相机像平面精准位于同一平面上，利用三角形关系，得到像素点距离相机的距离 ；通过S5中所得待测物体的最小四边形顶点坐标，计算出各顶点距离相机的距离，即得出了各顶点现实世界的三维坐标，使用欧式方程，计算出四个顶点间的真实距离，至此实现了对物体尺寸的测量。
  2. 根据权利要求1所述的基于双目视觉的立体摄影测量方法，其特征在于，所述步骤二中将矫正后的照片图像进行连续4次下采样，下采样操作缩小了图片尺寸，并保留了部分有效信息，获得5张不同尺寸的图像。
  3. 根据权利要求1所述的基于双目视觉的立体摄影测量方法，其特征在于，所述步骤三中的代价聚合使用最小生成树把图像视为一个四联通区域的图，将两像素点形成的边的权值定义为像素点的灰度差值，通过自底向上和自顶向下方法遍历最小生成树，得到最小生成树聚合代价；然后使用扫描线优化聚合策略，依次从图像不同方向，利用前一像素点聚合代价确定当前像素点聚合代价，最后利用各尺寸图像的聚合代价获取原始尺寸图像的最优聚合代价。
  4. 根据权利要求1所述的基于双目视觉的立体摄影测量方法，其特征在于，所述步骤四中视差计算为选取每个像素的最小聚合代价所对应的视差值作为最终视差，视差优化采用一致性检查、唯一性检测及中值滤波方法，其中，一致性检查用于检查视差图中的不一致性像素，唯一性检测用于检测视差图中的不唯一性像素，而中值滤波用于平滑视差图中的噪声。
  5. 根据权利要求1所述的基于双目视觉的立体摄影测量方法，其特征在于，所述步骤五利用图像分割技术对矫正后的图像进行图像分割，得到待测物体的轮廓信息，并使用最小四边形框出物体轮廓，获取四边形四个顶点的坐标。
PCT/CN2023/104336 2023-04-27 2023-06-30 基于双目视觉的立体摄影测量方法 WO2024032233A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US18/541,978 US11959744B1 (en) 2023-04-27 2023-12-15 Stereophotogrammetric method based on binocular vision

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202310464266.0A CN116188558B (zh) 2023-04-27 2023-04-27 基于双目视觉的立体摄影测量方法
CN202310464266.0 2023-04-27

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/541,978 Continuation US11959744B1 (en) 2023-04-27 2023-12-15 Stereophotogrammetric method based on binocular vision

Publications (1)

Publication Number Publication Date
WO2024032233A1 true WO2024032233A1 (zh) 2024-02-15

Family

ID=86452580

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/104336 WO2024032233A1 (zh) 2023-04-27 2023-06-30 基于双目视觉的立体摄影测量方法

Country Status (4)

Country Link
US (1) US11959744B1 (zh)
CN (1) CN116188558B (zh)
WO (1) WO2024032233A1 (zh)
ZA (1) ZA202308186B (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116188558B (zh) * 2023-04-27 2023-07-11 华北理工大学 基于双目视觉的立体摄影测量方法
CN116721109B (zh) * 2023-08-11 2023-11-03 合肥图迅电子科技有限公司 一种双目视觉图像半全局匹配方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150271472A1 (en) * 2014-03-24 2015-09-24 Lips Incorporation System and method for stereoscopic photography
CN108682026A (zh) * 2018-03-22 2018-10-19 辽宁工业大学 一种基于多匹配基元融合的双目视觉立体匹配方法
CN111260597A (zh) * 2020-01-10 2020-06-09 大连理工大学 一种多波段立体相机的视差图像融合方法
CN113763269A (zh) * 2021-08-30 2021-12-07 上海工程技术大学 一种用于双目图像的立体匹配方法
CN116188558A (zh) * 2023-04-27 2023-05-30 华北理工大学 基于双目视觉的立体摄影测量方法

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8619082B1 (en) * 2012-08-21 2013-12-31 Pelican Imaging Corporation Systems and methods for parallax detection and correction in images captured using array cameras that contain occlusions using subsets of images to perform depth estimation
CN103985128B (zh) * 2014-05-23 2017-03-15 南京理工大学 一种基于颜色内相关和自适应支撑权重的立体匹配方法
CN104794713B (zh) * 2015-04-15 2017-07-11 同济大学 基于arm和双目视觉的温室作物数字化成像方法
CN106355570B (zh) * 2016-10-21 2019-03-19 昆明理工大学 一种结合深度特征的双目立体视觉匹配方法
CN106780590B (zh) * 2017-01-03 2019-12-24 成都通甲优博科技有限责任公司 一种深度图的获取方法及系统
CN109493373B (zh) * 2018-11-07 2020-11-10 上海为森车载传感技术有限公司 一种基于双目立体视觉的立体匹配方法
CN109887021B (zh) * 2019-01-19 2023-06-06 天津大学 基于跨尺度的随机游走立体匹配方法
CN110473217B (zh) * 2019-07-25 2022-12-06 沈阳工业大学 一种基于Census变换的双目立体匹配方法
CN111754588B (zh) * 2020-06-30 2024-03-29 江南大学 一种基于方差的自适应窗口大小的双目视觉匹配方法
CN111833393A (zh) * 2020-07-05 2020-10-27 桂林电子科技大学 一种基于边缘信息的双目立体匹配方法
EP3937137A1 (en) * 2020-07-07 2022-01-12 X-Rite Europe GmbH Visualizing the appearances of at least two materials
CN114120012A (zh) * 2021-11-29 2022-03-01 江苏科技大学 一种基于多特征融合和树形结构代价聚合的立体匹配方法
CN114255286B (zh) * 2022-02-28 2022-05-13 常州罗博斯特机器人有限公司 一种多视角双目视觉感知的目标尺寸测量方法
CN115995074A (zh) * 2022-12-26 2023-04-21 淮安中科晶上智能网联研究院有限公司 一种基于改进半全局立体匹配算法的无人车测距方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150271472A1 (en) * 2014-03-24 2015-09-24 Lips Incorporation System and method for stereoscopic photography
CN108682026A (zh) * 2018-03-22 2018-10-19 辽宁工业大学 一种基于多匹配基元融合的双目视觉立体匹配方法
CN111260597A (zh) * 2020-01-10 2020-06-09 大连理工大学 一种多波段立体相机的视差图像融合方法
CN113763269A (zh) * 2021-08-30 2021-12-07 上海工程技术大学 一种用于双目图像的立体匹配方法
CN116188558A (zh) * 2023-04-27 2023-05-30 华北理工大学 基于双目视觉的立体摄影测量方法

Also Published As

Publication number Publication date
CN116188558A (zh) 2023-05-30
CN116188558B (zh) 2023-07-11
ZA202308186B (en) 2023-09-27
US11959744B1 (en) 2024-04-16

Similar Documents

Publication Publication Date Title
CN110569704B (zh) 一种基于立体视觉的多策略自适应车道线检测方法
WO2024032233A1 (zh) 基于双目视觉的立体摄影测量方法
CN107767442B (zh) 一种基于Kinect和双目视觉的脚型三维重建与测量方法
US20220207776A1 (en) Disparity image fusion method for multiband stereo cameras
CN111563564B (zh) 基于深度学习的散斑图像逐像素匹配方法
CN110288642B (zh) 基于相机阵列的三维物体快速重建方法
KR100513055B1 (ko) 변이지도 및 깊이지도의 융합을 통한 3차원 장면 모델생성 장치 및 그 방법
CN112053432B (zh) 一种基于结构光与偏振的双目视觉三维重建方法
JP4095491B2 (ja) 距離測定装置、距離測定方法、及び距離測定プログラム
CN113034568B (zh) 一种机器视觉深度估计方法、装置、系统
US9025862B2 (en) Range image pixel matching method
CN110956661B (zh) 基于双向单应矩阵的可见光与红外相机动态位姿计算方法
CN107588721A (zh) 一种基于双目视觉的零件多尺寸的测量方法及系统
CN113129430B (zh) 基于双目结构光的水下三维重建方法
CN108613637A (zh) 一种基于参考图像的结构光系统解相方法及系统
CN111996883B (zh) 一种检测公路路面宽度的方法
KR101714224B1 (ko) 센서 융합 기반 3차원 영상 복원 장치 및 방법
CN107374638A (zh) 一种基于双目视觉模块的身高测量系统及方法
CN116958419A (zh) 一种基于波前编码的双目立体视觉三维重建系统和方法
CN116295113A (zh) 一种融合条纹投影的偏振三维成像方法
WO2022057556A1 (zh) 一种基于深度学习的端到端散斑投影三维测量方法
CN110487254B (zh) 一种用于rov的水下目标尺寸快速测量方法
Um et al. Three-dimensional scene reconstruction using multiview images and depth camera
CN113052898B (zh) 基于主动式双目相机的点云和强反光目标实时定位方法
Delmas et al. Stereo camera visual odometry for moving urban environments

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23851460

Country of ref document: EP

Kind code of ref document: A1