CN102253057B - Endoscope system and measurement method using endoscope system - Google Patents

Endoscope system and measurement method using endoscope system Download PDF

Info

Publication number
CN102253057B
CN102253057B CN 201110092627 CN201110092627A CN102253057B CN 102253057 B CN102253057 B CN 102253057B CN 201110092627 CN201110092627 CN 201110092627 CN 201110092627 A CN201110092627 A CN 201110092627A CN 102253057 B CN102253057 B CN 102253057B
Authority
CN
Grant status
Grant
Patent type
Prior art keywords
endoscope
system
measurement
method
endoscope system
Prior art date
Application number
CN 201110092627
Other languages
Chinese (zh)
Other versions
CN102253057A (en )
Inventor
宋展
郑翔
聂磊
陆丕清
黎文富
李涛
Original Assignee
中国科学院深圳先进技术研究院
深圳市亚泰光电技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Grant date

Links

Abstract

一种内窥镜系统,其特征在于,至少包括:图像采集装置,用于拍摄视频图像,包括主摄像头以及副摄像头,所述主摄像头的分辨率高于副摄像头的分辨率;控制器,用于控制所述主摄像头以及副摄像头的开启,并传输所述视频图像;图像处理装置,用于根据用户指令获取视频图像上的待测点,并对所述视频图像进行计算得到视频图像中待测点的三维信息。 An endoscope system, characterized in that, at least comprising: image capture means for capturing a video image, comprising a main camera and the sub camera, the main camera resolution higher than the resolution of the sub-camera; a controller, with for controlling the opening of the main camera and the sub camera, and transmission of the video image; an image processing means for obtaining a point to be measured on the video image in accordance with a user instruction, the video image and the video image to be calculated three-dimensional information of the measuring point. 上述内窥镜系统及使用内窥镜系统的测量方法,在图像采集装置中设置主摄像头和副摄像头,为获取待测点的三维信息开启主摄像头和副摄像头进行测量,主摄像头与副摄像头的结合大大地降低了内窥镜系统的成本,降低了图像采集装置的体积。 The endoscope system and a measuring method using the endoscope system, set the main camera and the sub camera in the image pickup device, the opening of the main camera and the sub camera to acquire three-dimensional information of the target point is measured, the main camera and the sub camera in combination greatly reduces the cost of the endoscope system, reducing the volume of the image pickup device.

Description

内窥镜系统及使用内窥镜系统的测量方法【技术领域】[0001] 本发明涉及计算机视觉技术,特别是涉及一种内窥系统及使用内窥镜系统的测量方法。 The endoscope system using the measuring method and an endoscope system TECHNICAL FIELD [0001] The present invention relates to computer vision, and more particularly to a method for measuring the use of an endoscopic system and endoscope system. 【背景技术】[0002] 内窥镜由于存在着能够无损地检测到微小物体的优点,并能观察到人无法到达的地方等优点,被广泛地应用于各个领域中,例如,对建筑行业中墙壁的检测,在油气工业中管道内部情况的检测,在汽车工业中发动机维修的检测以及安防领域中爆炸物的检测等领域都广泛地使用了内窥镜。 BACKGROUND OF THE INVENTION [0002] due to the presence of the endoscope can be detected without damage to the advantages of the minute object, and can be observed where people can not reach, etc., it is widely used in various fields, for example, for the construction industry detecting a wall, in the case of oil and gas industry for detecting an interior of the pipe, detecting in the automotive industry and engine maintenance detection of explosives in the field of security, and other fields are widely used endoscope. [0003] 在内窥镜的发展过程中,简单的二维影像不能够提供足够多的信息,使得可提供三维信息的内窥镜迅速发展。 [0003] In the development of the endoscope, a simple two-dimensional image can not provide enough information, so that the endoscope can provide three-dimensional information rapidly. [0004] 根据工作原理,将可提供三维信息的内窥镜划分为虚拟现实式以及光学式。 [0004] According to the principle, will provide three-dimensional information of the endoscope is divided into virtual reality type and an optical type. 其中, 虚拟现实式的内窥镜是通过光栅分别将左右摄像机所拍摄的图片送入使用者的左右眼,从而形成立体影像。 Wherein the virtual reality type endoscope by the user is fed to the left and right gratings image captured by the camera right and left eyes, thereby forming a three-dimensional image. 可是,虚拟现实式的内窥镜需要使用都佩戴特殊的立体视觉设备,如立体视觉头盔、眼镜等,无法精确地进行三维测量,且价格非常地昂贵。 However, virtual reality-type endoscope need to use all wear special stereoscopic vision devices, such as stereo vision helmets, glasses, etc., can not accurately measure the three-dimensional, and the price is very expensive. 光学式的内窥镜是通过两个内窥镜组成立体视觉测量系统,得利用精确的运动控制装置或者结构光装置来重构三组信息,成本非常高。 Optical endoscope is composed of an endoscope by two stereo vision measurement system, give accurate motion apparatus using structured light apparatus or a control message reconstructing three groups, the cost is very high. 【发明内容】[0005] 基于此,有必要提供一种可降低成本的内窥镜系统。 SUMMARY OF THE INVENTION [0005] Based on this, it is necessary to reduce the cost of providing an endoscope system. [0006] 此外,还有必要提供一种应用了上述内窥镜系统的使用内窥镜系统的测量方法。 [0006] In addition, there is a need to provide an application method for measuring the endoscope system using the endoscope system. [0007] —种使用内窥镜系统的测量方法,包括以下步骤:[0008] 分别通过主摄像头以及副摄像头拍摄得到视频图像,所述主摄像头的分辨率高于副摄像头的分辨率;[0009] 计算所述视频图像中的视差图像;[0010] 根据用户指令获取主视频图像上的待测点;[0011] 根据所述视差图像对所述副视频图像进行计算得到主视频图像中待测点的三维信息。 [0007] - measurement system using an endoscope species, comprising the steps of: [0008] are captured by the main camera and the sub camera to obtain a video image having a resolution higher than the resolution of the camera main sub camera; [0009 ] calculating a disparity image of the video image; [0010] Gets the point to be measured on the main video image in accordance with a user instruction; [0011] primary video image in the obtained parallax images to the sub-video image is calculated based on measured three-dimensional information points. [0012] 优选地,还包括:[0013] 关闭所述副摄像头,显示所述主摄像头中的画面。 [0012] Preferably, further comprising: [0013] close the secondary camera, the main camera screen display. [0014] 优选地,所述分别通过主摄像头以及副摄像头拍摄得到视频图像的步骤之前还包括:[0015] 对所述主摄像头以及副摄像头进行标定得到标定参数的步骤。 [0014] Preferably, prior to each of said main camera and the sub camera capturing video images obtained in step by further comprising: [0015] the main camera and the sub camera calibration parameters obtained in the calibration step. [0016] 优选地,所述视频图像包括王摄像头拍摄的王视频图像以及副摄像头拍摄的副视频图像,所述计算所述视频图像中的视差图像的步骤为:[0017] 通过局部滑动窗口方式或全局方式对主视频图像和副视频图像进行匹配得到视差图像。 Step [0016] Preferably, the video image comprises sub picture video Wang Wang image captured by the camera and the sub camera shooting, calculates the parallax images in a video image as: [0017] By way of a local sliding window or globally main video image and sub-video image matching obtain parallax images. [0018] 优选地,所述根据用户指令获取视频图像上的待测点的步骤为:[0019] 获取用户在主视频图像上选择的待测点;[0020] 所述根据所述视差图像对所述视频图像进行计算得到视频图像中待测点的三维信息的步骤为:[0021] 根据视差图像以及主视频图像上的待测点得到待测点在副视频图像中的匹配占.[0022] 通过标定参数及副视频图像中的匹配点进行计算得到所述待测点的三维信息。 [0018] Preferably, the acquisition step according to a user instruction points to be measured on the video image: [0019] obtaining test point selected by the user on the main video image; [0020] The parallax images according to the said step of calculating to obtain a three-dimensional video image information in the video image of the target point is: [0021] to give the target point in the sub-video image matching accounts according to the point to be measured parallax image and a main image video [0022. ] measured to obtain three-dimensional information of the point by point calibration parameters and the matching sub-video image are calculated. [0023] 上述内窥镜系统及使用内窥镜系统的测量方法,在图像采集装置中设置主摄像头和副摄像头,为获取待测点的三维信息开启主摄像头和副摄像头进行测量,主摄像头与副摄像头的结合大大地降低了内窥镜系统的成本,降低了图像采集装置的体积。 [0023] The measuring method of the endoscope system and the use of an endoscope system provided main camera and the sub camera in the image pickup device, the opening of the main camera and the sub camera is measured to obtain three-dimensional information of the target point, the main camera and binding sub-camera greatly reduces the cost of the endoscope system, reducing the volume of the image pickup device. [0024] 上述内窥镜系统及使用内窥镜系统的测量方法中,在用户通过视频图像进行观察时关闭副摄像头,仅通过主摄像头拍摄视频图像,并显示,极大地节省了系统资源。 [0024] The measuring method and an endoscope system using the endoscope system described above, the closing when the user observed by the sub video camera, a video image captured by only the master camera, and displayed significant savings in system resources. 【附图说明】[0025] 图1为一个实施例中内窥镜系统的结构示意图;[0026] 图2为图1中图像采集装置的结构示意图;[0027] 图3为一个实施例中主视频图像的示意图;[0028] 图4为图3中与主视频图像对应的副视频图像的示意图;[0029] 图5为一个实施例中待测点的三维信息计算的示意图;[0030] 图6为一个实施例中使用内窥镜系统的测量方法的流程图;[0031] 图7为图6中根据视差图像对视频图像进行计算得到视频图像中待测点的三维信息的流程图;[0032] 图8为一个实施例中内窥镜系统的应用示意图;[0033] 图9为图8中图像采集装置的另一角度示意图。 BRIEF DESCRIPTION [0025] Figure 1 is a schematic view of an endoscope system according to an embodiment; 1 a schematic view of the structure of the image pickup apparatus [0026] FIG. 2 is a diagram; [0027] Example 3 is an embodiment of the main a schematic view of a video image; [0028] FIG. 4 is a schematic view in FIG 3 the main video image corresponding to the sub-video image; [0029] FIG. 5 is a schematic three-dimensional information of the target point calculated in the embodiment of the embodiment; [0030] FIG. 6 is a flowchart illustrating a measurement method using the endoscope system according to one embodiment; [0031] FIG. 7 is a flowchart showing the calculated three-dimensional information in the video image of the target point in FIG. 6 according to the video image of parallax images; [ 0032] FIG. 8 is a schematic diagram of a application example of the endoscope system of the embodiment; [0033] in another aspect of FIG. 9 in FIG. 8 is a schematic view of an image pickup device. 【具体实施方式】[0034]图1示出了一个实施例中内窥镜系统的详细结构,该内窥镜系统包括图像采集装置10、控制器30以及图像处理装置50。 DETAILED DESCRIPTION OF EMBODIMENTS [0034] FIG 1 illustrates a detailed structure of an endoscope system according to an embodiment, the endoscope system comprising an image acquisition apparatus 10, a controller 30 and an image processing apparatus 50. [0035] 结合参阅图2,图像采集装置10用于拍摄视频图像,包括主摄像头101以及副摄像头103,主摄像头101的分辨率高于副摄像头103的分辨率。 [0035] Referring to FIG 2, the image capture device 10 for capturing video images, comprising a main camera 101 and sub-camera 103, main camera 101 resolution higher than the resolution of the sub-camera 103. 本实施例中,图像采集装置10 置于内窥镜系统的前端,集成了主摄像头101和副摄像头103,主摄像头101的分辨率较高, 副摄像头103的分辨率较低。 In this embodiment, the image pickup apparatus 10 in the distal end of the endoscope system, an integrated main camera 101 and sub-camera 103, main camera 101 of higher resolution and lower resolution sub-camera 103. 例如,主摄像头101的型号为640 X 480像素,30帧/秒,直径约4.5毫米;副摄像头103的型号为320X240像素,30帧/秒,直径约2.5毫米。 For example, the main camera 101 model is 640 X 480 pixels, 30 frames / sec, about 4.5 mm in diameter; the sub camera is 320X240 pixels 103 model, 30 frames / sec, about 2.5 millimeters in diameter. 与传统的单摄像头内窥镜系统相比,图像采集装置10通过增加低成本、小体积的副摄像头103保证了图像质量,并实现系统的三维测量功能。 Compared with the traditional single camera endoscope system, the image pickup apparatus 10 by increasing the cost, a small volume of the sub-camera 103 to ensure the quality of the image, and three-dimensional measurement function of the system. [0036] 在进行测量之前,需要进行内窥镜系统的标定,开启了的主摄像头101拍摄得到主标定图像,副摄像头103拍摄得到副标定图像。 [0036] Before measurement, calibration of the endoscope system of the main imaging head 101 opens the main photographed calibration image 103 captured by the sub camera calibration sub-picture. 具体地,将平面靶标图像放置于内窥镜系统的前端,同时开启主摄像头101和副摄像头103,分别拍摄多组不同位置不同角度的标定图像,其中,每一组标定图像包括了主标定图像和副标定图像,每一主标定图像都有与其对应的副标定图像。 Specifically, the planar target image was placed on the distal end of the endoscope system, and turn on the main camera 101 and sub-camera 103, respectively photographed calibration image different plurality of sets of different position angles, wherein each set of calibration images comprises a main calibration image and a secondary labeling image, each calibration image sub master calibration has corresponding thereto. 为保证标定的准确性,主标定图像与副标定图像均不少于三张。 To ensure the accuracy of the calibration, the calibration of the main image and sub-image calibration are no less than three. 平面靶标图像可以是棋盘格图像、圆形图像或同心圆图像。 The target image may be a planar checkerboard image, image or concentric circular images. [0037] 图像采集装置10中还包括了用于辅助照明的光源。 [0037] The image pickup apparatus 10 further includes an auxiliary light source for illumination. 为辅助黑暗环境中的照明,光源优选为LED (Light Emitting Diode,发光二极管)光源。 Auxiliary illuminating a dark environment, the light source preferably is a LED (Light Emitting Diode, LED) light source. 光源的数量优选为2个。 Number of light sources is preferably 2. [0038] 控制器30,用于控制主摄像头101以及副摄像头103的开启,并传输视频图像。 [0038] The controller 30 for controlling the opening of the main camera 101 and sub-camera 103, and transmit video images. 本实施例中,在进行测量时,控制器30开启主摄像头101和副摄像头103。 When the embodiment, the present embodiment the measurement is performed, the controller 30 turn on the main camera 101 and sub-camera 103. 控制器30通过视频传输线与图像采集装置10相连,控制着主摄像头101和副摄像头103的开启或关闭。 The controller 30 is connected via the video transmission line 10 and the image acquisition device, controls the opening of the main camera 101 and sub-camera 103 on or off. 在实际的使用过程中,用户在大多仅需要使用一个摄像头进行观察,只有在需要进行测量时才需要同时开启两个摄像头,因此,当用户进行观察时,控制器30开启主摄像头101,关闭副摄像头103 ;当用户需要进行测量时,控制器30同时开启主摄像头101和副摄像头103,从而实现了系统资源的充分利用。 In actual use, the user often requiring only a single camera for observation, need to open the two cameras only when measurements are needed, therefore, when the user viewed, the controller 30 turn on the main camera 101, close the sub camera 103; when the user need to be measured, the controller 30 is turned on while the main camera 101 and sub-camera 103, enabling full use of system resources. 在优选的实施例中,控制器30为USB (Universal Serial Bus,通用串行总线)控制器。 In a preferred embodiment, the controller 30 is a USB (Universal Serial Bus, Universal Serial Bus) controller. [0039] 图像处理装置50,用于根据用户指令获取视频图像上的待测点,并对视频图像进行计算得到视频图像中待测点的三维信息。 [0039] The image processing apparatus 50 for acquiring a video image on the point to be measured according to a user instruction, the video image and the video image is calculated to obtain three-dimensional information of the point to be measured. 本实施例中,视频图像包括主摄像头101拍摄的主视频图像和副摄像头103拍摄的副视频图像,用户在图像处理装置50所显示的主视频图像中选择待测点,图像处理装置50根据用户指令得到需要测量的待测点。 In this embodiment, the video image includes a main video image and the sub camera main camera 101 captured 103 captured sub-video image, the user selects a site to be tested on the main video image processing apparatus 50 displayed, the image processing apparatus 50 according to a user to give instructions to the target point to be measured. [0040] 为进行内窥镜系统的标定,图像处理装置50根据主标定图像及副标定图像计算得到标定参数。 [0040] 50 to obtain the calibration parameters for the calibration, the image processing system of an endoscope apparatus according to the master calibration image and a calibration sub-image calculation. [0041 ] 分别从主标定图像和副标定图像中提取特征点,利用主标定图像和副标定图像中的特征点以及已知的平面靶标图像中的坐标即可计算出标定参数。 [0041] respectively, from the main image and the sub-calibrated calibration image feature points are extracted using the calibration parameters to calculate calibration feature point of the main image and the sub image and the calibration with known coordinates in the plane of the target image. 标定参数中包括了主摄像头101与副摄像头103之间的相对位置关系以及摄像头内部参数。 Calibration parameters included in the relative positional relationship between the camera and the internal parameters between the main camera and the sub camera 101 103. 相对位置关系包括了旋转参数和平移参数等,摄像头内部参数包括了摄像头的焦距、光心位置以及畸变参数。 Rotating relative positional relationship parameters include translation parameters and the like, the camera internal parameters include the focal length of the camera, the position of the optical center and distortion parameters. 主标定图像与副标定图像中的特征点是依据平面靶标图像的图像类型决定的,例如,若平面靶标图像为棋盘图像,则将图像中的角点作为特征点;若平面靶标图像为圆形图像或者同心圆图像,则将图像中的圆心作为特征点。 Primary calibration image feature point sub-calibration image are determined according to image type planar target image, for example, if the planar target images corner checkerboard image, then the image as a feature point; if planar target image circular image or an image of concentric circles, then the center of the image as a feature point. [0042] 在图像采集装置10拍摄了视频图像之后,图像处理装置50采用局部滑动窗口方式或全局方式对主视频图像和副视频图像进行匹配得到视差图像,根据用户在主视频图像上选择的待测点以及视差图像得到待测点在副视频图像中的匹配点,并根据标定参数及副视频图像中的匹配点进行计算得到待测点的三维信息。 After the [0042] captured video images in the image pickup apparatus 10, the image processing apparatus 50 using a local sliding window or globally main video image and sub-video image matching obtained parallax images to be selected by the user on the main video image in accordance with parallax measuring point and the target point matching point to obtain an image in the sub-video image, and three-dimensional information according to the calculated target point and the matching point calibration parameter sub video images. [0043] 具体地,如图3和图4所示,局部滑动窗口方式对主视频图像和副视频图像进行匹配得到视差图像的过程中,设主视频图像300和副视频图像400中对应点均平行于V轴,对于主视频图像300上的任一点(U,V),将滑动窗口移动到点(U,V),得到的图像编码为向量w ;对于副视频图像400上的任一点(u+d, V),将滑动窗口移动到点(u+d, V),得到的图像编码为向量w'。 [0043] Specifically, as shown in FIGS. 3 and 4, a local sliding window embodiment of a main video image and sub-video image matching the parallax images obtained in the process, provided the main video and sub video image 300 image 400 corresponding points are V is parallel to the axis, to any of the video image on the main point 300 (the U-, V), the sliding window moves to point (the U-, V), obtained as a vector image encoding W; for any of the sub-video image point 400 ( u + d, V), the sliding window is moved to the point (u + d, V), an image obtained by coding a vector w '. 通过点积或夹角等方法计算w与w'的相似度与w'的相似度可作为主视频图像中点(U,V)与副视频图像中点(u+d,v)的相似度,不断改变d的大小,以求出相似度最高的一点(u+d_,v)作为主图像中点(U,V)的匹配点,进而生成视差图像。 W and w calculated by the angle method or the like dot product similarity 'similarity with w' can be used as the midpoint of the main video image (U, V) and the sub-video image similarity midpoint (u + d, v) of , changing the size d of the highest degree of similarity in order to point (u + d_, v) as the midpoint of the main image (U, V) of the matching points, thereby generating the parallax image. 通过局部滑动窗口方式可对生成的视差图像进行校正,极大地提高了视差图像的质量。 It can be corrected disparity image generated by the local sliding window embodiment, which greatly improves the quality of parallax images. [0044] 为了进一步提高视差图像的质量,在通过了局部滑动窗口方式得到视差图像之后,还可通过相邻匹配点的位置关系对生成的视差图像进行校正。 [0044] In order to further improve the quality of parallax images, by way of sliding window after local parallax images obtained, also corrects the parallax image generated by the positional relationship between adjacent matching point. [0045] 在通过全局方式对主视频图像和副视频图像进行匹配得到视差图像的过程中,为求解匹配点,构造图像匹配能量来进行主视频图像与副视频图像的匹配,为计算出视差图像D的能量公式E(D),计算过程如以下公式所示:[0046] [0045] for the main video image and sub-video image in a global manner matching obtained parallax image process for solving the matching point, configured image matching energy to a main video and the sub video matching images, the calculated parallax image the energy of the formula D E (D), calculation shown in the following formula: [0046]

Figure CN102253057BD00061

[0047] 其中,C(p,Dp)表示主视频图像中任一点P与副视频图像中点P+Dp的匹配误差, [0047] wherein, C (p, Dp) represents a main video image of any point P and the sub-matching error video image of the midpoint of P + Dp,

Figure CN102253057BD00062

表示主视频图像中任一点P与其领域像素Np的微小视差的惩罚因子, It represents the main video image to any point P in its field of pixels Np slight disparity penalty factor,

Figure CN102253057BD00063

代表主视频图像中像素点q与其领域像素Np的较大视差惩罚能量,P2与梯度幅值有关。 It represents the main video image and its pixel q pixels Np large disparity fields energy penalty, P2 related to the magnitude of the gradient. 通过全局方式进行匹配的过程中将使得整体图像的匹配效果比较局部滑动窗口方式的匹配效果好。 In the matching process to match a global manner so that the overall effect of good images compare local sliding window mode matching effect. [0048] 通过求解以上视差图像中能量的最小化即可直接求解出视差图像。 [0048] By solving the above parallax of parallax images can be directly solved for minimizing the energy in an image. [0049] 另一实施例中,用户还可进行手工匹配,以减少由于重复的纹理信息所带来的误匹配和匹配失败。 [0049] In another embodiment, the user may manually to match, in order to reduce the repetitive texture information brought mismatched and fails to match. [0050] 用户在主视频图像上选择待测点,图像处理装置50由视差图像得到副视频图像上的匹配点,根据该副视频图像上的匹配点与标定参数进行计算即可得到待测点的三维信肩、O[0051] 例如,如图5所示,待测点的三维信息计算过程可以是:设Cf为主摄像头101中心,为副摄像头103中心,主摄像头101与副摄像头103之间的中心间距为T,X1为主摄像头101上的待测点,&为副摄像头103上的匹配点。 [0050] The user selects the point to be measured on the main video image, the image processing apparatus 50 to obtain a matching point in the sub-video image by the parallax image, and the calculated calibration parameters can be obtained on the point to be measured according to the matching point of the sub video image three-dimensional letters shoulder, O [0051] for example, FIG. 5, the three-dimensional information calculation process of the target point may be: provided Cf main camera 101 center, sub-camera 103 center the head, the main camera 101 and sub-camera 103, center spacing between the point to be measured based on the imaging head 101 is T, X1, & matching point in the sub-camera 103. 根据相似三角形可以得到 Can be obtained according to the similar triangles

Figure CN102253057BD00064

求解可得到Z = -TfA1-XrO由上可知,在已标定T、f以及主视频图像上任一点P中X1和&之前的情况下即可计算出深度信息Z,再通过标定参数即可得到三维信息(X,y, z)。 Obtained solving Z = -TfA1-XrO seen from the above, the calibrated T, f and a main video image can be calculated at any point prior to the case where X1 is P and the depth information & Z, then the calibration parameters can be obtained by a three-dimensional information (X, y, z). [0052]另一实施例中,上述内窥镜系统的控制器30关闭副摄像头103,图像采集装置10 通过主摄像头101拍摄视频图像,并在图像处理装置50中显示视频图像。 [0052] In another embodiment, the endoscope system controller 30 closes the sub-camera 103, the image capture apparatus 10 captures video images through the main camera 101 and the video image 50 in the image processing apparatus. 在用户进行观察时,仅打开主摄像头101即可得到高分辨率的视频图像。 When a user to observe only the opening 101 of the master camera video image can be obtained high resolution. 在优选的实施例中,控制器30通过USB输出线与图像处理装置50连接。 In a preferred embodiment, the controller 30 is connected via a USB cable and output the image processing apparatus 50. 由于副摄像头103在大部分的时间中都处于闲置状态,仅在测量时开启,该USB输出线可以仅设置I条,主摄像头101与副摄像头103共用一条USB输出线与图像处理装置50相连。 Since the sub-camera 103 most of the time are in the idle state, is switched on only when the measurement, the USB output lines may be provided only I bar connected to the main camera 101 and sub-camera 103 share a USB output lines of the image processing apparatus 50. [0053] 此外,还有必要提供一种使用内窥镜系统的测量方法,如图6所示,该测量方法包括以下步骤:[0054] 步骤S10,分别通过主摄像头以及副摄像头拍摄得到视频图像,主摄像头的分辨率高于副摄像头的分辨率。 [0053] In addition, there is a need to provide a measurement method using the endoscope system shown in Figure 6, the measuring method comprising the steps of: [0054] Step S10, the respective photographing through the main camera and the sub camera to obtain a video image , the main camera resolution higher than the resolution of the sub-camera. 本实施例中,在进行测量管道裂缝、危险易燃易爆物品等测量时, 开启主摄像头和副摄像头进行视频图像的拍摄,其中,主摄像头的分辨率较高,副摄像头的分辨率较低。 When the present embodiment, during the measurement pipe fracture, inflammable and explosive dangerous articles and the like, to open the main camera and the sub camera for capturing video images, wherein the higher resolution of the main camera, the camera lower resolution sub . 例如,主摄像头的型号为640X480像素,30帧/秒,直径约4.5毫米;副摄像头的型号为320X240像素,30帧/秒,直径约2.5毫米。 For example, the main camera model 640X480 pixels, 30 frames / sec, about 4.5 mm in diameter; the sub camera model of 320X240 pixels, 30 frames / sec, about 2.5 millimeters in diameter. [0055] 上述使用内窥镜系统的测量方法中,在分别通过主摄像头以及副摄像头拍摄得到视频图像的步骤之前还包括了对主摄像头以及副摄像头进行标定得到标定参数的步骤。 Before [0055] The above-described measuring method using an endoscope system, respectively, through the main camera and the sub camera video image photographed step further comprises the step of the main camera and the sub camera is calibrated to obtain the calibration parameters. 本实施例中,标定参数中包括了主摄像头与副摄像头之间的相对位置关系以及摄像头内部参数。 In this embodiment, calibration parameters included in the relative positional relationship between the camera and the internal parameters between the main camera and the sub camera. 相对位置关系包括了旋转参数和平移参数等,摄像头内部参数包括了摄像头的焦距、光心位置以及畸变参数。 Rotating relative positional relationship parameters include translation parameters and the like, the camera internal parameters include the focal length of the camera, the position of the optical center and distortion parameters. 开启了的主摄像头拍摄得到主标定图像,副摄像头拍摄得到副标定图像。 Opening the main camera main photographed calibration image captured by the sub camera calibration sub-picture. 具体地,同时开启主摄像头和副摄像头进行平面靶标图像的拍摄,以得到多组标定图像。 Specifically, at the same time open the main camera and the sub camera for photographing the target image plane to obtain a plurality of sets of calibration images. 其中,每一组标定图像包括了主标定图像和副标定图像,每一主标定图像都有与其对应的副标定图像。 Wherein each group comprises a calibration image of the main image and the sub-calibrated calibration image, each calibration image sub master calibration has corresponding thereto. 为保证标定的准确性,主标定图像与副标定图像均不少于三张。 To ensure the accuracy of the calibration, the calibration of the main image and sub-image calibration are no less than three. 平面靶标图像可以是棋盘格图像、圆形图像或同心圆图像。 The target image may be a planar checkerboard image, image or concentric circular images. [0056] 分别从主标定图像和副标定图像中提取特征点,利用主标定图像和副标定图像的特征点以及已知的平面靶标图像中的坐标即可计算出标定参数。 [0056] respectively, from the main image and the sub-calibrated calibration image feature points are extracted, the calibration parameters can be calculated using the main image and the sub-calibrated calibration image feature point coordinates are known and the plane of the target image. 标定参数中包括了主摄像头与副摄像头之间的相对位置关系以及内部参数。 Calibration parameters include the internal parameters and the relative positional relationship between the main camera and the sub camera. 主标定图像与副标定图像中的特征点是依据平面靶标图像决定的,例如,若平面靶标图像为棋盘图像,则将图像中的角点作为特征点;若平面靶标图像为圆形图像或者同心圆图像,则将图像中的圆心作为特征点。 Primary calibration feature point image sub calibration image is based on planar target image determined, for example, if the planar target images corner checkerboard image, then the image as a feature point; if planar target image as a circular image or a concentric circle image, then the center of the image as a feature point. [0057] 步骤S30,计算视频图像中的视差图像。 [0057] Step S30, the calculation of the parallax images in a video image. 本实施例中,计算视频图像中的视差图像的步骤为:通过局部滑动窗口方式或全局方式对主视频图像和副视频图像进行匹配得到视差图像。 In this embodiment the step of calculating the parallax image as the video image: matching obtained parallax images to the main video image and sub-video images by local sliding window or globally. [0058] 具体地,局部滑动窗口方式对主视频图像和副视频图像进行匹配得到视差图像的过程中,在主视频图像上,将滑动窗口移动到任一点,得到该点的图像编码向量,在副视频图像上,同样将滑动窗口移动到任一点,并得到相应的图像编码向量。 [0058] Specifically, the local sliding window embodiment of a main video image and sub-video images obtained parallax image matching process, on the main video image, the sliding window moves to any point to obtain an image vector of the coding point, and in image, likewise the sub video window sliding movement to any point, and a vector encoding the corresponding image. 通过点积或夹角等方法计算主视频图像中图像编码向量与副视频图像中图像编码向量之间的相似度,不断变换副视频图像中的点,以得到与主视频图像中图像编码向量相似度最高的一点作为匹配点, 从而生成视差图像。 Dot product or method of calculating the similarity between the angle between the main video image and sub-image video coding vector coding vector image, changing point of the sub video images, main video image to obtain the image similar to the coding vector the highest point as the matching point, so as to generate a parallax image. 通过局部调动窗口方式可对生成的视差图像进行校正,极大地提高了视差图像的质量。 It can be corrected disparity image generated by the partial transfer window mode, which greatly improves the quality of parallax images. [0059] 为了进一步提高视差图像的质量,在通过了局部滑动窗口方式对视差图像进行校正后,还可通过相邻匹配点的位置关系对视差图像进行校正。 [0059] In order to further improve the quality of parallax images, after correcting the parallax images by the local sliding window mode, the image may be corrected by the positional relationship between the adjacent parallax matching point. [0060] 在通过全局方式对主视频图像和副视频图像进行匹配得到视差图像的过程中,为求解匹配点,构造图像匹配能量来进行主视频图像与副视频图像的匹配,为计算出视差图像D的能量公式E(D),计算过程如以下公式所示:[0061].=Σ(Χ/Ά,Η Σ ,顺厂久I=1]+ Σ "/[h.,—",卜「- P q-^-Np.q^Np _[0062] 其中,C(p,Dp)表示主视频图像中任一点P与副视频图像中点p+Dp的匹配误差,Σ —Al=1]表示主视频图像中任一点P与其领域像素Np的惩罚因子,Σ - Hl > 1I代表主视频图像中像素点q与其领域像素Np的较大视差惩罚能量,P2与梯度幅值有关。通过全局方式进行`匹配的过程中将使得整体图像的匹配效果比较局部滑动窗口方式的匹配效果好。[0063] 通过求解以上视差图像中能量的最小化即可直接求解出视差图像。[0064] 另一实施例中,用户还可进行手工匹配,以减少由于重复的纹理信息所 [0060] for the main video image and sub-video image in a global manner matching obtained parallax image process for solving the matching point, configured image matching energy to a main video and the sub video matching images, the calculated parallax image D energy equation E (D), calculation shown in the following formula: [0061] = Σ (Χ / Ά, Η Σ, cis factory For I = 1] + Σ "/[h.,-", Bu. "- P q - ^ - Np.q ^ Np _ [0062] where, C (p, Dp) represents a main video image at any point in the midpoint P and the sub-video image matching error p + Dp, Σ -Al = 1 ] represents the main video image to any point P in its field of pixels Np penalty factor, Σ - Hl> 1I represents the main video image and its pixel q pixels Np large disparity fields energy penalty, P2 and by the global gradient magnitude related. `matching process performed in the embodiment such that the overall effect of the image matching is better match the local effect of the sliding window embodiment. [0063] can be solved by solving the parallax image minimizing parallax image energy directly above. [0064] another embodiment, the user may manually to match, in order to reduce the repetition of the texture information 来的误匹配和匹配失败。[0065] 步骤S50,根据用户指令获取视频图像上的待测点。本实施例中,根据用户指令获取视频图像上的待测点的步骤为:获取用户在主视频图像上选择的待测点。[0066] 步骤S70,根据视差图像对视频图像进行计算得到视频图像中待测点的三维信息。 本实施例中,用户在主视频图像上选择待测点,由视差图像得到副视频图像上的匹配点,根据该副视频图像上的匹配点与标定参数进行计算即可得到待测点的三维信息。[0067] 在一个具体的实施例中,如图7所示,根据视差图像对视频图像进行计算得到视频图像中待测点的三维信息的步骤为:[0068] 步骤S701,根据视差图像以及主视频图像上的待测点得到待测点在副视频图像中的匹配点。本实施例中,由视差图像即可得到副视频图像中与待测点对应的匹配点。[0069] 步骤S703,通过标定参数及 And matching to mismatching failure [0065] step S50, the point to be measured is acquired according to a user instruction on a video image according to the present embodiment, the step of obtaining test point on a video image according to a user instruction: Get user in the main selected video image point to be measured. [0066] step S70, the parallax images to obtain the video image information is calculated according to the three-dimensional video image of the target point. in this embodiment, the user selects the point to be measured on the main video image, to give matching points on the parallax image sub video images, three-dimensional information obtained point to be measured can be calculated based on the matching points on the sub video images and calibration parameters. [0067] in a particular embodiment, FIG. 7 step, the obtained three-dimensional information in the video image of the target point on the parallax image for the video image is calculated according to: [0068] step S701, the obtained point measured according to the point to be measured on the secondary video in the main video image and parallax images matching point in the image. in this embodiment, the parallax images can be obtained with a sub-video image matching point corresponding to the point to be measured. [0069] step S703, the calibration parameters and by 副视频图像中的匹配点进行计算待测点的三维信息。 本实施例中,三维信息中记录了待测点的三维空间坐标。[0070] 此外,根据用户的需求,还可根据待测点的三维空间坐标计算出深度、高度、面积以及截面等信息。[0071] 其他实施例中,上述测量方法还包括了关闭副摄像头,显示主摄像头中的画面的步骤。本实施例中,在用户进行观察时,仅打开主摄像头即可得到高分辨率的视频图像。[0072] 下面结合一个实施例来详细阐述上述内窥镜系统及使用内窥镜系统的测量方法的应用过程。 Matching point in the sub-three-dimensional video image information calculation point under test. In this embodiment, the three-dimensional information recorded in the three-dimensional coordinates of the point to be measured. [0070] Further, according to the needs of users, according to the site to be tested may be the three-dimensional coordinates of the calculated information depth, height, area, and cross-section, etc. [0071] other embodiments, the above-described measuring method further comprises the step of turning off the sub camera, display the main camera of the screen. in this embodiment, the user when viewed, can only open the main camera of high resolution video images. [0072] a connection with the following examples detail the endoscope system using application process and method for measuring an endoscope system. 该实施例中,如图8和图9所示,在对内窥镜系统进行了标定后,将图像采集装置10移动待测物体周围能够清晰成像的位置,该位置可根据镜头焦距而定,例如,可以是距离待测物体50〜100毫米的位置。 In this embodiment, as shown in FIG. 8 and FIG. 9, after the calibration of the endoscope system, the image pickup apparatus 10 moves around the object to be detected can be clearly imaged position, this position can be determined according to the lens focal length, For example, the distance may be 50~100 mm measured position of the object. [0073] 在光源105的辅助下,若用户需要对待测物体进行观测,则控制器30只需要开启主摄像头101,通过主摄像头101所拍摄的视频图像进行观测。 [0073] In the auxiliary light source 105, if the user needs to be treated observation object to be measured, the controller 30 only needs to turn on the main camera 101, video images captured by the main camera 101 is observed. [0074] 若用户需要测量待测物体中某一点的位置,则控制器30将同时开启主摄像头101 和副摄像头103,并分别拍摄得到主视频图像和副视频图像。 [0074] If the user needs to measure the position of a point in the object to be measured, the controller 30 will also open the main camera 101 and sub-camera 103 are photographed and the main video image and sub-video image. 图像通过局部滑动窗口方式或全局方式对主视频图像和副视频图像进行匹配得到视差图像。 Main image video image and the sub video by local sliding window or globally matching the parallax image obtained. [0075] 用户对图像处理装置50所显示的主视频图像进行待测点的选择,图像处理装置50根据用户所选择的待测点由视差图像得到副视频图像上的匹配点,根据副视频图像上匹配点与标定参数进行计算即可得到待测点的三维信息。 [0075] user to the main video image displayed by the image processing apparatus 50 selected test point, the image processing apparatus 50 to obtain a matching point in the sub-video image by the parallax image selected by the user according to the site to be tested, according to the sub video and the matching point on the calibration parameters to calculate three-dimensional information to obtain the point to be measured. [0076] 上述内窥镜系统及使用内窥镜系统的测量方法,在图像采集装置中设置主摄像头和副摄像头,为获取待测点的三维信息开启主摄像头和副摄像头进行测量,主摄像头与副摄像头的结合大大地降低了内窥镜系统的成本,降低了图像采集装置的体积。 [0076] The measuring method of the endoscope system and the use of an endoscope system provided main camera and the sub camera in the image pickup device, the opening of the main camera and the sub camera is measured to obtain three-dimensional information of the target point, the main camera and binding sub-camera greatly reduces the cost of the endoscope system, reducing the volume of the image pickup device. [0077] 上述内窥镜系统及使用内窥镜系统的测量方法中,在用户通过视频图像进行观察时关闭副摄像头,仅通过主摄像头拍摄视频图像,并显示,极大地节省了系统资源[0078] 以上所述实施例仅表达了本发明的几种实施方式,其描述较为具体和详细,但并不能因此而理解为对本发明专利范围的限制。 [0077] The method of measurement and the endoscope system using an endoscope system, when a user closes the image observed by the sub video camera, a video image captured by only the master camera, and displayed significant savings in system resources [0078 ] the above embodiments are only expressed several embodiments of the present invention, and detailed description thereof is more specific, but can not therefore be understood as limiting the scope of the present invention. 应当指出的是,对于本领域的普通技术人员来说,在不脱离本发明构思的前提下,还可以做出若干变形和改进,这些都属于本发明的保护范围。 It should be noted that those of ordinary skill in the art, without departing from the spirit of the present invention, can make various changes and modifications, which fall within the protection scope of the present invention. 因此,本发明专利的保护范围应以所附权利要求为准。 Therefore, the protection scope of the present invention should be subject to the appended claims.

Claims (5)

  1. 1.一种使用内窥镜系统的测量方法,包括以下步骤:分别通过主摄像头以及副摄像头拍摄得到视频图像,所述主摄像头的分辨率高于副摄像头的分辨率,所述视频图像包括王摄像头拍摄的王视频图像和副摄像头拍摄的副视频图像;计算所述视频图像中的视差图像;根据用户指令获取主视频图像上的待测点;根据所述视差图像对所述副视频图像进行计算得到主视频图像中待测点的三维信息。 An endoscope system using the measuring method, comprising the steps of: respectively captured by the main camera and the sub camera to obtain a video image having a resolution higher than the resolution of the camera main sub camera, said video image comprising Wang sub video camera capturing the video image and the sub Wang captured by the camera; calculating a disparity image of the video image; obtaining point to be measured on the main video image in accordance with a user instruction; for the sub video image according to the image parallax calculate three-dimensional information of the main video image point under test.
  2. 2.根据权利要求1所述的使用内窥镜系统的测量方法,其特征在于,还包括:关闭所述副摄像头,显示所述主摄像头中的画面。 The measuring method using the endoscope system according to claim 1, characterized in that, further comprising: closing the sub camera, the main camera screen display.
  3. 3.根据权利要求1所述的使用内窥镜系统的测量方法,其特征在于,所述分别通过主摄像头以及副摄像头拍摄得到视频图像的步骤之前还包括:对所述主摄像头以及副摄像头进行标定得到标定参数的步骤。 The measuring method of using the endoscope system according to claim, characterized in that the respectively captured by the main camera and the sub camera video image before the step of obtaining further comprises: the main camera and the sub camera for calibration parameters obtained calibration step.
  4. 4.根据权利要求2所述的使用内窥镜系统的测量方法,其特征在于,所述视频图像包括王摄像头拍摄的王视频图像以及副摄像头拍摄的副视频图像,所述计算所述视频图像中的视差图像的步骤为:通过局部滑动窗口方式或全局方式对主视频图像和副视频图像进行匹配得到视差图像。 4. The method of claim 2 measuring endoscope system according to claim, wherein said video image comprises sub-video image captured by the camera Wang Wang video images taken by a camera and a sub, the computing of the video image step parallax image is: matching the main parallax image to obtain video image and the sub video by local sliding window or globally.
  5. 5.根据权利要求3所述的使用内窥镜系统的测量方法,其特征在于,所述根据用户指令获取主视频图像上的待测点的步骤为:获取用户在主视频图像上选择的待测点;所述根据所述视差图像对所述副视频图像进行计算得到主视频图像中待测点的三维信息的步骤为:根据视差图像以及主视频图像上的待测点得到待测点在副视频图像中的匹配点;通过标定参数及副视频图像中的匹配点进行计算得到所述待测点的三维信息。 The measuring method of using the endoscope system of claim 3, characterized in that the step point to be measured on the main video image is acquired according to a user instruction: Get be selected by the user on the main video image measuring point; obtained three-dimensional information of the main video image, the parallax image of the target point of the sub video image is calculated according to the steps of: obtain a point to be measured according to the test points on the parallax image, and a main video image matching point in the sub video images; three-dimensional information of the measured points obtained by the matching points in the image of the calibration parameters are calculated and the sub video.
CN 201110092627 2011-04-13 2011-04-13 Endoscope system and measurement method using endoscope system CN102253057B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN 201110092627 CN102253057B (en) 2011-04-13 2011-04-13 Endoscope system and measurement method using endoscope system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN 201110092627 CN102253057B (en) 2011-04-13 2011-04-13 Endoscope system and measurement method using endoscope system

Publications (2)

Publication Number Publication Date
CN102253057A true CN102253057A (en) 2011-11-23
CN102253057B true CN102253057B (en) 2013-06-05

Family

ID=44980446

Family Applications (1)

Application Number Title Priority Date Filing Date
CN 201110092627 CN102253057B (en) 2011-04-13 2011-04-13 Endoscope system and measurement method using endoscope system

Country Status (1)

Country Link
CN (1) CN102253057B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103505218B (en) * 2013-10-10 2015-06-24 南京普派医疗科技有限公司 Method for measuring focus tissue size through endoscope
CN104236515B (en) * 2014-09-11 2017-01-18 山东省计量科学研究院 Electronic Endoscope viewing angle automatic measuring apparatus
CN105120135A (en) * 2015-08-25 2015-12-02 努比亚技术有限公司 Binocular camera

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0316244A1 (en) 1987-11-12 1989-05-17 Welch Allyn, Inc. Video equipped endoscope with needle probe
CN1283086A (en) 1997-11-04 2001-02-07 赛特莱恩技术有限公司 Video rectoscope
CN2621596Y (en) 2003-05-21 2004-06-30 高龙关 Simple health endoscope
CN200970224Y (en) 2006-11-09 2007-11-07 上海大学 Three dimension positioning system for medical endoscope body
CN201248687Y (en) 2008-07-08 2009-06-03 欧普康光电(厦门)有限公司 Medical endoscope adopting a digital image device
CN101558987A (en) 2009-05-22 2009-10-21 俭 朱 Fingertip video speculum with protective sleeve

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3791777B2 (en) * 2001-12-28 2006-06-28 オリンパス株式会社 Electronic endoscope

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0316244A1 (en) 1987-11-12 1989-05-17 Welch Allyn, Inc. Video equipped endoscope with needle probe
CN1283086A (en) 1997-11-04 2001-02-07 赛特莱恩技术有限公司 Video rectoscope
CN2621596Y (en) 2003-05-21 2004-06-30 高龙关 Simple health endoscope
CN200970224Y (en) 2006-11-09 2007-11-07 上海大学 Three dimension positioning system for medical endoscope body
CN201248687Y (en) 2008-07-08 2009-06-03 欧普康光电(厦门)有限公司 Medical endoscope adopting a digital image device
CN101558987A (en) 2009-05-22 2009-10-21 俭 朱 Fingertip video speculum with protective sleeve

Also Published As

Publication number Publication date Type
CN102253057A (en) 2011-11-23 application

Similar Documents

Publication Publication Date Title
US20110122251A1 (en) Comparison of Infrared Images
US20130188834A1 (en) Gaze point detection method and gaze point detection device
JP2010259605A (en) Visual line measuring device and visual line measuring program
CN1932841A (en) Petoscope based on bionic oculus and method thereof
US20110249117A1 (en) Imaging device, distance measuring method, and non-transitory computer-readable recording medium storing a program
JP2007163367A (en) Camera information analyzer
CN102395036A (en) Apparatus and method for providing 3D augmented reality
CN102042825A (en) Three-dimensional imaging measurement system combining planar array imaging with laser scanning
CN102156537A (en) Equipment and method for detecting head posture
Niola et al. A method for the calibration of a 3-D laser scanner
JP2000287223A (en) Method and device for three-dimensional data input
CN103824318A (en) Multi-camera-array depth perception method
CN102207371A (en) Three-dimensional point coordinate measuring method and measuring apparatus thereof
US20080074648A1 (en) Method and Apparatus for Three-Dimensional Measurement of Objects in an Extended angular range
CN201072403Y (en) Device for measuring solid particle three-dimensional concentration field and velocity field in gas/solid two-phase stream
CN102081296A (en) Device and method for quickly positioning compound-eye vision imitated moving target and synchronously acquiring panoramagram
CN102692213A (en) Traffic accident field surveying instrument based on active omnidirectional visual sensor
JP2003130621A (en) Method and system for measuring three-dimensional shape
CN103954221A (en) Binocular photogrammetry method of large flexible structure vibration displacement
CN101464132A (en) Position confirming method and apparatus
CN101301236A (en) Eyesight protection system based on three-dimensional camera shooting and method
JP2009250740A (en) Image synthesizing method and visual inspecting device of tire
US20140300703A1 (en) Image processing apparatus, image processing method, and program
JP2012028949A (en) Image processing device and control method of the same
CN103528520A (en) Binocular vision-based synchronous operation lifting system detection system and method

Legal Events

Date Code Title Description
C06 Publication
C10 Request of examination as to substance
C14 Granted